system

Abstract

Provide a system. [Solution] receiving medical information input by a user using a terminal; A means for searching for an appropriate specialist based on the medical information; means for acquiring schedule information of the specialist and confirming an appointment; a video call means for connecting the user and the specialist at the appointment time; means for saving the medical treatment details after the medical treatment session ends and notifying the user of the saving; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The disparity in access to medical services between urban and remote areas is a major barrier to receiving quality medical care. Patients also face difficulties in quickly finding an appropriate specialist, making an appointment, and receiving ongoing follow-up care. This can lead to delays in appropriate treatment and waste of medical resources. To address these challenges, more efficient and user-friendly methods of accessing medical care are needed. [Means for solving the problem]

[0005] The present invention provides a system that allows a user to input medical information using a terminal, search for a necessary specialist, and make an appointment. Specifically, the system includes a means for receiving the medical information input by the user using a terminal and searching for an appropriate specialist based on the medical information, a means for acquiring the specialist's schedule information and confirming the appointment, a video call means for connecting the user with the specialist at the scheduled appointment time, and a means for saving the medical treatment details after the medical treatment session ends and notifying the user. Furthermore, the video call means establishes a secure video call session between the user and the specialist, and the medical treatment details storage means enables the next medical treatment schedule and follow-up notifications to be set, thereby realizing continuous medical support.

[0006] "User" refers to an individual who receives medical services using the system.

[0007] "Terminal" refers to an electronic device such as a smartphone, PC, or tablet that a user uses to access the system and use medical services.

[0008] "Medical information" refers to information entered by the user that is necessary to receive medical services, such as medical condition, symptoms, desired consultation time, and medical insurance information.

[0009] "Specialist" refers to a medical professional who has advanced knowledge and skills in a specific medical field and provides medical services to users.

[0010] "Search methods" refers to algorithms and technologies used to find appropriate specialists based on the medical information entered by the user.

[0011] "Schedule information" refers to the dates and times when a specialist is available for consultation and the time slots available for appointments, and is information used to confirm a consultation appointment with a user.

[0012] "Means for confirming appointments" refers to the system functionality for determining and confirming a date and time for a consultation based on the user's and specialist's preferences and availability.

[0013] "Video calling means" refers to a communication technology that connects a user and a specialist via audio and video over the Internet, enabling remote medical treatment.

[0014] A "medical session" refers to a series of consultations or examinations that a user has with a specialist using video communication means.

[0015] "Consultation" refers to the information, diagnoses, prescriptions, and treatment plans recorded by a specialist about a user during a consultation session.

[0016] "Means of notification" refers to the technology or method used to communicate information to users, such as medical treatment details, appointment confirmations, and follow-up reminders.

[0017] "Establishing a video call session" refers to a technical means that provides a secure and reliable communication environment for real-time audio and video consultation between a user and a specialist.

[0018] "Medical content storage means" refers to a system function that stores important information such as the results of medical sessions, prescriptions, and follow-up plans in a database, allowing them to be referenced and updated as needed.

[0019] "Follow-up notifications" refers to notification techniques that remind users of upcoming medical appointments and ongoing treatment plan information. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] The present invention is a system that enables users living in remote locations to receive high-quality medical services. The system supports the process of users entering medical information using a terminal, searching for an appropriate specialist, and making an appointment. The system also has the function of receiving medical treatment via video call, storing medical treatment results, and performing follow-up.

[0042] System configuration

[0043] The system has the following main functions:

[0044] 1. User registration and authentication function

[0045] 2. Doctor search and appointment function

[0046] 3. Video call medical consultation function

[0047] 4. Medical records and follow-up features

[0048] Program processing flow

[0049] 1. User registration and authentication function

[0050] On the device: The user launches the app or website and enters the required personal information.

[0051] Server: Receives the entered information, stores it in a database, and sends a verification link to the user's email address.

[0052] User: Receives a verification email and clicks on the link to activate the account.

[0053] Server: Verify the link and activate the user's account.

[0054] Examples:

[0055] User A accesses the account remotely, provides the necessary information, and creates an account. After receiving the authentication email, the user clicks the link to activate the account.

[0056] 2. Doctor search and appointment function

[0057] Device: Enter symptoms into the app on your device and set search criteria.

[0058] Server: Based on the input information, the server filters the doctor information in the database, generates a list of appropriate doctors, and sends it to the terminal.

[0059] Terminal: Shows the user a list of doctors and lets them select the appointment time they prefer.

[0060] Server: Confirms the selected reservation information, updates the schedule, and sends a reservation confirmation notification to the user.

[0061] Examples:

[0062] User B complains of stomach pain and searches for a gastroenterologist. He selects an appropriate doctor from the search results and makes an appointment for 2:00 PM the next day.

[0063] 3. Video call medical consultation function

[0064] Device: The user launches the app at the scheduled time and prepares to connect.

[0065] Server: Establishes a secure communication channel and initiates the video call session.

[0066] Terminal: Displays the video call screen, allowing the user and doctor to communicate in real time.

[0067] User: Engages in medical conversations with doctors and receives guidance and prescriptions.

[0068] Examples:

[0069] User C launches the app at the scheduled time and receives a consultation via video call with Doctor D. Doctor D makes an appropriate diagnosis and issues a prescription.

[0070] 4. Medical records and follow-up features

[0071] Server: After the consultation session is over, it stores the details in a database and sets reminders for upcoming consultations and follow-ups.

[0072] Terminal: Allows users to check their medical history and schedule their next appointment if necessary.

[0073] Users: Receive follow-up notifications and prepare for upcoming appointments.

[0074] Examples:

[0075] User E opens the app after the consultation and checks the medical history. The system sends a reminder for a follow-up appointment in one week, and the user makes the next appointment.

[0076] This system allows users living in remote areas to easily receive high-quality medical services, and through continuous follow-up, users can efficiently manage their health.

[0077] The processing flow will be explained below.

[0078] User registration and authentication functions

[0079] Step 1:

[0080] On the device: The user launches the app or website and clicks the "Sign up" button.

[0081] Step 2:

[0082] Terminal: Displays a form where the user enters required information such as name, address, contact information, and health insurance information.

[0083] Step 3:

[0084] User: Enter the required information and click the "Submit" button.

[0085] Step 4:

[0086] Terminal: Checks the entered information for any error messages, and if there are no problems, sends the information to the server.

[0087] Step 5:

[0088] Server: Stores the received user information in a database.

[0089] Step 6:

[0090] Server: Sends a confirmation email containing a verification link to the user's email address.

[0091] Step 7:

[0092] User: Receives verification email and clicks link.

[0093] Step 8:

[0094] Server: Checks the validity of the link and activates the user account.

[0095] Step 9:

[0096] Server: Sends account activation notification to the user.

[0097] ---

[0098] Doctor search and appointment function

[0099] Step 1:

[0100] Terminal: The user logs in and displays the symptom entry screen.

[0101] Step 2:

[0102] User: Enter symptoms and desired specialty and click the "Search" button.

[0103] Step 3:

[0104] Terminal: Sends the entered information to the server.

[0105] Step 4:

[0106] Server: Searches for doctor information in the database and generates a list of doctors that match the criteria.

[0107] Step 5:

[0108] Server: Sends doctor list and schedule information to the device.

[0109] Step 6:

[0110] Terminal: Displays a list of doctors to the user as search results.

[0111] Step 7:

[0112] User: Select the desired doctor and appointment date and time, and click the "Make an appointment" button.

[0113] Step 8:

[0114] Device: Send the selected information back to the server.

[0115] Step 9:

[0116] Server: Stores appointment information in a database and updates doctor schedules.

[0117] Step 10:

[0118] Server: Sends appointment confirmation notifications to users and doctors.

[0119] ---

[0120] Video call consultation function

[0121] Step 1:

[0122] On your device: Launch the app at the scheduled time and click the "Start video call" button.

[0123] Step 2:

[0124] Terminal: Displays the video call screen and sends a connection request to the server.

[0125] Step 3:

[0126] Server: Connects the user and doctor to a secure video call session.

[0127] Step 4:

[0128] Device: Initiate a video call that relays audio and video.

[0129] Step 5:

[0130] User: Receive medical treatment via video call with a doctor.

[0131] Step 6:

[0132] Server: Monitors the stability of the connection and handles errors as needed.

[0133] ---

[0134] Medical records and follow-up features

[0135] Step 1:

[0136] Server: After the video call session ends, the doctor's medical details are saved in a database.

[0137] Step 2:

[0138] Server: Sends a notification to the user that the medical record has been saved.

[0139] Step 3:

[0140] Terminal: Provides a screen where users can check their medical history.

[0141] Step 4:

[0142] Server: Schedule upcoming appointments and set follow-up reminders.

[0143] Step 5:

[0144] User: Receive reminders and schedule upcoming appointments.

[0145] Example 1

[0146] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0147] It is important to make it easy for people living in remote areas or users who have difficulty visiting medical institutions to receive appropriate medical services. There is also a need for a system that efficiently performs follow-up care and health management after medical treatment. However, current systems do not adequately meet these needs, so a system that can provide comprehensive medical services is needed.

[0148] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0149] In this invention, the server includes means for receiving medical information input by a user using a device, means for searching for an appropriate doctor based on the medical information, means for acquiring the doctor's schedule information and confirming the appointment, means for video calling the user and the doctor at the appointment time, means for saving the medical treatment details after the medical treatment session ends, means for setting a follow-up notification based on the medical record, and means for sending the follow-up notification to the user, thereby enabling the user to receive appropriate medical services even from a remote location and enabling follow-up after the medical treatment and long-term health management.

[0150] A "device" is an electronic device used by a user to input medical information, such as a smartphone, tablet, or computer.

[0151] "Medical information" refers to information about symptoms, medical history, and current health status that a user inputs using a device.

[0152] The "server" is a computer system that receives input medical information, searches for and schedules appropriate doctors, and manages video calls and medical records.

[0153] A "physician" is a medical professional who is qualified in a particular specialty and provides medical treatment and diagnosis to users.

[0154] "Schedule information" is information about the doctor's available dates and times and free times.

[0155] "Booking" is the process of confirming a consultation date and time selected by the user and securing a consultation session with a doctor.

[0156] The "video call means" is a video call function that connects the user and a doctor in real time to provide medical care.

[0157] A "medical session" is a series of processes in which a doctor provides medical care to a user via video call or the like.

[0158] "Medical content" refers to information such as diagnoses, instructions, and prescriptions given during a medical session.

[0159] A "follow-up notification" is a communication sent to a user after a medical examination to inform them of the next medical examination or follow-up appointment.

[0160] "Medical records" are data that store information such as a user's past medical history, diagnosis results, and prescriptions.

[0161] The present invention is a system that aims to enable users living in remote locations to receive high-quality medical services. This system provides functions that allow users to input medical information using a device, search for and make appointments with appropriate doctors, receive medical treatment via video call, save medical treatment details, and perform follow-ups. An embodiment of this system will be described in detail below.

[0162] User registration and authentication functions

[0163] A user launches an app or website using a device such as a smartphone or PC and enters personal information such as their name, email address, phone number, and password. The entered information is sent from the device to a server and stored in a database (e.g., MySQL (registered trademark)) using SSL / TLS encrypted communication. The server then sends an authentication link to the user's email address, and the user activates their account by clicking the link.

[0164] Doctor search and appointment function

[0165] The user enters symptoms (e.g., abdominal pain) into the app on their device and sets search criteria (e.g., gastroenterology). This information is sent to the server, which filters corresponding specialists from a database (e.g., PostgreSQL). The filtered list of doctors is sent to the device in JSON format, and the user selects the doctor of their choice and makes an appointment. The reservation information is managed by the server, which uses a push notification service (e.g., Firebase) to send a reservation confirmation to the user.

[0166] Video call consultation function

[0167] When the time for the appointment arrives, the user launches the app on their device to prepare for the video call. The server then establishes a secure communication channel (e.g., WebRTC) and starts a video call session between the user and the doctor. The user and doctor communicate in real time through the video call, providing treatment, guidance, and issuing prescriptions.

[0168] Medical records and follow-up features

[0169] After the video call session ends, the server saves the medical record in a database (e.g., AWS® RDS). The saved medical record is used to set reminders for the next medical appointment and follow-up. The user can check the medical history and next follow-up information through the app. The server also generates and sends follow-up notifications to the user.

[0170] Specific examples

[0171] User A uses the app to register personal information, receives a verification email, and clicks the link. He enters his symptoms of abdominal pain and searches for and makes an appointment with a gastroenterologist. When the appointment time arrives, he starts a video call and receives a diagnosis and prescription from the doctor. After the appointment, he checks the medical record, receives a follow-up notification, and prepares for the next appointment.

[0172] Prompt Sentence Examples

[0173] "What is the process for User A to launch the app to register for the first time, enter their personal information, receive a verification email, and click the link to activate their account?"

[0174] This system enables users living in remote areas to easily receive high-quality medical services, and allows for efficient health management through continuous follow-up.

[0175] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0176] Step 1:

[0177] Enter user information

[0178] Input: A user uses a device (smartphone or computer) to open an app or website and enter personal information (such as name, email address, phone number, or password).

[0179] Specific behavior: The user enters the required information into the device's input form and clicks the "Submit" button.

[0180] Output: The entered data is sent from the terminal to the server.

[0181] Step 2:

[0182] Information stored in a database

[0183] Input: Personal information sent from your device.

[0184] Specific operation: The server receives the received personal information using SSL / TLS encrypted communication, then stores the received data in a database (e.g., MySQL) and flags the data as successful.

[0185] Output: Personal information stored in a database and a verification email prepared.

[0186] Step 3:

[0187] Generate and send the authentication link

[0188] Input: User's email address and other personal information stored on the server.

[0189] What happens: The server generates a unique token for the user, generates an email containing an authentication link, and then uses a mail server (e.g., SendGrid) to send the authentication email.

[0190] Output: An email containing a verification link will be sent to the user's email address.

[0191] Step 4:

[0192] Receiving emails and clicking links

[0193] Input: The email containing the authentication link.

[0194] What happens: The user opens the email using their email client (e.g. Gmail) and clicks on the authentication link.

[0195] Output: The action of the user clicking the authentication link.

[0196] Step 5:

[0197] Activate your account

[0198] Input: Authentication link click event.

[0199] What happens: The server verifies the token in the authentication link, and if it matches, activates the user's account. If activation is successful, it generates a success message and sends it to the user.

[0200] Output: Activated user account and a success message.

[0201] Step 6:

[0202] Enter symptoms and set search conditions

[0203] Input: Symptoms and search terms entered by the user (e.g., gastroenterology).

[0204] Specific operation: The user enters symptoms and search criteria into the device app and clicks the "Search" button.

[0205] Output: The symptoms and search criteria are sent to the server.

[0206] Step 7:

[0207] Database Search and Filtering

[0208] Input: Symptoms and search criteria submitted by the user.

[0209] Specific operation: The server searches for doctor information in a database (e.g., PostgreSQL) and filters it based on the entered conditions. As a result, it generates a list of suitable doctors.

[0210] Output: A filtered list of doctors is generated.

[0211] Step 8:

[0212] Generate and submit a doctor list

[0213] Input: Filtered list of doctors.

[0214] Specific operation: The server converts the filtered doctor list into JSON format and sends it to the terminal.

[0215] Output: Doctor list in JSON format sent to the terminal.

[0216] Step 9:

[0217] Select a consultation time

[0218] Input: Doctor list displayed on terminal.

[0219] Specific operation: The user selects the doctor of their choice and the appointment date and time in the calendar format, and then the selection is sent to the server.

[0220] Output: The selected appointment date and time and doctor information are sent to the server.

[0221] Step 10:

[0222] Reservation updates and notifications

[0223] Input: User selected appointment date and time and doctor information.

[0224] What happens: The server updates the reservation schedule and generates a reservation confirmation notification, which is sent to the user using a push notification service (e.g., Firebase).

[0225] Output: Pushed booking confirmation notification.

[0226] Step 11:

[0227] Launching the app and preparing to connect

[0228] Input: Appointment time arrives.

[0229] Specific behavior: The user launches the app at the scheduled time and prepares to connect to the video call. The app requests access to the camera and microphone.

[0230] Output: Video call ready to connect.

[0231] Step 12:

[0232] Establishing a communication channel

[0233] Input: The user's ready to connect state.

[0234] What happens: The server establishes a secure communication channel (e.g., WebRTC) and connects the user and doctor to the same session.

[0235] Output: Establishment of a secure video call session.

[0236] Step 13:

[0237] Starting a video call session

[0238] Input: The established communication channel.

[0239] What happens: The video call screen appears in the user's app and a video call session with the doctor begins.

[0240] Output: Start of video call session.

[0241] Step 14:

[0242] Real-time communication

[0243] Input: A video call session.

[0244] Specific operation: The user communicates with the doctor in real time, explaining symptoms and asking questions, and the doctor makes a diagnosis based on the user's symptoms.

[0245] Output: Real-time communication results of the medical consultation.

[0246] Step 15:

[0247] Providing diagnosis and prescriptions

[0248] Input: Real-time communication results.

[0249] What it does: The doctor will explain the diagnosis and issue a prescription if necessary. The diagnosis and prescription information will be provided online.

[0250] Output: Diagnostic results and prescription information.

[0251] Step 16:

[0252] Ending and recording the medical session

[0253] Input: End of medical session.

[0254] What it does: After the video call session ends, the server logs the medical details, including the doctor's diagnosis, prescriptions, and the next recommended visit date and time.

[0255] Output: Recorded medical details.

[0256] Step 17:

[0257] Medical record database storage

[0258] Input: Recorded medical details.

[0259] What it does: The server stores medical records in a database (e.g., AWS RDS). The stored data is encrypted and only accessible to the user and their doctor.

[0260] Output: Medical records stored in a database.

[0261] Step 18:

[0262] Follow-up notification settings

[0263] Input: Medical record.

[0264] What it does: The server uses medical records to schedule reminders for upcoming appointments and follow-ups, adjusting the details of the reminders to fit the user's schedule.

[0265] Output: The set follow-up reminder.

[0266] Step 19:

[0267] Checking medical history

[0268] Input: Stored medical records.

[0269] What it does: The user accesses their medical history through the app, which shows past diagnoses, prescriptions, and next follow-up appointments.

[0270] Output: Displayed medical history.

[0271] Step 20:

[0272] Receive follow-up notifications

[0273] Input: The follow-up reminder that was set.

[0274] Specific operation: A follow-up notification is sent from the server at the specified date and time, and the user receives the notification. The user prepares for the next consultation.

[0275] Output: The follow-up notification sent to the user.

[0276] (Application example 1)

[0277] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0278] The problem that this invention aims to solve is to provide users living in remote areas with a means to efficiently access and learn appropriate medical educational content, in addition to enabling them to receive high-quality medical services. This will enable the creation of an environment in which users can not only access specialists, but also consistently use educational content to deepen their medical knowledge.

[0279] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0280] In this invention, the server

[0281] means for receiving medical information input by a user using a terminal;

[0282] A means for searching for an appropriate specialist based on the medical information;

[0283] means for acquiring schedule information of the specialist and confirming an appointment;

[0284] a video call means for connecting the user and the specialist at the appointment time;

[0285] means for saving the medical treatment details after the medical treatment session ends and notifying the user of the saving;

[0286] a means for a user to input search keywords to filter and display medical education content;

[0287] A means for conducting live lectures using video calls;

[0288] A way to keep a record of what you've watched and set reminders for upcoming lectures and follow-ups,

[0289] This allows users to receive medical treatment from specialists and efficiently access and learn high-quality medical education content.

[0290] "Medical information" refers to information such as health status, symptoms, and medical history entered by the user through the terminal.

[0291] A "specialist" is a medical professional with advanced knowledge and skills in a particular medical field.

[0292] "Schedule information" is information that indicates the dates and times when specialists are available for consultation and the reservation status.

[0293] A "video call" is a means for a user in a remote location and a specialist to communicate in real time using audio and video via the Internet.

[0294] "Medical content" refers to information about diagnosis and treatment plans recorded during video calls or face-to-face medical sessions.

[0295] "Notifications" are a means of informing users about medical treatments and follow-up appointments.

[0296] A "search keyword" is an input character string that a user uses when searching for medical education content.

[0297] "Filtering" is the process of sorting and displaying relevant medical education content based on search keywords.

[0298] "Medical education content" refers to information such as videos, articles, and lecture materials for learning medical knowledge and skills.

[0299] "Live Lecture" means an educational session that takes place in real time and is delivered via video call.

[0300] "Reminder" is a function that notifies users so that they do not forget their next lecture or medical appointment.

[0301] Overall system configuration

[0302] The system of the present invention is designed to enable users to receive high-quality medical services even when they are not in the field. This system mainly uses a smartphone app and is built by combining front-end and back-end technologies.

[0303] Hardware and Software Configuration

[0304] Hardware

[0305] User devices: Mainly smartphones are used.

[0306] Servers: Includes database and application servers.

[0307] software

[0308] Front-end: React Native (smartphone app development)

[0309] Backend: Node.js (application server), MongoDB (database)

[0310] Communication: WebRTC (secure communication for video calls)

[0311] Authentication: Firebase Authentication (user authentication)

[0312] Feature details

[0313] 1. User registration and authentication function

[0314] The user launches the smartphone app and enters the necessary personal information.

[0315] The server receives the entered information and stores it in a database (MongoDB). Firebase Authentication is used to send an authentication email to the user.

[0316] The user receives a verification email and clicks on the link to activate the account. The server verifies the link and activates the user's account.

[0317] 2. Doctor search and appointment function

[0318] The user terminal transmits search keywords based on the symptoms to the server.

[0319] The server queries the database (MongoDB) and generates a list of relevant specialists, which is then sent to the user's device.

[0320] The user selects the desired specialist from a list of doctors and makes an appointment.

[0321] The server stores the user's reservation information and sends a reservation confirmation notification.

[0322] 3. Video call medical consultation function

[0323] The user launches the smartphone app at the scheduled time and prepares to connect to the video call.

[0324] The server establishes a secure communication channel using WebRTC and initiates the video call session.

[0325] Users and specialists can conduct consultations in real time via video calls.

[0326] 4. Medical records and follow-up features

[0327] After the consultation session, the server saves the details of the consultation in a database and sets reminders for the next consultation or follow-up.

[0328] The user terminal provides an interface that allows the user to check their medical history and make their next appointment.

[0329] 5. Medical education content search and browsing function

[0330] The user terminal transmits search keywords to the server to filter medical education content.

[0331] The server retrieves the relevant content from a database (MongoDB) and returns it to the user's terminal.

[0332] Users can view the filtered content through a smartphone app.

[0333] 6. Live lecture function via video call

[0334] The user launches the smartphone app based on the schedule of the live lecture and prepares the video call connection.

[0335] The server uses WebRTC to establish a secure communication channel and conduct the video call session for the live lecture.

[0336] Users can listen to lectures in real time and ask questions and hold discussions.

[0337] 7. Content saving and follow-up features

[0338] The server stores a record of the content the user has viewed in a database and sets reminders for upcoming lectures and follow-ups.

[0339] The user terminal provides an interface that allows the user to check their viewing history and receive reminders for upcoming lectures.

[0340] Specific examples

[0341] For example, if a user wants to learn about the topic of "diabetes," they can enter search keywords into the app and related videos and articles will be displayed. Additionally, WebRTC can be used to attend live seminars by specialists, and the content of these seminars will be stored in a database.

[0342] Prompt Sentence Examples

[0343] Search for diabetes education content to get a list of related videos and articles, collect real-time user interest in these contents, and recommend the most viewed content.

[0344] This allows users to not only receive medical treatment from specialists, but also to effectively access and learn from high-quality medical education content.

[0345] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0346] Step 1:

[0347] User registration and authentication functions

[0348] Input: The user enters personal information into the device and presses the submit button.

[0349] Server: Receives personal information sent by the user and saves it in a database (MongoDB). After saving, sends an authentication email to the user via Firebase Authentication.

[0350] Output: A verification email will be sent to the user's email address.

[0351] User: Receives verification email and clicks verification link.

[0352] Server: Confirms the click on the authentication link and activates the user's account.

[0353] What happens: The server verifies the sender's email address and records the date and time the verification link is clicked.

[0354] Step 2:

[0355] Doctor search and appointment function

[0356] Input: The user enters the symptoms into the terminal and presses the search button.

[0357] Server: Receives the input symptom information and queries the database (MongoDB) to generate a list of relevant specialists.

[0358] Output: The filtered list of specialists is sent to the user terminal.

[0359] User: Selects the desired specialist from a list of specialists and schedules an appointment.

[0360] Server: Receives the reservation information, stores it in a database, and sends a reservation confirmation to the user.

[0361] Specific operation: The server updates the appointment schedule based on the date and time, specialist ID, and user ID, and calls the notification function.

[0362] Step 3:

[0363] Video call consultation function

[0364] Input: At the scheduled time, the user launches the app on their device and prepares for the video call.

[0365] Server: Receives the video call request and establishes a secure communication channel using WebRTC.

[0366] Output: A video call session is started and the video call screen is displayed on the user's device.

[0367] User: Consults with specialists via video call.

[0368] What it does: The server manages WebRTC peer connections and ensures communication encryption.

[0369] Step 4:

[0370] Medical records and follow-up features

[0371] Input: After the medical session ends, the server receives the medical details.

[0372] Server: Receives medical records, stores them in a database, and sets follow-up reminders.

[0373] Output: The medical details are saved in the database and a notification is sent to the user's device.

[0374] Users: Receive notifications and can check their medical history.

[0375] What it does: The server tags the stored visits and schedules reminders for upcoming appointments.

[0376] Step 5:

[0377] Medical education content search and browsing function

[0378] Input: The user enters search keywords into the device and presses the search button.

[0379] Server: Receives search keywords, queries the database, and filters relevant medical education content.

[0380] Output: The filtered medical education content is sent to the user terminal.

[0381] User: Views filtered content on a smartphone app.

[0382] What it does: The server ranks the search results and prioritizes the most relevant content to present to the user.

[0383] Step 6:

[0384] Live lecture function via video call

[0385] Input: User launches the app and prepares based on the schedule of a live lecture.

[0386] Server: Receives the video call request and establishes a secure communication channel using WebRTC.

[0387] Output: A video call session of a live lecture is started and the video is displayed on the user's device.

[0388] Users: Attend live lectures, ask questions and participate in discussions.

[0389] Specific operation: The server monitors the progress of live lectures in real time and notifies you if any problems occur.

[0390] Step 7:

[0391] Content saving and follow-up features

[0392] Input: The server receives information about the content that was viewed.

[0393] Server: Stores records of viewed content in a database and sets reminders for upcoming lectures and follow-ups.

[0394] Output: The viewing history is stored in a database and a reminder for the next lecture is sent to the user's device.

[0395] User: Check viewing history and prepare for the next lecture.

[0396] What it does: The server creates statistics on viewing history and analyzes the topics that users are most interested in.

[0397] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0398] The present invention is a system that enables users living in remote locations to receive high-quality medical services. The system includes a process where users input medical information using a terminal, search for an appropriate specialist, and make an appointment. It also includes functions for receiving medical treatment via video call, saving medical results and providing follow-up, and using an emotion engine to reflect the user's emotional state in the medical treatment.

[0399] System configuration

[0400] The system has the following main functions:

[0401] 1. User registration and authentication function

[0402] 2. Doctor search and appointment function

[0403] 3. Video call medical consultation function

[0404] 4. Medical records and follow-up features

[0405] 5. Emotion recognition function using emotion engine

[0406] Program processing flow

[0407] 1. User registration and authentication function

[0408] On the device: The user launches the app or website and enters the required personal information.

[0409] Server: Receives the entered information, stores it in a database, and sends a verification link to the user's email address.

[0410] User: Receives a verification email and clicks on the link to activate the account.

[0411] Server: Verify the link and activate the user's account.

[0412] Examples:

[0413] User A accesses the account remotely, provides the necessary information, and creates an account. After receiving the authentication email, the user clicks the link to activate the account.

[0414] 2. Doctor search and appointment function

[0415] Device: Enter symptoms into the app on your device and set search criteria.

[0416] Server: Based on the input information, the server filters the doctor information in the database and generates a list of doctors that meet the criteria. This list is then sent to the terminal.

[0417] Terminal: Shows the user a list of doctors and lets them select the appointment time they prefer.

[0418] Server: Confirms the selected reservation information, updates the schedule, and sends a reservation confirmation notification to the user.

[0419] Examples:

[0420] User B complains of stomach pain and searches for a gastroenterologist. He selects an appropriate doctor from the search results and makes an appointment for 2:00 PM the next day.

[0421] 3. Video call medical consultation function

[0422] Device: The user launches the app at the scheduled time and prepares to connect.

[0423] Server: Establishes a secure communication channel and initiates the video call session.

[0424] Terminal: Displays the video call screen, allowing the user and doctor to communicate in real time.

[0425] User: Engages in medical conversations with doctors and receives guidance and prescriptions.

[0426] Examples:

[0427] User C launches the app at the scheduled time and receives a consultation via video call with Doctor D. Doctor D makes an appropriate diagnosis and issues a prescription.

[0428] 4. Medical records and follow-up features

[0429] Server: After the consultation session is over, it stores the details in a database and sets reminders for upcoming consultations and follow-ups.

[0430] Terminal: Allows users to check their medical history and schedule their next appointment if necessary.

[0431] Users: Receive follow-up notifications and prepare for upcoming appointments.

[0432] Examples:

[0433] User E opens the app after the consultation and checks the medical history. The system sends a reminder for a follow-up appointment in one week, and the user makes the next appointment.

[0434] 5. Emotion recognition function using emotion engine

[0435] Device: Captures the user's facial expressions and voice in real time during a video call.

[0436] Server: The emotion engine analyzes the captured data and recognizes the user's emotional state.

[0437] Server: Provides the doctor with the recognized emotional information and reflects it in the medical treatment.

[0438] Server: Stores medical records including emotional information in a database.

[0439] Examples:

[0440] When User F has a video call with a doctor, the emotion engine recognizes that the user is anxious from their facial expression. This information is provided to the doctor, who takes the user's mental state into consideration during the consultation. The emotional information is recorded in the medical treatment details and prescription.

[0441] This system allows users living in remote areas to easily receive high-quality medical services. In addition, by providing medical treatment based on the user's emotional state, more personalized medical services can be provided.

[0442] The processing flow will be explained below.

[0443] Emotion recognition function using emotion engine

[0444] Step 1:

[0445] Device: Show a popup to get the user's camera and microphone permissions before starting a video call.

[0446] Step 2:

[0447] User: Allow use of camera and microphone.

[0448] Step 3:

[0449] Device: After obtaining permission, it displays the video call screen and captures the user's facial expressions and voice in real time.

[0450] Step 4:

[0451] Terminal: Sends captured facial expression and voice data to the server.

[0452] Step 5:

[0453] Server: Analyzes the received facial expression and voice data and uses an emotion engine to recognize the user's emotional state.

[0454] Step 6:

[0455] Server: Processes the recognized emotion information and displays it on the doctor's screen in real time.

[0456] Step 7:

[0457] Terminal: The doctor checks the user's emotional state and responds accordingly.

[0458] Step 8:

[0459] Server: Stores the details of the medical session, including emotional information, in a database.

[0460] Step 9:

[0461] Server: Uses the stored emotional information to inform the next medical appointment or follow-up.

[0462] Specific examples

[0463] Video call consultations

[0464] Step 1:

[0465] Device: User F clicks "Start video call" on the screen for starting a video call.

[0466] Step 2:

[0467] On your device: A pop-up will appear asking you to allow access to your camera and microphone.

[0468] Step 3:

[0469] User: Allow use of camera and microphone.

[0470] Step 4:

[0471] Terminal: The video call screen is displayed, and User F's facial expressions and voice are captured.

[0472] Step 5:

[0473] Device: Captured data is sent to the server.

[0474] Step 6:

[0475] Server: Analyzes the received data, and the emotion engine recognizes User F's emotions as both calm and anxious.

[0476] Step 7:

[0477] Server: The recognized emotion information is displayed on Doctor D's screen.

[0478] Step 8:

[0479] Terminal: Doctor D checks User F's emotional state and provides medical treatment while taking it into consideration.

[0480] Step 9:

[0481] Server: Stores the results of the medical session and emotional information in a database.

[0482] Step 10:

[0483] Server: Based on the stored emotional information, sends notifications and reminders to User F that will be helpful for their next medical appointment.

[0484] This system enables medical treatment that takes into account the user's emotional state, enabling more personalized medical services to be provided. Furthermore, by storing emotional information and utilizing it for the next medical treatment, continuous and consistent medical support can be achieved.

[0485] Example 2

[0486] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0487] In order for users living in remote areas to receive high-quality medical services, they need a system that smoothly handles the entire process, from searching for specialists and making appointments to video call consultations, saving medical records, and providing follow-ups. However, existing systems lack the functionality to reflect the user's emotional state in medical treatment, making them insufficient for providing more personalized medical services. Therefore, there is a need for a system that can analyze the user's emotional state in real time and reflect that in medical treatment.

[0488] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0489] In this invention, the server includes means for receiving medical information input by a user using a terminal, means for searching for an appropriate medical professional based on the medical information, means for acquiring the professional's schedule information and confirming an appointment, means for establishing a secure video call session between the user and the medical professional, means for saving the medical treatment details after the medical treatment session ends and notifying the user, means for capturing the user's facial expressions and voice data during the video call and performing emotion analysis, and means for providing the medical professional with the results of the emotion analysis and reflecting them in the medical treatment details, thereby enabling users in remote locations to receive high-quality and personalized medical services.

[0490] "User" refers to an individual who uses the system to input medical information and receive medical treatment.

[0491] "Terminal" refers to an information processing device such as a smartphone, PC, or tablet used by a user.

[0492] "Medical information" refers to all information necessary for medical treatment, such as medical condition, symptoms, and past medical history, entered by the user.

[0493] "Medical professional" refers to a doctor or nurse who has specialized knowledge and skills in a particular medical field.

[0494] "Schedule information" refers to information such as the availability of medical professionals and appointment availability.

[0495] "Video calling" refers to a means of real-time video and audio communication between a user and a medical professional via the Internet.

[0496] "Medical content" refers to the doctor's diagnosis, treatment plan, prescription information, etc. recorded during a medical session.

[0497] "Emotion analysis" refers to the process of analyzing a user's facial expressions and voice data during a video call to identify the user's emotional state.

[0498] "Emotional state" refers to information that indicates the user's psychological state or mood.

[0499] "Follow-up Notification" means a notification sent to a user for a follow-up or reminder of an upcoming medical appointment.

[0500] The present invention is a system that enables users living in remote locations to receive high-quality medical services. The system involves a process in which users input medical information using a terminal, search for an appropriate medical professional, and receive medical treatment via video call. In addition to functions for saving and following up on medical treatment details, the system also has an emotion analysis function that reflects the user's emotional state in medical treatment.

[0501] System configuration

[0502] 1. User registration and authentication function

[0503] On the device: The user launches an app or website and enters personal information such as their name, email address, and password.

[0504] Server: Receives the entered information, stores it in a database (e.g., MySQL), and sends an email containing a verification link to the user's email address.

[0505] User: Receives a verification email and clicks on the link to activate the account.

[0506] Server: Verify the link and activate the user's account.

[0507] 2. Doctor search and appointment function

[0508] Device: Enter symptoms into the device app and set search criteria.

[0509] Server: Based on the input information, it searches a doctor database (for example, ElasticSearch (registered trademark)) and generates a list of doctors that meet the criteria. It then sends this list to the terminal.

[0510] Terminal: Shows the user a list of doctors and lets them select the appointment time they prefer.

[0511] Server: Confirms the selected reservation information, saves the schedule information to the database, and sends a reservation confirmation notification to the user.

[0512] 3. Video call medical consultation function

[0513] On the device: At the scheduled time, the user launches the app and prepares for the video call.

[0514] Server: Uses technologies such as WebRTC to establish a secure video call session and connect users with medical professionals.

[0515] Terminal: Real-time communication between the user and medical professionals.

[0516] 4. Medical records and follow-up features

[0517] Server: After the consultation session is over, it stores the consultation details in a database and sets reminders for the next consultation or follow-up.

[0518] Terminal: Allows users to check their medical history and schedule their next appointment if necessary.

[0519] 5. Emotion recognition function using emotion engine

[0520] Device: Captures facial and audio data in real time during a video call.

[0521] Server: Analyzes the captured data using an emotion analysis engine (e.g., Microsoft® Azure® Face API) to identify the user's emotional state.

[0522] Server: Providing recognized emotion information to medical professionals to improve the quality of medical care.

[0523] Server: Stores medical records including emotional information in a database.

[0524] Specific examples

[0525] User A accesses the system and creates an account. He receives a verification email and clicks the link to activate his account. He then complains of abdominal pain, searches for a gastroenterologist, and makes an appointment for 2:00 p.m. the next day. He launches the app at the scheduled time and consults with the doctor via video call. The consultation details are saved in the database, and a reminder to return to the doctor one week later is sent.

[0526] Examples of prompt statements

[0527] Enter the following prompt for the generative AI model:

[0528] Please explain the steps a user takes to access the platform remotely and create an account. Include the process by which a user enters symptoms, searches for a doctor, and makes an appointment using the medical platform. Also, please explain the process for receiving medical advice in real time via video call, as well as the steps for saving medical records and scheduling the next follow-up appointment. Also, please include a function that uses an emotion engine to recognize the user's emotional state and reflect it in medical treatment. Please clearly indicate specific tools and technologies (e.g., WebRTC, MySQL, Microsoft Azure Face API, etc.).

[0529] This system enables users living in remote areas to receive high-quality, personalized medical services.

[0530] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0531] System program processing flow

[0532] 1. User registration and authentication function

[0533] Step 1: Enter your user information

[0534] User: Accesses an application or website from a smartphone or computer device and enters personal information such as name, email address, and password.

[0535] Input: The user enters information using a keyboard or touch screen.

[0536] Output: The input information is ready to be used.

[0537] Specific behavior: The user fills in the required information in the form and clicks the "Register" button.

[0538] Step 2: Send and store user information

[0539] Terminal: Encrypts the entered data and sends it to the server via a secure communication protocol (e.g., HTTPS).

[0540] Server: Stores the received data in a database (e.g., MySQL).

[0541] Input: Encrypted user information.

[0542] Output: A confirmation that the data was saved correctly to the database.

[0543] Specific operation: When the device clicks the send button, the data is encrypted and sent to the server. After receiving it, the server stores the data in a database.

[0544] Step 3: Sending a verification email

[0545] Server: Uses a mail server to send an email containing a verification link to the email address entered by the user.

[0546] Input: The user's email address.

[0547] Output: A confirmation that the verification email was sent successfully.

[0548] Specific operation: The server automatically generates an authentication email and sends it to the user via the mail server.

[0549] Step 4: Click on the authentication link

[0550] Users: Open the verification email they received and click the link to activate their account.

[0551] Input: The link in the verification email.

[0552] Output: A confirmation that the account has been activated.

[0553] What happens: The user checks their email, clicks the link, and is redirected to a page to activate their account.

[0554] Step 5: Activate your account

[0555] Server: Checks the information in the authentication link and updates the user's status in the database to "active".

[0556] Input: Authentication link information.

[0557] Output: Confirmation that the user account has been activated.

[0558] What happens: The server validates the link and updates the user status in the database.

[0559] 2. Doctor search and appointment function

[0560] Step 1: Enter symptoms and set search criteria

[0561] User: Enters search criteria into the application, such as symptoms and desired appointment date and time.

[0562] Input: Symptom information and desired date and time of consultation entered by the user.

[0563] Output: The search criteria has been set.

[0564] Specific behavior: The user enters their symptoms and desired appointment date into the app's search form and clicks the "Search" button.

[0565] Step 2: Filtering physician information

[0566] Server: Based on the input information, it searches a doctor database (e.g., Elasticsearch) and generates a list of doctors that match the criteria.

[0567] Input: The search criteria entered by the user.

[0568] Output: A list of doctors who match the criteria.

[0569] Specific operation: The server executes a database query to retrieve physician data that matches the criteria.

[0570] Step 3: View the Doctor List

[0571] Terminal: Displays the list of doctors received from the server to the user.

[0572] Input: Doctor list sent from the server.

[0573] Output: A list of doctors displayed on the user interface.

[0574] Specific behavior: The device displays the list in the user interface so that the user can check it.

[0575] Step 4: Choose your appointment time

[0576] User: Select the desired appointment time and confirm the appointment.

[0577] Input: Selected appointment times and doctor information.

[0578] Output: Reservation information is confirmed.

[0579] Specific actions: The user selects a doctor and appointment time and clicks the "Make appointment" button.

[0580] Step 5: Confirm and notify reservation information

[0581] Server: Saves the selected reservation information to a database and sends a reservation confirmation notification to the user.

[0582] Input: Booking information selected by the user.

[0583] Output: The reservation information is saved to the database and a notification is sent to the user.

[0584] Specific behavior: The server records the reservation information in a database and sends a reservation confirmation email.

[0585] 3. Video call medical consultation function

[0586] Step 1: Prepare for the video call

[0587] User: Launches the application at the scheduled time and prepares for the video call.

[0588] Input: User's app launch action.

[0589] Output: Ready to start a video call.

[0590] Specific behavior: The user launches the app and clicks the start video call button.

[0591] Step 2: Establishing communication channels

[0592] Server: Establish a secure video call channel using WebRTC.

[0593] Input: User's request to start a video call.

[0594] Output: A secure video call session is established.

[0595] What happens: The server initializes a WebRTC session and connects the user with a medical professional.

[0596] Step 3: Start a video call

[0597] Device: The user and medical professionals communicate in real time via a video call screen.

[0598] Enter: a secure video call session.

[0599] Output: Real-time medical consultation with video and audio.

[0600] What happens: The device displays a video call window and the user and medical professional begin talking in real time.

[0601] Step 4: Proceeding with the treatment

[0602] Users: Engage in medical conversations with healthcare professionals and receive guidance and prescriptions.

[0603] Input: Medical consultation details.

[0604] Output: The consultation is completed and any necessary instructions or prescriptions are provided.

[0605] What it does: A user answers a question or gets advice from a medical professional.

[0606] 4. Medical records and follow-up features

[0607] Step 1: Save your medical records

[0608] Server: After the medical session is over, the medical details entered by the medical professional are stored in a database.

[0609] Input: Medical record data.

[0610] Output: Medical information saved in the database.

[0611] Specific operation: The server stores the medical information in a database.

[0612] Step 2: Set up follow-up

[0613] Server: Sets reminders for upcoming appointments and follow-ups in the system.

[0614] Input: Follow-up appointment information.

[0615] Output: The next appointment reminder has been set.

[0616] What happens: The server saves the reminder information in a database.

[0617] Step 3: Review medical history

[0618] User: Checks medical history in the application and schedules next appointments if necessary.

[0619] Input: Medical History Request Operation.

[0620] Output: Medical history displayed.

[0621] What happens: The user views their medical history in the app and clicks the "Schedule a follow-up appointment" button.

[0622] Step 4: Receive follow-up notifications

[0623] Users: Receive follow-up notifications and prepare for upcoming appointments.

[0624] Enter: Follow-up notification.

[0625] Output: The notification has been confirmed and the next appointment is ready.

[0626] What happens: The user receives a notification and sees what's next on their schedule.

[0627] 5. Emotion recognition function using emotion engine

[0628] Step 1: Capture facial expressions and voice

[0629] Device: Captures the user's facial expressions and voice in real time during a video call.

[0630] Input: User's facial and voice data.

[0631] Output: Captured facial expression and speech data.

[0632] What happens: Your device captures data using the camera and microphone.

[0633] Step 2: Analyze emotional state

[0634] Server: Analyzes the captured data using emotion recognition software (e.g., Microsoft Azure's Face API) to identify the user's emotional state.

[0635] Input: Captured facial and speech data.

[0636] Output: Data about the user's emotional state.

[0637] Specific operation: The server calls the API and obtains emotion data.

[0638] Step 3: Providing emotional information

[0639] Server: Provides recognized emotional information to medical professionals and reflects it in medical treatment.

[0640] Input: Emotional state data.

[0641] Output: Emotional information provided to a medical professional.

[0642] Specific operation: The server displays the emotional information on the medical professional's screen.

[0643] Step 4: Preserving medical records

[0644] Server: Stores medical records including emotional information in a database.

[0645] Input: Emotional information and medical details data.

[0646] Output: Medical records containing emotion data stored in a database.

[0647] Specific operation: The server integrates the emotion data and medical details and stores them in a database.

[0648] summary

[0649] These specific processing steps and operations enable users living in remote locations to receive high-quality, personalized medical services. Smooth coordination between each step ensures that the entire medical treatment process is carried out efficiently.

[0650] (Application example 2)

[0651] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0652] Conventional telemedicine systems allow users to receive high-quality medical services from remote locations, but they have the drawback of not reflecting the emotional state of the individual in the medical treatment, making it difficult to provide treatment that fully takes into account the psychological state of the user. There is also the risk of personal medical information being leaked during video calls. It is necessary to solve these issues and provide safe and personalized medical services.

[0653] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0654] In this invention, the server includes means for receiving medical information input by a user using a terminal, means for searching for an appropriate specialist based on the medical information, means for acquiring the specialist's schedule information and confirming the appointment, means for video calling to connect the user with the specialist at the appointment time, means for capturing the user's facial expressions and voice during the video call and analyzing the emotional state, and means for saving the medical treatment details after the medical treatment session ends and notifying the user. This enables personalized medical treatment that takes into account the user's emotional state and also provides medical services in a secure communication environment.

[0655] A "terminal" is an electronic device that allows a user to input medical information, communicate with specialists, and receive medical treatment.

[0656] "Medical information" refers to information entered by the user, such as their medical condition, symptoms, medical history, etc.

[0657] A "specialist" is a doctor with advanced knowledge and experience in a particular medical field.

[0658] "Schedule information" is information that indicates the dates and times when specialists are available for consultation, the reservation status, and so on.

[0659] "Video communication means" refers to audio and video communication means for real-time communication between the user and the specialist.

[0660] "Emotion recognition means" is a technology that analyzes a user's facial expressions and voice during a video call to recognize their emotional state.

[0661] A "medical session" refers to a period of time during which a medical consultation is conducted between a user and a specialist.

[0662] "Medical content" refers to a record of the diagnosis, treatment, instruction, etc. provided during a medical session.

[0663] "Notification" is a means of communicating information to inform users of medical results and the need for follow-up.

[0664] An embodiment of the present invention provides a system that enables a user in a remote location to safely receive high-quality medical services. A specific method for implementing this system will be described below.

[0665] 1. Basic configuration

[0666] This system consists of a user device, a specialist device, and a server that links these devices. The user device is a smartphone, tablet, or PC, and the specialist device uses similar devices. The server is located on the cloud and manages the entire system.

[0667] 2. Enter medical information and search for specialists

[0668] Users input medical information using their own devices. This medical information includes specific medical conditions, symptoms, medical history, etc., and is received by a server. The server has a means for searching for an appropriate specialist based on the received medical information. A pre-built doctor database is used to search for a specialist. Search criteria also include the specialist's qualifications, field of expertise, and user ratings.

[0669] 3. Confirmation of reservation

[0670] After a specialist is found, the server retrieves the specialist's schedule information and displays available appointment dates and times on the user's device. When the user selects the desired appointment time, the server confirms the appointment based on that information. When the appointment is confirmed, a notification is sent to both the user and the specialist.

[0671] 4. Video calling methods

[0672] When the appointment time arrives, the server establishes a video call session between the user and the specialist's device. This video call is highly secure, using end-to-end encryption technology, such as OpenSSL and WebRTC.

[0673] 5. Emotion recognition means

[0674] During a video call, the user's device uses a camera and microphone to capture the user's facial expressions and voice, and sends the data to the server. The server then uses an emotion engine to analyze the user's facial expressions and voice data and recognize their emotional state. This emotional information is provided to specialists in real time and reflected in their medical treatment. The emotional information is also saved as part of the medical record.

[0675] 6. Storage and notification of medical information

[0676] After the consultation session ends, the server stores the consultation details in a database. This data includes the specialist's diagnosis, advice, prescriptions, and the user's emotional state. The system also has the ability to automatically schedule the next consultation and set follow-up notifications, allowing users to receive reminders and be better prepared for their next appointment.

[0677] Specific examples

[0678] User A feels unwell and enters medical information about abdominal pain, which is received by the server. The server searches for gastroenterologists and displays the schedules of several candidates. User A selects 3:00 p.m. the following day and the appointment is confirmed. When the appointment time arrives, a video call begins between User A and Specialist B. During the call, the emotion engine recognizes the anxious expression on User A's face and displays this information to Specialist B. After the consultation, the details of the consultation and emotional information are saved, and User A is notified of the next follow-up appointment.

[0679] Prompt Sentence Examples

[0680] User A logs in using the application, enters their symptoms, and searches for a doctor. The AI ​​model recommends the most suitable doctor and schedules a video call. During the call, a facial expression recognition engine analyzes the user's emotional state, and appropriate responses are made within the video call. The session data is encrypted and stored, and a notification for the next follow-up is set.

[0681] This allows users to safely receive medical care from specialists even from remote locations, and provides personalized medical services that take emotional state into account, resulting in a higher quality medical experience.

[0682] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0683] Step 1:

[0684] The user inputs medical information using the terminal.

[0685] Input: User information about medical condition, symptoms, and medical history.

[0686] Data processing and calculation: Preprocessing to standardize the information entered by the user and store it in a database in a format that conforms to the format.

[0687] Output: Standardized medical information data.

[0688] What happens: A user launches an application, fills out a form, and hits the submit button. This information is sent to the server for preprocessing.

[0689] Step 2:

[0690] A server receives the medical information and executes a procedure to search for an appropriate specialist.

[0691] Input: User's medical information data.

[0692] Data processing and calculation: Using AI algorithms to filter and rank the most suitable specialists based on medical information.

[0693] Output: A list of suitable specialists.

[0694] Specific operation: The server analyzes the received medical information and lists the most suitable doctors from a specialist database.

[0695] Step 3:

[0696] The server retrieves the specialist's schedule information and presents it to the user.

[0697] Input: A list of appropriate specialists.

[0698] Data processing and calculation: Retrieves specialist schedule information from the database and displays it as a choice for the user.

[0699] Output: The available schedule presented to the user.

[0700] Specific operation: The server retrieves the specialist's schedule information from the database, sends the available appointment time slots to the user's terminal, and displays them as options for the user.

[0701] Step 4:

[0702] The user selects the desired reservation time, and the server confirms the reservation.

[0703] Input: The appointment time selected by the user.

[0704] Data processing and calculation: The selected reservation information is saved in the database and notified to the specialist and the user.

[0705] Output: Booking confirmation notification.

[0706] Specific operation: The user selects the desired time and presses the "Confirm appointment" button. The server updates the appointment information and sends a notification to the specialist and the user.

[0707] Step 5:

[0708] When the appointment time arrives, the server establishes a video call means connecting the user with the specialist.

[0709] Input: Appointment time, user, and specialist connection information.

[0710] Data processing and calculation: Encrypting the video call session and establishing the connection.

[0711] Output: A secure video call session.

[0712] Specific Operation: The server initiates a video call session and establishes a secure communication channel between the user and the specialist.

[0713] Step 6:

[0714] During a video call, the terminal captures the user's facial expressions and voice and performs emotion recognition.

[0715] Input: User's facial expression data and voice data.

[0716] Data processing and calculation: Analyze the user's emotional state using an emotion recognition engine.

[0717] Output: User's emotional information.

[0718] Specific operation: The camera and microphone on the user's device capture facial expressions and voice, and send this data to the server, which then analyzes the data using the emotion engine.

[0719] Step 7:

[0720] After the consultation session is over, the server stores the consultation and sends follow-up notifications to the user.

[0721] Input: Clinical content data and emotional information.

[0722] Data processing and calculation: storing medical details and emotional information in the database and setting notifications for the next follow-up.

[0723] Output: Stored medical records and follow-up notifications.

[0724] Specific operation: The server saves the medical results in the database, sets the next follow-up notification, and sends a notification to the user.

[0725] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0726] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0727] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0728] [Second embodiment]

[0729] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0730] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0731] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0732] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0733] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0734] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0735] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0736] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0737] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0738] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0739] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0740] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0741] The present invention is a system that enables users living in remote locations to receive high-quality medical services. The system supports the process of users entering medical information using a terminal, searching for an appropriate specialist, and making an appointment. The system also has the function of receiving medical treatment via video call, storing medical treatment results, and performing follow-up.

[0742] System configuration

[0743] The system has the following main functions:

[0744] 1. User registration and authentication function

[0745] 2. Doctor search and appointment function

[0746] 3. Video call medical consultation function

[0747] 4. Medical records and follow-up features

[0748] Program processing flow

[0749] 1. User registration and authentication function

[0750] On the device: The user launches the app or website and enters the required personal information.

[0751] Server: Receives the entered information, stores it in a database, and sends a verification link to the user's email address.

[0752] User: Receives a verification email and clicks on the link to activate the account.

[0753] Server: Verify the link and activate the user's account.

[0754] Examples:

[0755] User A accesses the account remotely, provides the necessary information, and creates an account. After receiving the authentication email, the user clicks the link to activate the account.

[0756] 2. Doctor search and appointment function

[0757] Device: Enter symptoms into the app on your device and set search criteria.

[0758] Server: Based on the input information, the server filters the doctor information in the database, generates a list of appropriate doctors, and sends it to the terminal.

[0759] Terminal: Shows the user a list of doctors and lets them select the appointment time they prefer.

[0760] Server: Confirms the selected reservation information, updates the schedule, and sends a reservation confirmation notification to the user.

[0761] Examples:

[0762] User B complains of stomach pain and searches for a gastroenterologist. He selects an appropriate doctor from the search results and makes an appointment for 2:00 PM the next day.

[0763] 3. Video call medical consultation function

[0764] Device: The user launches the app at the scheduled time and prepares to connect.

[0765] Server: Establishes a secure communication channel and initiates the video call session.

[0766] Terminal: Displays the video call screen, allowing the user and doctor to communicate in real time.

[0767] User: Engages in medical conversations with doctors and receives guidance and prescriptions.

[0768] Examples:

[0769] User C launches the app at the scheduled time and receives a consultation via video call with Doctor D. Doctor D makes an appropriate diagnosis and issues a prescription.

[0770] 4. Medical records and follow-up features

[0771] Server: After the consultation session is over, it stores the details in a database and sets reminders for upcoming consultations and follow-ups.

[0772] Terminal: Allows users to check their medical history and schedule their next appointment if necessary.

[0773] Users: Receive follow-up notifications and prepare for upcoming appointments.

[0774] Examples:

[0775] User E opens the app after the consultation and checks the medical history. The system sends a reminder for a follow-up appointment in one week, and the user makes the next appointment.

[0776] This system allows users living in remote areas to easily receive high-quality medical services, and through continuous follow-up, users can efficiently manage their health.

[0777] The processing flow will be explained below.

[0778] User registration and authentication functions

[0779] Step 1:

[0780] On the device: The user launches the app or website and clicks the "Sign up" button.

[0781] Step 2:

[0782] Terminal: Displays a form where the user enters required information such as name, address, contact information, and health insurance information.

[0783] Step 3:

[0784] User: Enter the required information and click the "Submit" button.

[0785] Step 4:

[0786] Terminal: Checks the entered information for any error messages, and if there are no problems, sends the information to the server.

[0787] Step 5:

[0788] Server: Stores the received user information in a database.

[0789] Step 6:

[0790] Server: Sends a confirmation email containing a verification link to the user's email address.

[0791] Step 7:

[0792] User: Receives verification email and clicks link.

[0793] Step 8:

[0794] Server: Checks the validity of the link and activates the user account.

[0795] Step 9:

[0796] Server: Sends account activation notification to the user.

[0797] ---

[0798] Doctor search and appointment function

[0799] Step 1:

[0800] Terminal: The user logs in and displays the symptom entry screen.

[0801] Step 2:

[0802] User: Enter symptoms and desired specialty and click the "Search" button.

[0803] Step 3:

[0804] Terminal: Sends the entered information to the server.

[0805] Step 4:

[0806] Server: Searches for doctor information in the database and generates a list of doctors that match the criteria.

[0807] Step 5:

[0808] Server: Sends doctor list and schedule information to the device.

[0809] Step 6:

[0810] Terminal: Displays a list of doctors to the user as search results.

[0811] Step 7:

[0812] User: Select the desired doctor and appointment date and time, and click the "Make an appointment" button.

[0813] Step 8:

[0814] Device: Send the selected information back to the server.

[0815] Step 9:

[0816] Server: Stores appointment information in a database and updates doctor schedules.

[0817] Step 10:

[0818] Server: Sends appointment confirmation notifications to users and doctors.

[0819] ---

[0820] Video call consultation function

[0821] Step 1:

[0822] On your device: Launch the app at the scheduled time and click the "Start video call" button.

[0823] Step 2:

[0824] Terminal: Displays the video call screen and sends a connection request to the server.

[0825] Step 3:

[0826] Server: Connects the user and doctor to a secure video call session.

[0827] Step 4:

[0828] Device: Initiate a video call that relays audio and video.

[0829] Step 5:

[0830] User: Receive medical treatment via video call with a doctor.

[0831] Step 6:

[0832] Server: Monitors the stability of the connection and handles errors as needed.

[0833] ---

[0834] Medical records and follow-up features

[0835] Step 1:

[0836] Server: After the video call session ends, the doctor's medical details are saved in a database.

[0837] Step 2:

[0838] Server: Sends a notification to the user that the medical record has been saved.

[0839] Step 3:

[0840] Terminal: Provides a screen where users can check their medical history.

[0841] Step 4:

[0842] Server: Schedule upcoming appointments and set follow-up reminders.

[0843] Step 5:

[0844] User: Receive reminders and schedule upcoming appointments.

[0845] Example 1

[0846] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0847] It is important to make it easy for people living in remote areas or users who have difficulty visiting medical institutions to receive appropriate medical services. There is also a need for a system that efficiently performs follow-up care and health management after medical treatment. However, current systems do not adequately meet these needs, so a system that can provide comprehensive medical services is needed.

[0848] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0849] In this invention, the server includes means for receiving medical information input by a user using a device, means for searching for an appropriate doctor based on the medical information, means for acquiring the doctor's schedule information and confirming the appointment, means for video calling the user and the doctor at the appointment time, means for saving the medical treatment details after the medical treatment session ends, means for setting a follow-up notification based on the medical record, and means for sending the follow-up notification to the user, thereby enabling the user to receive appropriate medical services even from a remote location and enabling follow-up after the medical treatment and long-term health management.

[0850] A "device" is an electronic device used by a user to input medical information, such as a smartphone, tablet, or computer.

[0851] "Medical information" refers to information about symptoms, medical history, and current health status that a user inputs using a device.

[0852] The "server" is a computer system that receives input medical information, searches for and schedules appropriate doctors, and manages video calls and medical records.

[0853] A "physician" is a medical professional who is qualified in a particular specialty and provides medical treatment and diagnosis to users.

[0854] "Schedule information" is information about the doctor's available dates and times and free times.

[0855] "Booking" is the process of confirming a consultation date and time selected by the user and securing a consultation session with a doctor.

[0856] The "video call means" is a video call function that connects the user and a doctor in real time to provide medical care.

[0857] A "medical session" is a series of processes in which a doctor provides medical care to a user via video call or the like.

[0858] "Medical content" refers to information such as diagnoses, instructions, and prescriptions given during a medical session.

[0859] A "follow-up notification" is a communication sent to a user after a medical examination to inform them of the next medical examination or follow-up appointment.

[0860] "Medical records" are data that store information such as a user's past medical history, diagnosis results, and prescriptions.

[0861] The present invention is a system that aims to enable users living in remote locations to receive high-quality medical services. This system provides functions that allow users to input medical information using a device, search for and make appointments with appropriate doctors, receive medical treatment via video call, save medical treatment details, and perform follow-ups. An embodiment of this system will be described in detail below.

[0862] User registration and authentication functions

[0863] A user launches an app or website using a device such as a smartphone or PC and enters personal information such as their name, email address, phone number, and password. The entered information is sent from the device to a server and stored in a database (e.g., MySQL) using SSL / TLS encrypted communication. The server then sends an authentication link to the user's email address, and the user activates their account by clicking the link.

[0864] Doctor search and appointment function

[0865] The user enters symptoms (e.g., abdominal pain) into the app on their device and sets search criteria (e.g., gastroenterology). This information is sent to the server, which filters corresponding specialists from a database (e.g., PostgreSQL). The filtered list of doctors is sent to the device in JSON format, and the user selects the doctor of their choice and makes an appointment. The reservation information is managed by the server, which uses a push notification service (e.g., Firebase) to send a reservation confirmation to the user.

[0866] Video call consultation function

[0867] When the time for the appointment arrives, the user launches the app on their device to prepare for the video call. The server then establishes a secure communication channel (e.g., WebRTC) and starts a video call session between the user and the doctor. The user and doctor communicate in real time through the video call, providing treatment, guidance, and issuing prescriptions.

[0868] Medical records and follow-up features

[0869] After the video call session ends, the server saves the medical record to a database (e.g., AWS RDS). The saved medical record is used to set reminders for the next medical appointment and follow-up. The user can check the medical history and next follow-up information through the app. The server also generates and sends follow-up notifications to the user.

[0870] Specific examples

[0871] User A uses the app to register personal information, receives a verification email, and clicks the link. He enters his symptoms of abdominal pain and searches for and makes an appointment with a gastroenterologist. When the appointment time arrives, he starts a video call and receives a diagnosis and prescription from the doctor. After the appointment, he checks the medical record, receives a follow-up notification, and prepares for the next appointment.

[0872] Prompt Sentence Examples

[0873] "What is the process for User A to launch the app to register for the first time, enter their personal information, receive a verification email, and click the link to activate their account?"

[0874] This system enables users living in remote areas to easily receive high-quality medical services, and allows for efficient health management through continuous follow-up.

[0875] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0876] Step 1:

[0877] Enter user information

[0878] Input: A user uses a device (smartphone or computer) to open an app or website and enter personal information (such as name, email address, phone number, or password).

[0879] Specific behavior: The user enters the required information into the device's input form and clicks the "Submit" button.

[0880] Output: The entered data is sent from the terminal to the server.

[0881] Step 2:

[0882] Information stored in a database

[0883] Input: Personal information sent from your device.

[0884] Specific operation: The server receives the received personal information using SSL / TLS encrypted communication, then stores the received data in a database (e.g., MySQL) and flags the data as successful.

[0885] Output: Personal information stored in a database and a verification email prepared.

[0886] Step 3:

[0887] Generate and send the authentication link

[0888] Input: User's email address and other personal information stored on the server.

[0889] What happens: The server generates a unique token for the user, generates an email containing an authentication link, and then uses a mail server (e.g., SendGrid) to send the authentication email.

[0890] Output: An email containing a verification link will be sent to the user's email address.

[0891] Step 4:

[0892] Receiving emails and clicking links

[0893] Input: The email containing the authentication link.

[0894] What happens: The user opens the email using their email client (e.g. Gmail) and clicks on the authentication link.

[0895] Output: The action of the user clicking the authentication link.

[0896] Step 5:

[0897] Activate your account

[0898] Input: Authentication link click event.

[0899] What happens: The server verifies the token in the authentication link, and if it matches, activates the user's account. If activation is successful, it generates a success message and sends it to the user.

[0900] Output: Activated user account and a success message.

[0901] Step 6:

[0902] Enter symptoms and set search conditions

[0903] Input: Symptoms and search terms entered by the user (e.g., gastroenterology).

[0904] Specific operation: The user enters symptoms and search criteria into the device app and clicks the "Search" button.

[0905] Output: The symptoms and search criteria are sent to the server.

[0906] Step 7:

[0907] Database Search and Filtering

[0908] Input: Symptoms and search criteria submitted by the user.

[0909] Specific operation: The server searches for doctor information in a database (e.g., PostgreSQL) and filters it based on the entered conditions. As a result, it generates a list of suitable doctors.

[0910] Output: A filtered list of doctors is generated.

[0911] Step 8:

[0912] Generate and submit a doctor list

[0913] Input: Filtered list of doctors.

[0914] Specific operation: The server converts the filtered doctor list into JSON format and sends it to the terminal.

[0915] Output: Doctor list in JSON format sent to the terminal.

[0916] Step 9:

[0917] Select a consultation time

[0918] Input: Doctor list displayed on terminal.

[0919] Specific operation: The user selects the doctor of their choice and the appointment date and time in the calendar format, and then the selection is sent to the server.

[0920] Output: The selected appointment date and time and doctor information are sent to the server.

[0921] Step 10:

[0922] Reservation updates and notifications

[0923] Input: User selected appointment date and time and doctor information.

[0924] What happens: The server updates the reservation schedule and generates a reservation confirmation notification, which is sent to the user using a push notification service (e.g., Firebase).

[0925] Output: Pushed booking confirmation notification.

[0926] Step 11:

[0927] Launching the app and preparing to connect

[0928] Input: Appointment time arrives.

[0929] Specific behavior: The user launches the app at the scheduled time and prepares to connect to the video call. The app requests access to the camera and microphone.

[0930] Output: Video call ready to connect.

[0931] Step 12:

[0932] Establishing a communication channel

[0933] Input: The user's ready to connect state.

[0934] What happens: The server establishes a secure communication channel (e.g., WebRTC) and connects the user and doctor to the same session.

[0935] Output: Establishment of a secure video call session.

[0936] Step 13:

[0937] Starting a video call session

[0938] Input: The established communication channel.

[0939] What happens: The video call screen appears in the user's app and a video call session with the doctor begins.

[0940] Output: Start of video call session.

[0941] Step 14:

[0942] Real-time communication

[0943] Input: A video call session.

[0944] Specific operation: The user communicates with the doctor in real time, explaining symptoms and asking questions, and the doctor makes a diagnosis based on the user's symptoms.

[0945] Output: Real-time communication results of the medical consultation.

[0946] Step 15:

[0947] Providing diagnosis and prescriptions

[0948] Input: Real-time communication results.

[0949] What it does: The doctor will explain the diagnosis and issue a prescription if necessary. The diagnosis and prescription information will be provided online.

[0950] Output: Diagnostic results and prescription information.

[0951] Step 16:

[0952] Ending and recording the medical session

[0953] Input: End of medical session.

[0954] What it does: After the video call session ends, the server logs the medical details, including the doctor's diagnosis, prescriptions, and the next recommended visit date and time.

[0955] Output: Recorded medical details.

[0956] Step 17:

[0957] Medical record database storage

[0958] Input: Recorded medical details.

[0959] What it does: The server stores medical records in a database (e.g., AWS RDS). The stored data is encrypted and only accessible to the user and their doctor.

[0960] Output: Medical records stored in a database.

[0961] Step 18:

[0962] Follow-up notification settings

[0963] Input: Medical record.

[0964] What it does: The server uses medical records to schedule reminders for upcoming appointments and follow-ups, adjusting the details of the reminders to fit the user's schedule.

[0965] Output: The set follow-up reminder.

[0966] Step 19:

[0967] Checking medical history

[0968] Input: Stored medical records.

[0969] What it does: The user accesses their medical history through the app, which shows past diagnoses, prescriptions, and next follow-up appointments.

[0970] Output: Displayed medical history.

[0971] Step 20:

[0972] Receive follow-up notifications

[0973] Input: The follow-up reminder that was set.

[0974] Specific operation: A follow-up notification is sent from the server at the specified date and time, and the user receives the notification. The user prepares for the next consultation.

[0975] Output: The follow-up notification sent to the user.

[0976] (Application example 1)

[0977] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0978] The problem that this invention aims to solve is to provide users living in remote areas with a means to efficiently access and learn appropriate medical educational content, in addition to enabling them to receive high-quality medical services. This will enable the creation of an environment in which users can not only access specialists, but also consistently use educational content to deepen their medical knowledge.

[0979] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0980] In this invention, the server

[0981] means for receiving medical information input by a user using a terminal;

[0982] A means for searching for an appropriate specialist based on the medical information;

[0983] means for acquiring schedule information of the specialist and confirming an appointment;

[0984] a video call means for connecting the user and the specialist at the appointment time;

[0985] means for saving the medical treatment details after the medical treatment session ends and notifying the user of the saving;

[0986] a means for a user to input search keywords to filter and display medical education content;

[0987] A means for conducting live lectures using video calls;

[0988] A way to keep a record of what you've watched and set reminders for upcoming lectures and follow-ups,

[0989] This allows users to receive medical treatment from specialists and efficiently access and learn high-quality medical education content.

[0990] "Medical information" refers to information such as health status, symptoms, and medical history entered by the user through the terminal.

[0991] A "specialist" is a medical professional with advanced knowledge and skills in a particular medical field.

[0992] "Schedule information" is information that indicates the dates and times when specialists are available for consultation and the reservation status.

[0993] A "video call" is a means for a user in a remote location and a specialist to communicate in real time using audio and video via the Internet.

[0994] "Medical content" refers to information about diagnosis and treatment plans recorded during video calls or face-to-face medical sessions.

[0995] "Notifications" are a means of informing users about medical treatments and follow-up appointments.

[0996] A "search keyword" is an input character string that a user uses when searching for medical education content.

[0997] "Filtering" is the process of sorting and displaying relevant medical education content based on search keywords.

[0998] "Medical education content" refers to information such as videos, articles, and lecture materials for learning medical knowledge and skills.

[0999] "Live Lecture" means an educational session that takes place in real time and is delivered via video call.

[1000] "Reminder" is a function that notifies users so that they do not forget their next lecture or medical appointment.

[1001] Overall system configuration

[1002] The system of the present invention is designed to enable users to receive high-quality medical services even when they are not in the field. This system mainly uses a smartphone app and is built by combining front-end and back-end technologies.

[1003] Hardware and Software Configuration

[1004] Hardware

[1005] User devices: Mainly smartphones are used.

[1006] Servers: Includes database and application servers.

[1007] software

[1008] Front-end: React Native (smartphone app development)

[1009] Backend: Node.js (application server), MongoDB (database)

[1010] Communication: WebRTC (secure communication for video calls)

[1011] Authentication: Firebase Authentication (user authentication)

[1012] Feature details

[1013] 1. User registration and authentication function

[1014] The user launches the smartphone app and enters the necessary personal information.

[1015] The server receives the entered information and stores it in a database (MongoDB). Firebase Authentication is used to send an authentication email to the user.

[1016] The user receives a verification email and clicks on the link to activate the account. The server verifies the link and activates the user's account.

[1017] 2. Doctor search and appointment function

[1018] The user terminal transmits search keywords based on the symptoms to the server.

[1019] The server queries the database (MongoDB) and generates a list of relevant specialists, which is then sent to the user's device.

[1020] The user selects the desired specialist from a list of doctors and makes an appointment.

[1021] The server stores the user's reservation information and sends a reservation confirmation notification.

[1022] 3. Video call medical consultation function

[1023] The user launches the smartphone app at the scheduled time and prepares to connect to the video call.

[1024] The server establishes a secure communication channel using WebRTC and initiates the video call session.

[1025] Users and specialists can conduct consultations in real time via video calls.

[1026] 4. Medical records and follow-up features

[1027] After the consultation session, the server saves the details of the consultation in a database and sets reminders for the next consultation or follow-up.

[1028] The user terminal provides an interface that allows the user to check their medical history and make their next appointment.

[1029] 5. Medical education content search and browsing function

[1030] The user terminal transmits search keywords to the server to filter medical education content.

[1031] The server retrieves the relevant content from a database (MongoDB) and returns it to the user's terminal.

[1032] Users can view the filtered content through a smartphone app.

[1033] 6. Live lecture function via video call

[1034] The user launches the smartphone app based on the schedule of the live lecture and prepares the video call connection.

[1035] The server uses WebRTC to establish a secure communication channel and conduct the video call session for the live lecture.

[1036] Users can listen to lectures in real time and ask questions and hold discussions.

[1037] 7. Content saving and follow-up features

[1038] The server stores a record of the content the user has viewed in a database and sets reminders for upcoming lectures and follow-ups.

[1039] The user terminal provides an interface that allows the user to check their viewing history and receive reminders for upcoming lectures.

[1040] Specific examples

[1041] For example, if a user wants to learn about the topic of "diabetes," they can enter search keywords into the app and related videos and articles will be displayed. Additionally, WebRTC can be used to attend live seminars by specialists, and the content of these seminars will be stored in a database.

[1042] Prompt Sentence Examples

[1043] Search for diabetes education content to get a list of related videos and articles, collect real-time user interest in these contents, and recommend the most viewed content.

[1044] This allows users to not only receive medical treatment from specialists, but also to effectively access and learn from high-quality medical education content.

[1045] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1046] Step 1:

[1047] User registration and authentication functions

[1048] Input: The user enters personal information into the device and presses the submit button.

[1049] Server: Receives personal information sent by the user and saves it in a database (MongoDB). After saving, sends an authentication email to the user via Firebase Authentication.

[1050] Output: A verification email will be sent to the user's email address.

[1051] User: Receives verification email and clicks verification link.

[1052] Server: Confirms the click on the authentication link and activates the user's account.

[1053] What happens: The server verifies the sender's email address and records the date and time the verification link is clicked.

[1054] Step 2:

[1055] Doctor search and appointment function

[1056] Input: The user enters the symptoms into the terminal and presses the search button.

[1057] Server: Receives the input symptom information and queries the database (MongoDB) to generate a list of relevant specialists.

[1058] Output: The filtered list of specialists is sent to the user terminal.

[1059] User: Selects the desired specialist from a list of specialists and schedules an appointment.

[1060] Server: Receives the reservation information, stores it in a database, and sends a reservation confirmation to the user.

[1061] Specific operation: The server updates the appointment schedule based on the date and time, specialist ID, and user ID, and calls the notification function.

[1062] Step 3:

[1063] Video call consultation function

[1064] Input: At the scheduled time, the user launches the app on their device and prepares for the video call.

[1065] Server: Receives the video call request and establishes a secure communication channel using WebRTC.

[1066] Output: A video call session is started and the video call screen is displayed on the user's device.

[1067] User: Consults with specialists via video call.

[1068] What it does: The server manages WebRTC peer connections and ensures communication encryption.

[1069] Step 4:

[1070] Medical records and follow-up features

[1071] Input: After the medical session ends, the server receives the medical details.

[1072] Server: Receives medical records, stores them in a database, and sets follow-up reminders.

[1073] Output: The medical details are saved in the database and a notification is sent to the user's device.

[1074] Users: Receive notifications and can check their medical history.

[1075] What it does: The server tags the stored visits and schedules reminders for upcoming appointments.

[1076] Step 5:

[1077] Medical education content search and browsing function

[1078] Input: The user enters search keywords into the device and presses the search button.

[1079] Server: Receives search keywords, queries the database, and filters relevant medical education content.

[1080] Output: The filtered medical education content is sent to the user terminal.

[1081] User: Views filtered content on a smartphone app.

[1082] What it does: The server ranks the search results and prioritizes the most relevant content to present to the user.

[1083] Step 6:

[1084] Live lecture function via video call

[1085] Input: User launches the app and prepares based on the schedule of a live lecture.

[1086] Server: Receives the video call request and establishes a secure communication channel using WebRTC.

[1087] Output: A video call session of a live lecture is started and the video is displayed on the user's device.

[1088] Users: Attend live lectures, ask questions and participate in discussions.

[1089] Specific operation: The server monitors the progress of live lectures in real time and notifies you if any problems occur.

[1090] Step 7:

[1091] Content saving and follow-up features

[1092] Input: The server receives information about the content that was viewed.

[1093] Server: Stores records of viewed content in a database and sets reminders for upcoming lectures and follow-ups.

[1094] Output: The viewing history is stored in a database and a reminder for the next lecture is sent to the user's device.

[1095] User: Check viewing history and prepare for the next lecture.

[1096] What it does: The server creates statistics on viewing history and analyzes the topics that users are most interested in.

[1097] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1098] The present invention is a system that enables users living in remote locations to receive high-quality medical services. The system includes a process where users input medical information using a terminal, search for an appropriate specialist, and make an appointment. It also includes functions for receiving medical treatment via video call, saving medical results and providing follow-up, and using an emotion engine to reflect the user's emotional state in the medical treatment.

[1099] System configuration

[1100] The system has the following main functions:

[1101] 1. User registration and authentication function

[1102] 2. Doctor search and appointment function

[1103] 3. Video call medical consultation function

[1104] 4. Medical records and follow-up features

[1105] 5. Emotion recognition function using emotion engine

[1106] Program processing flow

[1107] 1. User registration and authentication function

[1108] On the device: The user launches the app or website and enters the required personal information.

[1109] Server: Receives the entered information, stores it in a database, and sends a verification link to the user's email address.

[1110] User: Receives a verification email and clicks on the link to activate the account.

[1111] Server: Verify the link and activate the user's account.

[1112] Examples:

[1113] User A accesses the account remotely, provides the necessary information, and creates an account. After receiving the authentication email, the user clicks the link to activate the account.

[1114] 2. Doctor search and appointment function

[1115] Device: Enter symptoms into the app on your device and set search criteria.

[1116] Server: Based on the input information, the server filters the doctor information in the database and generates a list of doctors that meet the criteria. This list is then sent to the terminal.

[1117] Terminal: Shows the user a list of doctors and lets them select the appointment time they prefer.

[1118] Server: Confirms the selected reservation information, updates the schedule, and sends a reservation confirmation notification to the user.

[1119] Examples:

[1120] User B complains of stomach pain and searches for a gastroenterologist. He selects an appropriate doctor from the search results and makes an appointment for 2:00 PM the next day.

[1121] 3. Video call medical consultation function

[1122] Device: The user launches the app at the scheduled time and prepares to connect.

[1123] Server: Establishes a secure communication channel and initiates the video call session.

[1124] Terminal: Displays the video call screen, allowing the user and doctor to communicate in real time.

[1125] User: Engages in medical conversations with doctors and receives guidance and prescriptions.

[1126] Examples:

[1127] User C launches the app at the scheduled time and receives a consultation via video call with Doctor D. Doctor D makes an appropriate diagnosis and issues a prescription.

[1128] 4. Medical records and follow-up features

[1129] Server: After the consultation session is over, it stores the details in a database and sets reminders for upcoming consultations and follow-ups.

[1130] Terminal: Allows users to check their medical history and schedule their next appointment if necessary.

[1131] Users: Receive follow-up notifications and prepare for upcoming appointments.

[1132] Examples:

[1133] User E opens the app after the consultation and checks the medical history. The system sends a reminder for a follow-up appointment in one week, and the user makes the next appointment.

[1134] 5. Emotion recognition function using emotion engine

[1135] Device: Captures the user's facial expressions and voice in real time during a video call.

[1136] Server: The emotion engine analyzes the captured data and recognizes the user's emotional state.

[1137] Server: Provides the doctor with the recognized emotional information and reflects it in the medical treatment.

[1138] Server: Stores medical records including emotional information in a database.

[1139] Examples:

[1140] When User F has a video call with a doctor, the emotion engine recognizes that the user is anxious from their facial expression. This information is provided to the doctor, who takes the user's mental state into consideration during the consultation. The emotional information is recorded in the medical treatment details and prescription.

[1141] This system allows users living in remote areas to easily receive high-quality medical services. In addition, by providing medical treatment based on the user's emotional state, more personalized medical services can be provided.

[1142] The processing flow will be explained below.

[1143] Emotion recognition function using emotion engine

[1144] Step 1:

[1145] Device: Show a popup to get the user's camera and microphone permissions before starting a video call.

[1146] Step 2:

[1147] User: Allow use of camera and microphone.

[1148] Step 3:

[1149] Device: After obtaining permission, it displays the video call screen and captures the user's facial expressions and voice in real time.

[1150] Step 4:

[1151] Terminal: Sends captured facial expression and voice data to the server.

[1152] Step 5:

[1153] Server: Analyzes the received facial expression and voice data and uses an emotion engine to recognize the user's emotional state.

[1154] Step 6:

[1155] Server: Processes the recognized emotion information and displays it on the doctor's screen in real time.

[1156] Step 7:

[1157] Terminal: The doctor checks the user's emotional state and responds accordingly.

[1158] Step 8:

[1159] Server: Stores the details of the medical session, including emotional information, in a database.

[1160] Step 9:

[1161] Server: Uses the stored emotional information to inform the next medical appointment or follow-up.

[1162] Specific examples

[1163] Video call consultations

[1164] Step 1:

[1165] Device: User F clicks "Start video call" on the screen for starting a video call.

[1166] Step 2:

[1167] On your device: A pop-up will appear asking you to allow access to your camera and microphone.

[1168] Step 3:

[1169] User: Allow use of camera and microphone.

[1170] Step 4:

[1171] Terminal: The video call screen is displayed, and User F's facial expressions and voice are captured.

[1172] Step 5:

[1173] Device: Captured data is sent to the server.

[1174] Step 6:

[1175] Server: Analyzes the received data, and the emotion engine recognizes User F's emotions as both calm and anxious.

[1176] Step 7:

[1177] Server: The recognized emotion information is displayed on Doctor D's screen.

[1178] Step 8:

[1179] Terminal: Doctor D checks User F's emotional state and provides medical treatment while taking it into consideration.

[1180] Step 9:

[1181] Server: Stores the results of the medical session and emotional information in a database.

[1182] Step 10:

[1183] Server: Based on the stored emotional information, sends notifications and reminders to User F that will be helpful for their next medical appointment.

[1184] This system enables medical treatment that takes into account the user's emotional state, enabling more personalized medical services to be provided. Furthermore, by storing emotional information and utilizing it for the next medical treatment, continuous and consistent medical support can be achieved.

[1185] Example 2

[1186] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1187] In order for users living in remote areas to receive high-quality medical services, they need a system that smoothly handles the entire process, from searching for specialists and making appointments to video call consultations, saving medical records, and providing follow-ups. However, existing systems lack the functionality to reflect the user's emotional state in medical treatment, making them insufficient for providing more personalized medical services. Therefore, there is a need for a system that can analyze the user's emotional state in real time and reflect that in medical treatment.

[1188] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1189] In this invention, the server includes means for receiving medical information input by a user using a terminal, means for searching for an appropriate medical professional based on the medical information, means for acquiring the professional's schedule information and confirming an appointment, means for establishing a secure video call session between the user and the medical professional, means for saving the medical treatment details after the medical treatment session ends and notifying the user, means for capturing the user's facial expressions and voice data during the video call and performing emotion analysis, and means for providing the medical professional with the results of the emotion analysis and reflecting them in the medical treatment details, thereby enabling users in remote locations to receive high-quality and personalized medical services.

[1190] "User" refers to an individual who uses the system to input medical information and receive medical treatment.

[1191] "Terminal" refers to an information processing device such as a smartphone, PC, or tablet used by a user.

[1192] "Medical information" refers to all information necessary for medical treatment, such as medical condition, symptoms, and past medical history, entered by the user.

[1193] "Medical professional" refers to a doctor or nurse who has specialized knowledge and skills in a particular medical field.

[1194] "Schedule information" refers to information such as the availability of medical professionals and appointment availability.

[1195] "Video calling" refers to a means of real-time video and audio communication between a user and a medical professional via the Internet.

[1196] "Medical content" refers to the doctor's diagnosis, treatment plan, prescription information, etc. recorded during a medical session.

[1197] "Emotion analysis" refers to the process of analyzing a user's facial expressions and voice data during a video call to identify the user's emotional state.

[1198] "Emotional state" refers to information that indicates the user's psychological state or mood.

[1199] "Follow-up Notification" means a notification sent to a user for a follow-up or reminder of an upcoming medical appointment.

[1200] The present invention is a system that enables users living in remote locations to receive high-quality medical services. The system involves a process in which users input medical information using a terminal, search for an appropriate medical professional, and receive medical treatment via video call. In addition to functions for saving and following up on medical treatment details, the system also has an emotion analysis function that reflects the user's emotional state in medical treatment.

[1201] System configuration

[1202] 1. User registration and authentication function

[1203] On the device: The user launches an app or website and enters personal information such as their name, email address, and password.

[1204] Server: Receives the entered information, stores it in a database (e.g., MySQL), and sends an email containing a verification link to the user's email address.

[1205] User: Receives a verification email and clicks on the link to activate the account.

[1206] Server: Verify the link and activate the user's account.

[1207] 2. Doctor search and appointment function

[1208] Device: Enter symptoms into the device app and set search criteria.

[1209] Server: Based on the input information, it searches a doctor database (e.g., Elasticsearch) and generates a list of doctors that match the criteria. It then sends this list to the terminal.

[1210] Terminal: Shows the user a list of doctors and lets them select the appointment time they prefer.

[1211] Server: Confirms the selected reservation information, saves the schedule information to the database, and sends a reservation confirmation notification to the user.

[1212] 3. Video call medical consultation function

[1213] On the device: At the scheduled time, the user launches the app and prepares for the video call.

[1214] Server: Uses technologies such as WebRTC to establish a secure video call session and connect users with medical professionals.

[1215] Terminal: Real-time communication between the user and medical professionals.

[1216] 4. Medical records and follow-up features

[1217] Server: After the consultation session is over, it stores the consultation details in a database and sets reminders for the next consultation or follow-up.

[1218] Terminal: Allows users to check their medical history and schedule their next appointment if necessary.

[1219] 5. Emotion recognition function using emotion engine

[1220] Device: Captures facial and audio data in real time during a video call.

[1221] Server: Analyzes the captured data using an emotion analysis engine (e.g., Microsoft Azure's Face API) to identify the user's emotional state.

[1222] Server: Providing recognized emotion information to medical professionals to improve the quality of medical care.

[1223] Server: Stores medical records including emotional information in a database.

[1224] Specific examples

[1225] User A accesses the system and creates an account. He receives a verification email and clicks the link to activate his account. He then complains of abdominal pain, searches for a gastroenterologist, and makes an appointment for 2:00 p.m. the next day. He launches the app at the scheduled time and consults with the doctor via video call. The consultation details are saved in the database, and a reminder to return to the doctor one week later is sent.

[1226] Examples of prompt statements

[1227] Enter the following prompt for the generative AI model:

[1228] Please explain the steps a user takes to access the platform remotely and create an account. Include the process by which a user enters symptoms, searches for a doctor, and makes an appointment using the medical platform. Also, please explain the process for receiving medical advice in real time via video call, as well as the steps for saving medical records and scheduling the next follow-up appointment. Also, please include a function that uses an emotion engine to recognize the user's emotional state and reflect it in medical treatment. Please clearly indicate specific tools and technologies (e.g., WebRTC, MySQL, Microsoft Azure Face API, etc.).

[1229] This system enables users living in remote areas to receive high-quality, personalized medical services.

[1230] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1231] System program processing flow

[1232] 1. User registration and authentication function

[1233] Step 1: Enter your user information

[1234] User: Accesses an application or website from a smartphone or computer device and enters personal information such as name, email address, and password.

[1235] Input: The user enters information using a keyboard or touch screen.

[1236] Output: The input information is ready to be used.

[1237] Specific behavior: The user fills in the required information in the form and clicks the "Register" button.

[1238] Step 2: Send and store user information

[1239] Terminal: Encrypts the entered data and sends it to the server via a secure communication protocol (e.g., HTTPS).

[1240] Server: Stores the received data in a database (e.g., MySQL).

[1241] Input: Encrypted user information.

[1242] Output: A confirmation that the data was saved correctly to the database.

[1243] Specific operation: When the device clicks the send button, the data is encrypted and sent to the server. After receiving it, the server stores the data in a database.

[1244] Step 3: Sending a verification email

[1245] Server: Uses a mail server to send an email containing a verification link to the email address entered by the user.

[1246] Input: The user's email address.

[1247] Output: A confirmation that the verification email was sent successfully.

[1248] Specific operation: The server automatically generates an authentication email and sends it to the user via the mail server.

[1249] Step 4: Click on the authentication link

[1250] Users: Open the verification email they received and click the link to activate their account.

[1251] Input: The link in the verification email.

[1252] Output: A confirmation that the account has been activated.

[1253] What happens: The user checks their email, clicks the link, and is redirected to a page to activate their account.

[1254] Step 5: Activate your account

[1255] Server: Checks the information in the authentication link and updates the user's status in the database to "active".

[1256] Input: Authentication link information.

[1257] Output: Confirmation that the user account has been activated.

[1258] What happens: The server validates the link and updates the user status in the database.

[1259] 2. Doctor search and appointment function

[1260] Step 1: Enter symptoms and set search criteria

[1261] User: Enters search criteria into the application, such as symptoms and desired appointment date and time.

[1262] Input: Symptom information and desired date and time of consultation entered by the user.

[1263] Output: The search criteria has been set.

[1264] Specific behavior: The user enters their symptoms and desired appointment date into the app's search form and clicks the "Search" button.

[1265] Step 2: Filtering physician information

[1266] Server: Based on the input information, it searches a doctor database (e.g., Elasticsearch) and generates a list of doctors that match the criteria.

[1267] Input: The search criteria entered by the user.

[1268] Output: A list of doctors who match the criteria.

[1269] Specific operation: The server executes a database query to retrieve physician data that matches the criteria.

[1270] Step 3: View the Doctor List

[1271] Terminal: Displays the list of doctors received from the server to the user.

[1272] Input: Doctor list sent from the server.

[1273] Output: A list of doctors displayed on the user interface.

[1274] Specific behavior: The device displays the list in the user interface so that the user can check it.

[1275] Step 4: Choose your appointment time

[1276] User: Select the desired appointment time and confirm the appointment.

[1277] Input: Selected appointment times and doctor information.

[1278] Output: Reservation information is confirmed.

[1279] Specific actions: The user selects a doctor and appointment time and clicks the "Make appointment" button.

[1280] Step 5: Confirm and notify reservation information

[1281] Server: Saves the selected reservation information to a database and sends a reservation confirmation notification to the user.

[1282] Input: Booking information selected by the user.

[1283] Output: The reservation information is saved to the database and a notification is sent to the user.

[1284] Specific behavior: The server records the reservation information in a database and sends a reservation confirmation email.

[1285] 3. Video call medical consultation function

[1286] Step 1: Prepare for the video call

[1287] User: Launches the application at the scheduled time and prepares for the video call.

[1288] Input: User's app launch action.

[1289] Output: Ready to start a video call.

[1290] Specific behavior: The user launches the app and clicks the start video call button.

[1291] Step 2: Establishing communication channels

[1292] Server: Establish a secure video call channel using WebRTC.

[1293] Input: User's request to start a video call.

[1294] Output: A secure video call session is established.

[1295] What happens: The server initializes a WebRTC session and connects the user with a medical professional.

[1296] Step 3: Start a video call

[1297] Device: The user and medical professionals communicate in real time via a video call screen.

[1298] Enter: a secure video call session.

[1299] Output: Real-time medical consultation with video and audio.

[1300] What happens: The device displays a video call window and the user and medical professional begin talking in real time.

[1301] Step 4: Proceeding with the treatment

[1302] Users: Engage in medical conversations with healthcare professionals and receive guidance and prescriptions.

[1303] Input: Medical consultation details.

[1304] Output: The consultation is completed and any necessary instructions or prescriptions are provided.

[1305] What it does: A user answers a question or gets advice from a medical professional.

[1306] 4. Medical records and follow-up features

[1307] Step 1: Save your medical records

[1308] Server: After the medical session is over, the medical details entered by the medical professional are stored in a database.

[1309] Input: Medical record data.

[1310] Output: Medical information saved in the database.

[1311] Specific operation: The server stores the medical information in a database.

[1312] Step 2: Set up follow-up

[1313] Server: Sets reminders for upcoming appointments and follow-ups in the system.

[1314] Input: Follow-up appointment information.

[1315] Output: The next appointment reminder has been set.

[1316] What happens: The server saves the reminder information in a database.

[1317] Step 3: Review medical history

[1318] User: Checks medical history in the application and schedules next appointments if necessary.

[1319] Input: Medical History Request Operation.

[1320] Output: Medical history displayed.

[1321] What happens: The user views their medical history in the app and clicks the "Schedule a follow-up appointment" button.

[1322] Step 4: Receive follow-up notifications

[1323] Users: Receive follow-up notifications and prepare for upcoming appointments.

[1324] Enter: Follow-up notification.

[1325] Output: The notification has been confirmed and the next appointment is ready.

[1326] What happens: The user receives a notification and sees what's next on their schedule.

[1327] 5. Emotion recognition function using emotion engine

[1328] Step 1: Capture facial expressions and voice

[1329] Device: Captures the user's facial expressions and voice in real time during a video call.

[1330] Input: User's facial and voice data.

[1331] Output: Captured facial expression and speech data.

[1332] What happens: Your device captures data using the camera and microphone.

[1333] Step 2: Analyze emotional state

[1334] Server: Analyzes the captured data using emotion recognition software (e.g., Microsoft Azure's Face API) to identify the user's emotional state.

[1335] Input: Captured facial and speech data.

[1336] Output: Data about the user's emotional state.

[1337] Specific operation: The server calls the API and obtains emotion data.

[1338] Step 3: Providing emotional information

[1339] Server: Provides recognized emotional information to medical professionals and reflects it in medical treatment.

[1340] Input: Emotional state data.

[1341] Output: Emotional information provided to a medical professional.

[1342] Specific operation: The server displays the emotional information on the medical professional's screen.

[1343] Step 4: Preserving medical records

[1344] Server: Stores medical records including emotional information in a database.

[1345] Input: Emotional information and medical details data.

[1346] Output: Medical records containing emotion data stored in a database.

[1347] Specific operation: The server integrates the emotion data and medical details and stores them in a database.

[1348] summary

[1349] These specific processing steps and operations enable users living in remote locations to receive high-quality, personalized medical services. Smooth coordination between each step ensures that the entire medical treatment process is carried out efficiently.

[1350] (Application example 2)

[1351] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1352] Conventional telemedicine systems allow users to receive high-quality medical services from remote locations, but they have the drawback of not reflecting the emotional state of the individual in the medical treatment, making it difficult to provide treatment that fully takes into account the psychological state of the user. There is also the risk of personal medical information being leaked during video calls. It is necessary to solve these issues and provide safe and personalized medical services.

[1353] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1354] In this invention, the server includes means for receiving medical information input by a user using a terminal, means for searching for an appropriate specialist based on the medical information, means for acquiring the specialist's schedule information and confirming the appointment, means for video calling to connect the user with the specialist at the appointment time, means for capturing the user's facial expressions and voice during the video call and analyzing the emotional state, and means for saving the medical treatment details after the medical treatment session ends and notifying the user. This enables personalized medical treatment that takes into account the user's emotional state and also provides medical services in a secure communication environment.

[1355] A "terminal" is an electronic device that allows a user to input medical information, communicate with specialists, and receive medical treatment.

[1356] "Medical information" refers to information entered by the user, such as their medical condition, symptoms, medical history, etc.

[1357] A "specialist" is a doctor with advanced knowledge and experience in a particular medical field.

[1358] "Schedule information" is information that indicates the dates and times when specialists are available for consultation, the reservation status, and so on.

[1359] "Video communication means" refers to audio and video communication means for real-time communication between the user and the specialist.

[1360] "Emotion recognition means" is a technology that analyzes a user's facial expressions and voice during a video call to recognize their emotional state.

[1361] A "medical session" refers to a period of time during which a medical consultation is conducted between a user and a specialist.

[1362] "Medical content" refers to a record of the diagnosis, treatment, instruction, etc. provided during a medical session.

[1363] "Notification" is a means of communicating information to inform users of medical results and the need for follow-up.

[1364] An embodiment of the present invention provides a system that enables a user in a remote location to safely receive high-quality medical services. A specific method for implementing this system will be described below.

[1365] 1. Basic configuration

[1366] This system consists of a user device, a specialist device, and a server that links these devices. The user device is a smartphone, tablet, or PC, and the specialist device uses similar devices. The server is located on the cloud and manages the entire system.

[1367] 2. Enter medical information and search for specialists

[1368] Users input medical information using their own devices. This medical information includes specific medical conditions, symptoms, medical history, etc., and is received by a server. The server has a means for searching for an appropriate specialist based on the received medical information. A pre-built doctor database is used to search for a specialist. Search criteria also include the specialist's qualifications, field of expertise, and user ratings.

[1369] 3. Confirmation of reservation

[1370] After a specialist is found, the server retrieves the specialist's schedule information and displays available appointment dates and times on the user's device. When the user selects the desired appointment time, the server confirms the appointment based on that information. When the appointment is confirmed, a notification is sent to both the user and the specialist.

[1371] 4. Video calling methods

[1372] When the appointment time arrives, the server establishes a video call session between the user and the specialist's device. This video call is highly secure, using end-to-end encryption technology, such as OpenSSL and WebRTC.

[1373] 5. Emotion recognition means

[1374] During a video call, the user's device uses a camera and microphone to capture the user's facial expressions and voice, and sends the data to the server. The server then uses an emotion engine to analyze the user's facial expressions and voice data and recognize their emotional state. This emotional information is provided to specialists in real time and reflected in their medical treatment. The emotional information is also saved as part of the medical record.

[1375] 6. Storage and notification of medical information

[1376] After the consultation session ends, the server stores the consultation details in a database. This data includes the specialist's diagnosis, advice, prescriptions, and the user's emotional state. The system also has the ability to automatically schedule the next consultation and set follow-up notifications, allowing users to receive reminders and be better prepared for their next appointment.

[1377] Specific examples

[1378] User A feels unwell and enters medical information about abdominal pain, which is received by the server. The server searches for gastroenterologists and displays the schedules of several candidates. User A selects 3:00 p.m. the following day and the appointment is confirmed. When the appointment time arrives, a video call begins between User A and Specialist B. During the call, the emotion engine recognizes the anxious expression on User A's face and displays this information to Specialist B. After the consultation, the details of the consultation and emotional information are saved, and User A is notified of the next follow-up appointment.

[1379] Prompt Sentence Examples

[1380] User A logs in using the application, enters their symptoms, and searches for a doctor. The AI ​​model recommends the most suitable doctor and schedules a video call. During the call, a facial expression recognition engine analyzes the user's emotional state, and appropriate responses are made within the video call. The session data is encrypted and stored, and a notification for the next follow-up is set.

[1381] This allows users to safely receive medical care from specialists even from remote locations, and provides personalized medical services that take emotional state into account, resulting in a higher quality medical experience.

[1382] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1383] Step 1:

[1384] The user inputs medical information using the terminal.

[1385] Input: User information about medical condition, symptoms, and medical history.

[1386] Data processing and calculation: Preprocessing to standardize the information entered by the user and store it in a database in a format that conforms to the format.

[1387] Output: Standardized medical information data.

[1388] What happens: A user launches an application, fills out a form, and hits the submit button. This information is sent to the server for preprocessing.

[1389] Step 2:

[1390] A server receives the medical information and executes a procedure to search for an appropriate specialist.

[1391] Input: User's medical information data.

[1392] Data processing and calculation: Using AI algorithms to filter and rank the most suitable specialists based on medical information.

[1393] Output: A list of suitable specialists.

[1394] Specific operation: The server analyzes the received medical information and lists the most suitable doctors from a specialist database.

[1395] Step 3:

[1396] The server retrieves the specialist's schedule information and presents it to the user.

[1397] Input: A list of appropriate specialists.

[1398] Data processing and calculation: Retrieves specialist schedule information from the database and displays it as a choice for the user.

[1399] Output: The available schedule presented to the user.

[1400] Specific operation: The server retrieves the specialist's schedule information from the database, sends the available appointment time slots to the user's terminal, and displays them as options for the user.

[1401] Step 4:

[1402] The user selects the desired reservation time, and the server confirms the reservation.

[1403] Input: The appointment time selected by the user.

[1404] Data processing and calculation: The selected reservation information is saved in the database and notified to the specialist and the user.

[1405] Output: Booking confirmation notification.

[1406] Specific operation: The user selects the desired time and presses the "Confirm appointment" button. The server updates the appointment information and sends a notification to the specialist and the user.

[1407] Step 5:

[1408] When the appointment time arrives, the server establishes a video call means connecting the user with the specialist.

[1409] Input: Appointment time, user, and specialist connection information.

[1410] Data processing and calculation: Encrypting the video call session and establishing the connection.

[1411] Output: A secure video call session.

[1412] Specific Operation: The server initiates a video call session and establishes a secure communication channel between the user and the specialist.

[1413] Step 6:

[1414] During a video call, the terminal captures the user's facial expressions and voice and performs emotion recognition.

[1415] Input: User's facial expression data and voice data.

[1416] Data processing and calculation: Analyze the user's emotional state using an emotion recognition engine.

[1417] Output: User's emotional information.

[1418] Specific operation: The camera and microphone on the user's device capture facial expressions and voice, and send this data to the server, which then analyzes the data using the emotion engine.

[1419] Step 7:

[1420] After the consultation session is over, the server stores the consultation and sends follow-up notifications to the user.

[1421] Input: Clinical content data and emotional information.

[1422] Data processing and calculation: storing medical details and emotional information in the database and setting notifications for the next follow-up.

[1423] Output: Stored medical records and follow-up notifications.

[1424] Specific operation: The server saves the medical results in the database, sets the next follow-up notification, and sends a notification to the user.

[1425] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1426] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1427] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1428] [Third embodiment]

[1429] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1430] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1431] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1432] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1433] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1434] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1435] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1436] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1437] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1438] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1439] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1440] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1441] The present invention is a system that enables users living in remote locations to receive high-quality medical services. The system supports the process of users entering medical information using a terminal, searching for an appropriate specialist, and making an appointment. The system also has the function of receiving medical treatment via video call, storing medical treatment results, and performing follow-up.

[1442] System configuration

[1443] The system has the following main functions:

[1444] 1. User registration and authentication function

[1445] 2. Doctor search and appointment function

[1446] 3. Video call medical consultation function

[1447] 4. Medical records and follow-up features

[1448] Program processing flow

[1449] 1. User registration and authentication function

[1450] On the device: The user launches the app or website and enters the required personal information.

[1451] Server: Receives the entered information, stores it in a database, and sends a verification link to the user's email address.

[1452] User: Receives a verification email and clicks on the link to activate the account.

[1453] Server: Verify the link and activate the user's account.

[1454] Examples:

[1455] User A accesses the account remotely, provides the necessary information, and creates an account. After receiving the authentication email, the user clicks the link to activate the account.

[1456] 2. Doctor search and appointment function

[1457] Device: Enter symptoms into the app on your device and set search criteria.

[1458] Server: Based on the input information, the server filters the doctor information in the database, generates a list of appropriate doctors, and sends it to the terminal.

[1459] Terminal: Shows the user a list of doctors and lets them select the appointment time they prefer.

[1460] Server: Confirms the selected reservation information, updates the schedule, and sends a reservation confirmation notification to the user.

[1461] Examples:

[1462] User B complains of stomach pain and searches for a gastroenterologist. He selects an appropriate doctor from the search results and makes an appointment for 2:00 PM the next day.

[1463] 3. Video call medical consultation function

[1464] Device: The user launches the app at the scheduled time and prepares to connect.

[1465] Server: Establishes a secure communication channel and initiates the video call session.

[1466] Terminal: Displays the video call screen, allowing the user and doctor to communicate in real time.

[1467] User: Engages in medical conversations with doctors and receives guidance and prescriptions.

[1468] Examples:

[1469] User C launches the app at the scheduled time and receives a consultation via video call with Doctor D. Doctor D makes an appropriate diagnosis and issues a prescription.

[1470] 4. Medical records and follow-up features

[1471] Server: After the consultation session is over, it stores the details in a database and sets reminders for upcoming consultations and follow-ups.

[1472] Terminal: Allows users to check their medical history and schedule their next appointment if necessary.

[1473] Users: Receive follow-up notifications and prepare for upcoming appointments.

[1474] Examples:

[1475] User E opens the app after the consultation and checks the medical history. The system sends a reminder for a follow-up appointment in one week, and the user makes the next appointment.

[1476] This system allows users living in remote areas to easily receive high-quality medical services, and through continuous follow-up, users can efficiently manage their health.

[1477] The processing flow will be explained below.

[1478] User registration and authentication functions

[1479] Step 1:

[1480] On the device: The user launches the app or website and clicks the "Sign up" button.

[1481] Step 2:

[1482] Terminal: Displays a form where the user enters required information such as name, address, contact information, and health insurance information.

[1483] Step 3:

[1484] User: Enter the required information and click the "Submit" button.

[1485] Step 4:

[1486] Terminal: Checks the entered information for any error messages, and if there are no problems, sends the information to the server.

[1487] Step 5:

[1488] Server: Stores the received user information in a database.

[1489] Step 6:

[1490] Server: Sends a confirmation email containing a verification link to the user's email address.

[1491] Step 7:

[1492] User: Receives verification email and clicks link.

[1493] Step 8:

[1494] Server: Checks the validity of the link and activates the user account.

[1495] Step 9:

[1496] Server: Sends account activation notification to the user.

[1497] ---

[1498] Doctor search and appointment function

[1499] Step 1:

[1500] Terminal: The user logs in and displays the symptom entry screen.

[1501] Step 2:

[1502] User: Enter symptoms and desired specialty and click the "Search" button.

[1503] Step 3:

[1504] Terminal: Sends the entered information to the server.

[1505] Step 4:

[1506] Server: Searches for doctor information in the database and generates a list of doctors that match the criteria.

[1507] Step 5:

[1508] Server: Sends doctor list and schedule information to the device.

[1509] Step 6:

[1510] Terminal: Displays a list of doctors to the user as search results.

[1511] Step 7:

[1512] User: Select the desired doctor and appointment date and time, and click the "Make an appointment" button.

[1513] Step 8:

[1514] Device: Send the selected information back to the server.

[1515] Step 9:

[1516] Server: Stores appointment information in a database and updates doctor schedules.

[1517] Step 10:

[1518] Server: Sends appointment confirmation notifications to users and doctors.

[1519] ---

[1520] Video call consultation function

[1521] Step 1:

[1522] On your device: Launch the app at the scheduled time and click the "Start video call" button.

[1523] Step 2:

[1524] Terminal: Displays the video call screen and sends a connection request to the server.

[1525] Step 3:

[1526] Server: Connects the user and doctor to a secure video call session.

[1527] Step 4:

[1528] Device: Initiate a video call that relays audio and video.

[1529] Step 5:

[1530] User: Receive medical treatment via video call with a doctor.

[1531] Step 6:

[1532] Server: Monitors the stability of the connection and handles errors as needed.

[1533] ---

[1534] Medical records and follow-up features

[1535] Step 1:

[1536] Server: After the video call session ends, the doctor's medical details are saved in a database.

[1537] Step 2:

[1538] Server: Sends a notification to the user that the medical record has been saved.

[1539] Step 3:

[1540] Terminal: Provides a screen where users can check their medical history.

[1541] Step 4:

[1542] Server: Schedule upcoming appointments and set follow-up reminders.

[1543] Step 5:

[1544] User: Receive reminders and schedule upcoming appointments.

[1545] Example 1

[1546] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1547] It is important to make it easy for people living in remote areas or users who have difficulty visiting medical institutions to receive appropriate medical services. There is also a need for a system that efficiently performs follow-up care and health management after medical treatment. However, current systems do not adequately meet these needs, so a system that can provide comprehensive medical services is needed.

[1548] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1549] In this invention, the server includes means for receiving medical information input by a user using a device, means for searching for an appropriate doctor based on the medical information, means for acquiring the doctor's schedule information and confirming the appointment, means for video calling the user and the doctor at the appointment time, means for saving the medical treatment details after the medical treatment session ends, means for setting a follow-up notification based on the medical record, and means for sending the follow-up notification to the user, thereby enabling the user to receive appropriate medical services even from a remote location and enabling follow-up after the medical treatment and long-term health management.

[1550] A "device" is an electronic device used by a user to input medical information, such as a smartphone, tablet, or computer.

[1551] "Medical information" refers to information about symptoms, medical history, and current health status that a user inputs using a device.

[1552] The "server" is a computer system that receives input medical information, searches for and schedules appropriate doctors, and manages video calls and medical records.

[1553] A "physician" is a medical professional who is qualified in a particular specialty and provides medical treatment and diagnosis to users.

[1554] "Schedule information" is information about the doctor's available dates and times and free times.

[1555] "Booking" is the process of confirming a consultation date and time selected by the user and securing a consultation session with a doctor.

[1556] The "video call means" is a video call function that connects the user and a doctor in real time to provide medical care.

[1557] A "medical session" is a series of processes in which a doctor provides medical care to a user via video call or the like.

[1558] "Medical content" refers to information such as diagnoses, instructions, and prescriptions given during a medical session.

[1559] A "follow-up notification" is a communication sent to a user after a medical examination to inform them of the next medical examination or follow-up appointment.

[1560] "Medical records" are data that store information such as a user's past medical history, diagnosis results, and prescriptions.

[1561] The present invention is a system that aims to enable users living in remote locations to receive high-quality medical services. This system provides functions that allow users to input medical information using a device, search for and make appointments with appropriate doctors, receive medical treatment via video call, save medical treatment details, and perform follow-ups. An embodiment of this system will be described in detail below.

[1562] User registration and authentication functions

[1563] A user launches an app or website using a device such as a smartphone or PC and enters personal information such as their name, email address, phone number, and password. The entered information is sent from the device to a server and stored in a database (e.g., MySQL) using SSL / TLS encrypted communication. The server then sends an authentication link to the user's email address, and the user activates their account by clicking the link.

[1564] Doctor search and appointment function

[1565] The user enters symptoms (e.g., abdominal pain) into the app on their device and sets search criteria (e.g., gastroenterology). This information is sent to the server, which filters corresponding specialists from a database (e.g., PostgreSQL). The filtered list of doctors is sent to the device in JSON format, and the user selects the doctor of their choice and makes an appointment. The reservation information is managed by the server, which uses a push notification service (e.g., Firebase) to send a reservation confirmation to the user.

[1566] Video call consultation function

[1567] When the time for the appointment arrives, the user launches the app on their device to prepare for the video call. The server then establishes a secure communication channel (e.g., WebRTC) and starts a video call session between the user and the doctor. The user and doctor communicate in real time through the video call, providing treatment, guidance, and issuing prescriptions.

[1568] Medical records and follow-up features

[1569] After the video call session ends, the server saves the medical record to a database (e.g., AWS RDS). The saved medical record is used to set reminders for the next medical appointment and follow-up. The user can check the medical history and next follow-up information through the app. The server also generates and sends follow-up notifications to the user.

[1570] Specific examples

[1571] User A uses the app to register personal information, receives a verification email, and clicks the link. He enters his symptoms of abdominal pain and searches for and makes an appointment with a gastroenterologist. When the appointment time arrives, he starts a video call and receives a diagnosis and prescription from the doctor. After the appointment, he checks the medical record, receives a follow-up notification, and prepares for the next appointment.

[1572] Prompt Sentence Examples

[1573] "What is the process for User A to launch the app to register for the first time, enter their personal information, receive a verification email, and click the link to activate their account?"

[1574] This system enables users living in remote areas to easily receive high-quality medical services, and allows for efficient health management through continuous follow-up.

[1575] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1576] Step 1:

[1577] Enter user information

[1578] Input: A user uses a device (smartphone or computer) to open an app or website and enter personal information (such as name, email address, phone number, or password).

[1579] Specific behavior: The user enters the required information into the device's input form and clicks the "Submit" button.

[1580] Output: The entered data is sent from the terminal to the server.

[1581] Step 2:

[1582] Information stored in a database

[1583] Input: Personal information sent from your device.

[1584] Specific operation: The server receives the received personal information using SSL / TLS encrypted communication, then stores the received data in a database (e.g., MySQL) and flags the data as successful.

[1585] Output: Personal information stored in a database and a verification email prepared.

[1586] Step 3:

[1587] Generate and send the authentication link

[1588] Input: User's email address and other personal information stored on the server.

[1589] What happens: The server generates a unique token for the user, generates an email containing an authentication link, and then uses a mail server (e.g., SendGrid) to send the authentication email.

[1590] Output: An email containing a verification link will be sent to the user's email address.

[1591] Step 4:

[1592] Receiving emails and clicking links

[1593] Input: The email containing the authentication link.

[1594] What happens: The user opens the email using their email client (e.g. Gmail) and clicks on the authentication link.

[1595] Output: The action of the user clicking the authentication link.

[1596] Step 5:

[1597] Activate your account

[1598] Input: Authentication link click event.

[1599] What happens: The server verifies the token in the authentication link, and if it matches, activates the user's account. If activation is successful, it generates a success message and sends it to the user.

[1600] Output: Activated user account and a success message.

[1601] Step 6:

[1602] Enter symptoms and set search conditions

[1603] Input: Symptoms and search terms entered by the user (e.g., gastroenterology).

[1604] Specific operation: The user enters symptoms and search criteria into the device app and clicks the "Search" button.

[1605] Output: The symptoms and search criteria are sent to the server.

[1606] Step 7:

[1607] Database Search and Filtering

[1608] Input: Symptoms and search criteria submitted by the user.

[1609] Specific operation: The server searches for doctor information in a database (e.g., PostgreSQL) and filters it based on the entered conditions. As a result, it generates a list of suitable doctors.

[1610] Output: A filtered list of doctors is generated.

[1611] Step 8:

[1612] Generate and submit a doctor list

[1613] Input: Filtered list of doctors.

[1614] Specific operation: The server converts the filtered doctor list into JSON format and sends it to the terminal.

[1615] Output: Doctor list in JSON format sent to the terminal.

[1616] Step 9:

[1617] Select a consultation time

[1618] Input: Doctor list displayed on terminal.

[1619] Specific operation: The user selects the doctor of their choice and the appointment date and time in the calendar format, and then the selection is sent to the server.

[1620] Output: The selected appointment date and time and doctor information are sent to the server.

[1621] Step 10:

[1622] Reservation updates and notifications

[1623] Input: User selected appointment date and time and doctor information.

[1624] What happens: The server updates the reservation schedule and generates a reservation confirmation notification, which is sent to the user using a push notification service (e.g., Firebase).

[1625] Output: Pushed booking confirmation notification.

[1626] Step 11:

[1627] Launching the app and preparing to connect

[1628] Input: Appointment time arrives.

[1629] Specific behavior: The user launches the app at the scheduled time and prepares to connect to the video call. The app requests access to the camera and microphone.

[1630] Output: Video call ready to connect.

[1631] Step 12:

[1632] Establishing a communication channel

[1633] Input: The user's ready to connect state.

[1634] What happens: The server establishes a secure communication channel (e.g., WebRTC) and connects the user and doctor to the same session.

[1635] Output: Establishment of a secure video call session.

[1636] Step 13:

[1637] Starting a video call session

[1638] Input: The established communication channel.

[1639] What happens: The video call screen appears in the user's app and a video call session with the doctor begins.

[1640] Output: Start of video call session.

[1641] Step 14:

[1642] Real-time communication

[1643] Input: A video call session.

[1644] Specific operation: The user communicates with the doctor in real time, explaining symptoms and asking questions, and the doctor makes a diagnosis based on the user's symptoms.

[1645] Output: Real-time communication results of the medical consultation.

[1646] Step 15:

[1647] Providing diagnosis and prescriptions

[1648] Input: Real-time communication results.

[1649] What it does: The doctor will explain the diagnosis and issue a prescription if necessary. The diagnosis and prescription information will be provided online.

[1650] Output: Diagnostic results and prescription information.

[1651] Step 16:

[1652] Ending and recording the medical session

[1653] Input: End of medical session.

[1654] What it does: After the video call session ends, the server logs the medical details, including the doctor's diagnosis, prescriptions, and the next recommended visit date and time.

[1655] Output: Recorded medical details.

[1656] Step 17:

[1657] Medical record database storage

[1658] Input: Recorded medical details.

[1659] What it does: The server stores medical records in a database (e.g., AWS RDS). The stored data is encrypted and only accessible to the user and their doctor.

[1660] Output: Medical records stored in a database.

[1661] Step 18:

[1662] Follow-up notification settings

[1663] Input: Medical record.

[1664] What it does: The server uses medical records to schedule reminders for upcoming appointments and follow-ups, adjusting the details of the reminders to fit the user's schedule.

[1665] Output: The set follow-up reminder.

[1666] Step 19:

[1667] Checking medical history

[1668] Input: Stored medical records.

[1669] What it does: The user accesses their medical history through the app, which shows past diagnoses, prescriptions, and next follow-up appointments.

[1670] Output: Displayed medical history.

[1671] Step 20:

[1672] Receive follow-up notifications

[1673] Input: The follow-up reminder that was set.

[1674] Specific operation: A follow-up notification is sent from the server at the specified date and time, and the user receives the notification. The user prepares for the next consultation.

[1675] Output: The follow-up notification sent to the user.

[1676] (Application example 1)

[1677] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1678] The problem that this invention aims to solve is to provide users living in remote areas with a means to efficiently access and learn appropriate medical educational content, in addition to enabling them to receive high-quality medical services. This will enable the creation of an environment in which users can not only access specialists, but also consistently use educational content to deepen their medical knowledge.

[1679] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1680] In this invention, the server

[1681] means for receiving medical information input by a user using a terminal;

[1682] A means for searching for an appropriate specialist based on the medical information;

[1683] means for acquiring schedule information of the specialist and confirming an appointment;

[1684] a video call means for connecting the user and the specialist at the appointment time;

[1685] means for saving the medical treatment details after the medical treatment session ends and notifying the user of the saving;

[1686] a means for a user to input search keywords to filter and display medical education content;

[1687] A means for conducting live lectures using video calls;

[1688] A way to keep a record of what you've watched and set reminders for upcoming lectures and follow-ups,

[1689] This allows users to receive medical treatment from specialists and efficiently access and learn high-quality medical education content.

[1690] "Medical information" refers to information such as health status, symptoms, and medical history entered by the user through the terminal.

[1691] A "specialist" is a medical professional with advanced knowledge and skills in a particular medical field.

[1692] "Schedule information" is information that indicates the dates and times when specialists are available for consultation and the reservation status.

[1693] A "video call" is a means for a user in a remote location and a specialist to communicate in real time using audio and video via the Internet.

[1694] "Medical content" refers to information about diagnosis and treatment plans recorded during video calls or face-to-face medical sessions.

[1695] "Notifications" are a means of informing users about medical treatments and follow-up appointments.

[1696] A "search keyword" is an input character string that a user uses when searching for medical education content.

[1697] "Filtering" is the process of sorting and displaying relevant medical education content based on search keywords.

[1698] "Medical education content" refers to information such as videos, articles, and lecture materials for learning medical knowledge and skills.

[1699] "Live Lecture" means an educational session that takes place in real time and is delivered via video call.

[1700] "Reminder" is a function that notifies users so that they do not forget their next lecture or medical appointment.

[1701] Overall system configuration

[1702] The system of the present invention is designed to enable users to receive high-quality medical services even when they are not in the field. This system mainly uses a smartphone app and is built by combining front-end and back-end technologies.

[1703] Hardware and Software Configuration

[1704] Hardware

[1705] User devices: Mainly smartphones are used.

[1706] Servers: Includes database and application servers.

[1707] software

[1708] Front-end: React Native (smartphone app development)

[1709] Backend: Node.js (application server), MongoDB (database)

[1710] Communication: WebRTC (secure communication for video calls)

[1711] Authentication: Firebase Authentication (user authentication)

[1712] Feature details

[1713] 1. User registration and authentication function

[1714] The user launches the smartphone app and enters the necessary personal information.

[1715] The server receives the entered information and stores it in a database (MongoDB). Firebase Authentication is used to send an authentication email to the user.

[1716] The user receives a verification email and clicks on the link to activate the account. The server verifies the link and activates the user's account.

[1717] 2. Doctor search and appointment function

[1718] The user terminal transmits search keywords based on the symptoms to the server.

[1719] The server queries the database (MongoDB) and generates a list of relevant specialists, which is then sent to the user's device.

[1720] The user selects the desired specialist from a list of doctors and makes an appointment.

[1721] The server stores the user's reservation information and sends a reservation confirmation notification.

[1722] 3. Video call medical consultation function

[1723] The user launches the smartphone app at the scheduled time and prepares to connect to the video call.

[1724] The server establishes a secure communication channel using WebRTC and initiates the video call session.

[1725] Users and specialists can conduct consultations in real time via video calls.

[1726] 4. Medical records and follow-up features

[1727] After the consultation session, the server saves the details of the consultation in a database and sets reminders for the next consultation or follow-up.

[1728] The user terminal provides an interface that allows the user to check their medical history and make their next appointment.

[1729] 5. Medical education content search and browsing function

[1730] The user terminal transmits search keywords to the server to filter medical education content.

[1731] The server retrieves the relevant content from a database (MongoDB) and returns it to the user's terminal.

[1732] Users can view the filtered content through a smartphone app.

[1733] 6. Live lecture function via video call

[1734] The user launches the smartphone app based on the schedule of the live lecture and prepares the video call connection.

[1735] The server uses WebRTC to establish a secure communication channel and conduct the video call session for the live lecture.

[1736] Users can listen to lectures in real time and ask questions and hold discussions.

[1737] 7. Content saving and follow-up features

[1738] The server stores a record of the content the user has viewed in a database and sets reminders for upcoming lectures and follow-ups.

[1739] The user terminal provides an interface that allows the user to check their viewing history and receive reminders for upcoming lectures.

[1740] Specific examples

[1741] For example, if a user wants to learn about the topic of "diabetes," they can enter search keywords into the app and related videos and articles will be displayed. Additionally, WebRTC can be used to attend live seminars by specialists, and the content of these seminars will be stored in a database.

[1742] Prompt Sentence Examples

[1743] Search for diabetes education content to get a list of related videos and articles, collect real-time user interest in these contents, and recommend the most viewed content.

[1744] This allows users to not only receive medical treatment from specialists, but also to effectively access and learn from high-quality medical education content.

[1745] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1746] Step 1:

[1747] User registration and authentication functions

[1748] Input: The user enters personal information into the device and presses the submit button.

[1749] Server: Receives personal information sent by the user and saves it in a database (MongoDB). After saving, sends an authentication email to the user via Firebase Authentication.

[1750] Output: A verification email will be sent to the user's email address.

[1751] User: Receives verification email and clicks verification link.

[1752] Server: Confirms the click on the authentication link and activates the user's account.

[1753] What happens: The server verifies the sender's email address and records the date and time the verification link is clicked.

[1754] Step 2:

[1755] Doctor search and appointment function

[1756] Input: The user enters the symptoms into the terminal and presses the search button.

[1757] Server: Receives the input symptom information and queries the database (MongoDB) to generate a list of relevant specialists.

[1758] Output: The filtered list of specialists is sent to the user terminal.

[1759] User: Selects the desired specialist from a list of specialists and schedules an appointment.

[1760] Server: Receives the reservation information, stores it in a database, and sends a reservation confirmation to the user.

[1761] Specific operation: The server updates the appointment schedule based on the date and time, specialist ID, and user ID, and calls the notification function.

[1762] Step 3:

[1763] Video call consultation function

[1764] Input: At the scheduled time, the user launches the app on their device and prepares for the video call.

[1765] Server: Receives the video call request and establishes a secure communication channel using WebRTC.

[1766] Output: A video call session is started and the video call screen is displayed on the user's device.

[1767] User: Consults with specialists via video call.

[1768] What it does: The server manages WebRTC peer connections and ensures communication encryption.

[1769] Step 4:

[1770] Medical records and follow-up features

[1771] Input: After the medical session ends, the server receives the medical details.

[1772] Server: Receives medical records, stores them in a database, and sets follow-up reminders.

[1773] Output: The medical details are saved in the database and a notification is sent to the user's device.

[1774] Users: Receive notifications and can check their medical history.

[1775] What it does: The server tags the stored visits and schedules reminders for upcoming appointments.

[1776] Step 5:

[1777] Medical education content search and browsing function

[1778] Input: The user enters search keywords into the device and presses the search button.

[1779] Server: Receives search keywords, queries the database, and filters relevant medical education content.

[1780] Output: The filtered medical education content is sent to the user terminal.

[1781] User: Views filtered content on a smartphone app.

[1782] What it does: The server ranks the search results and prioritizes the most relevant content to present to the user.

[1783] Step 6:

[1784] Live lecture function via video call

[1785] Input: User launches the app and prepares based on the schedule of a live lecture.

[1786] Server: Receives the video call request and establishes a secure communication channel using WebRTC.

[1787] Output: A video call session of a live lecture is started and the video is displayed on the user's device.

[1788] Users: Attend live lectures, ask questions and participate in discussions.

[1789] Specific operation: The server monitors the progress of live lectures in real time and notifies you if any problems occur.

[1790] Step 7:

[1791] Content saving and follow-up features

[1792] Input: The server receives information about the content that was viewed.

[1793] Server: Stores records of viewed content in a database and sets reminders for upcoming lectures and follow-ups.

[1794] Output: The viewing history is stored in a database and a reminder for the next lecture is sent to the user's device.

[1795] User: Check viewing history and prepare for the next lecture.

[1796] What it does: The server creates statistics on viewing history and analyzes the topics that users are most interested in.

[1797] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1798] The present invention is a system that enables users living in remote locations to receive high-quality medical services. The system includes a process where users input medical information using a terminal, search for an appropriate specialist, and make an appointment. It also includes functions for receiving medical treatment via video call, saving medical results and providing follow-up, and using an emotion engine to reflect the user's emotional state in the medical treatment.

[1799] System configuration

[1800] The system has the following main functions:

[1801] 1. User registration and authentication function

[1802] 2. Doctor search and appointment function

[1803] 3. Video call medical consultation function

[1804] 4. Medical records and follow-up features

[1805] 5. Emotion recognition function using emotion engine

[1806] Program processing flow

[1807] 1. User registration and authentication function

[1808] On the device: The user launches the app or website and enters the required personal information.

[1809] Server: Receives the entered information, stores it in a database, and sends a verification link to the user's email address.

[1810] User: Receives a verification email and clicks on the link to activate the account.

[1811] Server: Verify the link and activate the user's account.

[1812] Examples:

[1813] User A accesses the account remotely, provides the necessary information, and creates an account. After receiving the authentication email, the user clicks the link to activate the account.

[1814] 2. Doctor search and appointment function

[1815] Device: Enter symptoms into the app on your device and set search criteria.

[1816] Server: Based on the input information, the server filters the doctor information in the database and generates a list of doctors that meet the criteria. This list is then sent to the terminal.

[1817] Terminal: Shows the user a list of doctors and lets them select the appointment time they prefer.

[1818] Server: Confirms the selected reservation information, updates the schedule, and sends a reservation confirmation notification to the user.

[1819] Examples:

[1820] User B complains of stomach pain and searches for a gastroenterologist. He selects an appropriate doctor from the search results and makes an appointment for 2:00 PM the next day.

[1821] 3. Video call medical consultation function

[1822] Device: The user launches the app at the scheduled time and prepares to connect.

[1823] Server: Establishes a secure communication channel and initiates the video call session.

[1824] Terminal: Displays the video call screen, allowing the user and doctor to communicate in real time.

[1825] User: Engages in medical conversations with doctors and receives guidance and prescriptions.

[1826] Examples:

[1827] User C launches the app at the scheduled time and receives a consultation via video call with Doctor D. Doctor D makes an appropriate diagnosis and issues a prescription.

[1828] 4. Medical records and follow-up features

[1829] Server: After the consultation session is over, it stores the details in a database and sets reminders for upcoming consultations and follow-ups.

[1830] Terminal: Allows users to check their medical history and schedule their next appointment if necessary.

[1831] Users: Receive follow-up notifications and prepare for upcoming appointments.

[1832] Examples:

[1833] User E opens the app after the consultation and checks the medical history. The system sends a reminder for a follow-up appointment in one week, and the user makes the next appointment.

[1834] 5. Emotion recognition function using emotion engine

[1835] Device: Captures the user's facial expressions and voice in real time during a video call.

[1836] Server: The emotion engine analyzes the captured data and recognizes the user's emotional state.

[1837] Server: Provides the doctor with the recognized emotional information and reflects it in the medical treatment.

[1838] Server: Stores medical records including emotional information in a database.

[1839] Examples:

[1840] When User F has a video call with a doctor, the emotion engine recognizes that the user is anxious from their facial expression. This information is provided to the doctor, who takes the user's mental state into consideration during the consultation. The emotional information is recorded in the medical treatment details and prescription.

[1841] This system allows users living in remote areas to easily receive high-quality medical services. In addition, by providing medical treatment based on the user's emotional state, more personalized medical services can be provided.

[1842] The processing flow will be explained below.

[1843] Emotion recognition function using emotion engine

[1844] Step 1:

[1845] Device: Show a popup to get the user's camera and microphone permissions before starting a video call.

[1846] Step 2:

[1847] User: Allow use of camera and microphone.

[1848] Step 3:

[1849] Device: After obtaining permission, it displays the video call screen and captures the user's facial expressions and voice in real time.

[1850] Step 4:

[1851] Terminal: Sends captured facial expression and voice data to the server.

[1852] Step 5:

[1853] Server: Analyzes the received facial expression and voice data and uses an emotion engine to recognize the user's emotional state.

[1854] Step 6:

[1855] Server: Processes the recognized emotion information and displays it on the doctor's screen in real time.

[1856] Step 7:

[1857] Terminal: The doctor checks the user's emotional state and responds accordingly.

[1858] Step 8:

[1859] Server: Stores the details of the medical session, including emotional information, in a database.

[1860] Step 9:

[1861] Server: Uses the stored emotional information to inform the next medical appointment or follow-up.

[1862] Specific examples

[1863] Video call consultations

[1864] Step 1:

[1865] Device: User F clicks "Start video call" on the screen for starting a video call.

[1866] Step 2:

[1867] On your device: A pop-up will appear asking you to allow access to your camera and microphone.

[1868] Step 3:

[1869] User: Allow use of camera and microphone.

[1870] Step 4:

[1871] Terminal: The video call screen is displayed, and User F's facial expressions and voice are captured.

[1872] Step 5:

[1873] Device: Captured data is sent to the server.

[1874] Step 6:

[1875] Server: Analyzes the received data, and the emotion engine recognizes User F's emotions as both calm and anxious.

[1876] Step 7:

[1877] Server: The recognized emotion information is displayed on Doctor D's screen.

[1878] Step 8:

[1879] Terminal: Doctor D checks User F's emotional state and provides medical treatment while taking it into consideration.

[1880] Step 9:

[1881] Server: Stores the results of the medical session and emotional information in a database.

[1882] Step 10:

[1883] Server: Based on the stored emotional information, sends notifications and reminders to User F that will be helpful for their next medical appointment.

[1884] This system enables medical treatment that takes into account the user's emotional state, enabling more personalized medical services to be provided. Furthermore, by storing emotional information and utilizing it for the next medical treatment, continuous and consistent medical support can be achieved.

[1885] Example 2

[1886] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1887] In order for users living in remote areas to receive high-quality medical services, they need a system that smoothly handles the entire process, from searching for specialists and making appointments to video call consultations, saving medical records, and providing follow-ups. However, existing systems lack the functionality to reflect the user's emotional state in medical treatment, making them insufficient for providing more personalized medical services. Therefore, there is a need for a system that can analyze the user's emotional state in real time and reflect that in medical treatment.

[1888] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1889] In this invention, the server includes means for receiving medical information input by a user using a terminal, means for searching for an appropriate medical professional based on the medical information, means for acquiring the professional's schedule information and confirming an appointment, means for establishing a secure video call session between the user and the medical professional, means for saving the medical treatment details after the medical treatment session ends and notifying the user, means for capturing the user's facial expressions and voice data during the video call and performing emotion analysis, and means for providing the medical professional with the results of the emotion analysis and reflecting them in the medical treatment details, thereby enabling users in remote locations to receive high-quality and personalized medical services.

[1890] "User" refers to an individual who uses the system to input medical information and receive medical treatment.

[1891] "Terminal" refers to an information processing device such as a smartphone, PC, or tablet used by a user.

[1892] "Medical information" refers to all information necessary for medical treatment, such as medical condition, symptoms, and past medical history, entered by the user.

[1893] "Medical professional" refers to a doctor or nurse who has specialized knowledge and skills in a particular medical field.

[1894] "Schedule information" refers to information such as the availability of medical professionals and appointment availability.

[1895] "Video calling" refers to a means of real-time video and audio communication between a user and a medical professional via the Internet.

[1896] "Medical content" refers to the doctor's diagnosis, treatment plan, prescription information, etc. recorded during a medical session.

[1897] "Emotion analysis" refers to the process of analyzing a user's facial expressions and voice data during a video call to identify the user's emotional state.

[1898] "Emotional state" refers to information that indicates the user's psychological state or mood.

[1899] "Follow-up Notification" means a notification sent to a user for a follow-up or reminder of an upcoming medical appointment.

[1900] The present invention is a system that enables users living in remote locations to receive high-quality medical services. The system involves a process in which users input medical information using a terminal, search for an appropriate medical professional, and receive medical treatment via video call. In addition to functions for saving and following up on medical treatment details, the system also has an emotion analysis function that reflects the user's emotional state in medical treatment.

[1901] System configuration

[1902] 1. User registration and authentication function

[1903] On the device: The user launches an app or website and enters personal information such as their name, email address, and password.

[1904] Server: Receives the entered information, stores it in a database (e.g., MySQL), and sends an email containing a verification link to the user's email address.

[1905] User: Receives a verification email and clicks on the link to activate the account.

[1906] Server: Verify the link and activate the user's account.

[1907] 2. Doctor search and appointment function

[1908] Device: Enter symptoms into the device app and set search criteria.

[1909] Server: Based on the input information, it searches a doctor database (e.g., Elasticsearch) and generates a list of doctors that match the criteria. It then sends this list to the terminal.

[1910] Terminal: Shows the user a list of doctors and lets them select the appointment time they prefer.

[1911] Server: Confirms the selected reservation information, saves the schedule information to the database, and sends a reservation confirmation notification to the user.

[1912] 3. Video call medical consultation function

[1913] On the device: At the scheduled time, the user launches the app and prepares for the video call.

[1914] Server: Uses technologies such as WebRTC to establish a secure video call session and connect users with medical professionals.

[1915] Terminal: Real-time communication between the user and medical professionals.

[1916] 4. Medical records and follow-up features

[1917] Server: After the consultation session is over, it stores the consultation details in a database and sets reminders for the next consultation or follow-up.

[1918] Terminal: Allows users to check their medical history and schedule their next appointment if necessary.

[1919] 5. Emotion recognition function using emotion engine

[1920] Device: Captures facial and audio data in real time during a video call.

[1921] Server: Analyzes the captured data using an emotion analysis engine (e.g., Microsoft Azure's Face API) to identify the user's emotional state.

[1922] Server: Providing recognized emotion information to medical professionals to improve the quality of medical care.

[1923] Server: Stores medical records including emotional information in a database.

[1924] Specific examples

[1925] User A accesses the system and creates an account. He receives a verification email and clicks the link to activate his account. He then complains of abdominal pain, searches for a gastroenterologist, and makes an appointment for 2:00 p.m. the next day. He launches the app at the scheduled time and consults with the doctor via video call. The consultation details are saved in the database, and a reminder to return to the doctor one week later is sent.

[1926] Examples of prompt statements

[1927] Enter the following prompt for the generative AI model:

[1928] Please explain the steps a user takes to access the platform remotely and create an account. Include the process by which a user enters symptoms, searches for a doctor, and makes an appointment using the medical platform. Also, please explain the process for receiving medical advice in real time via video call, as well as the steps for saving medical records and scheduling the next follow-up appointment. Also, please include a function that uses an emotion engine to recognize the user's emotional state and reflect it in medical treatment. Please clearly indicate specific tools and technologies (e.g., WebRTC, MySQL, Microsoft Azure Face API, etc.).

[1929] This system enables users living in remote areas to receive high-quality, personalized medical services.

[1930] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1931] System program processing flow

[1932] 1. User registration and authentication function

[1933] Step 1: Enter your user information

[1934] User: Accesses an application or website from a smartphone or computer device and enters personal information such as name, email address, and password.

[1935] Input: The user enters information using a keyboard or touch screen.

[1936] Output: The input information is ready to be used.

[1937] Specific behavior: The user fills in the required information in the form and clicks the "Register" button.

[1938] Step 2: Send and store user information

[1939] Terminal: Encrypts the entered data and sends it to the server via a secure communication protocol (e.g., HTTPS).

[1940] Server: Stores the received data in a database (e.g., MySQL).

[1941] Input: Encrypted user information.

[1942] Output: A confirmation that the data was saved correctly to the database.

[1943] Specific operation: When the device clicks the send button, the data is encrypted and sent to the server. After receiving it, the server stores the data in a database.

[1944] Step 3: Sending a verification email

[1945] Server: Uses a mail server to send an email containing a verification link to the email address entered by the user.

[1946] Input: The user's email address.

[1947] Output: A confirmation that the verification email was sent successfully.

[1948] Specific operation: The server automatically generates an authentication email and sends it to the user via the mail server.

[1949] Step 4: Click on the authentication link

[1950] Users: Open the verification email they received and click the link to activate their account.

[1951] Input: The link in the verification email.

[1952] Output: A confirmation that the account has been activated.

[1953] What happens: The user checks their email, clicks the link, and is redirected to a page to activate their account.

[1954] Step 5: Activate your account

[1955] Server: Checks the information in the authentication link and updates the user's status in the database to "active".

[1956] Input: Authentication link information.

[1957] Output: Confirmation that the user account has been activated.

[1958] What happens: The server validates the link and updates the user status in the database.

[1959] 2. Doctor search and appointment function

[1960] Step 1: Enter symptoms and set search criteria

[1961] User: Enters search criteria into the application, such as symptoms and desired appointment date and time.

[1962] Input: Symptom information and desired date and time of consultation entered by the user.

[1963] Output: The search criteria has been set.

[1964] Specific behavior: The user enters their symptoms and desired appointment date into the app's search form and clicks the "Search" button.

[1965] Step 2: Filtering physician information

[1966] Server: Based on the input information, it searches a doctor database (e.g., Elasticsearch) and generates a list of doctors that match the criteria.

[1967] Input: The search criteria entered by the user.

[1968] Output: A list of doctors who match the criteria.

[1969] Specific operation: The server executes a database query to retrieve physician data that matches the criteria.

[1970] Step 3: View the Doctor List

[1971] Terminal: Displays the list of doctors received from the server to the user.

[1972] Input: Doctor list sent from the server.

[1973] Output: A list of doctors displayed on the user interface.

[1974] Specific behavior: The device displays the list in the user interface so that the user can check it.

[1975] Step 4: Choose your appointment time

[1976] User: Select the desired appointment time and confirm the appointment.

[1977] Input: Selected appointment times and doctor information.

[1978] Output: Reservation information is confirmed.

[1979] Specific actions: The user selects a doctor and appointment time and clicks the "Make appointment" button.

[1980] Step 5: Confirm and notify reservation information

[1981] Server: Saves the selected reservation information to a database and sends a reservation confirmation notification to the user.

[1982] Input: Booking information selected by the user.

[1983] Output: The reservation information is saved to the database and a notification is sent to the user.

[1984] Specific behavior: The server records the reservation information in a database and sends a reservation confirmation email.

[1985] 3. Video call medical consultation function

[1986] Step 1: Prepare for the video call

[1987] User: Launches the application at the scheduled time and prepares for the video call.

[1988] Input: User's app launch action.

[1989] Output: Ready to start a video call.

[1990] Specific behavior: The user launches the app and clicks the start video call button.

[1991] Step 2: Establishing communication channels

[1992] Server: Establish a secure video call channel using WebRTC.

[1993] Input: User's request to start a video call.

[1994] Output: A secure video call session is established.

[1995] What happens: The server initializes a WebRTC session and connects the user with a medical professional.

[1996] Step 3: Start a video call

[1997] Device: The user and medical professionals communicate in real time via a video call screen.

[1998] Enter: a secure video call session.

[1999] Output: Real-time medical consultation with video and audio.

[2000] What happens: The device displays a video call window and the user and medical professional begin talking in real time.

[2001] Step 4: Proceeding with the treatment

[2002] Users: Engage in medical conversations with healthcare professionals and receive guidance and prescriptions.

[2003] Input: Medical consultation details.

[2004] Output: The consultation is completed and any necessary instructions or prescriptions are provided.

[2005] What it does: A user answers a question or gets advice from a medical professional.

[2006] 4. Medical records and follow-up features

[2007] Step 1: Save your medical records

[2008] Server: After the medical session is over, the medical details entered by the medical professional are stored in a database.

[2009] Input: Medical record data.

[2010] Output: Medical information saved in the database.

[2011] Specific operation: The server stores the medical information in a database.

[2012] Step 2: Set up follow-up

[2013] Server: Sets reminders for upcoming appointments and follow-ups in the system.

[2014] Input: Follow-up appointment information.

[2015] Output: The next appointment reminder has been set.

[2016] What happens: The server saves the reminder information in a database.

[2017] Step 3: Review medical history

[2018] User: Checks medical history in the application and schedules next appointments if necessary.

[2019] Input: Medical History Request Operation.

[2020] Output: Medical history displayed.

[2021] What happens: The user views their medical history in the app and clicks the "Schedule a follow-up appointment" button.

[2022] Step 4: Receive follow-up notifications

[2023] Users: Receive follow-up notifications and prepare for upcoming appointments.

[2024] Enter: Follow-up notification.

[2025] Output: The notification has been confirmed and the next appointment is ready.

[2026] What happens: The user receives a notification and sees what's next on their schedule.

[2027] 5. Emotion recognition function using emotion engine

[2028] Step 1: Capture facial expressions and voice

[2029] Device: Captures the user's facial expressions and voice in real time during a video call.

[2030] Input: User's facial and voice data.

[2031] Output: Captured facial expression and speech data.

[2032] What happens: Your device captures data using the camera and microphone.

[2033] Step 2: Analyze emotional state

[2034] Server: Analyzes the captured data using emotion recognition software (e.g., Microsoft Azure's Face API) to identify the user's emotional state.

[2035] Input: Captured facial and speech data.

[2036] Output: Data about the user's emotional state.

[2037] Specific operation: The server calls the API and obtains emotion data.

[2038] Step 3: Providing emotional information

[2039] Server: Provides recognized emotional information to medical professionals and reflects it in medical treatment.

[2040] Input: Emotional state data.

[2041] Output: Emotional information provided to a medical professional.

[2042] Specific operation: The server displays the emotional information on the medical professional's screen.

[2043] Step 4: Preserving medical records

[2044] Server: Stores medical records including emotional information in a database.

[2045] Input: Emotional information and medical details data.

[2046] Output: Medical records containing emotion data stored in a database.

[2047] Specific operation: The server integrates the emotion data and medical details and stores them in a database.

[2048] summary

[2049] These specific processing steps and operations enable users living in remote locations to receive high-quality, personalized medical services. Smooth coordination between each step ensures that the entire medical treatment process is carried out efficiently.

[2050] (Application example 2)

[2051] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2052] Conventional telemedicine systems allow users to receive high-quality medical services from remote locations, but they have the drawback of not reflecting the emotional state of the individual in the medical treatment, making it difficult to provide treatment that fully takes into account the psychological state of the user. There is also the risk of personal medical information being leaked during video calls. It is necessary to solve these issues and provide safe and personalized medical services.

[2053] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2054] In this invention, the server includes means for receiving medical information input by a user using a terminal, means for searching for an appropriate specialist based on the medical information, means for acquiring the specialist's schedule information and confirming the appointment, means for video calling to connect the user with the specialist at the appointment time, means for capturing the user's facial expressions and voice during the video call and analyzing the emotional state, and means for saving the medical treatment details after the medical treatment session ends and notifying the user. This enables personalized medical treatment that takes into account the user's emotional state and also provides medical services in a secure communication environment.

[2055] A "terminal" is an electronic device that allows a user to input medical information, communicate with specialists, and receive medical treatment.

[2056] "Medical information" refers to information entered by the user, such as their medical condition, symptoms, medical history, etc.

[2057] A "specialist" is a doctor with advanced knowledge and experience in a particular medical field.

[2058] "Schedule information" is information that indicates the dates and times when specialists are available for consultation, the reservation status, and so on.

[2059] "Video communication means" refers to audio and video communication means for real-time communication between the user and the specialist.

[2060] "Emotion recognition means" is a technology that analyzes a user's facial expressions and voice during a video call to recognize their emotional state.

[2061] A "medical session" refers to a period of time during which a medical consultation is conducted between a user and a specialist.

[2062] "Medical content" refers to a record of the diagnosis, treatment, instruction, etc. provided during a medical session.

[2063] "Notification" is a means of communicating information to inform users of medical results and the need for follow-up.

[2064] An embodiment of the present invention provides a system that enables a user in a remote location to safely receive high-quality medical services. A specific method for implementing this system will be described below.

[2065] 1. Basic configuration

[2066] This system consists of a user device, a specialist device, and a server that links these devices. The user device is a smartphone, tablet, or PC, and the specialist device uses similar devices. The server is located on the cloud and manages the entire system.

[2067] 2. Enter medical information and search for specialists

[2068] Users input medical information using their own devices. This medical information includes specific medical conditions, symptoms, medical history, etc., and is received by a server. The server has a means for searching for an appropriate specialist based on the received medical information. A pre-built doctor database is used to search for a specialist. Search criteria also include the specialist's qualifications, field of expertise, and user ratings.

[2069] 3. Confirmation of reservation

[2070] After a specialist is found, the server retrieves the specialist's schedule information and displays available appointment dates and times on the user's device. When the user selects the desired appointment time, the server confirms the appointment based on that information. When the appointment is confirmed, a notification is sent to both the user and the specialist.

[2071] 4. Video calling methods

[2072] When the appointment time arrives, the server establishes a video call session between the user and the specialist's device. This video call is highly secure, using end-to-end encryption technology, such as OpenSSL and WebRTC.

[2073] 5. Emotion recognition means

[2074] During a video call, the user's device uses a camera and microphone to capture the user's facial expressions and voice, and sends the data to the server. The server then uses an emotion engine to analyze the user's facial expressions and voice data and recognize their emotional state. This emotional information is provided to specialists in real time and reflected in their medical treatment. The emotional information is also saved as part of the medical record.

[2075] 6. Storage and notification of medical information

[2076] After the consultation session ends, the server stores the consultation details in a database. This data includes the specialist's diagnosis, advice, prescriptions, and the user's emotional state. The system also has the ability to automatically schedule the next consultation and set follow-up notifications, allowing users to receive reminders and be better prepared for their next appointment.

[2077] Specific examples

[2078] User A feels unwell and enters medical information about abdominal pain, which is received by the server. The server searches for gastroenterologists and displays the schedules of several candidates. User A selects 3:00 p.m. the following day and the appointment is confirmed. When the appointment time arrives, a video call begins between User A and Specialist B. During the call, the emotion engine recognizes the anxious expression on User A's face and displays this information to Specialist B. After the consultation, the details of the consultation and emotional information are saved, and User A is notified of the next follow-up appointment.

[2079] Prompt Sentence Examples

[2080] User A logs in using the application, enters their symptoms, and searches for a doctor. The AI ​​model recommends the most suitable doctor and schedules a video call. During the call, a facial expression recognition engine analyzes the user's emotional state, and appropriate responses are made within the video call. The session data is encrypted and stored, and a notification for the next follow-up is set.

[2081] This allows users to safely receive medical care from specialists even from remote locations, and provides personalized medical services that take emotional state into account, resulting in a higher quality medical experience.

[2082] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2083] Step 1:

[2084] The user inputs medical information using the terminal.

[2085] Input: User information about medical condition, symptoms, and medical history.

[2086] Data processing and calculation: Preprocessing to standardize the information entered by the user and store it in a database in a format that conforms to the format.

[2087] Output: Standardized medical information data.

[2088] What happens: A user launches an application, fills out a form, and hits the submit button. This information is sent to the server for preprocessing.

[2089] Step 2:

[2090] A server receives the medical information and executes a procedure to search for an appropriate specialist.

[2091] Input: User's medical information data.

[2092] Data processing and calculation: Using AI algorithms to filter and rank the most suitable specialists based on medical information.

[2093] Output: A list of suitable specialists.

[2094] Specific operation: The server analyzes the received medical information and lists the most suitable doctors from a specialist database.

[2095] Step 3:

[2096] The server retrieves the specialist's schedule information and presents it to the user.

[2097] Input: A list of appropriate specialists.

[2098] Data processing and calculation: Retrieves specialist schedule information from the database and displays it as a choice for the user.

[2099] Output: The available schedule presented to the user.

[2100] Specific operation: The server retrieves the specialist's schedule information from the database, sends the available appointment time slots to the user's terminal, and displays them as options for the user.

[2101] Step 4:

[2102] The user selects the desired reservation time, and the server confirms the reservation.

[2103] Input: The appointment time selected by the user.

[2104] Data processing and calculation: The selected reservation information is saved in the database and notified to the specialist and the user.

[2105] Output: Booking confirmation notification.

[2106] Specific operation: The user selects the desired time and presses the "Confirm appointment" button. The server updates the appointment information and sends a notification to the specialist and the user.

[2107] Step 5:

[2108] When the appointment time arrives, the server establishes a video call means connecting the user with the specialist.

[2109] Input: Appointment time, user, and specialist connection information.

[2110] Data processing and calculation: Encrypting the video call session and establishing the connection.

[2111] Output: A secure video call session.

[2112] Specific Operation: The server initiates a video call session and establishes a secure communication channel between the user and the specialist.

[2113] Step 6:

[2114] During a video call, the terminal captures the user's facial expressions and voice and performs emotion recognition.

[2115] Input: User's facial expression data and voice data.

[2116] Data processing and calculation: Analyze the user's emotional state using an emotion recognition engine.

[2117] Output: User's emotional information.

[2118] Specific operation: The camera and microphone on the user's device capture facial expressions and voice, and send this data to the server, which then analyzes the data using the emotion engine.

[2119] Step 7:

[2120] After the consultation session is over, the server stores the consultation and sends follow-up notifications to the user.

[2121] Input: Clinical content data and emotional information.

[2122] Data processing and calculation: storing medical details and emotional information in the database and setting notifications for the next follow-up.

[2123] Output: Stored medical records and follow-up notifications.

[2124] Specific operation: The server saves the medical results in the database, sets the next follow-up notification, and sends a notification to the user.

[2125] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2126] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2127] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2128] [Fourth embodiment]

[2129] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2130] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2131] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2132] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2133] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2134] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2135] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2136] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2137] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2138] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2139] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2140] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2141] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2142] The present invention is a system that enables users living in remote locations to receive high-quality medical services. The system supports the process of users entering medical information using a terminal, searching for an appropriate specialist, and making an appointment. The system also has the function of receiving medical treatment via video call, storing medical treatment results, and performing follow-up.

[2143] System configuration

[2144] The system has the following main functions:

[2145] 1. User registration and authentication function

[2146] 2. Doctor search and appointment function

[2147] 3. Video call medical consultation function

[2148] 4. Medical records and follow-up features

[2149] Program processing flow

[2150] 1. User registration and authentication function

[2151] On the device: The user launches the app or website and enters the required personal information.

[2152] Server: Receives the entered information, stores it in a database, and sends a verification link to the user's email address.

[2153] User: Receives a verification email and clicks on the link to activate the account.

[2154] Server: Verify the link and activate the user's account.

[2155] Examples:

[2156] User A accesses the account remotely, provides the necessary information, and creates an account. After receiving the authentication email, the user clicks the link to activate the account.

[2157] 2. Doctor search and appointment function

[2158] Device: Enter symptoms into the app on your device and set search criteria.

[2159] Server: Based on the input information, the server filters the doctor information in the database, generates a list of appropriate doctors, and sends it to the terminal.

[2160] Terminal: Shows the user a list of doctors and lets them select the appointment time they prefer.

[2161] Server: Confirms the selected reservation information, updates the schedule, and sends a reservation confirmation notification to the user.

[2162] Examples:

[2163] User B complains of stomach pain and searches for a gastroenterologist. He selects an appropriate doctor from the search results and makes an appointment for 2:00 PM the next day.

[2164] 3. Video call medical consultation function

[2165] Device: The user launches the app at the scheduled time and prepares to connect.

[2166] Server: Establishes a secure communication channel and initiates the video call session.

[2167] Terminal: Displays the video call screen, allowing the user and doctor to communicate in real time.

[2168] User: Engages in medical conversations with doctors and receives guidance and prescriptions.

[2169] Examples:

[2170] User C launches the app at the scheduled time and receives a consultation via video call with Doctor D. Doctor D makes an appropriate diagnosis and issues a prescription.

[2171] 4. Medical records and follow-up features

[2172] Server: After the consultation session is over, it stores the details in a database and sets reminders for upcoming consultations and follow-ups.

[2173] Terminal: Allows users to check their medical history and schedule their next appointment if necessary.

[2174] Users: Receive follow-up notifications and prepare for upcoming appointments.

[2175] Examples:

[2176] User E opens the app after the consultation and checks the medical history. The system sends a reminder for a follow-up appointment in one week, and the user makes the next appointment.

[2177] This system allows users living in remote areas to easily receive high-quality medical services, and through continuous follow-up, users can efficiently manage their health.

[2178] The processing flow will be explained below.

[2179] User registration and authentication functions

[2180] Step 1:

[2181] On the device: The user launches the app or website and clicks the "Sign up" button.

[2182] Step 2:

[2183] Terminal: Displays a form where the user enters required information such as name, address, contact information, and health insurance information.

[2184] Step 3:

[2185] User: Enter the required information and click the "Submit" button.

[2186] Step 4:

[2187] Terminal: Checks the entered information for any error messages, and if there are no problems, sends the information to the server.

[2188] Step 5:

[2189] Server: Stores the received user information in a database.

[2190] Step 6:

[2191] Server: Sends a confirmation email containing a verification link to the user's email address.

[2192] Step 7:

[2193] User: Receives verification email and clicks link.

[2194] Step 8:

[2195] Server: Checks the validity of the link and activates the user account.

[2196] Step 9:

[2197] Server: Sends account activation notification to the user.

[2198] ---

[2199] Doctor search and appointment function

[2200] Step 1:

[2201] Terminal: The user logs in and displays the symptom entry screen.

[2202] Step 2:

[2203] User: Enter symptoms and desired specialty and click the "Search" button.

[2204] Step 3:

[2205] Terminal: Sends the entered information to the server.

[2206] Step 4:

[2207] Server: Searches for doctor information in the database and generates a list of doctors that match the criteria.

[2208] Step 5:

[2209] Server: Sends doctor list and schedule information to the device.

[2210] Step 6:

[2211] Terminal: Displays a list of doctors to the user as search results.

[2212] Step 7:

[2213] User: Select the desired doctor and appointment date and time, and click the "Make an appointment" button.

[2214] Step 8:

[2215] Device: Send the selected information back to the server.

[2216] Step 9:

[2217] Server: Stores appointment information in a database and updates doctor schedules.

[2218] Step 10:

[2219] Server: Sends appointment confirmation notifications to users and doctors.

[2220] ---

[2221] Video call consultation function

[2222] Step 1:

[2223] On your device: Launch the app at the scheduled time and click the "Start video call" button.

[2224] Step 2:

[2225] Terminal: Displays the video call screen and sends a connection request to the server.

[2226] Step 3:

[2227] Server: Connects the user and doctor to a secure video call session.

[2228] Step 4:

[2229] Device: Initiate a video call that relays audio and video.

[2230] Step 5:

[2231] User: Receive medical treatment via video call with a doctor.

[2232] Step 6:

[2233] Server: Monitors the stability of the connection and handles errors as needed.

[2234] ---

[2235] Medical records and follow-up features

[2236] Step 1:

[2237] Server: After the video call session ends, the doctor's medical details are saved in a database.

[2238] Step 2:

[2239] Server: Sends a notification to the user that the medical record has been saved.

[2240] Step 3:

[2241] Terminal: Provides a screen where users can check their medical history.

[2242] Step 4:

[2243] Server: Schedule upcoming appointments and set follow-up reminders.

[2244] Step 5:

[2245] User: Receive reminders and schedule upcoming appointments.

[2246] Example 1

[2247] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2248] It is important to make it easy for people living in remote areas or users who have difficulty visiting medical institutions to receive appropriate medical services. There is also a need for a system that efficiently performs follow-up care and health management after medical treatment. However, current systems do not adequately meet these needs, so a system that can provide comprehensive medical services is needed.

[2249] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2250] In this invention, the server includes means for receiving medical information input by a user using a device, means for searching for an appropriate doctor based on the medical information, means for acquiring the doctor's schedule information and confirming the appointment, means for video calling the user and the doctor at the appointment time, means for saving the medical treatment details after the medical treatment session ends, means for setting a follow-up notification based on the medical record, and means for sending the follow-up notification to the user, thereby enabling the user to receive appropriate medical services even from a remote location and enabling follow-up after the medical treatment and long-term health management.

[2251] A "device" is an electronic device used by a user to input medical information, such as a smartphone, tablet, or computer.

[2252] "Medical information" refers to information about symptoms, medical history, and current health status that a user inputs using a device.

[2253] The "server" is a computer system that receives input medical information, searches for and schedules appropriate doctors, and manages video calls and medical records.

[2254] A "physician" is a medical professional who is qualified in a particular specialty and provides medical treatment and diagnosis to users.

[2255] "Schedule information" is information about the doctor's available dates and times and free times.

[2256] "Booking" is the process of confirming a consultation date and time selected by the user and securing a consultation session with a doctor.

[2257] The "video call means" is a video call function that connects the user and a doctor in real time to provide medical care.

[2258] A "medical session" is a series of processes in which a doctor provides medical care to a user via video call or the like.

[2259] "Medical content" refers to information such as diagnoses, instructions, and prescriptions given during a medical session.

[2260] A "follow-up notification" is a communication sent to a user after a medical examination to inform them of the next medical examination or follow-up appointment.

[2261] "Medical records" are data that store information such as a user's past medical history, diagnosis results, and prescriptions.

[2262] The present invention is a system that aims to enable users living in remote locations to receive high-quality medical services. This system provides functions that allow users to input medical information using a device, search for and make appointments with appropriate doctors, receive medical treatment via video call, save medical treatment details, and perform follow-ups. An embodiment of this system will be described in detail below.

[2263] User registration and authentication functions

[2264] A user launches an app or website using a device such as a smartphone or PC and enters personal information such as their name, email address, phone number, and password. The entered information is sent from the device to a server and stored in a database (e.g., MySQL) using SSL / TLS encrypted communication. The server then sends an authentication link to the user's email address, and the user activates their account by clicking the link.

[2265] Doctor search and appointment function

[2266] The user enters symptoms (e.g., abdominal pain) into the app on their device and sets search criteria (e.g., gastroenterology). This information is sent to the server, which filters corresponding specialists from a database (e.g., PostgreSQL). The filtered list of doctors is sent to the device in JSON format, and the user selects the doctor of their choice and makes an appointment. The reservation information is managed by the server, which uses a push notification service (e.g., Firebase) to send a reservation confirmation to the user.

[2267] Video call consultation function

[2268] When the time for the appointment arrives, the user launches the app on their device to prepare for the video call. The server then establishes a secure communication channel (e.g., WebRTC) and starts a video call session between the user and the doctor. The user and doctor communicate in real time through the video call, providing treatment, guidance, and issuing prescriptions.

[2269] Medical records and follow-up features

[2270] After the video call session ends, the server saves the medical record to a database (e.g., AWS RDS). The saved medical record is used to set reminders for the next medical appointment and follow-up. The user can check the medical history and next follow-up information through the app. The server also generates and sends follow-up notifications to the user.

[2271] Specific examples

[2272] User A uses the app to register personal information, receives a verification email, and clicks the link. He enters his symptoms of abdominal pain and searches for and makes an appointment with a gastroenterologist. When the appointment time arrives, he starts a video call and receives a diagnosis and prescription from the doctor. After the appointment, he checks the medical record, receives a follow-up notification, and prepares for the next appointment.

[2273] Prompt Sentence Examples

[2274] "What is the process for User A to launch the app to register for the first time, enter their personal information, receive a verification email, and click the link to activate their account?"

[2275] This system enables users living in remote areas to easily receive high-quality medical services, and allows for efficient health management through continuous follow-up.

[2276] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2277] Step 1:

[2278] Enter user information

[2279] Input: A user uses a device (smartphone or computer) to open an app or website and enter personal information (such as name, email address, phone number, or password).

[2280] Specific behavior: The user enters the required information into the device's input form and clicks the "Submit" button.

[2281] Output: The entered data is sent from the terminal to the server.

[2282] Step 2:

[2283] Information stored in a database

[2284] Input: Personal information sent from your device.

[2285] Specific operation: The server receives the received personal information using SSL / TLS encrypted communication, then stores the received data in a database (e.g., MySQL) and flags the data as successful.

[2286] Output: Personal information stored in a database and a verification email prepared.

[2287] Step 3:

[2288] Generate and send the authentication link

[2289] Input: User's email address and other personal information stored on the server.

[2290] What happens: The server generates a unique token for the user, generates an email containing an authentication link, and then uses a mail server (e.g., SendGrid) to send the authentication email.

[2291] Output: An email containing a verification link will be sent to the user's email address.

[2292] Step 4:

[2293] Receiving emails and clicking links

[2294] Input: The email containing the authentication link.

[2295] What happens: The user opens the email using their email client (e.g. Gmail) and clicks on the authentication link.

[2296] Output: The action of the user clicking the authentication link.

[2297] Step 5:

[2298] Activate your account

[2299] Input: Authentication link click event.

[2300] What happens: The server verifies the token in the authentication link, and if it matches, activates the user's account. If activation is successful, it generates a success message and sends it to the user.

[2301] Output: Activated user account and a success message.

[2302] Step 6:

[2303] Enter symptoms and set search conditions

[2304] Input: Symptoms and search terms entered by the user (e.g., gastroenterology).

[2305] Specific operation: The user enters symptoms and search criteria into the device app and clicks the "Search" button.

[2306] Output: The symptoms and search criteria are sent to the server.

[2307] Step 7:

[2308] Database Search and Filtering

[2309] Input: Symptoms and search criteria submitted by the user.

[2310] Specific operation: The server searches for doctor information in a database (e.g., PostgreSQL) and filters it based on the entered conditions. As a result, it generates a list of suitable doctors.

[2311] Output: A filtered list of doctors is generated.

[2312] Step 8:

[2313] Generate and submit a doctor list

[2314] Input: Filtered list of doctors.

[2315] Specific operation: The server converts the filtered doctor list into JSON format and sends it to the terminal.

[2316] Output: Doctor list in JSON format sent to the terminal.

[2317] Step 9:

[2318] Select a consultation time

[2319] Input: Doctor list displayed on terminal.

[2320] Specific operation: The user selects the doctor of their choice and the appointment date and time in the calendar format, and then the selection is sent to the server.

[2321] Output: The selected appointment date and time and doctor information are sent to the server.

[2322] Step 10:

[2323] Reservation updates and notifications

[2324] Input: User selected appointment date and time and doctor information.

[2325] What happens: The server updates the reservation schedule and generates a reservation confirmation notification, which is sent to the user using a push notification service (e.g., Firebase).

[2326] Output: Pushed booking confirmation notification.

[2327] Step 11:

[2328] Launching the app and preparing to connect

[2329] Input: Appointment time arrives.

[2330] Specific behavior: The user launches the app at the scheduled time and prepares to connect to the video call. The app requests access to the camera and microphone.

[2331] Output: Video call ready to connect.

[2332] Step 12:

[2333] Establishing a communication channel

[2334] Input: The user's ready to connect state.

[2335] What happens: The server establishes a secure communication channel (e.g., WebRTC) and connects the user and doctor to the same session.

[2336] Output: Establishment of a secure video call session.

[2337] Step 13:

[2338] Starting a video call session

[2339] Input: The established communication channel.

[2340] What happens: The video call screen appears in the user's app and a video call session with the doctor begins.

[2341] Output: Start of video call session.

[2342] Step 14:

[2343] Real-time communication

[2344] Input: A video call session.

[2345] Specific operation: The user communicates with the doctor in real time, explaining symptoms and asking questions, and the doctor makes a diagnosis based on the user's symptoms.

[2346] Output: Real-time communication results of the medical consultation.

[2347] Step 15:

[2348] Providing diagnosis and prescriptions

[2349] Input: Real-time communication results.

[2350] What it does: The doctor will explain the diagnosis and issue a prescription if necessary. The diagnosis and prescription information will be provided online.

[2351] Output: Diagnostic results and prescription information.

[2352] Step 16:

[2353] Ending and recording the medical session

[2354] Input: End of medical session.

[2355] What it does: After the video call session ends, the server logs the medical details, including the doctor's diagnosis, prescriptions, and the next recommended visit date and time.

[2356] Output: Recorded medical details.

[2357] Step 17:

[2358] Medical record database storage

[2359] Input: Recorded medical details.

[2360] What it does: The server stores medical records in a database (e.g., AWS RDS). The stored data is encrypted and only accessible to the user and their doctor.

[2361] Output: Medical records stored in a database.

[2362] Step 18:

[2363] Follow-up notification settings

[2364] Input: Medical record.

[2365] What it does: The server uses medical records to schedule reminders for upcoming appointments and follow-ups, adjusting the details of the reminders to fit the user's schedule.

[2366] Output: The set follow-up reminder.

[2367] Step 19:

[2368] Checking medical history

[2369] Input: Stored medical records.

[2370] What it does: The user accesses their medical history through the app, which shows past diagnoses, prescriptions, and next follow-up appointments.

[2371] Output: Displayed medical history.

[2372] Step 20:

[2373] Receive follow-up notifications

[2374] Input: The follow-up reminder that was set.

[2375] Specific operation: A follow-up notification is sent from the server at the specified date and time, and the user receives the notification. The user prepares for the next consultation.

[2376] Output: The follow-up notification sent to the user.

[2377] (Application example 1)

[2378] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2379] The problem that this invention aims to solve is to provide users living in remote areas with a means to efficiently access and learn appropriate medical educational content, in addition to enabling them to receive high-quality medical services. This will enable the creation of an environment in which users can not only access specialists, but also consistently use educational content to deepen their medical knowledge.

[2380] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2381] In this invention, the server

[2382] means for receiving medical information input by a user using a terminal;

[2383] A means for searching for an appropriate specialist based on the medical information;

[2384] means for acquiring schedule information of the specialist and confirming an appointment;

[2385] a video call means for connecting the user and the specialist at the appointment time;

[2386] means for saving the medical treatment details after the medical treatment session ends and notifying the user of the saving;

[2387] a means for a user to input search keywords to filter and display medical education content;

[2388] A means for conducting live lectures using video calls;

[2389] A way to keep a record of what you've watched and set reminders for upcoming lectures and follow-ups,

[2390] This allows users to receive medical treatment from specialists and efficiently access and learn high-quality medical education content.

[2391] "Medical information" refers to information such as health status, symptoms, and medical history entered by the user through the terminal.

[2392] A "specialist" is a medical professional with advanced knowledge and skills in a particular medical field.

[2393] "Schedule information" is information that indicates the dates and times when specialists are available for consultation and the reservation status.

[2394] A "video call" is a means for a user in a remote location and a specialist to communicate in real time using audio and video via the Internet.

[2395] "Medical content" refers to information about diagnosis and treatment plans recorded during video calls or face-to-face medical sessions.

[2396] "Notifications" are a means of informing users about medical treatments and follow-up appointments.

[2397] A "search keyword" is an input character string that a user uses when searching for medical education content.

[2398] "Filtering" is the process of sorting and displaying relevant medical education content based on search keywords.

[2399] "Medical education content" refers to information such as videos, articles, and lecture materials for learning medical knowledge and skills.

[2400] "Live Lecture" means an educational session that takes place in real time and is delivered via video call.

[2401] "Reminder" is a function that notifies users so that they do not forget their next lecture or medical appointment.

[2402] Overall system configuration

[2403] The system of the present invention is designed to enable users to receive high-quality medical services even when they are not in the field. This system mainly uses a smartphone app and is built by combining front-end and back-end technologies.

[2404] Hardware and Software Configuration

[2405] Hardware

[2406] User devices: Mainly smartphones are used.

[2407] Servers: Includes database and application servers.

[2408] software

[2409] Front-end: React Native (smartphone app development)

[2410] Backend: Node.js (application server), MongoDB (database)

[2411] Communication: WebRTC (secure communication for video calls)

[2412] Authentication: Firebase Authentication (user authentication)

[2413] Feature details

[2414] 1. User registration and authentication function

[2415] The user launches the smartphone app and enters the necessary personal information.

[2416] The server receives the entered information and stores it in a database (MongoDB). Firebase Authentication is used to send an authentication email to the user.

[2417] The user receives a verification email and clicks on the link to activate the account. The server verifies the link and activates the user's account.

[2418] 2. Doctor search and appointment function

[2419] The user terminal transmits search keywords based on the symptoms to the server.

[2420] The server queries the database (MongoDB) and generates a list of relevant specialists, which is then sent to the user's device.

[2421] The user selects the desired specialist from a list of doctors and makes an appointment.

[2422] The server stores the user's reservation information and sends a reservation confirmation notification.

[2423] 3. Video call medical consultation function

[2424] The user launches the smartphone app at the scheduled time and prepares to connect to the video call.

[2425] The server establishes a secure communication channel using WebRTC and initiates the video call session.

[2426] Users and specialists can conduct consultations in real time via video calls.

[2427] 4. Medical records and follow-up features

[2428] After the consultation session, the server saves the details of the consultation in a database and sets reminders for the next consultation or follow-up.

[2429] The user terminal provides an interface that allows the user to check their medical history and make their next appointment.

[2430] 5. Medical education content search and browsing function

[2431] The user terminal transmits search keywords to the server to filter medical education content.

[2432] The server retrieves the relevant content from a database (MongoDB) and returns it to the user's terminal.

[2433] Users can view the filtered content through a smartphone app.

[2434] 6. Live lecture function via video call

[2435] The user launches the smartphone app based on the schedule of the live lecture and prepares the video call connection.

[2436] The server uses WebRTC to establish a secure communication channel and conduct the video call session for the live lecture.

[2437] Users can listen to lectures in real time and ask questions and hold discussions.

[2438] 7. Content saving and follow-up features

[2439] The server stores a record of the content the user has viewed in a database and sets reminders for upcoming lectures and follow-ups.

[2440] The user terminal provides an interface that allows the user to check their viewing history and receive reminders for upcoming lectures.

[2441] Specific examples

[2442] For example, if a user wants to learn about the topic of "diabetes," they can enter search keywords into the app and related videos and articles will be displayed. Additionally, WebRTC can be used to attend live seminars by specialists, and the content of these seminars will be stored in a database.

[2443] Prompt Sentence Examples

[2444] Search for diabetes education content to get a list of related videos and articles, collect real-time user interest in these contents, and recommend the most viewed content.

[2445] This allows users to not only receive medical treatment from specialists, but also to effectively access and learn from high-quality medical education content.

[2446] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2447] Step 1:

[2448] User registration and authentication functions

[2449] Input: The user enters personal information into the device and presses the submit button.

[2450] Server: Receives personal information sent by the user and saves it in a database (MongoDB). After saving, sends an authentication email to the user via Firebase Authentication.

[2451] Output: A verification email will be sent to the user's email address.

[2452] User: Receives verification email and clicks verification link.

[2453] Server: Confirms the click on the authentication link and activates the user's account.

[2454] What happens: The server verifies the sender's email address and records the date and time the verification link is clicked.

[2455] Step 2:

[2456] Doctor search and appointment function

[2457] Input: The user enters the symptoms into the terminal and presses the search button.

[2458] Server: Receives the input symptom information and queries the database (MongoDB) to generate a list of relevant specialists.

[2459] Output: The filtered list of specialists is sent to the user terminal.

[2460] User: Selects the desired specialist from a list of specialists and schedules an appointment.

[2461] Server: Receives the reservation information, stores it in a database, and sends a reservation confirmation to the user.

[2462] Specific operation: The server updates the appointment schedule based on the date and time, specialist ID, and user ID, and calls the notification function.

[2463] Step 3:

[2464] Video call consultation function

[2465] Input: At the scheduled time, the user launches the app on their device and prepares for the video call.

[2466] Server: Receives the video call request and establishes a secure communication channel using WebRTC.

[2467] Output: A video call session is started and the video call screen is displayed on the user's device.

[2468] User: Consults with specialists via video call.

[2469] What it does: The server manages WebRTC peer connections and ensures communication encryption.

[2470] Step 4:

[2471] Medical records and follow-up features

[2472] Input: After the medical session ends, the server receives the medical details.

[2473] Server: Receives medical records, stores them in a database, and sets follow-up reminders.

[2474] Output: The medical details are saved in the database and a notification is sent to the user's device.

[2475] Users: Receive notifications and can check their medical history.

[2476] What it does: The server tags the stored visits and schedules reminders for upcoming appointments.

[2477] Step 5:

[2478] Medical education content search and browsing function

[2479] Input: The user enters search keywords into the device and presses the search button.

[2480] Server: Receives search keywords, queries the database, and filters relevant medical education content.

[2481] Output: The filtered medical education content is sent to the user terminal.

[2482] User: Views filtered content on a smartphone app.

[2483] What it does: The server ranks the search results and prioritizes the most relevant content to present to the user.

[2484] Step 6:

[2485] Live lecture function via video call

[2486] Input: User launches the app and prepares based on the schedule of a live lecture.

[2487] Server: Receives the video call request and establishes a secure communication channel using WebRTC.

[2488] Output: A video call session of a live lecture is started and the video is displayed on the user's device.

[2489] Users: Attend live lectures, ask questions and participate in discussions.

[2490] Specific operation: The server monitors the progress of live lectures in real time and notifies you if any problems occur.

[2491] Step 7:

[2492] Content saving and follow-up features

[2493] Input: The server receives information about the content that was viewed.

[2494] Server: Stores records of viewed content in a database and sets reminders for upcoming lectures and follow-ups.

[2495] Output: The viewing history is stored in a database and a reminder for the next lecture is sent to the user's device.

[2496] User: Check viewing history and prepare for the next lecture.

[2497] What it does: The server creates statistics on viewing history and analyzes the topics that users are most interested in.

[2498] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2499] The present invention is a system that enables users living in remote locations to receive high-quality medical services. The system includes a process where users input medical information using a terminal, search for an appropriate specialist, and make an appointment. It also includes functions for receiving medical treatment via video call, saving medical results and providing follow-up, and using an emotion engine to reflect the user's emotional state in the medical treatment.

[2500] System configuration

[2501] The system has the following main functions:

[2502] 1. User registration and authentication function

[2503] 2. Doctor search and appointment function

[2504] 3. Video call medical consultation function

[2505] 4. Medical records and follow-up features

[2506] 5. Emotion recognition function using emotion engine

[2507] Program processing flow

[2508] 1. User registration and authentication function

[2509] On the device: The user launches the app or website and enters the required personal information.

[2510] Server: Receives the entered information, stores it in a database, and sends a verification link to the user's email address.

[2511] User: Receives a verification email and clicks on the link to activate the account.

[2512] Server: Verify the link and activate the user's account.

[2513] Examples:

[2514] User A accesses the account remotely, provides the necessary information, and creates an account. After receiving the authentication email, the user clicks the link to activate the account.

[2515] 2. Doctor search and appointment function

[2516] Device: Enter symptoms into the app on your device and set search criteria.

[2517] Server: Based on the input information, the server filters the doctor information in the database and generates a list of doctors that meet the criteria. This list is then sent to the terminal.

[2518] Terminal: Shows the user a list of doctors and lets them select the appointment time they prefer.

[2519] Server: Confirms the selected reservation information, updates the schedule, and sends a reservation confirmation notification to the user.

[2520] Examples:

[2521] User B complains of stomach pain and searches for a gastroenterologist. He selects an appropriate doctor from the search results and makes an appointment for 2:00 PM the next day.

[2522] 3. Video call medical consultation function

[2523] Device: The user launches the app at the scheduled time and prepares to connect.

[2524] Server: Establishes a secure communication channel and initiates the video call session.

[2525] Terminal: Displays the video call screen, allowing the user and doctor to communicate in real time.

[2526] User: Engages in medical conversations with doctors and receives guidance and prescriptions.

[2527] Examples:

[2528] User C launches the app at the scheduled time and receives a consultation via video call with Doctor D. Doctor D makes an appropriate diagnosis and issues a prescription.

[2529] 4. Medical records and follow-up features

[2530] Server: After the consultation session is over, it stores the details in a database and sets reminders for upcoming consultations and follow-ups.

[2531] Terminal: Allows users to check their medical history and schedule their next appointment if necessary.

[2532] Users: Receive follow-up notifications and prepare for upcoming appointments.

[2533] Examples:

[2534] User E opens the app after the consultation and checks the medical history. The system sends a reminder for a follow-up appointment in one week, and the user makes the next appointment.

[2535] 5. Emotion recognition function using emotion engine

[2536] Device: Captures the user's facial expressions and voice in real time during a video call.

[2537] Server: The emotion engine analyzes the captured data and recognizes the user's emotional state.

[2538] Server: Provides the doctor with the recognized emotional information and reflects it in the medical treatment.

[2539] Server: Stores medical records including emotional information in a database.

[2540] Examples:

[2541] When User F has a video call with a doctor, the emotion engine recognizes that the user is anxious from their facial expression. This information is provided to the doctor, who takes the user's mental state into consideration during the consultation. The emotional information is recorded in the medical treatment details and prescription.

[2542] This system allows users living in remote areas to easily receive high-quality medical services. In addition, by providing medical treatment based on the user's emotional state, more personalized medical services can be provided.

[2543] The processing flow will be explained below.

[2544] Emotion recognition function using emotion engine

[2545] Step 1:

[2546] Device: Show a popup to get the user's camera and microphone permissions before starting a video call.

[2547] Step 2:

[2548] User: Allow use of camera and microphone.

[2549] Step 3:

[2550] Device: After obtaining permission, it displays the video call screen and captures the user's facial expressions and voice in real time.

[2551] Step 4:

[2552] Terminal: Sends captured facial expression and voice data to the server.

[2553] Step 5:

[2554] Server: Analyzes the received facial expression and voice data and uses an emotion engine to recognize the user's emotional state.

[2555] Step 6:

[2556] Server: Processes the recognized emotion information and displays it on the doctor's screen in real time.

[2557] Step 7:

[2558] Terminal: The doctor checks the user's emotional state and responds accordingly.

[2559] Step 8:

[2560] Server: Stores the details of the medical session, including emotional information, in a database.

[2561] Step 9:

[2562] Server: Uses the stored emotional information to inform the next medical appointment or follow-up.

[2563] Specific examples

[2564] Video call consultations

[2565] Step 1:

[2566] Device: User F clicks "Start video call" on the screen for starting a video call.

[2567] Step 2:

[2568] On your device: A pop-up will appear asking you to allow access to your camera and microphone.

[2569] Step 3:

[2570] User: Allow use of camera and microphone.

[2571] Step 4:

[2572] Terminal: The video call screen is displayed, and User F's facial expressions and voice are captured.

[2573] Step 5:

[2574] Device: Captured data is sent to the server.

[2575] Step 6:

[2576] Server: Analyzes the received data, and the emotion engine recognizes User F's emotions as both calm and anxious.

[2577] Step 7:

[2578] Server: The recognized emotion information is displayed on Doctor D's screen.

[2579] Step 8:

[2580] Terminal: Doctor D checks User F's emotional state and provides medical treatment while taking it into consideration.

[2581] Step 9:

[2582] Server: Stores the results of the medical session and emotional information in a database.

[2583] Step 10:

[2584] Server: Based on the stored emotional information, sends notifications and reminders to User F that will be helpful for their next medical appointment.

[2585] This system enables medical treatment that takes into account the user's emotional state, enabling more personalized medical services to be provided. Furthermore, by storing emotional information and utilizing it for the next medical treatment, continuous and consistent medical support can be achieved.

[2586] Example 2

[2587] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2588] In order for users living in remote areas to receive high-quality medical services, they need a system that smoothly handles the entire process, from searching for specialists and making appointments to video call consultations, saving medical records, and providing follow-ups. However, existing systems lack the functionality to reflect the user's emotional state in medical treatment, making them insufficient for providing more personalized medical services. Therefore, there is a need for a system that can analyze the user's emotional state in real time and reflect that in medical treatment.

[2589] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2590] In this invention, the server includes means for receiving medical information input by a user using a terminal, means for searching for an appropriate medical professional based on the medical information, means for acquiring the professional's schedule information and confirming an appointment, means for establishing a secure video call session between the user and the medical professional, means for saving the medical treatment details after the medical treatment session ends and notifying the user, means for capturing the user's facial expressions and voice data during the video call and performing emotion analysis, and means for providing the medical professional with the results of the emotion analysis and reflecting them in the medical treatment details, thereby enabling users in remote locations to receive high-quality and personalized medical services.

[2591] "User" refers to an individual who uses the system to input medical information and receive medical treatment.

[2592] "Terminal" refers to an information processing device such as a smartphone, PC, or tablet used by a user.

[2593] "Medical information" refers to all information necessary for medical treatment, such as medical condition, symptoms, and past medical history, entered by the user.

[2594] "Medical professional" refers to a doctor or nurse who has specialized knowledge and skills in a particular medical field.

[2595] "Schedule information" refers to information such as the availability of medical professionals and appointment availability.

[2596] "Video calling" refers to a means of real-time video and audio communication between a user and a medical professional via the Internet.

[2597] "Medical content" refers to the doctor's diagnosis, treatment plan, prescription information, etc. recorded during a medical session.

[2598] "Emotion analysis" refers to the process of analyzing a user's facial expressions and voice data during a video call to identify the user's emotional state.

[2599] "Emotional state" refers to information that indicates the user's psychological state or mood.

[2600] "Follow-up Notification" means a notification sent to a user for a follow-up or reminder of an upcoming medical appointment.

[2601] The present invention is a system that enables users living in remote locations to receive high-quality medical services. The system involves a process in which users input medical information using a terminal, search for an appropriate medical professional, and receive medical treatment via video call. In addition to functions for saving and following up on medical treatment details, the system also has an emotion analysis function that reflects the user's emotional state in medical treatment.

[2602] System configuration

[2603] 1. User registration and authentication function

[2604] On the device: The user launches an app or website and enters personal information such as their name, email address, and password.

[2605] Server: Receives the entered information, stores it in a database (e.g., MySQL), and sends an email containing a verification link to the user's email address.

[2606] User: Receives a verification email and clicks on the link to activate the account.

[2607] Server: Verify the link and activate the user's account.

[2608] 2. Doctor search and appointment function

[2609] Device: Enter symptoms into the device app and set search criteria.

[2610] Server: Based on the input information, it searches a doctor database (e.g., Elasticsearch) and generates a list of doctors that match the criteria. It then sends this list to the terminal.

[2611] Terminal: Shows the user a list of doctors and lets them select the appointment time they prefer.

[2612] Server: Confirms the selected reservation information, saves the schedule information to the database, and sends a reservation confirmation notification to the user.

[2613] 3. Video call medical consultation function

[2614] On the device: At the scheduled time, the user launches the app and prepares for the video call.

[2615] Server: Uses technologies such as WebRTC to establish a secure video call session and connect users with medical professionals.

[2616] Terminal: Real-time communication between the user and medical professionals.

[2617] 4. Medical records and follow-up features

[2618] Server: After the consultation session is over, it stores the consultation details in a database and sets reminders for the next consultation or follow-up.

[2619] Terminal: Allows users to check their medical history and schedule their next appointment if necessary.

[2620] 5. Emotion recognition function using emotion engine

[2621] Device: Captures facial and audio data in real time during a video call.

[2622] Server: Analyzes the captured data using an emotion analysis engine (e.g., Microsoft Azure's Face API) to identify the user's emotional state.

[2623] Server: Providing recognized emotion information to medical professionals to improve the quality of medical care.

[2624] Server: Stores medical records including emotional information in a database.

[2625] Specific examples

[2626] User A accesses the system and creates an account. He receives a verification email and clicks the link to activate his account. He then complains of abdominal pain, searches for a gastroenterologist, and makes an appointment for 2:00 p.m. the next day. He launches the app at the scheduled time and consults with the doctor via video call. The consultation details are saved in the database, and a reminder to return to the doctor one week later is sent.

[2627] Examples of prompt statements

[2628] Enter the following prompt for the generative AI model:

[2629] Please explain the steps a user takes to access the platform remotely and create an account. Include the process by which a user enters symptoms, searches for a doctor, and makes an appointment using the medical platform. Also, please explain the process for receiving medical advice in real time via video call, as well as the steps for saving medical records and scheduling the next follow-up appointment. Also, please include a function that uses an emotion engine to recognize the user's emotional state and reflect it in medical treatment. Please clearly indicate specific tools and technologies (e.g., WebRTC, MySQL, Microsoft Azure Face API, etc.).

[2630] This system enables users living in remote areas to receive high-quality, personalized medical services.

[2631] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2632] System program processing ...

Claims

1. receiving medical information input by a user using a terminal; A means for searching for an appropriate specialist based on the medical information; means for acquiring schedule information of the specialist and confirming an appointment; a video call means for connecting the user and the specialist at the appointment time; means for saving the medical treatment details after the medical treatment session ends and notifying the user of the saving; A system including:

2. 10. The system of claim 1, wherein said video calling means establishes a secure video calling session between a user and a specialist.

3. 2. The system according to claim 1, wherein the means for storing medical treatment details allows the setting of a next medical treatment schedule and follow-up notifications.

Images