System

Abstract

A system is provided.SOLUTION: A system, comprising: means for a user to input a diagnostic result; means for a server to pre-process the diagnostic result and extract key information; means for the server to interpret the diagnostic result using a AI model and generate therapy options; means for the server to present the interpretation result and the therapy options to the user; means for the server to search for recommended healthcare providers based on the diagnostic result and the therapy options; and means for a device to display the recommended healthcare providers to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern medicine, many patients often receive treatment without fully understanding their doctor's diagnosis. This lack of understanding not only leads to high medical costs and the risk of incorrect treatment, but can also lead to insufficient supervision by family members, especially the elderly and children. Furthermore, patients who do not have time to seek a second opinion from another doctor find it difficult to make a satisfactory decision. Therefore, there is a need for a system that can eliminate information asymmetry in medical knowledge and help patients receive appropriate treatment. [Means for solving the problem]

[0005] To solve this problem, the present invention provides a system that enables patients to quickly obtain a second opinion using AI based on a doctor's diagnosis. The system of the present invention includes the following means:

[0006] 1. A means for the user to input diagnostic data (e.g., an interface that accepts input in text, voice, or image format).

[0007] 2. The means by which the server pre-processes the diagnostic data and extracts key information (e.g., using natural language processing, speech recognition, or optical character recognition techniques).

[0008] 3. A means for the server to utilize the AI ​​model to interpret diagnostic results and generate treatment options.

[0009] 4. The means by which the server presents the interpretation results and treatment options to the user (e.g., organizing the results and displaying them in a format suitable for the user interface).

[0010] 5. A means for the server to search for recommended medical institutions based on the diagnosis and treatment options (e.g., using internal and external databases to find medical institutions).

[0011] 6. A means by which the terminal displays recommended medical institutions to the user.

[0012] This will enable patients to accurately understand their diagnostic results, quickly obtain information about appropriate treatment options and recommended medical institutions, and make decisions to receive treatment that satisfies their needs.

[0013] "User" refers to an individual or patient who uses the system to input diagnostic data and receive information.

[0014] "Server" refers to the computer system that processes diagnostic data, interprets it using AI, and organizes the information.

[0015] "Diagnostic Data" refers to the results of a diagnosis made by a physician to a patient, and may include information in text, audio, or image format.

[0016] "Preprocessing" refers to a series of processes for analyzing input diagnostic data and extracting important information.

[0017] "Natural language processing" refers to the technology of analyzing text data and extracting meaningful information.

[0018] "Speech recognition" refers to the technology of converting voice data into text.

[0019] "Optical character recognition" refers to the technology of extracting text information from image data.

[0020] "AI model" refers to an algorithm or computational method that uses artificial intelligence techniques to interpret diagnostic results and generate treatment options.

[0021] "Interpretation result" refers to information that represents the interpretation of the diagnostic content generated by the AI ​​model from the diagnostic data.

[0022] "Treatment options" refer to the treatment options offered to a patient.

[0023] "Recommended medical institution" refers to a medical institution that provides appropriate medical services to patients based on diagnostic results and treatment options.

[0024] "Terminal" refers to an electronic device such as a computer, smartphone, or tablet that a user uses to receive diagnosis results and information about recommended medical institutions.

[0025] "User interface" refers to the screen and input means that users use to operate the system. [Brief explanation of the drawings]

[0026] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0027] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0028] First, the terms used in the following description will be explained.

[0029] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0030] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0031] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0032] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0033] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0034] [First embodiment]

[0035] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0036] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0037] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0038] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0039] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0040] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0041] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0042] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0043] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0044] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0045] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0046] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0047] This invention is a system that utilizes AI to enable patients to quickly obtain a second opinion based on a doctor's diagnosis. The system works through a series of processes: the user inputs diagnostic data, the server analyzes the data and provides treatment options, and finally the terminal displays the results.

[0048] Specific system functions

[0049] Data Entry Method

[0050] Users can input diagnostic data into the system using devices such as smartphones and PCs. Diagnostic data can be input by text input, voice recording, or uploading a scanned image. For example, users can input the text of the diagnostic report directly, record the diagnostic results by voice and upload them, or take an image of the diagnostic report and upload it.

[0051] Data preprocessing methods

[0052] The server preprocesses the diagnostic data received from the user. Specifically, if it is text data, it uses natural language processing (NLP) techniques to extract important medical information. If it is audio data, it uses speech recognition technology to convert the speech into text and then further analyzes the text. If it is image data, it uses optical character recognition (OCR) technology to extract text information.

[0053] AI-based diagnostic interpretation methods

[0054] The server inputs the preprocessed data into the AI ​​model. The designed AI model interprets the diagnostic data based on the medical database and past diagnostic results. This interpretation includes summarizing the diagnostic results, analyzing the current condition, and proposing multiple treatment options. For example, if a patient is at risk for high blood pressure and diabetes based on their symptoms and diagnostic results, the model will suggest specific treatment options for each.

[0055] Presentation of results

[0056] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the device, which displays the results through a user interface and provides the information in a format that is easy for the user to understand, including an interface where the user can view treatment options and a screen that displays a summary of the diagnosis.

[0057] Search methods for recommended medical institutions

[0058] The server searches for nearby appropriate medical facilities based on the diagnosis and treatment options. It uses internal and external databases to identify medical facilities that match the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[0059] Specific examples

[0060] For example, if a user inputs a diagnosis in text format such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing to extract key information about blood pressure and diabetes. The AI ​​model then interprets the diagnosis and generates treatment options such as "use of antihypertensive medication for high blood pressure" or "dietary therapy to prevent diabetes." This information is then organized and displayed in an easy-to-understand format on the user's device. Furthermore, the system recommends the nearest internal medicine or specialist medical institution based on the user's location.

[0061] As described above, the present invention is a system that enables users to easily understand diagnostic results and quickly obtain satisfactory treatment options. This system eliminates information asymmetry for users, making it easier for them to make decisions regarding appropriate treatment.

[0062] The processing flow will be explained below.

[0063] Step 1:

[0064] The user enters the diagnostic data. In this step, the user uses a device such as a smartphone or PC to enter the contents of the diagnostic report. The input method can be text input, voice recording, or uploading an image of the diagnostic report. For example, if the user takes a photo of the diagnostic report and uploads it to the system, the image will be sent.

[0065] Step 2:

[0066] The server preprocesses the diagnostic data received from the user. If it is text data, it uses natural language processing (NLP) techniques to extract important information. If it is voice data, it uses voice recognition technology to convert it into text. If it is image data, it uses optical character recognition (OCR) technology to extract text information. For example, in the case of image data, OCR technology extracts text information from the image.

[0067] Step 3:

[0068] The server inputs the preprocessed data into the AI ​​model, which then analyzes the diagnostic data based on medical databases and past diagnostic results. This analysis includes summarizing the diagnostic results, interpreting the current condition, and proposing multiple treatment options. For example, if a patient is at risk for high blood pressure and diabetes based on their symptoms and diagnostic results, the model will suggest specific treatment options for each.

[0069] Step 4:

[0070] The server organizes the interpretation results and treatment options from the AI ​​model and sends them to the device. In this step, the server formats the interpretation results in a format that is easy for the user to understand and prepares the results in a format suitable for the user interface. For example, a summary of the diagnosis results and multiple treatment options are organized in a list format.

[0071] Step 5:

[0072] The device receives the information sent from the server and displays the results to the user. In this step, the results are presented in an easy-to-understand manner through a user interface, for example, providing the user with specific treatment options or a summary of the diagnosis.

[0073] Step 6:

[0074] The server searches for recommended medical institutions based on the diagnosis and treatment options. This step uses internal and external databases to find appropriate medical institutions that match the user's location. For example, it lists nearby internal medicine clinics and specialists.

[0075] Step 7:

[0076] The terminal receives the information on recommended medical institutions sent from the server and displays it to the user. In this step, a list of recommended medical institutions is displayed through the user interface. For example, the location, medical department, contact information, etc. of the medical institution are displayed.

[0077] Example 1

[0078] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0079] In the medical field, it is difficult for patients to quickly and accurately obtain a second opinion. In particular, when handling diagnostic data in various formats, processing it is complex and time-consuming. In addition, it is currently difficult to accurately recommend appropriate treatment options and medical institutions. This has led to the problem that patients are unable to quickly obtain satisfactory treatment options.

[0080] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0081] In this invention, the server includes a means for preprocessing diagnostic data and extracting important information, a means for interpreting diagnostic results using a generative AI model and generating treatment options, and a means for presenting the interpretation results and treatment options to a user, thereby enabling the user to quickly analyze and understand diagnostic data in various formats and obtain appropriate treatment options and recommendations for medical institutions.

[0082] "User" refers to the person who uses the system to input diagnostic data and operate the terminal that displays diagnostic results and treatment options.

[0083] "Server" refers to a computing device that preprocesses diagnostic data, analyzes it using AI models, organizes the results, and searches for medical institutions.

[0084] "Terminal" refers to the device through which a user views diagnostic results, treatment options, and recommended medical institutions, including smartphones and PCs.

[0085] "Diagnostic data" refers to information including the doctor's diagnosis, and is expressed in the form of text, audio, image, or the like.

[0086] "Preprocessing" is the process of preparing diagnostic data to extract important information, and uses natural language processing, speech recognition, and optical character recognition technologies.

[0087] A "generative AI model" refers to an algorithm that uses input diagnostic data, references medical databases and past diagnostic results, and generates a summary of the diagnostic results and multiple treatment options.

[0088] "Natural language processing (NLP)" refers to the technology of extracting meaningful information from text data.

[0089] "Voice recognition technology" refers to technology that converts voice data into text data.

[0090] "Optical character recognition (OCR)" refers to the technology of extracting character information from image data.

[0091] "Diagnostic interpretation" refers to the process by which an AI model summarizes diagnostic results and suggests treatment options based on pre-processed diagnostic data.

[0092] "Treatment options" refer to treatment methods and procedures proposed based on diagnostic results.

[0093] "Recommended Medical Institution" refers to a medical institution recommended to the user based on the diagnosis and treatment options.

[0094] "Interface" refers to the screens and operating means through which a user can view and select diagnostic results and treatment options via a terminal.

[0095] This invention relates to a system that allows users to quickly obtain a second opinion based on a doctor's diagnosis using AI. The system works through a series of processes: the user inputs diagnostic data, the server analyzes the data, provides treatment options, and finally the terminal displays the results.

[0096] First, the user inputs diagnostic data using a device such as a smartphone or PC. Diagnostic data can be input by text input, voice recording, or uploading a scanned image. For example, the user can input the text of the diagnostic report directly, record the diagnostic results by voice and upload them, or take an image of the diagnostic report and upload it.

[0097] The server then preprocesses the diagnostic data received from the user. Specifically, if it is text data, it uses natural language processing (NLP) techniques to extract key medical information. If it is audio data, it uses speech recognition technology to convert the speech into text and then further analyzes the text. If it is image data, it uses optical character recognition (OCR) technology to extract text information.

[0098] The server inputs the preprocessed data into a generative AI model. The designed AI model interprets the diagnostic data based on medical databases and past diagnostic results. This interpretation includes summarizing the diagnostic results, analyzing the current condition, and proposing multiple treatment options. For example, if a patient is at risk for high blood pressure and diabetes based on their symptoms and diagnostic results, the model will suggest specific treatment options for each.

[0099] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the device, which displays the results through a user interface and provides the information in a format that is easy for the user to understand, including an interface where the user can view treatment options and a screen that displays a summary of the diagnosis.

[0100] The server then searches for appropriate nearby medical facilities based on the diagnosis and treatment options. Using internal and external databases, it identifies medical facilities based on the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[0101] For example, if a user enters a diagnosis in text format such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing to extract key information about blood pressure and diabetes. The AI ​​model then interprets the diagnosis and generates treatment options such as "use of antihypertensive medication for high blood pressure" or "dietary therapy to prevent diabetes." This information is then organized and displayed in an easy-to-understand format on the user's device. Furthermore, the system recommends the nearest internal medicine or specialist medical facility based on the user's location.

[0102] An example of a prompt is as follows:

[0103] "Analyze the following diagnostic data and provide possible diagnoses and treatment options: 'A patient has been diagnosed with high blood pressure and possible diabetes.'"

[0104] This invention is a system that enables users to easily understand diagnostic results and quickly obtain satisfactory treatment options, thereby eliminating information asymmetry and making it easier for users to make decisions regarding appropriate treatment.

[0105] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0106] Step 1: Data entry

[0107] Users enter diagnostic data using a smartphone or computer by entering text, recording a voice message, or uploading a scanned image.

[0108] Specifically, users can manually enter the text of the medical certificate, record it as audio and upload it, or take an image of the medical certificate and upload it to the system.

[0109] Input: Diagnostic data (text, audio, images)

[0110] Output: Raw diagnostic data sent to the server

[0111] Step 2: Preprocessing the data

[0112] The server preprocesses the diagnostic data received from the user: if it is text data, it uses natural language processing (NLP) techniques to extract key medical information; if it is audio data, it uses speech recognition technology to convert speech to text; and if it is image data, it uses optical character recognition (OCR) technology to extract text information.

[0113] Input: Raw diagnostic data

[0114] Output: Preprocessed diagnostic data (text with key medical information extracted)

[0115] Step 3: AI-based diagnostic interpretation

[0116] The server inputs the pre-processed data into a generative AI model, which interprets the data based on medical databases and past diagnostic results to generate a summary of the diagnosis and multiple treatment options.

[0117] For example, it identifies the risk of high blood pressure and diabetes and suggests specific treatments for each.

[0118] Input: Preprocessed diagnostic data

[0119] Output: Diagnostic interpretation and treatment options from the AI ​​model

[0120] Step 4: Presenting the results

[0121] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the terminal.

[0122] The device displays the results through a user interface, providing information in a format that is easy for the user to understand, including an interface that allows the user to easily browse treatment options and a screen that displays a summary of the diagnosis.

[0123] Input: Diagnostic interpretation results and treatment options from the AI ​​model

[0124] Output: The results displayed on the user's terminal

[0125] Step 5: Find a recommended medical institution

[0126] The server searches for nearby appropriate medical facilities based on the diagnosis and treatment options, using internal and external databases to identify and present a list of recommended medical facilities based on the user's location.

[0127] Input: Diagnosis, treatment options, user location

[0128] Output: List of recommended medical institutions

[0129] (Application example 1)

[0130] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0131] In modern medicine, it is important for patients to seek accurate and prompt second opinions, but in many cases, obtaining an appropriate opinion is difficult due to a lack of medical expertise or physical distance. Efficiently processing diagnostic data in different formats (text, audio, images) and providing optimal treatment options is also a major challenge. Furthermore, there are limited ways to provide information to patients in a visually understandable manner, resulting in delayed access to appropriate treatment.

[0132] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0133] In this invention, the server includes a means for a user to input diagnostic data, a means for the server to preprocess the diagnostic data and extract important information, and a means for the server to use a generative AI model to interpret the diagnostic results and generate prompts for generating treatment options. This enables the diagnostic data to be analyzed quickly and accurately, and optimal treatment options to be provided to the user. Furthermore, since the user can easily input diagnostic data in text, voice, or image formats, the system can accommodate different data formats and improve information accessibility. Furthermore, by using the generated prompts, the generative AI model can propose highly accurate treatment options, resulting in the rapid recommendation of the most appropriate medical institution, thereby shortening the time it takes for patients to receive appropriate treatment.

[0134] "Diagnostic Data" means information about a patient's medical condition, medical history, test results, and other information provided in text, audio, or image format.

[0135] "Server" refers to a computer system that preprocesses diagnostic data collected from users and analyzes and diagnoses it using a generative AI model.

[0136] A "generative AI model" refers to an artificial intelligence algorithm that uses large amounts of medical data to generate diagnostic results and treatment options based on input data.

[0137] A "prompt sentence" is an input sentence that prompts the generative AI model to generate an appropriate response, and is generated based on diagnostic data.

[0138] "Text" refers to information in a format consisting of strings of letters, numbers, symbols, etc.

[0139] "Voice" refers to information in the form of data that is a recording of what the user is saying.

[0140] An "image" is a data format that contains visual information and is obtained by scanning or photographing.

[0141] "Natural language processing" refers to the technology of processing and analyzing the information and structure contained in text data.

[0142] "Speech recognition" refers to the technology of converting voice data into text data.

[0143] "Optical character recognition" refers to the technology of extracting character information from image data.

[0144] "Treatment options" refer to the treatment methods and specific medical procedures proposed based on the diagnostic results.

[0145] "User interface" refers to an interactive screen or operating means for a user to operate a system.

[0146] The present invention relates to a system that quickly and accurately analyzes diagnostic data and provides appropriate treatment options. The system works through a series of processes: a user inputs diagnostic data using an application installed on a smartphone, smart glasses, a head-mounted display, or a robot, the data is preprocessed on a server, treatment options are generated using a generative AI model, and the results are finally displayed on a terminal.

[0147] Explanation of the process

[0148] 1. User data entry

[0149] Users use devices such as smartphones to input diagnostic data into the system. This data can be input as text, voice, or images. For example, a user might input information such as "I've been diagnosed with high blood pressure" in text format into an application.

[0150] 2. Data preprocessing on the server

[0151] The server preprocesses the diagnostic data received from the user. For text data, it uses natural language processing (NLP) techniques to extract key medical information. For audio data, it uses speech recognition technology to convert the speech to text, which is then processed in the same way as text data. For image data, it uses optical character recognition (OCR) technology to extract text information.

[0152] 3. Diagnostic interpretation using generative AI models

[0153] The server uses a generative AI model based on the preprocessed data to interpret the diagnosis results. At this time, a prompt is generated and input into the AI ​​model to generate the diagnosis results and treatment options. For example, a prompt might be generated such as, "You have been diagnosed with high blood pressure. You have been diagnosed as being at risk for lifestyle-related diseases."

[0154] 4. Displaying the results

[0155] The server organizes the diagnostic interpretation results and treatment options generated by the generative AI model and sends them to the device, which displays the results through a user interface and provides the information in a format that is easy for the user to understand.

[0156] 5. Search for recommended medical institutions

[0157] The server searches for appropriate nearby medical facilities based on the diagnosis and treatment options, using internal and external databases to identify medical facilities that match the user's location and present them as a recommended list.

[0158] Hardware and software used

[0159] The main technologies used in this system include natural language processing (NLP), speech recognition, optical character recognition (OCR), and generative AI models. Specifically, it uses Spacy for NLP, Google Cloud Speech API for speech recognition, Tesseract for OCR, and a Transformer-based model (e.g., rinna / japanese-gpt2-medium) for the generative AI model.

[0160] Specific examples

[0161] When a user enters text data such as "I've been diagnosed with high blood pressure," the server preprocesses the data using natural language processing technology to extract important information. Next, based on this information, the server generates a prompt statement: "I've been diagnosed with high blood pressure. I've been told I'm at risk for lifestyle-related diseases." This prompt statement is then input into a generative AI model, which retrieves optimal treatment options and hospital recommendations. Finally, the results are displayed in an easy-to-understand format on the user's device, allowing the user to quickly and appropriately select treatment.

[0162] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0163] Step 1:

[0164] The user enters diagnostic data.

[0165] Specifically, a user inputs diagnostic data into the system using a device such as a smartphone or PC. This diagnostic data can be provided in the form of text, voice, or images. For example, a user may input text data such as "I have been diagnosed with high blood pressure." The input data is then sent to the server.

[0166] Step 2:

[0167] The server pre-processes the diagnostic data.

[0168] The server pre-processes the diagnostic data received from the user, specifically using natural language processing (NLP), speech recognition, and optical character recognition (OCR) technologies.

[0169] For text data, NLP techniques (such as Spacy) are used to extract important medical information. For example, if the input is "diagnosed with high blood pressure," the keywords "blood pressure" and "diagnosis" are extracted.

[0170] For voice data, the Google Cloud Speech API is used to convert the speech to text, which is then processed in the same way as text data. If the voice input is "I've been diagnosed with high blood pressure," speech recognition will produce the same text.

[0171] For image data, Tesseract is used to extract text information from the image. If you upload an image of a medical certificate, the IMAGE TO TEXT technique will retrieve the text as well.

[0172] Step 3:

[0173] The server uses a generative AI model to interpret the diagnosis and generate prompts to generate treatment options.

[0174] Based on the preprocessed data, the server uses a generative AI model (e.g., rinna / japanese-gpt2-medium) to interpret the diagnosis results and generate prompts to generate treatment options. Specifically, the preprocessed text data is used to generate the prompt, "You have been diagnosed with high blood pressure. You have been deemed to be at risk for lifestyle-related diseases."

[0175] Step 4:

[0176] The server inputs the generated prompts into the AI ​​to generate treatment options.

[0177] The generated prompt sentences are input into a generative AI model, which then generates a diagnosis and treatment options based on the prompt sentence. For example, in response to the prompt sentence "You have been diagnosed with high blood pressure and have been told you are at risk for lifestyle-related diseases," treatment options such as "use of antihypertensive medication" and "improvement of diet" are output.

[0178] Step 5:

[0179] The server presents the interpretation results and treatment options to the user.

[0180] The server organizes the generated diagnostic interpretation results and treatment options and sends them to the user's device. The device displays them through a user interface. For example, the interpretation result may be "Your blood pressure is high. You are at risk of lifestyle-related diseases," with treatment options such as "Use antihypertensive medication and improve your diet."

[0181] Step 6:

[0182] The server searches for recommended medical institutions based on the diagnosis and treatment options.

[0183] Based on the diagnosis and treatment options, the server searches for appropriate nearby medical institutions taking into account the user's location information, using internal and external medical institution databases to identify medical institutions that meet the user's criteria.

[0184] Step 7:

[0185] The terminal displays recommended medical institutions to the user.

[0186] Information about recommended medical institutions is sent to the device, which then displays it through a user interface. For example, "Nearby Internal Medicine Clinics: XX Internal Medicine Clinic" is displayed in list format.

[0187] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0188] This invention combines an emotion engine with a system that allows users to quickly obtain a second opinion based on a doctor's diagnosis using AI. This system functions through a series of processes: the user inputs diagnostic data, the server analyzes the data, provides treatment options, and finally the device displays the results. Furthermore, the emotion engine recognizes the user's emotional state and reflects it in the analysis results and information presentation.

[0189] Specific system functions

[0190] Data Entry Method

[0191] Users can input diagnostic data into the system using devices such as smartphones and PCs. Diagnostic data can be input by text input, voice recording, or uploading a scanned image. For example, if a user takes a photo of a medical certificate and uploads it to the system, the image will be sent.

[0192] Data preprocessing methods

[0193] The server preprocesses the diagnostic data received from the user. Specifically, if it is text data, it uses natural language processing (NLP) technology to extract important medical information. If it is audio data, it uses speech recognition technology to convert the speech into text, which is then further analyzed. If it is image data, it uses optical character recognition (OCR) technology to extract text information. For example, if it is image data, OCR technology extracts text information from the image.

[0194] emotion recognition means

[0195] The server uses an emotion engine to recognize the user's emotions. It analyzes emotions from text and voice data entered by the user and obtains their emotional state. For example, it can recognize whether the user is feeling anxious from the tone of their voice and the words they use.

[0196] AI-based diagnostic interpretation methods

[0197] The server inputs the preprocessed data into the AI ​​model. The AI ​​model analyzes the diagnostic data based on a medical database and past diagnostic results. This analysis includes a summary of the diagnostic results, an interpretation of the current condition, and multiple treatment options. Furthermore, the presentation of the interpretation results and treatment options can be adjusted based on the user's emotional state. For example, if a user is feeling anxious, the AI ​​model will present information in a more understandable and reassuring manner.

[0198] Presentation of results

[0199] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the device in a format that is adjusted based on the user's emotional state. The device then displays the results through a user interface and provides information in a format that is easy for the user to understand. For example, if the user is feeling anxious, the device will explain the interpretation results more carefully and display a reassuring message.

[0200] Search methods for recommended medical institutions

[0201] The server searches for nearby appropriate medical facilities based on the diagnosis and treatment options. It uses internal and external databases to identify medical facilities that match the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[0202] Specific examples

[0203] For example, if a user inputs a diagnosis in text format, such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing to extract key information about blood pressure and diabetes. The emotion engine then evaluates the user's emotional state. Next, an AI model interprets the diagnosis and generates treatment options, such as "antihypertensive medication for high blood pressure" or "dietary therapy for diabetes prevention." If the system recognizes that the user is feeling anxious, specific and detailed explanations are added to the results to provide reassurance. This information is then organized and displayed on the user's device in an easy-to-understand format. Furthermore, the system recommends the nearest internal medicine or specialist clinic based on the user's location. This format allows users to more easily understand the diagnosis and consider treatment options with peace of mind.

[0204] The processing flow will be explained below.

[0205] Step 1:

[0206] The user inputs the diagnostic data. The user inputs the contents of the diagnostic report using a smartphone or PC. This input method includes text input, voice recording, or uploading an image of the diagnostic report. For example, if the user takes a photo of the diagnostic report and uploads it to the system, the image data is sent to the system.

[0207] Step 2:

[0208] The server preprocesses the diagnostic data received from the user. For text data, natural language processing (NLP) techniques are used to extract key medical information. For audio data, speech recognition technology is used to convert the audio into text and then analyze the text. For image data, optical character recognition (OCR) technology is used to extract text information. For example, for image data, OCR technology extracts text information from the image and organizes its content.

[0209] Step 3:

[0210] The server recognizes the user's emotions using an emotion engine. The emotion engine analyzes emotions from the text and voice data entered by the user and determines the user's emotional state. For example, it evaluates whether the user is feeling anxious based on the content of the text, choice of words, and tone of voice.

[0211] Step 4:

[0212] The server inputs the preprocessed data into the AI ​​model to generate a diagnostic interpretation. The AI ​​model analyzes the diagnostic data based on a medical database and past diagnostic results. This analysis includes a summary of the diagnostic results, an interpretation of the current condition, and suggested treatment options. Furthermore, the presentation of the interpretation results and treatment options is adjusted based on the user's emotional state. For example, if the system recognizes that the user is feeling anxious, it will change the presentation format to provide reassurance.

[0213] Step 5:

[0214] The server organizes the interpretation results and treatment options from the AI ​​model and sends them to the device. The server then formats the interpretation results to present them to the user in an easy-to-understand manner. The interpretation results and treatment options are displayed in a format suitable for the user interface. For example, a summary of the diagnosis results and multiple treatment options are organized in a list format, and a message to reassure the user is also displayed.

[0215] Step 6:

[0216] The device receives the information sent from the server and displays the results to the user. A summary of the diagnosis, treatment options, and recommended medical institutions are clearly displayed through the user interface. For example, treatment options are presented in a format that is easy for the user to understand, and additional information can be added to provide special reassurance if the user is feeling anxious.

[0217] Step 7:

[0218] The server searches for recommended medical institutions based on the diagnosis and treatment options. It uses internal and external databases to find appropriate medical institutions that match the user's location. For example, based on the user's location, it may list nearby internal medicine clinics and specialists.

[0219] Step 8:

[0220] The terminal receives the information on recommended medical institutions sent from the server and displays it to the user. A list of recommended medical institutions is displayed through the user interface, including the medical institution's location, medical specialty, contact information, etc. It is designed to make it easy for users to check the information on recommended medical institutions and make appointments if necessary, for example.

[0221] Example 2

[0222] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0223] The present invention aims to provide a system that allows users to obtain a quick and accurate second opinion without having to go to a medical institution. Another objective of the present invention is to realize a system that takes into consideration the emotional state of the user when presenting diagnostic results and provides a sense of security.

[0224] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0225] In this invention, the server includes a means for preprocessing the diagnostic data and extracting important information, a means for recognizing the user's emotions, and a means for interpreting the diagnostic results and generating treatment options using an AI model, thereby enabling the user to obtain a quick and accurate second opinion without having to go to a medical institution.

[0226] "User" refers to an individual who uses the system to input and utilize diagnostic data.

[0227] "Terminal" refers to a device, such as a smartphone or PC, through which a user inputs diagnostic data and receives the results.

[0228] "Server" refers to the central processing unit that processes data, analyzes, and transmits results for the entire system.

[0229] "Diagnostic Data" means any form of information, including text, audio, and images, entered by a user as a medical diagnosis.

[0230] "Pre-processing" refers to the initial data processing performed by the server to analyze the diagnostic data and extract important information.

[0231] "Natural language processing (NLP)" refers to the technology for analyzing text data and extracting important information.

[0232] "Speech recognition" refers to the technology for converting voice data into text.

[0233] "Optical character recognition (OCR)" refers to a technology for extracting text information from image data.

[0234] An "emotion engine" refers to technology that analyzes emotions from user input data and recognizes that emotional state.

[0235] "AI model" refers to artificial intelligence technology that analyzes diagnostic results and generates treatment options based on medical data.

[0236] "Diagnosis result" refers to a medical interpretation based on the user's diagnostic data analyzed by the AI ​​model.

[0237] "Treatment options" refer to multiple treatment methods suggested based on diagnostic results.

[0238] "Recommended medical institution" refers to an appropriate medical institution that the server searches for and suggests to the user based on the diagnosis results and treatment options.

[0239] This invention combines emotion recognition technology with a system that utilizes AI technology to enable users to quickly obtain a second opinion based on a doctor's diagnosis. The system works through a series of processes: the user inputs diagnostic data, the server analyzes the data, provides treatment options, and finally the device displays the results. Furthermore, an emotion engine is used to recognize the user's emotional state and reflect this in the analysis results and information presentation.

[0240] First, the user enters diagnostic data into the system using a device such as a smartphone or PC. This diagnostic data can be provided by text entry, voice recording, or uploading a scanned image. Specifically, the user can also take a photo of the medical certificate and upload it to the system.

[0241] The server then preprocesses the diagnostic data received from the user. For text data, natural language processing (NLP) techniques are used to extract key medical information. For audio data, speech recognition techniques are used to convert the speech to text, which can then be further analyzed. For image data, optical character recognition (OCR) techniques are used to extract text information. This converts the diagnostic data into an analyzable format.

[0242] Furthermore, the server uses an emotion engine to recognize the user's emotions. It analyzes emotions from text and voice data entered by the user and obtains their emotional state. For example, it can recognize whether the user is feeling anxious from the tone of their voice and the words they use.

[0243] The server then inputs the preprocessed data and the user's emotional state into an AI model. The AI ​​model analyzes the diagnostic data based on a medical database and past diagnostic results. This analysis includes a summary of the diagnostic results, an interpretation of the user's current condition, and multiple treatment options. The AI ​​model also adjusts the presentation of the interpretation results and treatment options based on the user's emotional state. For example, if a user is feeling anxious, the AI ​​model can present information in a more understandable and reassuring way.

[0244] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends the adjusted results to the user's device based on the user's emotional state. The device then displays the results through a user interface, providing information in a format that is easy for the user to understand. For example, if the user is feeling anxious, the device will explain the interpretation results more carefully and display a reassuring message.

[0245] The server then searches for nearby appropriate medical facilities based on the diagnosis and treatment options. Using internal and external databases, it identifies medical facilities based on the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[0246] Specific examples

[0247] For example, if a user enters a diagnosis in text format such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing technology to extract important information about blood pressure and diabetes. The emotion engine evaluates the user's emotional state. Next, the AI ​​model interprets the diagnosis and generates treatment options such as "use of antihypertensive medication for high blood pressure" or "dietary therapy to prevent diabetes." If the system recognizes that the user is feeling anxious, it provides a detailed and detailed explanation when presenting the results. This information is organized and displayed in an easy-to-understand format on the user's device. Furthermore, the system recommends the nearest internal medicine or specialist medical institution based on the user's location information.

[0248] Prompt Sentence Examples

[0249] An example of a prompt to be input to the generative AI model is as follows:

[0250] A user has entered the following text data: "The patient has been diagnosed with high blood pressure and possible diabetes." Please provide the following information:

[0251] 1. Summary of diagnostic results

[0252] 2. Diagnostic interpretation

[0253] 3. Suggest treatment options, including reassuring explanations (assuming the user is feeling anxious).

[0254] In this way, the system can comprehensively analyze the user's input data and respond in a way that takes into account the necessary information and emotions.

[0255] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0256] Step 1:

[0257] Users access the system using devices such as smartphones or PCs and enter diagnostic data by either text entry, voice recording, or uploading scanned images.

[0258] Input: Text data, audio data, or image data of the medical certificate.

[0259] Output: Diagnostic data sent to the server.

[0260] Step 2:

[0261] The server preprocesses the diagnostic data it receives. Here, it distinguishes the type of data (text, image, audio) and performs appropriate preprocessing for each.

[0262] For text data, natural language processing (NLP) techniques are used to extract key medical information.

[0263] In the case of voice data, speech recognition technology is used to convert the voice into text.

[0264] For image data, optical character recognition (OCR) techniques are used to extract text information.

[0265] Input: Diagnostic data (text, audio, images).

[0266] Output: Preprocessed text data.

[0267] Step 3:

[0268] The server sends the pre-processed data to the emotion engine to analyze the user's emotional state.

[0269] For example, it recognizes whether a user is feeling anxious from the tone of their voice and text expressions.

[0270] Input: Preprocessed text data.

[0271] Output: The user's emotional state.

[0272] Step 4:

[0273] The server inputs the preprocessed data and emotional state into a generative AI model, which analyzes the data based on medical databases and past diagnostic results to generate a summary of the diagnosis, an interpretation of the patient's current condition, and multiple treatment options. It also adjusts the way the information is presented based on the patient's emotional state.

[0274] Input: Preprocessed text data, user emotional state.

[0275] Output: Diagnostic results, treatment options.

[0276] Step 5:

[0277] The server organizes the generated diagnostic results and treatment options, adjusts them according to the user's emotional state, and sends the organized information to the user's device.

[0278] Input: diagnosis, treatment options, and the user's emotional state.

[0279] Output: Tailored diagnostic results and treatment options.

[0280] Step 6:

[0281] The terminal displays the adjusted diagnosis results and treatment options sent from the server through a user interface.

[0282] For example, if a user is feeling anxious, a reassuring message or detailed explanation can be added.

[0283] Input: Adjusted diagnostic results and treatment options.

[0284] Output: Diagnostic results and treatment options displayed to the user.

[0285] Step 7:

[0286] Based on the diagnosis and treatment options, the server uses the user's location and internal and external databases to search for appropriate medical facilities nearby.

[0287] Input: Diagnosis results, treatment options, user location.

[0288] Output: A list of recommended medical institutions.

[0289] Step 8:

[0290] The terminal displays to the user a list of recommended medical institutions sent from the server.

[0291] For example, it displays the location of medical institutions on a map and provides detailed information.

[0292] Input: A list of recommended medical institutions.

[0293] Output: A list of recommended medical institutions that is displayed to the user.

[0294] (Application example 2)

[0295] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0296] Systems that allow users to quickly obtain a second opinion based on a doctor's diagnosis often have problems, such as a poor user interface, an irrational presentation of the diagnosis, or a lack of consideration for the user's emotional state. These problems make it difficult for users to understand the diagnosis information and take appropriate action while feeling sufficiently reassured.

[0297] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input diagnostic data in voice, text, or image format, means for the server to preprocess the diagnostic data and extract important information using natural language processing, speech recognition, or optical character recognition technology, means for the server to recognize the user's emotional state using an emotion engine, means for the server to interpret the diagnostic data using an artificial intelligence model and generate treatment options, means for the server to adjust and present the interpretation results and treatment options according to the user's emotional state, means for the server to search for recommended medical institutions based on the diagnostic results and treatment options, and means for the terminal to display recommended medical institutions to the user and present the results in an easy-to-understand and reassuring manner. This allows the user to quickly and safely obtain diagnostic information and treatment options according to their emotional state.

[0298] "Diagnostic Data" refers to information entered by a user in text, audio, or image format regarding health conditions and diagnostic results.

[0299] "Natural language processing" is a technique for extracting important medical information from diagnostic data in text format.

[0300] "Speech recognition" is a technology that converts diagnostic data in voice format into text and analyzes it.

[0301] "Optical character recognition" is a technology that extracts text information from diagnostic data in image format.

[0302] An "emotion engine" is software or a system that recognizes the emotional state from the user's input data and reflects that state.

[0303] An "artificial intelligence model" is an algorithm or technology that uses medical data to interpret diagnostic data and generate appropriate treatment options.

[0304] "Treatment options" are medical procedures or treatment options offered to the user.

[0305] A "recommended medical institution" is a medical facility identified based on diagnostic results and treatment options.

[0306] "Terminal" refers to a device such as a smartphone, PC, or smart glasses that allows users to display and operate diagnostic results and information.

[0307] The "emotional state of the user" refers to the psychological state of the user when diagnostic data is input or when the diagnostic results are presented.

[0308] This invention is a system that combines an emotion engine with a system that allows users to quickly obtain a second opinion based on a doctor's diagnosis using AI. This system uses a device such as smart glasses and functions through a series of processes including inputting diagnostic data, preprocessing, analysis, presenting results, and recommending medical institutions.

[0309] First, the user uses the smart glasses' voice recognition and camera functions to input diagnostic data in voice, text, or image format. For example, the user can provide information such as "I have high blood pressure and have been diagnosed with possible diabetes" through voice input, and also take an image of the diagnosis certificate with the camera and upload it.

[0310] The server pre-processes the input diagnostic data, converting speech to text using speech recognition and extracting key medical information using natural language processing (NLP) techniques, and extracting text information from image data using optical character recognition (OCR) techniques.

[0311] Next, the server uses an emotion engine to recognize the user's emotional state. It analyzes the user's emotions from voice input and text data to detect anxiety or relief. For example, if the user includes an expression such as "I'm worried," the emotion engine can detect anxiety.

[0312] Based on the analysis results, the server uses a generative AI model to interpret the diagnostic data and generate treatment options. This analysis includes a summary of the diagnostic results, an interpretation of the current condition, and multiple treatment options. The server adjusts the way the interpretation results and treatment options are displayed based on the user's emotional state. For example, if a user is feeling anxious, a more understandable and reassuring message will be displayed.

[0313] The generated diagnostic interpretation results and treatment options are sent from the server to the device. The device displays the results in an easy-to-understand manner through a user interface and, if necessary, provides a voice guide function. For example, the device could visually display the diagnostic results and, for anxious users, display a message such as, "Your condition is manageable. The following treatment options may be considered as next steps."

[0314] Finally, the server searches for appropriate medical institutions in the user's vicinity based on the diagnosis results and treatment options. It uses internal and external databases to identify the most suitable medical institutions and presents them on the device as a list of recommended medical institutions. Based on the information provided, the user can take action to receive appropriate medical treatment more quickly.

[0315] As a concrete example, consider the case where a user voice-inputs "I have high blood pressure and may have diabetes" into the smart glasses and provides a photo of the medical certificate. At this time, the server converts the voice to text using speech recognition technology, extracts important medical information using natural language processing, and then detects the user's anxiety using an emotion engine. The generative AI model generates treatment options such as antihypertensive medications and dietary therapy, and displays them on the smart glasses with reassuring explanations. It also simultaneously provides information recommending the nearest internal medicine doctor or specialist.

[0316] An example of a prompt sentence is, "Create a prompt sentence to provide a diagnosis and treatment options based on the health information provided by the user. For example, if the user enters, 'My blood pressure is high and I may have diabetes,' generate a diagnosis that includes treatment options such as antihypertensive medication and dietary therapy."

[0317] This system allows users to obtain a second opinion quickly and accurately, while still feeling at ease, and to choose appropriate medical action.

[0318] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0319] Step 1:

[0320] Entering diagnostic data

[0321] The user uses the voice recognition and camera functions of the smart glasses to input diagnostic data in the form of voice, text, or images. For example, the user can tell the system through voice input, "I have been diagnosed with high blood pressure and possible diabetes," and then take a picture of the diagnosis with the smart glasses' camera and upload it. The input in this case is voice data or image data.

[0322] Step 2:

[0323] Data Preprocessing

[0324] The server preprocesses the input diagnostic data. For voice data, it converts it into text using voice recognition technology and analyzes it using natural language processing (NLP) technology. Important medical information is extracted from the text-format diagnostic data. For image data, it extracts text information from the image using optical character recognition (OCR) technology. This preprocessing results in text data containing important medical information being output.

[0325] Step 3:

[0326] Recognition of emotional states

[0327] The server uses an emotion engine to recognize the user's emotional state. It analyzes the user's emotions from preprocessed text data and voice data and determines whether the user is in an emotional state such as anxiety or relief. The server generates emotional state data based on this.

[0328] Step 4:

[0329] Analyzing diagnostic data and generating treatment options

[0330] The server inputs the pre-processed diagnostic data and emotional state data into a generative AI model, which analyzes the diagnostic data based on a medical database and past diagnostic results to generate a summary of the diagnostic results, an interpretation of the current state, and treatment options. The generative AI model is used for this analysis, and the diagnostic interpretation and treatment options are obtained as outputs.

[0331] Step 5:

[0332] Emotion-based information presentation adjustment

[0333] The server adjusts the analysis results and treatment options based on the user's emotional state. Users who feel anxious are presented with reassuring messages and explanations, while users who feel at ease are presented with simple information. This results in tailored diagnostic results and treatment options.

[0334] Step 6:

[0335] Sending and displaying results

[0336] The server sends the adjusted diagnosis results and treatment options to the device, which then displays the results through a user interface (UI). Specifically, the diagnosis results and treatment options are visually displayed on the smart glasses' display, and audio guidance is provided as needed. At this time, information reflecting the user's input data is output.

[0337] Step 7:

[0338] Search for recommended medical institutions

[0339] The server searches for recommended medical institutions using internal and external databases based on the diagnosis results and treatment options, and identifies appropriate medical institutions based on the user's location information. The server generates this information as a recommendation list and sends it to the device.

[0340] Step 8:

[0341] Display of recommended medical institutions

[0342] The device will display recommended medical institutions to the user. The smart glasses display allows the user to view the nearest appropriate medical institution and its detailed information. Based on this information, the user can easily access the medical institution.

[0343] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0344] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0345] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0346] [Second embodiment]

[0347] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0348] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0349] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0350] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0351] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0352] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0353] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0354] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0355] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0356] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0357] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0358] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0359] This invention is a system that utilizes AI to enable patients to quickly obtain a second opinion based on a doctor's diagnosis. The system works through a series of processes: the user inputs diagnostic data, the server analyzes the data and provides treatment options, and finally the terminal displays the results.

[0360] Specific system functions

[0361] Data Entry Method

[0362] Users can input diagnostic data into the system using devices such as smartphones and PCs. Diagnostic data can be input by text input, voice recording, or uploading a scanned image. For example, users can input the text of the diagnostic report directly, record the diagnostic results by voice and upload them, or take an image of the diagnostic report and upload it.

[0363] Data preprocessing methods

[0364] The server preprocesses the diagnostic data received from the user. Specifically, if it is text data, it uses natural language processing (NLP) techniques to extract important medical information. If it is audio data, it uses speech recognition technology to convert the speech into text and then further analyzes the text. If it is image data, it uses optical character recognition (OCR) technology to extract text information.

[0365] AI-based diagnostic interpretation methods

[0366] The server inputs the preprocessed data into the AI ​​model. The designed AI model interprets the diagnostic data based on the medical database and past diagnostic results. This interpretation includes summarizing the diagnostic results, analyzing the current condition, and proposing multiple treatment options. For example, if a patient is at risk for high blood pressure and diabetes based on their symptoms and diagnostic results, the model will suggest specific treatment options for each.

[0367] Presentation of results

[0368] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the device, which displays the results through a user interface and provides the information in a format that is easy for the user to understand, including an interface where the user can view treatment options and a screen that displays a summary of the diagnosis.

[0369] Search methods for recommended medical institutions

[0370] The server searches for nearby appropriate medical facilities based on the diagnosis and treatment options. It uses internal and external databases to identify medical facilities that match the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[0371] Specific examples

[0372] For example, if a user inputs a diagnosis in text format such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing to extract key information about blood pressure and diabetes. The AI ​​model then interprets the diagnosis and generates treatment options such as "use of antihypertensive medication for high blood pressure" or "dietary therapy to prevent diabetes." This information is then organized and displayed in an easy-to-understand format on the user's device. Furthermore, the system recommends the nearest internal medicine or specialist medical institution based on the user's location.

[0373] As described above, the present invention is a system that enables users to easily understand diagnostic results and quickly obtain satisfactory treatment options. This system eliminates information asymmetry for users, making it easier for them to make decisions regarding appropriate treatment.

[0374] The processing flow will be explained below.

[0375] Step 1:

[0376] The user enters the diagnostic data. In this step, the user uses a device such as a smartphone or PC to enter the contents of the diagnostic report. The input method can be text input, voice recording, or uploading an image of the diagnostic report. For example, if the user takes a photo of the diagnostic report and uploads it to the system, the image will be sent.

[0377] Step 2:

[0378] The server preprocesses the diagnostic data received from the user. If it is text data, it uses natural language processing (NLP) techniques to extract important information. If it is voice data, it uses voice recognition technology to convert it into text. If it is image data, it uses optical character recognition (OCR) technology to extract text information. For example, in the case of image data, OCR technology extracts text information from the image.

[0379] Step 3:

[0380] The server inputs the preprocessed data into the AI ​​model, which then analyzes the diagnostic data based on medical databases and past diagnostic results. This analysis includes summarizing the diagnostic results, interpreting the current condition, and proposing multiple treatment options. For example, if a patient is at risk for high blood pressure and diabetes based on their symptoms and diagnostic results, the model will suggest specific treatment options for each.

[0381] Step 4:

[0382] The server organizes the interpretation results and treatment options from the AI ​​model and sends them to the device. In this step, the server formats the interpretation results in a format that is easy for the user to understand and prepares the results in a format suitable for the user interface. For example, a summary of the diagnosis results and multiple treatment options are organized in a list format.

[0383] Step 5:

[0384] The device receives the information sent from the server and displays the results to the user. In this step, the results are presented in an easy-to-understand manner through a user interface, for example, providing the user with specific treatment options or a summary of the diagnosis.

[0385] Step 6:

[0386] The server searches for recommended medical institutions based on the diagnosis and treatment options. This step uses internal and external databases to find appropriate medical institutions that match the user's location. For example, it lists nearby internal medicine clinics and specialists.

[0387] Step 7:

[0388] The terminal receives the information on recommended medical institutions sent from the server and displays it to the user. In this step, a list of recommended medical institutions is displayed through the user interface. For example, the location, medical department, contact information, etc. of the medical institution are displayed.

[0389] Example 1

[0390] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0391] In the medical field, it is difficult for patients to quickly and accurately obtain a second opinion. In particular, when handling diagnostic data in various formats, processing it is complex and time-consuming. In addition, it is currently difficult to accurately recommend appropriate treatment options and medical institutions. This has led to the problem that patients are unable to quickly obtain satisfactory treatment options.

[0392] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0393] In this invention, the server includes a means for preprocessing diagnostic data and extracting important information, a means for interpreting diagnostic results using a generative AI model and generating treatment options, and a means for presenting the interpretation results and treatment options to a user, thereby enabling the user to quickly analyze and understand diagnostic data in various formats and obtain appropriate treatment options and recommendations for medical institutions.

[0394] "User" refers to the person who uses the system to input diagnostic data and operate the terminal that displays diagnostic results and treatment options.

[0395] "Server" refers to a computing device that preprocesses diagnostic data, analyzes it using AI models, organizes the results, and searches for medical institutions.

[0396] "Terminal" refers to the device through which a user views diagnostic results, treatment options, and recommended medical institutions, including smartphones and PCs.

[0397] "Diagnostic data" refers to information including the doctor's diagnosis, and is expressed in the form of text, audio, image, or the like.

[0398] "Preprocessing" is the process of preparing diagnostic data to extract important information, and uses natural language processing, speech recognition, and optical character recognition technologies.

[0399] A "generative AI model" refers to an algorithm that uses input diagnostic data, references medical databases and past diagnostic results, and generates a summary of the diagnostic results and multiple treatment options.

[0400] "Natural language processing (NLP)" refers to the technology of extracting meaningful information from text data.

[0401] "Voice recognition technology" refers to technology that converts voice data into text data.

[0402] "Optical character recognition (OCR)" refers to the technology of extracting character information from image data.

[0403] "Diagnostic interpretation" refers to the process by which an AI model summarizes diagnostic results and suggests treatment options based on pre-processed diagnostic data.

[0404] "Treatment options" refer to treatment methods and procedures proposed based on diagnostic results.

[0405] "Recommended Medical Institution" refers to a medical institution recommended to the user based on the diagnosis and treatment options.

[0406] "Interface" refers to the screens and operating means through which a user can view and select diagnostic results and treatment options via a terminal.

[0407] This invention relates to a system that allows users to quickly obtain a second opinion based on a doctor's diagnosis using AI. The system works through a series of processes: the user inputs diagnostic data, the server analyzes the data, provides treatment options, and finally the terminal displays the results.

[0408] First, the user inputs diagnostic data using a device such as a smartphone or PC. Diagnostic data can be input by text input, voice recording, or uploading a scanned image. For example, the user can input the text of the diagnostic report directly, record the diagnostic results by voice and upload them, or take an image of the diagnostic report and upload it.

[0409] The server then preprocesses the diagnostic data received from the user. Specifically, if it is text data, it uses natural language processing (NLP) techniques to extract key medical information. If it is audio data, it uses speech recognition technology to convert the speech into text and then further analyzes the text. If it is image data, it uses optical character recognition (OCR) technology to extract text information.

[0410] The server inputs the preprocessed data into a generative AI model. The designed AI model interprets the diagnostic data based on medical databases and past diagnostic results. This interpretation includes summarizing the diagnostic results, analyzing the current condition, and proposing multiple treatment options. For example, if a patient is at risk for high blood pressure and diabetes based on their symptoms and diagnostic results, the model will suggest specific treatment options for each.

[0411] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the device, which displays the results through a user interface and provides the information in a format that is easy for the user to understand, including an interface where the user can view treatment options and a screen that displays a summary of the diagnosis.

[0412] The server then searches for appropriate nearby medical facilities based on the diagnosis and treatment options. Using internal and external databases, it identifies medical facilities based on the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[0413] For example, if a user enters a diagnosis in text format such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing to extract key information about blood pressure and diabetes. The AI ​​model then interprets the diagnosis and generates treatment options such as "use of antihypertensive medication for high blood pressure" or "dietary therapy to prevent diabetes." This information is then organized and displayed in an easy-to-understand format on the user's device. Furthermore, the system recommends the nearest internal medicine or specialist medical facility based on the user's location.

[0414] An example of a prompt is as follows:

[0415] "Analyze the following diagnostic data and provide possible diagnoses and treatment options: 'A patient has been diagnosed with high blood pressure and possible diabetes.'"

[0416] This invention is a system that enables users to easily understand diagnostic results and quickly obtain satisfactory treatment options, thereby eliminating information asymmetry and making it easier for users to make decisions regarding appropriate treatment.

[0417] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0418] Step 1: Data entry

[0419] Users enter diagnostic data using a smartphone or computer by entering text, recording a voice message, or uploading a scanned image.

[0420] Specifically, users can manually enter the text of the medical certificate, record it as audio and upload it, or take an image of the medical certificate and upload it to the system.

[0421] Input: Diagnostic data (text, audio, images)

[0422] Output: Raw diagnostic data sent to the server

[0423] Step 2: Preprocessing the data

[0424] The server preprocesses the diagnostic data received from the user: if it is text data, it uses natural language processing (NLP) techniques to extract key medical information; if it is audio data, it uses speech recognition technology to convert speech to text; and if it is image data, it uses optical character recognition (OCR) technology to extract text information.

[0425] Input: Raw diagnostic data

[0426] Output: Preprocessed diagnostic data (text with key medical information extracted)

[0427] Step 3: AI-based diagnostic interpretation

[0428] The server inputs the pre-processed data into a generative AI model, which interprets the data based on medical databases and past diagnostic results to generate a summary of the diagnosis and multiple treatment options.

[0429] For example, it identifies the risk of high blood pressure and diabetes and suggests specific treatments for each.

[0430] Input: Preprocessed diagnostic data

[0431] Output: Diagnostic interpretation and treatment options from the AI ​​model

[0432] Step 4: Presenting the results

[0433] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the terminal.

[0434] The device displays the results through a user interface, providing information in a format that is easy for the user to understand, including an interface that allows the user to easily browse treatment options and a screen that displays a summary of the diagnosis.

[0435] Input: Diagnostic interpretation results and treatment options from the AI ​​model

[0436] Output: The results displayed on the user's terminal

[0437] Step 5: Find a recommended medical institution

[0438] The server searches for nearby appropriate medical facilities based on the diagnosis and treatment options, using internal and external databases to identify and present a list of recommended medical facilities based on the user's location.

[0439] Input: Diagnosis, treatment options, user location

[0440] Output: List of recommended medical institutions

[0441] (Application example 1)

[0442] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0443] In modern medicine, it is important for patients to seek accurate and prompt second opinions, but in many cases, obtaining an appropriate opinion is difficult due to a lack of medical expertise or physical distance. Efficiently processing diagnostic data in different formats (text, audio, images) and providing optimal treatment options is also a major challenge. Furthermore, there are limited ways to provide information to patients in a visually understandable manner, resulting in delayed access to appropriate treatment.

[0444] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0445] In this invention, the server includes a means for a user to input diagnostic data, a means for the server to preprocess the diagnostic data and extract important information, and a means for the server to use a generative AI model to interpret the diagnostic results and generate prompts for generating treatment options. This enables the diagnostic data to be analyzed quickly and accurately, and optimal treatment options to be provided to the user. Furthermore, since the user can easily input diagnostic data in text, voice, or image formats, the system can accommodate different data formats and improve information accessibility. Furthermore, by using the generated prompts, the generative AI model can propose highly accurate treatment options, resulting in the rapid recommendation of the most appropriate medical institution, thereby shortening the time it takes for patients to receive appropriate treatment.

[0446] "Diagnostic Data" means information about a patient's medical condition, medical history, test results, and other information provided in text, audio, or image format.

[0447] "Server" refers to a computer system that preprocesses diagnostic data collected from users and analyzes and diagnoses it using a generative AI model.

[0448] A "generative AI model" refers to an artificial intelligence algorithm that uses large amounts of medical data to generate diagnostic results and treatment options based on input data.

[0449] A "prompt sentence" is an input sentence that prompts the generative AI model to generate an appropriate response, and is generated based on diagnostic data.

[0450] "Text" refers to information in a format consisting of strings of letters, numbers, symbols, etc.

[0451] "Voice" refers to information in the form of data that is a recording of what the user is saying.

[0452] An "image" is a data format that contains visual information and is obtained by scanning or photographing.

[0453] "Natural language processing" refers to the technology of processing and analyzing the information and structure contained in text data.

[0454] "Speech recognition" refers to the technology of converting voice data into text data.

[0455] "Optical character recognition" refers to the technology of extracting character information from image data.

[0456] "Treatment options" refer to the treatment methods and specific medical procedures proposed based on the diagnostic results.

[0457] "User interface" refers to an interactive screen or operating means for a user to operate a system.

[0458] The present invention relates to a system that quickly and accurately analyzes diagnostic data and provides appropriate treatment options. The system works through a series of processes: a user inputs diagnostic data using an application installed on a smartphone, smart glasses, a head-mounted display, or a robot, the data is preprocessed on a server, treatment options are generated using a generative AI model, and the results are finally displayed on a terminal.

[0459] Explanation of the process

[0460] 1. User data entry

[0461] Users use devices such as smartphones to input diagnostic data into the system. This data can be input as text, voice, or images. For example, a user might input information such as "I've been diagnosed with high blood pressure" in text format into an application.

[0462] 2. Data preprocessing on the server

[0463] The server preprocesses the diagnostic data received from the user. For text data, it uses natural language processing (NLP) techniques to extract key medical information. For audio data, it uses speech recognition technology to convert the speech to text, which is then processed in the same way as text data. For image data, it uses optical character recognition (OCR) technology to extract text information.

[0464] 3. Diagnostic interpretation using generative AI models

[0465] The server uses a generative AI model based on the preprocessed data to interpret the diagnosis results. At this time, a prompt is generated and input into the AI ​​model to generate the diagnosis results and treatment options. For example, a prompt might be generated such as, "You have been diagnosed with high blood pressure. You have been diagnosed as being at risk for lifestyle-related diseases."

[0466] 4. Displaying the results

[0467] The server organizes the diagnostic interpretation results and treatment options generated by the generative AI model and sends them to the device, which displays the results through a user interface and provides the information in a format that is easy for the user to understand.

[0468] 5. Search for recommended medical institutions

[0469] The server searches for appropriate nearby medical facilities based on the diagnosis and treatment options, using internal and external databases to identify medical facilities that match the user's location and present them as a recommended list.

[0470] Hardware and software used

[0471] The main technologies used in this system include natural language processing (NLP), speech recognition, optical character recognition (OCR), and generative AI models. Specifically, it uses Spacy for NLP, Google Cloud Speech API for speech recognition, Tesseract for OCR, and a Transformer-based model (e.g., rinna / japanese-gpt2-medium) for the generative AI model.

[0472] Specific examples

[0473] When a user enters text data such as "I've been diagnosed with high blood pressure," the server preprocesses the data using natural language processing technology to extract important information. Next, based on this information, the server generates a prompt statement: "I've been diagnosed with high blood pressure. I've been told I'm at risk for lifestyle-related diseases." This prompt statement is then input into a generative AI model, which retrieves optimal treatment options and hospital recommendations. Finally, the results are displayed in an easy-to-understand format on the user's device, allowing the user to quickly and appropriately select treatment.

[0474] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0475] Step 1:

[0476] The user enters diagnostic data.

[0477] Specifically, a user inputs diagnostic data into the system using a device such as a smartphone or PC. This diagnostic data can be provided in the form of text, voice, or images. For example, a user may input text data such as "I have been diagnosed with high blood pressure." The input data is then sent to the server.

[0478] Step 2:

[0479] The server pre-processes the diagnostic data.

[0480] The server pre-processes the diagnostic data received from the user, specifically using natural language processing (NLP), speech recognition, and optical character recognition (OCR) technologies.

[0481] For text data, NLP techniques (such as Spacy) are used to extract important medical information. For example, if the input is "diagnosed with high blood pressure," the keywords "blood pressure" and "diagnosis" are extracted.

[0482] For voice data, the Google Cloud Speech API is used to convert the speech to text, which is then processed in the same way as text data. If the voice input is "I've been diagnosed with high blood pressure," speech recognition will produce the same text.

[0483] For image data, Tesseract is used to extract text information from the image. If you upload an image of a medical certificate, the IMAGE TO TEXT technique will retrieve the text as well.

[0484] Step 3:

[0485] The server uses a generative AI model to interpret the diagnosis and generate prompts to generate treatment options.

[0486] Based on the preprocessed data, the server uses a generative AI model (e.g., rinna / japanese-gpt2-medium) to interpret the diagnosis results and generate prompts to generate treatment options. Specifically, the preprocessed text data is used to generate the prompt, "You have been diagnosed with high blood pressure. You have been deemed to be at risk for lifestyle-related diseases."

[0487] Step 4:

[0488] The server inputs the generated prompts into the AI ​​to generate treatment options.

[0489] The generated prompt sentences are input into a generative AI model, which then generates a diagnosis and treatment options based on the prompt sentence. For example, in response to the prompt sentence "You have been diagnosed with high blood pressure and have been told you are at risk for lifestyle-related diseases," treatment options such as "use of antihypertensive medication" and "improvement of diet" are output.

[0490] Step 5:

[0491] The server presents the interpretation results and treatment options to the user.

[0492] The server organizes the generated diagnostic interpretation results and treatment options and sends them to the user's device. The device displays them through a user interface. For example, the interpretation result may be "Your blood pressure is high. You are at risk of lifestyle-related diseases," with treatment options such as "Use antihypertensive medication and improve your diet."

[0493] Step 6:

[0494] The server searches for recommended medical institutions based on the diagnosis and treatment options.

[0495] Based on the diagnosis and treatment options, the server searches for appropriate nearby medical institutions taking into account the user's location information, using internal and external medical institution databases to identify medical institutions that meet the user's criteria.

[0496] Step 7:

[0497] The terminal displays recommended medical institutions to the user.

[0498] Information about recommended medical institutions is sent to the device, which then displays it through a user interface. For example, "Nearby Internal Medicine Clinics: XX Internal Medicine Clinic" is displayed in list format.

[0499] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0500] This invention combines an emotion engine with a system that allows users to quickly obtain a second opinion based on a doctor's diagnosis using AI. This system functions through a series of processes: the user inputs diagnostic data, the server analyzes the data, provides treatment options, and finally the device displays the results. Furthermore, the emotion engine recognizes the user's emotional state and reflects it in the analysis results and information presentation.

[0501] Specific system functions

[0502] Data Entry Method

[0503] Users can input diagnostic data into the system using devices such as smartphones and PCs. Diagnostic data can be input by text input, voice recording, or uploading a scanned image. For example, if a user takes a photo of a medical certificate and uploads it to the system, the image will be sent.

[0504] Data preprocessing methods

[0505] The server preprocesses the diagnostic data received from the user. Specifically, if it is text data, it uses natural language processing (NLP) technology to extract important medical information. If it is audio data, it uses speech recognition technology to convert the speech into text, which is then further analyzed. If it is image data, it uses optical character recognition (OCR) technology to extract text information. For example, if it is image data, OCR technology extracts text information from the image.

[0506] emotion recognition means

[0507] The server uses an emotion engine to recognize the user's emotions. It analyzes emotions from text and voice data entered by the user and obtains their emotional state. For example, it can recognize whether the user is feeling anxious from the tone of their voice and the words they use.

[0508] AI-based diagnostic interpretation methods

[0509] The server inputs the preprocessed data into the AI ​​model. The AI ​​model analyzes the diagnostic data based on a medical database and past diagnostic results. This analysis includes a summary of the diagnostic results, an interpretation of the current condition, and multiple treatment options. Furthermore, the presentation of the interpretation results and treatment options can be adjusted based on the user's emotional state. For example, if a user is feeling anxious, the AI ​​model will present information in a more understandable and reassuring manner.

[0510] Presentation of results

[0511] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the device in a format that is adjusted based on the user's emotional state. The device then displays the results through a user interface and provides information in a format that is easy for the user to understand. For example, if the user is feeling anxious, the device will explain the interpretation results more carefully and display a reassuring message.

[0512] Search methods for recommended medical institutions

[0513] The server searches for nearby appropriate medical facilities based on the diagnosis and treatment options. It uses internal and external databases to identify medical facilities that match the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[0514] Specific examples

[0515] For example, if a user inputs a diagnosis in text format, such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing to extract key information about blood pressure and diabetes. The emotion engine then evaluates the user's emotional state. Next, an AI model interprets the diagnosis and generates treatment options, such as "antihypertensive medication for high blood pressure" or "dietary therapy for diabetes prevention." If the system recognizes that the user is feeling anxious, specific and detailed explanations are added to the results to provide reassurance. This information is then organized and displayed on the user's device in an easy-to-understand format. Furthermore, the system recommends the nearest internal medicine or specialist clinic based on the user's location. This format allows users to more easily understand the diagnosis and consider treatment options with peace of mind.

[0516] The processing flow will be explained below.

[0517] Step 1:

[0518] The user inputs the diagnostic data. The user inputs the contents of the diagnostic report using a smartphone or PC. This input method includes text input, voice recording, or uploading an image of the diagnostic report. For example, if the user takes a photo of the diagnostic report and uploads it to the system, the image data is sent to the system.

[0519] Step 2:

[0520] The server preprocesses the diagnostic data received from the user. For text data, natural language processing (NLP) techniques are used to extract key medical information. For audio data, speech recognition technology is used to convert the audio into text and then analyze the text. For image data, optical character recognition (OCR) technology is used to extract text information. For example, for image data, OCR technology extracts text information from the image and organizes its content.

[0521] Step 3:

[0522] The server recognizes the user's emotions using an emotion engine. The emotion engine analyzes emotions from the text and voice data entered by the user and determines the user's emotional state. For example, it evaluates whether the user is feeling anxious based on the content of the text, choice of words, and tone of voice.

[0523] Step 4:

[0524] The server inputs the preprocessed data into the AI ​​model to generate a diagnostic interpretation. The AI ​​model analyzes the diagnostic data based on a medical database and past diagnostic results. This analysis includes a summary of the diagnostic results, an interpretation of the current condition, and suggested treatment options. Furthermore, the presentation of the interpretation results and treatment options is adjusted based on the user's emotional state. For example, if the system recognizes that the user is feeling anxious, it will change the presentation format to provide reassurance.

[0525] Step 5:

[0526] The server organizes the interpretation results and treatment options from the AI ​​model and sends them to the device. The server then formats the interpretation results to present them to the user in an easy-to-understand manner. The interpretation results and treatment options are displayed in a format suitable for the user interface. For example, a summary of the diagnosis results and multiple treatment options are organized in a list format, and a message to reassure the user is also displayed.

[0527] Step 6:

[0528] The device receives the information sent from the server and displays the results to the user. A summary of the diagnosis, treatment options, and recommended medical institutions are clearly displayed through the user interface. For example, treatment options are presented in a format that is easy for the user to understand, and additional information can be added to provide special reassurance if the user is feeling anxious.

[0529] Step 7:

[0530] The server searches for recommended medical institutions based on the diagnosis and treatment options. It uses internal and external databases to find appropriate medical institutions that match the user's location. For example, based on the user's location, it may list nearby internal medicine clinics and specialists.

[0531] Step 8:

[0532] The terminal receives the information on recommended medical institutions sent from the server and displays it to the user. A list of recommended medical institutions is displayed through the user interface, including the medical institution's location, medical specialty, contact information, etc. It is designed to make it easy for users to check the information on recommended medical institutions and make appointments if necessary, for example.

[0533] Example 2

[0534] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0535] The present invention aims to provide a system that allows users to obtain a quick and accurate second opinion without having to go to a medical institution. Another objective of the present invention is to realize a system that takes into consideration the emotional state of the user when presenting diagnostic results and provides a sense of security.

[0536] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0537] In this invention, the server includes a means for preprocessing the diagnostic data and extracting important information, a means for recognizing the user's emotions, and a means for interpreting the diagnostic results and generating treatment options using an AI model, thereby enabling the user to obtain a quick and accurate second opinion without having to go to a medical institution.

[0538] "User" refers to an individual who uses the system to input and utilize diagnostic data.

[0539] "Terminal" refers to a device, such as a smartphone or PC, through which a user inputs diagnostic data and receives the results.

[0540] "Server" refers to the central processing unit that processes data, analyzes, and transmits results for the entire system.

[0541] "Diagnostic Data" means any form of information, including text, audio, and images, entered by a user as a medical diagnosis.

[0542] "Pre-processing" refers to the initial data processing performed by the server to analyze the diagnostic data and extract important information.

[0543] "Natural language processing (NLP)" refers to the technology for analyzing text data and extracting important information.

[0544] "Speech recognition" refers to the technology for converting voice data into text.

[0545] "Optical character recognition (OCR)" refers to a technology for extracting text information from image data.

[0546] An "emotion engine" refers to technology that analyzes emotions from user input data and recognizes that emotional state.

[0547] "AI model" refers to artificial intelligence technology that analyzes diagnostic results and generates treatment options based on medical data.

[0548] "Diagnosis result" refers to a medical interpretation based on the user's diagnostic data analyzed by the AI ​​model.

[0549] "Treatment options" refer to multiple treatment methods suggested based on diagnostic results.

[0550] "Recommended medical institution" refers to an appropriate medical institution that the server searches for and suggests to the user based on the diagnosis results and treatment options.

[0551] This invention combines emotion recognition technology with a system that utilizes AI technology to enable users to quickly obtain a second opinion based on a doctor's diagnosis. The system works through a series of processes: the user inputs diagnostic data, the server analyzes the data, provides treatment options, and finally the device displays the results. Furthermore, an emotion engine is used to recognize the user's emotional state and reflect this in the analysis results and information presentation.

[0552] First, the user enters diagnostic data into the system using a device such as a smartphone or PC. This diagnostic data can be provided by text entry, voice recording, or uploading a scanned image. Specifically, the user can also take a photo of the medical certificate and upload it to the system.

[0553] The server then preprocesses the diagnostic data received from the user. For text data, natural language processing (NLP) techniques are used to extract key medical information. For audio data, speech recognition techniques are used to convert the speech to text, which can then be further analyzed. For image data, optical character recognition (OCR) techniques are used to extract text information. This converts the diagnostic data into an analyzable format.

[0554] Furthermore, the server uses an emotion engine to recognize the user's emotions. It analyzes emotions from text and voice data entered by the user and obtains their emotional state. For example, it can recognize whether the user is feeling anxious from the tone of their voice and the words they use.

[0555] The server then inputs the preprocessed data and the user's emotional state into an AI model. The AI ​​model analyzes the diagnostic data based on a medical database and past diagnostic results. This analysis includes a summary of the diagnostic results, an interpretation of the user's current condition, and multiple treatment options. The AI ​​model also adjusts the presentation of the interpretation results and treatment options based on the user's emotional state. For example, if a user is feeling anxious, the AI ​​model can present information in a more understandable and reassuring way.

[0556] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends the adjusted results to the user's device based on the user's emotional state. The device then displays the results through a user interface, providing information in a format that is easy for the user to understand. For example, if the user is feeling anxious, the device will explain the interpretation results more carefully and display a reassuring message.

[0557] The server then searches for nearby appropriate medical facilities based on the diagnosis and treatment options. Using internal and external databases, it identifies medical facilities based on the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[0558] Specific examples

[0559] For example, if a user enters a diagnosis in text format such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing technology to extract important information about blood pressure and diabetes. The emotion engine evaluates the user's emotional state. Next, the AI ​​model interprets the diagnosis and generates treatment options such as "use of antihypertensive medication for high blood pressure" or "dietary therapy to prevent diabetes." If the system recognizes that the user is feeling anxious, it provides a detailed and detailed explanation when presenting the results. This information is organized and displayed in an easy-to-understand format on the user's device. Furthermore, the system recommends the nearest internal medicine or specialist medical institution based on the user's location information.

[0560] Prompt Sentence Examples

[0561] An example of a prompt to be input to the generative AI model is as follows:

[0562] A user has entered the following text data: "The patient has been diagnosed with high blood pressure and possible diabetes." Please provide the following information:

[0563] 1. Summary of diagnostic results

[0564] 2. Diagnostic interpretation

[0565] 3. Suggest treatment options, including reassuring explanations (assuming the user is feeling anxious).

[0566] In this way, the system can comprehensively analyze the user's input data and respond in a way that takes into account the necessary information and emotions.

[0567] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0568] Step 1:

[0569] Users access the system using devices such as smartphones or PCs and enter diagnostic data by either text entry, voice recording, or uploading scanned images.

[0570] Input: Text data, audio data, or image data of the medical certificate.

[0571] Output: Diagnostic data sent to the server.

[0572] Step 2:

[0573] The server preprocesses the diagnostic data it receives. Here, it distinguishes the type of data (text, image, audio) and performs appropriate preprocessing for each.

[0574] For text data, natural language processing (NLP) techniques are used to extract key medical information.

[0575] In the case of voice data, speech recognition technology is used to convert the voice into text.

[0576] For image data, optical character recognition (OCR) techniques are used to extract text information.

[0577] Input: Diagnostic data (text, audio, images).

[0578] Output: Preprocessed text data.

[0579] Step 3:

[0580] The server sends the pre-processed data to the emotion engine to analyze the user's emotional state.

[0581] For example, it recognizes whether a user is feeling anxious from the tone of their voice and text expressions.

[0582] Input: Preprocessed text data.

[0583] Output: The user's emotional state.

[0584] Step 4:

[0585] The server inputs the preprocessed data and emotional state into a generative AI model, which analyzes the data based on medical databases and past diagnostic results to generate a summary of the diagnosis, an interpretation of the patient's current condition, and multiple treatment options. It also adjusts the way the information is presented based on the patient's emotional state.

[0586] Input: Preprocessed text data, user emotional state.

[0587] Output: Diagnostic results, treatment options.

[0588] Step 5:

[0589] The server organizes the generated diagnostic results and treatment options, adjusts them according to the user's emotional state, and sends the organized information to the user's device.

[0590] Input: diagnosis, treatment options, and the user's emotional state.

[0591] Output: Tailored diagnostic results and treatment options.

[0592] Step 6:

[0593] The terminal displays the adjusted diagnosis results and treatment options sent from the server through a user interface.

[0594] For example, if a user is feeling anxious, a reassuring message or detailed explanation can be added.

[0595] Input: Adjusted diagnostic results and treatment options.

[0596] Output: Diagnostic results and treatment options displayed to the user.

[0597] Step 7:

[0598] Based on the diagnosis and treatment options, the server uses the user's location and internal and external databases to search for appropriate medical facilities nearby.

[0599] Input: Diagnosis results, treatment options, user location.

[0600] Output: A list of recommended medical institutions.

[0601] Step 8:

[0602] The terminal displays to the user a list of recommended medical institutions sent from the server.

[0603] For example, it displays the location of medical institutions on a map and provides detailed information.

[0604] Input: A list of recommended medical institutions.

[0605] Output: A list of recommended medical institutions that is displayed to the user.

[0606] (Application example 2)

[0607] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0608] Systems that allow users to quickly obtain a second opinion based on a doctor's diagnosis often have problems, such as a poor user interface, an irrational presentation of the diagnosis, or a lack of consideration for the user's emotional state. These problems make it difficult for users to understand the diagnosis information and take appropriate action while feeling sufficiently reassured.

[0609] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input diagnostic data in voice, text, or image format, means for the server to preprocess the diagnostic data and extract important information using natural language processing, speech recognition, or optical character recognition technology, means for the server to recognize the user's emotional state using an emotion engine, means for the server to interpret the diagnostic data using an artificial intelligence model and generate treatment options, means for the server to adjust and present the interpretation results and treatment options according to the user's emotional state, means for the server to search for recommended medical institutions based on the diagnostic results and treatment options, and means for the terminal to display recommended medical institutions to the user and present the results in an easy-to-understand and reassuring manner. This allows the user to quickly and safely obtain diagnostic information and treatment options according to their emotional state.

[0610] "Diagnostic Data" refers to information entered by a user in text, audio, or image format regarding health conditions and diagnostic results.

[0611] "Natural language processing" is a technique for extracting important medical information from diagnostic data in text format.

[0612] "Speech recognition" is a technology that converts diagnostic data in voice format into text and analyzes it.

[0613] "Optical character recognition" is a technology that extracts text information from diagnostic data in image format.

[0614] An "emotion engine" is software or a system that recognizes the emotional state from the user's input data and reflects that state.

[0615] An "artificial intelligence model" is an algorithm or technology that uses medical data to interpret diagnostic data and generate appropriate treatment options.

[0616] "Treatment options" are medical procedures or treatment options offered to the user.

[0617] A "recommended medical institution" is a medical facility identified based on diagnostic results and treatment options.

[0618] "Terminal" refers to a device such as a smartphone, PC, or smart glasses that allows users to display and operate diagnostic results and information.

[0619] The "emotional state of the user" refers to the psychological state of the user when diagnostic data is input or when the diagnostic results are presented.

[0620] This invention is a system that combines an emotion engine with a system that allows users to quickly obtain a second opinion based on a doctor's diagnosis using AI. This system uses a device such as smart glasses and functions through a series of processes including inputting diagnostic data, preprocessing, analysis, presenting results, and recommending medical institutions.

[0621] First, the user uses the smart glasses' voice recognition and camera functions to input diagnostic data in voice, text, or image format. For example, the user can provide information such as "I have high blood pressure and have been diagnosed with possible diabetes" through voice input, and also take an image of the diagnosis certificate with the camera and upload it.

[0622] The server pre-processes the input diagnostic data, converting speech to text using speech recognition and extracting key medical information using natural language processing (NLP) techniques, and extracting text information from image data using optical character recognition (OCR) techniques.

[0623] Next, the server uses an emotion engine to recognize the user's emotional state. It analyzes the user's emotions from voice input and text data to detect anxiety or relief. For example, if the user includes an expression such as "I'm worried," the emotion engine can detect anxiety.

[0624] Based on the analysis results, the server uses a generative AI model to interpret the diagnostic data and generate treatment options. This analysis includes a summary of the diagnostic results, an interpretation of the current condition, and multiple treatment options. The server adjusts the way the interpretation results and treatment options are displayed based on the user's emotional state. For example, if a user is feeling anxious, a more understandable and reassuring message will be displayed.

[0625] The generated diagnostic interpretation results and treatment options are sent from the server to the device. The device displays the results in an easy-to-understand manner through a user interface and, if necessary, provides a voice guide function. For example, the device could visually display the diagnostic results and, for anxious users, display a message such as, "Your condition is manageable. The following treatment options may be considered as next steps."

[0626] Finally, the server searches for appropriate medical institutions in the user's vicinity based on the diagnosis results and treatment options. It uses internal and external databases to identify the most suitable medical institutions and presents them on the device as a list of recommended medical institutions. Based on the information provided, the user can take action to receive appropriate medical treatment more quickly.

[0627] As a concrete example, consider the case where a user voice-inputs "I have high blood pressure and may have diabetes" into the smart glasses and provides a photo of the medical certificate. At this time, the server converts the voice to text using speech recognition technology, extracts important medical information using natural language processing, and then detects the user's anxiety using an emotion engine. The generative AI model generates treatment options such as antihypertensive medications and dietary therapy, and displays them on the smart glasses with reassuring explanations. It also simultaneously provides information recommending the nearest internal medicine doctor or specialist.

[0628] An example of a prompt sentence is, "Create a prompt sentence to provide a diagnosis and treatment options based on the health information provided by the user. For example, if the user enters, 'My blood pressure is high and I may have diabetes,' generate a diagnosis that includes treatment options such as antihypertensive medication and dietary therapy."

[0629] This system allows users to obtain a second opinion quickly and accurately, while still feeling at ease, and to choose appropriate medical action.

[0630] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0631] Step 1:

[0632] Entering diagnostic data

[0633] The user uses the voice recognition and camera functions of the smart glasses to input diagnostic data in the form of voice, text, or images. For example, the user can tell the system through voice input, "I have been diagnosed with high blood pressure and possible diabetes," and then take a picture of the diagnosis with the smart glasses' camera and upload it. The input in this case is voice data or image data.

[0634] Step 2:

[0635] Data Preprocessing

[0636] The server preprocesses the input diagnostic data. For voice data, it converts it into text using voice recognition technology and analyzes it using natural language processing (NLP) technology. Important medical information is extracted from the text-format diagnostic data. For image data, it extracts text information from the image using optical character recognition (OCR) technology. This preprocessing results in text data containing important medical information being output.

[0637] Step 3:

[0638] Recognition of emotional states

[0639] The server uses an emotion engine to recognize the user's emotional state. It analyzes the user's emotions from preprocessed text data and voice data and determines whether the user is in an emotional state such as anxiety or relief. The server generates emotional state data based on this.

[0640] Step 4:

[0641] Analyzing diagnostic data and generating treatment options

[0642] The server inputs the pre-processed diagnostic data and emotional state data into a generative AI model, which analyzes the diagnostic data based on a medical database and past diagnostic results to generate a summary of the diagnostic results, an interpretation of the current state, and treatment options. The generative AI model is used for this analysis, and the diagnostic interpretation and treatment options are obtained as outputs.

[0643] Step 5:

[0644] Emotion-based information presentation adjustment

[0645] The server adjusts the analysis results and treatment options based on the user's emotional state. Users who feel anxious are presented with reassuring messages and explanations, while users who feel at ease are presented with simple information. This results in tailored diagnostic results and treatment options.

[0646] Step 6:

[0647] Sending and displaying results

[0648] The server sends the adjusted diagnosis results and treatment options to the device, which then displays the results through a user interface (UI). Specifically, the diagnosis results and treatment options are visually displayed on the smart glasses' display, and audio guidance is provided as needed. At this time, information reflecting the user's input data is output.

[0649] Step 7:

[0650] Search for recommended medical institutions

[0651] The server searches for recommended medical institutions using internal and external databases based on the diagnosis results and treatment options, and identifies appropriate medical institutions based on the user's location information. The server generates this information as a recommendation list and sends it to the device.

[0652] Step 8:

[0653] Display of recommended medical institutions

[0654] The device will display recommended medical institutions to the user. The smart glasses display allows the user to view the nearest appropriate medical institution and its detailed information. Based on this information, the user can easily access the medical institution.

[0655] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0656] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0657] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0658] [Third embodiment]

[0659] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0660] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0661] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0662] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0663] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0664] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0665] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0666] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0667] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0668] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0669] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0670] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0671] This invention is a system that utilizes AI to enable patients to quickly obtain a second opinion based on a doctor's diagnosis. The system works through a series of processes: the user inputs diagnostic data, the server analyzes the data and provides treatment options, and finally the terminal displays the results.

[0672] Specific system functions

[0673] Data Entry Method

[0674] Users can input diagnostic data into the system using devices such as smartphones and PCs. Diagnostic data can be input by text input, voice recording, or uploading a scanned image. For example, users can input the text of the diagnostic report directly, record the diagnostic results by voice and upload them, or take an image of the diagnostic report and upload it.

[0675] Data preprocessing methods

[0676] The server preprocesses the diagnostic data received from the user. Specifically, if it is text data, it uses natural language processing (NLP) techniques to extract important medical information. If it is audio data, it uses speech recognition technology to convert the speech into text and then further analyzes the text. If it is image data, it uses optical character recognition (OCR) technology to extract text information.

[0677] AI-based diagnostic interpretation methods

[0678] The server inputs the preprocessed data into the AI ​​model. The designed AI model interprets the diagnostic data based on the medical database and past diagnostic results. This interpretation includes summarizing the diagnostic results, analyzing the current condition, and proposing multiple treatment options. For example, if a patient is at risk for high blood pressure and diabetes based on their symptoms and diagnostic results, the model will suggest specific treatment options for each.

[0679] Presentation of results

[0680] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the device, which displays the results through a user interface and provides the information in a format that is easy for the user to understand, including an interface where the user can view treatment options and a screen that displays a summary of the diagnosis.

[0681] Search methods for recommended medical institutions

[0682] The server searches for nearby appropriate medical facilities based on the diagnosis and treatment options. It uses internal and external databases to identify medical facilities that match the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[0683] Specific examples

[0684] For example, if a user inputs a diagnosis in text format such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing to extract key information about blood pressure and diabetes. The AI ​​model then interprets the diagnosis and generates treatment options such as "use of antihypertensive medication for high blood pressure" or "dietary therapy to prevent diabetes." This information is then organized and displayed in an easy-to-understand format on the user's device. Furthermore, the system recommends the nearest internal medicine or specialist medical institution based on the user's location.

[0685] As described above, the present invention is a system that enables users to easily understand diagnostic results and quickly obtain satisfactory treatment options. This system eliminates information asymmetry for users, making it easier for them to make decisions regarding appropriate treatment.

[0686] The processing flow will be explained below.

[0687] Step 1:

[0688] The user enters the diagnostic data. In this step, the user uses a device such as a smartphone or PC to enter the contents of the diagnostic report. The input method can be text input, voice recording, or uploading an image of the diagnostic report. For example, if the user takes a photo of the diagnostic report and uploads it to the system, the image will be sent.

[0689] Step 2:

[0690] The server preprocesses the diagnostic data received from the user. If it is text data, it uses natural language processing (NLP) techniques to extract important information. If it is voice data, it uses voice recognition technology to convert it into text. If it is image data, it uses optical character recognition (OCR) technology to extract text information. For example, in the case of image data, OCR technology extracts text information from the image.

[0691] Step 3:

[0692] The server inputs the preprocessed data into the AI ​​model, which then analyzes the diagnostic data based on medical databases and past diagnostic results. This analysis includes summarizing the diagnostic results, interpreting the current condition, and proposing multiple treatment options. For example, if a patient is at risk for high blood pressure and diabetes based on their symptoms and diagnostic results, the model will suggest specific treatment options for each.

[0693] Step 4:

[0694] The server organizes the interpretation results and treatment options from the AI ​​model and sends them to the device. In this step, the server formats the interpretation results in a format that is easy for the user to understand and prepares the results in a format suitable for the user interface. For example, a summary of the diagnosis results and multiple treatment options are organized in a list format.

[0695] Step 5:

[0696] The device receives the information sent from the server and displays the results to the user. In this step, the results are presented in an easy-to-understand manner through a user interface, for example, providing the user with specific treatment options or a summary of the diagnosis.

[0697] Step 6:

[0698] The server searches for recommended medical institutions based on the diagnosis and treatment options. This step uses internal and external databases to find appropriate medical institutions that match the user's location. For example, it lists nearby internal medicine clinics and specialists.

[0699] Step 7:

[0700] The terminal receives the information on recommended medical institutions sent from the server and displays it to the user. In this step, a list of recommended medical institutions is displayed through the user interface. For example, the location, medical department, contact information, etc. of the medical institution are displayed.

[0701] Example 1

[0702] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0703] In the medical field, it is difficult for patients to quickly and accurately obtain a second opinion. In particular, when handling diagnostic data in various formats, processing it is complex and time-consuming. In addition, it is currently difficult to accurately recommend appropriate treatment options and medical institutions. This has led to the problem that patients are unable to quickly obtain satisfactory treatment options.

[0704] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0705] In this invention, the server includes a means for preprocessing diagnostic data and extracting important information, a means for interpreting diagnostic results using a generative AI model and generating treatment options, and a means for presenting the interpretation results and treatment options to a user, thereby enabling the user to quickly analyze and understand diagnostic data in various formats and obtain appropriate treatment options and recommendations for medical institutions.

[0706] "User" refers to the person who uses the system to input diagnostic data and operate the terminal that displays diagnostic results and treatment options.

[0707] "Server" refers to a computing device that preprocesses diagnostic data, analyzes it using AI models, organizes the results, and searches for medical institutions.

[0708] "Terminal" refers to the device through which a user views diagnostic results, treatment options, and recommended medical institutions, including smartphones and PCs.

[0709] "Diagnostic data" refers to information including the doctor's diagnosis, and is expressed in the form of text, audio, image, or the like.

[0710] "Preprocessing" is the process of preparing diagnostic data to extract important information, and uses natural language processing, speech recognition, and optical character recognition technologies.

[0711] A "generative AI model" refers to an algorithm that uses input diagnostic data, references medical databases and past diagnostic results, and generates a summary of the diagnostic results and multiple treatment options.

[0712] "Natural language processing (NLP)" refers to the technology of extracting meaningful information from text data.

[0713] "Voice recognition technology" refers to technology that converts voice data into text data.

[0714] "Optical character recognition (OCR)" refers to the technology of extracting character information from image data.

[0715] "Diagnostic interpretation" refers to the process by which an AI model summarizes diagnostic results and suggests treatment options based on pre-processed diagnostic data.

[0716] "Treatment options" refer to treatment methods and procedures proposed based on diagnostic results.

[0717] "Recommended Medical Institution" refers to a medical institution recommended to the user based on the diagnosis and treatment options.

[0718] "Interface" refers to the screens and operating means through which a user can view and select diagnostic results and treatment options via a terminal.

[0719] This invention relates to a system that allows users to quickly obtain a second opinion based on a doctor's diagnosis using AI. The system works through a series of processes: the user inputs diagnostic data, the server analyzes the data, provides treatment options, and finally the terminal displays the results.

[0720] First, the user inputs diagnostic data using a device such as a smartphone or PC. Diagnostic data can be input by text input, voice recording, or uploading a scanned image. For example, the user can input the text of the diagnostic report directly, record the diagnostic results by voice and upload them, or take an image of the diagnostic report and upload it.

[0721] The server then preprocesses the diagnostic data received from the user. Specifically, if it is text data, it uses natural language processing (NLP) techniques to extract key medical information. If it is audio data, it uses speech recognition technology to convert the speech into text and then further analyzes the text. If it is image data, it uses optical character recognition (OCR) technology to extract text information.

[0722] The server inputs the preprocessed data into a generative AI model. The designed AI model interprets the diagnostic data based on medical databases and past diagnostic results. This interpretation includes summarizing the diagnostic results, analyzing the current condition, and proposing multiple treatment options. For example, if a patient is at risk for high blood pressure and diabetes based on their symptoms and diagnostic results, the model will suggest specific treatment options for each.

[0723] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the device, which displays the results through a user interface and provides the information in a format that is easy for the user to understand, including an interface where the user can view treatment options and a screen that displays a summary of the diagnosis.

[0724] The server then searches for appropriate nearby medical facilities based on the diagnosis and treatment options. Using internal and external databases, it identifies medical facilities based on the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[0725] For example, if a user enters a diagnosis in text format such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing to extract key information about blood pressure and diabetes. The AI ​​model then interprets the diagnosis and generates treatment options such as "use of antihypertensive medication for high blood pressure" or "dietary therapy to prevent diabetes." This information is then organized and displayed in an easy-to-understand format on the user's device. Furthermore, the system recommends the nearest internal medicine or specialist medical facility based on the user's location.

[0726] An example of a prompt is as follows:

[0727] "Analyze the following diagnostic data and provide possible diagnoses and treatment options: 'A patient has been diagnosed with high blood pressure and possible diabetes.'"

[0728] This invention is a system that enables users to easily understand diagnostic results and quickly obtain satisfactory treatment options, thereby eliminating information asymmetry and making it easier for users to make decisions regarding appropriate treatment.

[0729] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0730] Step 1: Data entry

[0731] Users enter diagnostic data using a smartphone or computer by entering text, recording a voice message, or uploading a scanned image.

[0732] Specifically, users can manually enter the text of the medical certificate, record it as audio and upload it, or take an image of the medical certificate and upload it to the system.

[0733] Input: Diagnostic data (text, audio, images)

[0734] Output: Raw diagnostic data sent to the server

[0735] Step 2: Preprocessing the data

[0736] The server preprocesses the diagnostic data received from the user: if it is text data, it uses natural language processing (NLP) techniques to extract key medical information; if it is audio data, it uses speech recognition technology to convert speech to text; and if it is image data, it uses optical character recognition (OCR) technology to extract text information.

[0737] Input: Raw diagnostic data

[0738] Output: Preprocessed diagnostic data (text with key medical information extracted)

[0739] Step 3: AI-based diagnostic interpretation

[0740] The server inputs the pre-processed data into a generative AI model, which interprets the data based on medical databases and past diagnostic results to generate a summary of the diagnosis and multiple treatment options.

[0741] For example, it identifies the risk of high blood pressure and diabetes and suggests specific treatments for each.

[0742] Input: Preprocessed diagnostic data

[0743] Output: Diagnostic interpretation and treatment options from the AI ​​model

[0744] Step 4: Presenting the results

[0745] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the terminal.

[0746] The device displays the results through a user interface, providing information in a format that is easy for the user to understand, including an interface that allows the user to easily browse treatment options and a screen that displays a summary of the diagnosis.

[0747] Input: Diagnostic interpretation results and treatment options from the AI ​​model

[0748] Output: The results displayed on the user's terminal

[0749] Step 5: Find a recommended medical institution

[0750] The server searches for nearby appropriate medical facilities based on the diagnosis and treatment options, using internal and external databases to identify and present a list of recommended medical facilities based on the user's location.

[0751] Input: Diagnosis, treatment options, user location

[0752] Output: List of recommended medical institutions

[0753] (Application example 1)

[0754] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0755] In modern medicine, it is important for patients to seek accurate and prompt second opinions, but in many cases, obtaining an appropriate opinion is difficult due to a lack of medical expertise or physical distance. Efficiently processing diagnostic data in different formats (text, audio, images) and providing optimal treatment options is also a major challenge. Furthermore, there are limited ways to provide information to patients in a visually understandable manner, resulting in delayed access to appropriate treatment.

[0756] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0757] In this invention, the server includes a means for a user to input diagnostic data, a means for the server to preprocess the diagnostic data and extract important information, and a means for the server to use a generative AI model to interpret the diagnostic results and generate prompts for generating treatment options. This enables the diagnostic data to be analyzed quickly and accurately, and optimal treatment options to be provided to the user. Furthermore, since the user can easily input diagnostic data in text, voice, or image formats, the system can accommodate different data formats and improve information accessibility. Furthermore, by using the generated prompts, the generative AI model can propose highly accurate treatment options, resulting in the rapid recommendation of the most appropriate medical institution, thereby shortening the time it takes for patients to receive appropriate treatment.

[0758] "Diagnostic Data" means information about a patient's medical condition, medical history, test results, and other information provided in text, audio, or image format.

[0759] "Server" refers to a computer system that preprocesses diagnostic data collected from users and analyzes and diagnoses it using a generative AI model.

[0760] A "generative AI model" refers to an artificial intelligence algorithm that uses large amounts of medical data to generate diagnostic results and treatment options based on input data.

[0761] A "prompt sentence" is an input sentence that prompts the generative AI model to generate an appropriate response, and is generated based on diagnostic data.

[0762] "Text" refers to information in a format consisting of strings of letters, numbers, symbols, etc.

[0763] "Voice" refers to information in the form of data that is a recording of what the user is saying.

[0764] An "image" is a data format that contains visual information and is obtained by scanning or photographing.

[0765] "Natural language processing" refers to the technology of processing and analyzing the information and structure contained in text data.

[0766] "Speech recognition" refers to the technology of converting voice data into text data.

[0767] "Optical character recognition" refers to the technology of extracting character information from image data.

[0768] "Treatment options" refer to the treatment methods and specific medical procedures proposed based on the diagnostic results.

[0769] "User interface" refers to an interactive screen or operating means for a user to operate a system.

[0770] The present invention relates to a system that quickly and accurately analyzes diagnostic data and provides appropriate treatment options. The system works through a series of processes: a user inputs diagnostic data using an application installed on a smartphone, smart glasses, a head-mounted display, or a robot, the data is preprocessed on a server, treatment options are generated using a generative AI model, and the results are finally displayed on a terminal.

[0771] Explanation of the process

[0772] 1. User data entry

[0773] Users use devices such as smartphones to input diagnostic data into the system. This data can be input as text, voice, or images. For example, a user might input information such as "I've been diagnosed with high blood pressure" in text format into an application.

[0774] 2. Data preprocessing on the server

[0775] The server preprocesses the diagnostic data received from the user. For text data, it uses natural language processing (NLP) techniques to extract key medical information. For audio data, it uses speech recognition technology to convert the speech to text, which is then processed in the same way as text data. For image data, it uses optical character recognition (OCR) technology to extract text information.

[0776] 3. Diagnostic interpretation using generative AI models

[0777] The server uses a generative AI model based on the preprocessed data to interpret the diagnosis results. At this time, a prompt is generated and input into the AI ​​model to generate the diagnosis results and treatment options. For example, a prompt might be generated such as, "You have been diagnosed with high blood pressure. You have been diagnosed as being at risk for lifestyle-related diseases."

[0778] 4. Displaying the results

[0779] The server organizes the diagnostic interpretation results and treatment options generated by the generative AI model and sends them to the device, which displays the results through a user interface and provides the information in a format that is easy for the user to understand.

[0780] 5. Search for recommended medical institutions

[0781] The server searches for appropriate nearby medical facilities based on the diagnosis and treatment options, using internal and external databases to identify medical facilities that match the user's location and present them as a recommended list.

[0782] Hardware and software used

[0783] The main technologies used in this system include natural language processing (NLP), speech recognition, optical character recognition (OCR), and generative AI models. Specifically, it uses Spacy for NLP, Google Cloud Speech API for speech recognition, Tesseract for OCR, and a Transformer-based model (e.g., rinna / japanese-gpt2-medium) for the generative AI model.

[0784] Specific examples

[0785] When a user enters text data such as "I've been diagnosed with high blood pressure," the server preprocesses the data using natural language processing technology to extract important information. Next, based on this information, the server generates a prompt statement: "I've been diagnosed with high blood pressure. I've been told I'm at risk for lifestyle-related diseases." This prompt statement is then input into a generative AI model, which retrieves optimal treatment options and hospital recommendations. Finally, the results are displayed in an easy-to-understand format on the user's device, allowing the user to quickly and appropriately select treatment.

[0786] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0787] Step 1:

[0788] The user enters diagnostic data.

[0789] Specifically, a user inputs diagnostic data into the system using a device such as a smartphone or PC. This diagnostic data can be provided in the form of text, voice, or images. For example, a user may input text data such as "I have been diagnosed with high blood pressure." The input data is then sent to the server.

[0790] Step 2:

[0791] The server pre-processes the diagnostic data.

[0792] The server pre-processes the diagnostic data received from the user, specifically using natural language processing (NLP), speech recognition, and optical character recognition (OCR) technologies.

[0793] For text data, NLP techniques (such as Spacy) are used to extract important medical information. For example, if the input is "diagnosed with high blood pressure," the keywords "blood pressure" and "diagnosis" are extracted.

[0794] For voice data, the Google Cloud Speech API is used to convert the speech to text, which is then processed in the same way as text data. If the voice input is "I've been diagnosed with high blood pressure," speech recognition will produce the same text.

[0795] For image data, Tesseract is used to extract text information from the image. If you upload an image of a medical certificate, the IMAGE TO TEXT technique will retrieve the text as well.

[0796] Step 3:

[0797] The server uses a generative AI model to interpret the diagnosis and generate prompts to generate treatment options.

[0798] Based on the preprocessed data, the server uses a generative AI model (e.g., rinna / japanese-gpt2-medium) to interpret the diagnosis results and generate prompts to generate treatment options. Specifically, the preprocessed text data is used to generate the prompt, "You have been diagnosed with high blood pressure. You have been deemed to be at risk for lifestyle-related diseases."

[0799] Step 4:

[0800] The server inputs the generated prompts into the AI ​​to generate treatment options.

[0801] The generated prompt sentences are input into a generative AI model, which then generates a diagnosis and treatment options based on the prompt sentence. For example, in response to the prompt sentence "You have been diagnosed with high blood pressure and have been told you are at risk for lifestyle-related diseases," treatment options such as "use of antihypertensive medication" and "improvement of diet" are output.

[0802] Step 5:

[0803] The server presents the interpretation results and treatment options to the user.

[0804] The server organizes the generated diagnostic interpretation results and treatment options and sends them to the user's device. The device displays them through a user interface. For example, the interpretation result may be "Your blood pressure is high. You are at risk of lifestyle-related diseases," with treatment options such as "Use antihypertensive medication and improve your diet."

[0805] Step 6:

[0806] The server searches for recommended medical institutions based on the diagnosis and treatment options.

[0807] Based on the diagnosis and treatment options, the server searches for appropriate nearby medical institutions taking into account the user's location information, using internal and external medical institution databases to identify medical institutions that meet the user's criteria.

[0808] Step 7:

[0809] The terminal displays recommended medical institutions to the user.

[0810] Information about recommended medical institutions is sent to the device, which then displays it through a user interface. For example, "Nearby Internal Medicine Clinics: XX Internal Medicine Clinic" is displayed in list format.

[0811] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0812] This invention combines an emotion engine with a system that allows users to quickly obtain a second opinion based on a doctor's diagnosis using AI. This system functions through a series of processes: the user inputs diagnostic data, the server analyzes the data, provides treatment options, and finally the device displays the results. Furthermore, the emotion engine recognizes the user's emotional state and reflects it in the analysis results and information presentation.

[0813] Specific system functions

[0814] Data Entry Method

[0815] Users can input diagnostic data into the system using devices such as smartphones and PCs. Diagnostic data can be input by text input, voice recording, or uploading a scanned image. For example, if a user takes a photo of a medical certificate and uploads it to the system, the image will be sent.

[0816] Data preprocessing methods

[0817] The server preprocesses the diagnostic data received from the user. Specifically, if it is text data, it uses natural language processing (NLP) technology to extract important medical information. If it is audio data, it uses speech recognition technology to convert the speech into text, which is then further analyzed. If it is image data, it uses optical character recognition (OCR) technology to extract text information. For example, if it is image data, OCR technology extracts text information from the image.

[0818] emotion recognition means

[0819] The server uses an emotion engine to recognize the user's emotions. It analyzes emotions from text and voice data entered by the user and obtains their emotional state. For example, it can recognize whether the user is feeling anxious from the tone of their voice and the words they use.

[0820] AI-based diagnostic interpretation methods

[0821] The server inputs the preprocessed data into the AI ​​model. The AI ​​model analyzes the diagnostic data based on a medical database and past diagnostic results. This analysis includes a summary of the diagnostic results, an interpretation of the current condition, and multiple treatment options. Furthermore, the presentation of the interpretation results and treatment options can be adjusted based on the user's emotional state. For example, if a user is feeling anxious, the AI ​​model will present information in a more understandable and reassuring manner.

[0822] Presentation of results

[0823] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the device in a format that is adjusted based on the user's emotional state. The device then displays the results through a user interface and provides information in a format that is easy for the user to understand. For example, if the user is feeling anxious, the device will explain the interpretation results more carefully and display a reassuring message.

[0824] Search methods for recommended medical institutions

[0825] The server searches for nearby appropriate medical facilities based on the diagnosis and treatment options. It uses internal and external databases to identify medical facilities that match the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[0826] Specific examples

[0827] For example, if a user inputs a diagnosis in text format, such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing to extract key information about blood pressure and diabetes. The emotion engine then evaluates the user's emotional state. Next, an AI model interprets the diagnosis and generates treatment options, such as "antihypertensive medication for high blood pressure" or "dietary therapy for diabetes prevention." If the system recognizes that the user is feeling anxious, specific and detailed explanations are added to the results to provide reassurance. This information is then organized and displayed on the user's device in an easy-to-understand format. Furthermore, the system recommends the nearest internal medicine or specialist clinic based on the user's location. This format allows users to more easily understand the diagnosis and consider treatment options with peace of mind.

[0828] The processing flow will be explained below.

[0829] Step 1:

[0830] The user inputs the diagnostic data. The user inputs the contents of the diagnostic report using a smartphone or PC. This input method includes text input, voice recording, or uploading an image of the diagnostic report. For example, if the user takes a photo of the diagnostic report and uploads it to the system, the image data is sent to the system.

[0831] Step 2:

[0832] The server preprocesses the diagnostic data received from the user. For text data, natural language processing (NLP) techniques are used to extract key medical information. For audio data, speech recognition technology is used to convert the audio into text and then analyze the text. For image data, optical character recognition (OCR) technology is used to extract text information. For example, for image data, OCR technology extracts text information from the image and organizes its content.

[0833] Step 3:

[0834] The server recognizes the user's emotions using an emotion engine. The emotion engine analyzes emotions from the text and voice data entered by the user and determines the user's emotional state. For example, it evaluates whether the user is feeling anxious based on the content of the text, choice of words, and tone of voice.

[0835] Step 4:

[0836] The server inputs the preprocessed data into the AI ​​model to generate a diagnostic interpretation. The AI ​​model analyzes the diagnostic data based on a medical database and past diagnostic results. This analysis includes a summary of the diagnostic results, an interpretation of the current condition, and suggested treatment options. Furthermore, the presentation of the interpretation results and treatment options is adjusted based on the user's emotional state. For example, if the system recognizes that the user is feeling anxious, it will change the presentation format to provide reassurance.

[0837] Step 5:

[0838] The server organizes the interpretation results and treatment options from the AI ​​model and sends them to the device. The server then formats the interpretation results to present them to the user in an easy-to-understand manner. The interpretation results and treatment options are displayed in a format suitable for the user interface. For example, a summary of the diagnosis results and multiple treatment options are organized in a list format, and a message to reassure the user is also displayed.

[0839] Step 6:

[0840] The device receives the information sent from the server and displays the results to the user. A summary of the diagnosis, treatment options, and recommended medical institutions are clearly displayed through the user interface. For example, treatment options are presented in a format that is easy for the user to understand, and additional information can be added to provide special reassurance if the user is feeling anxious.

[0841] Step 7:

[0842] The server searches for recommended medical institutions based on the diagnosis and treatment options. It uses internal and external databases to find appropriate medical institutions that match the user's location. For example, based on the user's location, it may list nearby internal medicine clinics and specialists.

[0843] Step 8:

[0844] The terminal receives the information on recommended medical institutions sent from the server and displays it to the user. A list of recommended medical institutions is displayed through the user interface, including the medical institution's location, medical specialty, contact information, etc. It is designed to make it easy for users to check the information on recommended medical institutions and make appointments if necessary, for example.

[0845] Example 2

[0846] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0847] The present invention aims to provide a system that allows users to obtain a quick and accurate second opinion without having to go to a medical institution. Another objective of the present invention is to realize a system that takes into consideration the emotional state of the user when presenting diagnostic results and provides a sense of security.

[0848] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0849] In this invention, the server includes a means for preprocessing the diagnostic data and extracting important information, a means for recognizing the user's emotions, and a means for interpreting the diagnostic results and generating treatment options using an AI model, thereby enabling the user to obtain a quick and accurate second opinion without having to go to a medical institution.

[0850] "User" refers to an individual who uses the system to input and utilize diagnostic data.

[0851] "Terminal" refers to a device, such as a smartphone or PC, through which a user inputs diagnostic data and receives the results.

[0852] "Server" refers to the central processing unit that processes data, analyzes, and transmits results for the entire system.

[0853] "Diagnostic Data" means any form of information, including text, audio, and images, entered by a user as a medical diagnosis.

[0854] "Pre-processing" refers to the initial data processing performed by the server to analyze the diagnostic data and extract important information.

[0855] "Natural language processing (NLP)" refers to the technology for analyzing text data and extracting important information.

[0856] "Speech recognition" refers to the technology for converting voice data into text.

[0857] "Optical character recognition (OCR)" refers to a technology for extracting text information from image data.

[0858] An "emotion engine" refers to technology that analyzes emotions from user input data and recognizes that emotional state.

[0859] "AI model" refers to artificial intelligence technology that analyzes diagnostic results and generates treatment options based on medical data.

[0860] "Diagnosis result" refers to a medical interpretation based on the user's diagnostic data analyzed by the AI ​​model.

[0861] "Treatment options" refer to multiple treatment methods suggested based on diagnostic results.

[0862] "Recommended medical institution" refers to an appropriate medical institution that the server searches for and suggests to the user based on the diagnosis results and treatment options.

[0863] This invention combines emotion recognition technology with a system that utilizes AI technology to enable users to quickly obtain a second opinion based on a doctor's diagnosis. The system works through a series of processes: the user inputs diagnostic data, the server analyzes the data, provides treatment options, and finally the device displays the results. Furthermore, an emotion engine is used to recognize the user's emotional state and reflect this in the analysis results and information presentation.

[0864] First, the user enters diagnostic data into the system using a device such as a smartphone or PC. This diagnostic data can be provided by text entry, voice recording, or uploading a scanned image. Specifically, the user can also take a photo of the medical certificate and upload it to the system.

[0865] The server then preprocesses the diagnostic data received from the user. For text data, natural language processing (NLP) techniques are used to extract key medical information. For audio data, speech recognition techniques are used to convert the speech to text, which can then be further analyzed. For image data, optical character recognition (OCR) techniques are used to extract text information. This converts the diagnostic data into an analyzable format.

[0866] Furthermore, the server uses an emotion engine to recognize the user's emotions. It analyzes emotions from text and voice data entered by the user and obtains their emotional state. For example, it can recognize whether the user is feeling anxious from the tone of their voice and the words they use.

[0867] The server then inputs the preprocessed data and the user's emotional state into an AI model. The AI ​​model analyzes the diagnostic data based on a medical database and past diagnostic results. This analysis includes a summary of the diagnostic results, an interpretation of the user's current condition, and multiple treatment options. The AI ​​model also adjusts the presentation of the interpretation results and treatment options based on the user's emotional state. For example, if a user is feeling anxious, the AI ​​model can present information in a more understandable and reassuring way.

[0868] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends the adjusted results to the user's device based on the user's emotional state. The device then displays the results through a user interface, providing information in a format that is easy for the user to understand. For example, if the user is feeling anxious, the device will explain the interpretation results more carefully and display a reassuring message.

[0869] The server then searches for nearby appropriate medical facilities based on the diagnosis and treatment options. Using internal and external databases, it identifies medical facilities based on the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[0870] Specific examples

[0871] For example, if a user enters a diagnosis in text format such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing technology to extract important information about blood pressure and diabetes. The emotion engine evaluates the user's emotional state. Next, the AI ​​model interprets the diagnosis and generates treatment options such as "use of antihypertensive medication for high blood pressure" or "dietary therapy to prevent diabetes." If the system recognizes that the user is feeling anxious, it provides a detailed and detailed explanation when presenting the results. This information is organized and displayed in an easy-to-understand format on the user's device. Furthermore, the system recommends the nearest internal medicine or specialist medical institution based on the user's location information.

[0872] Prompt Sentence Examples

[0873] An example of a prompt to be input to the generative AI model is as follows:

[0874] A user has entered the following text data: "The patient has been diagnosed with high blood pressure and possible diabetes." Please provide the following information:

[0875] 1. Summary of diagnostic results

[0876] 2. Diagnostic interpretation

[0877] 3. Suggest treatment options, including reassuring explanations (assuming the user is feeling anxious).

[0878] In this way, the system can comprehensively analyze the user's input data and respond in a way that takes into account the necessary information and emotions.

[0879] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0880] Step 1:

[0881] Users access the system using devices such as smartphones or PCs and enter diagnostic data by either text entry, voice recording, or uploading scanned images.

[0882] Input: Text data, audio data, or image data of the medical certificate.

[0883] Output: Diagnostic data sent to the server.

[0884] Step 2:

[0885] The server preprocesses the diagnostic data it receives. Here, it distinguishes the type of data (text, image, audio) and performs appropriate preprocessing for each.

[0886] For text data, natural language processing (NLP) techniques are used to extract key medical information.

[0887] In the case of voice data, speech recognition technology is used to convert the voice into text.

[0888] For image data, optical character recognition (OCR) techniques are used to extract text information.

[0889] Input: Diagnostic data (text, audio, images).

[0890] Output: Preprocessed text data.

[0891] Step 3:

[0892] The server sends the pre-processed data to the emotion engine to analyze the user's emotional state.

[0893] For example, it recognizes whether a user is feeling anxious from the tone of their voice and text expressions.

[0894] Input: Preprocessed text data.

[0895] Output: The user's emotional state.

[0896] Step 4:

[0897] The server inputs the preprocessed data and emotional state into a generative AI model, which analyzes the data based on medical databases and past diagnostic results to generate a summary of the diagnosis, an interpretation of the patient's current condition, and multiple treatment options. It also adjusts the way the information is presented based on the patient's emotional state.

[0898] Input: Preprocessed text data, user emotional state.

[0899] Output: Diagnostic results, treatment options.

[0900] Step 5:

[0901] The server organizes the generated diagnostic results and treatment options, adjusts them according to the user's emotional state, and sends the organized information to the user's device.

[0902] Input: diagnosis, treatment options, and the user's emotional state.

[0903] Output: Tailored diagnostic results and treatment options.

[0904] Step 6:

[0905] The terminal displays the adjusted diagnosis results and treatment options sent from the server through a user interface.

[0906] For example, if a user is feeling anxious, a reassuring message or detailed explanation can be added.

[0907] Input: Adjusted diagnostic results and treatment options.

[0908] Output: Diagnostic results and treatment options displayed to the user.

[0909] Step 7:

[0910] Based on the diagnosis and treatment options, the server uses the user's location and internal and external databases to search for appropriate medical facilities nearby.

[0911] Input: Diagnosis results, treatment options, user location.

[0912] Output: A list of recommended medical institutions.

[0913] Step 8:

[0914] The terminal displays to the user a list of recommended medical institutions sent from the server.

[0915] For example, it displays the location of medical institutions on a map and provides detailed information.

[0916] Input: A list of recommended medical institutions.

[0917] Output: A list of recommended medical institutions that is displayed to the user.

[0918] (Application example 2)

[0919] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0920] Systems that allow users to quickly obtain a second opinion based on a doctor's diagnosis often have problems, such as a poor user interface, an irrational presentation of the diagnosis, or a lack of consideration for the user's emotional state. These problems make it difficult for users to understand the diagnosis information and take appropriate action while feeling sufficiently reassured.

[0921] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input diagnostic data in voice, text, or image format, means for the server to preprocess the diagnostic data and extract important information using natural language processing, speech recognition, or optical character recognition technology, means for the server to recognize the user's emotional state using an emotion engine, means for the server to interpret the diagnostic data using an artificial intelligence model and generate treatment options, means for the server to adjust and present the interpretation results and treatment options according to the user's emotional state, means for the server to search for recommended medical institutions based on the diagnostic results and treatment options, and means for the terminal to display recommended medical institutions to the user and present the results in an easy-to-understand and reassuring manner. This allows the user to quickly and safely obtain diagnostic information and treatment options according to their emotional state.

[0922] "Diagnostic Data" refers to information entered by a user in text, audio, or image format regarding health conditions and diagnostic results.

[0923] "Natural language processing" is a technique for extracting important medical information from diagnostic data in text format.

[0924] "Speech recognition" is a technology that converts diagnostic data in voice format into text and analyzes it.

[0925] "Optical character recognition" is a technology that extracts text information from diagnostic data in image format.

[0926] An "emotion engine" is software or a system that recognizes the emotional state from the user's input data and reflects that state.

[0927] An "artificial intelligence model" is an algorithm or technology that uses medical data to interpret diagnostic data and generate appropriate treatment options.

[0928] "Treatment options" are medical procedures or treatment options offered to the user.

[0929] A "recommended medical institution" is a medical facility identified based on diagnostic results and treatment options.

[0930] "Terminal" refers to a device such as a smartphone, PC, or smart glasses that allows users to display and operate diagnostic results and information.

[0931] The "emotional state of the user" refers to the psychological state of the user when diagnostic data is input or when the diagnostic results are presented.

[0932] This invention is a system that combines an emotion engine with a system that allows users to quickly obtain a second opinion based on a doctor's diagnosis using AI. This system uses a device such as smart glasses and functions through a series of processes including inputting diagnostic data, preprocessing, analysis, presenting results, and recommending medical institutions.

[0933] First, the user uses the smart glasses' voice recognition and camera functions to input diagnostic data in voice, text, or image format. For example, the user can provide information such as "I have high blood pressure and have been diagnosed with possible diabetes" through voice input, and also take an image of the diagnosis certificate with the camera and upload it.

[0934] The server pre-processes the input diagnostic data, converting speech to text using speech recognition and extracting key medical information using natural language processing (NLP) techniques, and extracting text information from image data using optical character recognition (OCR) techniques.

[0935] Next, the server uses an emotion engine to recognize the user's emotional state. It analyzes the user's emotions from voice input and text data to detect anxiety or relief. For example, if the user includes an expression such as "I'm worried," the emotion engine can detect anxiety.

[0936] Based on the analysis results, the server uses a generative AI model to interpret the diagnostic data and generate treatment options. This analysis includes a summary of the diagnostic results, an interpretation of the current condition, and multiple treatment options. The server adjusts the way the interpretation results and treatment options are displayed based on the user's emotional state. For example, if a user is feeling anxious, a more understandable and reassuring message will be displayed.

[0937] The generated diagnostic interpretation results and treatment options are sent from the server to the device. The device displays the results in an easy-to-understand manner through a user interface and, if necessary, provides a voice guide function. For example, the device could visually display the diagnostic results and, for anxious users, display a message such as, "Your condition is manageable. The following treatment options may be considered as next steps."

[0938] Finally, the server searches for appropriate medical institutions in the user's vicinity based on the diagnosis results and treatment options. It uses internal and external databases to identify the most suitable medical institutions and presents them on the device as a list of recommended medical institutions. Based on the information provided, the user can take action to receive appropriate medical treatment more quickly.

[0939] As a concrete example, consider the case where a user voice-inputs "I have high blood pressure and may have diabetes" into the smart glasses and provides a photo of the medical certificate. At this time, the server converts the voice to text using speech recognition technology, extracts important medical information using natural language processing, and then detects the user's anxiety using an emotion engine. The generative AI model generates treatment options such as antihypertensive medications and dietary therapy, and displays them on the smart glasses with reassuring explanations. It also simultaneously provides information recommending the nearest internal medicine doctor or specialist.

[0940] An example of a prompt sentence is, "Create a prompt sentence to provide a diagnosis and treatment options based on the health information provided by the user. For example, if the user enters, 'My blood pressure is high and I may have diabetes,' generate a diagnosis that includes treatment options such as antihypertensive medication and dietary therapy."

[0941] This system allows users to obtain a second opinion quickly and accurately, while still feeling at ease, and to choose appropriate medical action.

[0942] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0943] Step 1:

[0944] Entering diagnostic data

[0945] The user uses the voice recognition and camera functions of the smart glasses to input diagnostic data in the form of voice, text, or images. For example, the user can tell the system through voice input, "I have been diagnosed with high blood pressure and possible diabetes," and then take a picture of the diagnosis with the smart glasses' camera and upload it. The input in this case is voice data or image data.

[0946] Step 2:

[0947] Data Preprocessing

[0948] The server preprocesses the input diagnostic data. For voice data, it converts it into text using voice recognition technology and analyzes it using natural language processing (NLP) technology. Important medical information is extracted from the text-format diagnostic data. For image data, it extracts text information from the image using optical character recognition (OCR) technology. This preprocessing results in text data containing important medical information being output.

[0949] Step 3:

[0950] Recognition of emotional states

[0951] The server uses an emotion engine to recognize the user's emotional state. It analyzes the user's emotions from preprocessed text data and voice data and determines whether the user is in an emotional state such as anxiety or relief. The server generates emotional state data based on this.

[0952] Step 4:

[0953] Analyzing diagnostic data and generating treatment options

[0954] The server inputs the pre-processed diagnostic data and emotional state data into a generative AI model, which analyzes the diagnostic data based on a medical database and past diagnostic results to generate a summary of the diagnostic results, an interpretation of the current state, and treatment options. The generative AI model is used for this analysis, and the diagnostic interpretation and treatment options are obtained as outputs.

[0955] Step 5:

[0956] Emotion-based information presentation adjustment

[0957] The server adjusts the analysis results and treatment options based on the user's emotional state. Users who feel anxious are presented with reassuring messages and explanations, while users who feel at ease are presented with simple information. This results in tailored diagnostic results and treatment options.

[0958] Step 6:

[0959] Sending and displaying results

[0960] The server sends the adjusted diagnosis results and treatment options to the device, which then displays the results through a user interface (UI). Specifically, the diagnosis results and treatment options are visually displayed on the smart glasses' display, and audio guidance is provided as needed. At this time, information reflecting the user's input data is output.

[0961] Step 7:

[0962] Search for recommended medical institutions

[0963] The server searches for recommended medical institutions using internal and external databases based on the diagnosis results and treatment options, and identifies appropriate medical institutions based on the user's location information. The server generates this information as a recommendation list and sends it to the device.

[0964] Step 8:

[0965] Display of recommended medical institutions

[0966] The device will display recommended medical institutions to the user. The smart glasses display allows the user to view the nearest appropriate medical institution and its detailed information. Based on this information, the user can easily access the medical institution.

[0967] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0968] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0969] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0970] [Fourth embodiment]

[0971] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0972] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0973] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0974] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0975] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0976] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0977] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0978] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0979] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0980] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0981] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0982] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0983] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0984] This invention is a system that utilizes AI to enable patients to quickly obtain a second opinion based on a doctor's diagnosis. The system works through a series of processes: the user inputs diagnostic data, the server analyzes the data and provides treatment options, and finally the terminal displays the results.

[0985] Specific system functions

[0986] Data Entry Method

[0987] Users can input diagnostic data into the system using devices such as smartphones and PCs. Diagnostic data can be input by text input, voice recording, or uploading a scanned image. For example, users can input the text of the diagnostic report directly, record the diagnostic results by voice and upload them, or take an image of the diagnostic report and upload it.

[0988] Data preprocessing methods

[0989] The server preprocesses the diagnostic data received from the user. Specifically, if it is text data, it uses natural language processing (NLP) techniques to extract important medical information. If it is audio data, it uses speech recognition technology to convert the speech into text and then further analyzes the text. If it is image data, it uses optical character recognition (OCR) technology to extract text information.

[0990] AI-based diagnostic interpretation methods

[0991] The server inputs the preprocessed data into the AI ​​model. The designed AI model interprets the diagnostic data based on the medical database and past diagnostic results. This interpretation includes summarizing the diagnostic results, analyzing the current condition, and proposing multiple treatment options. For example, if a patient is at risk for high blood pressure and diabetes based on their symptoms and diagnostic results, the model will suggest specific treatment options for each.

[0992] Presentation of results

[0993] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the device, which displays the results through a user interface and provides the information in a format that is easy for the user to understand, including an interface where the user can view treatment options and a screen that displays a summary of the diagnosis.

[0994] Search methods for recommended medical institutions

[0995] The server searches for nearby appropriate medical facilities based on the diagnosis and treatment options. It uses internal and external databases to identify medical facilities that match the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[0996] Specific examples

[0997] For example, if a user inputs a diagnosis in text format such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing to extract key information about blood pressure and diabetes. The AI ​​model then interprets the diagnosis and generates treatment options such as "use of antihypertensive medication for high blood pressure" or "dietary therapy to prevent diabetes." This information is then organized and displayed in an easy-to-understand format on the user's device. Furthermore, the system recommends the nearest internal medicine or specialist medical institution based on the user's location.

[0998] As described above, the present invention is a system that enables users to easily understand diagnostic results and quickly obtain satisfactory treatment options. This system eliminates information asymmetry for users, making it easier for them to make decisions regarding appropriate treatment.

[0999] The processing flow will be explained below.

[1000] Step 1:

[1001] The user enters the diagnostic data. In this step, the user uses a device such as a smartphone or PC to enter the contents of the diagnostic report. The input method can be text input, voice recording, or uploading an image of the diagnostic report. For example, if the user takes a photo of the diagnostic report and uploads it to the system, the image will be sent.

[1002] Step 2:

[1003] The server preprocesses the diagnostic data received from the user. If it is text data, it uses natural language processing (NLP) techniques to extract important information. If it is voice data, it uses voice recognition technology to convert it into text. If it is image data, it uses optical character recognition (OCR) technology to extract text information. For example, in the case of image data, OCR technology extracts text information from the image.

[1004] Step 3:

[1005] The server inputs the preprocessed data into the AI ​​model, which then analyzes the diagnostic data based on medical databases and past diagnostic results. This analysis includes summarizing the diagnostic results, interpreting the current condition, and proposing multiple treatment options. For example, if a patient is at risk for high blood pressure and diabetes based on their symptoms and diagnostic results, the model will suggest specific treatment options for each.

[1006] Step 4:

[1007] The server organizes the interpretation results and treatment options from the AI ​​model and sends them to the device. In this step, the server formats the interpretation results in a format that is easy for the user to understand and prepares the results in a format suitable for the user interface. For example, a summary of the diagnosis results and multiple treatment options are organized in a list format.

[1008] Step 5:

[1009] The device receives the information sent from the server and displays the results to the user. In this step, the results are presented in an easy-to-understand manner through a user interface, for example, providing the user with specific treatment options or a summary of the diagnosis.

[1010] Step 6:

[1011] The server searches for recommended medical institutions based on the diagnosis and treatment options. This step uses internal and external databases to find appropriate medical institutions that match the user's location. For example, it lists nearby internal medicine clinics and specialists.

[1012] Step 7:

[1013] The terminal receives the information on recommended medical institutions sent from the server and displays it to the user. In this step, a list of recommended medical institutions is displayed through the user interface. For example, the location, medical department, contact information, etc. of the medical institution are displayed.

[1014] Example 1

[1015] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1016] In the medical field, it is difficult for patients to quickly and accurately obtain a second opinion. In particular, when handling diagnostic data in various formats, processing it is complex and time-consuming. In addition, it is currently difficult to accurately recommend appropriate treatment options and medical institutions. This has led to the problem that patients are unable to quickly obtain satisfactory treatment options.

[1017] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1018] In this invention, the server includes a means for preprocessing diagnostic data and extracting important information, a means for interpreting diagnostic results using a generative AI model and generating treatment options, and a means for presenting the interpretation results and treatment options to a user, thereby enabling the user to quickly analyze and understand diagnostic data in various formats and obtain appropriate treatment options and recommendations for medical institutions.

[1019] "User" refers to the person who uses the system to input diagnostic data and operate the terminal that displays diagnostic results and treatment options.

[1020] "Server" refers to a computing device that preprocesses diagnostic data, analyzes it using AI models, organizes the results, and searches for medical institutions.

[1021] "Terminal" refers to the device through which a user views diagnostic results, treatment options, and recommended medical institutions, including smartphones and PCs.

[1022] "Diagnostic data" refers to information including the doctor's diagnosis, and is expressed in the form of text, audio, image, or the like.

[1023] "Preprocessing" is the process of preparing diagnostic data to extract important information, and uses natural language processing, speech recognition, and optical character recognition technologies.

[1024] A "generative AI model" refers to an algorithm that uses input diagnostic data, references medical databases and past diagnostic results, and generates a summary of the diagnostic results and multiple treatment options.

[1025] "Natural language processing (NLP)" refers to the technology of extracting meaningful information from text data.

[1026] "Voice recognition technology" refers to technology that converts voice data into text data.

[1027] "Optical character recognition (OCR)" refers to the technology of extracting character information from image data.

[1028] "Diagnostic interpretation" refers to the process by which an AI model summarizes diagnostic results and suggests treatment options based on pre-processed diagnostic data.

[1029] "Treatment options" refer to treatment methods and procedures proposed based on diagnostic results.

[1030] "Recommended Medical Institution" refers to a medical institution recommended to the user based on the diagnosis and treatment options.

[1031] "Interface" refers to the screens and operating means through which a user can view and select diagnostic results and treatment options via a terminal.

[1032] This invention relates to a system that allows users to quickly obtain a second opinion based on a doctor's diagnosis using AI. The system works through a series of processes: the user inputs diagnostic data, the server analyzes the data, provides treatment options, and finally the terminal displays the results.

[1033] First, the user inputs diagnostic data using a device such as a smartphone or PC. Diagnostic data can be input by text input, voice recording, or uploading a scanned image. For example, the user can input the text of the diagnostic report directly, record the diagnostic results by voice and upload them, or take an image of the diagnostic report and upload it.

[1034] The server then preprocesses the diagnostic data received from the user. Specifically, if it is text data, it uses natural language processing (NLP) techniques to extract key medical information. If it is audio data, it uses speech recognition technology to convert the speech into text and then further analyzes the text. If it is image data, it uses optical character recognition (OCR) technology to extract text information.

[1035] The server inputs the preprocessed data into a generative AI model. The designed AI model interprets the diagnostic data based on medical databases and past diagnostic results. This interpretation includes summarizing the diagnostic results, analyzing the current condition, and proposing multiple treatment options. For example, if a patient is at risk for high blood pressure and diabetes based on their symptoms and diagnostic results, the model will suggest specific treatment options for each.

[1036] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the device, which displays the results through a user interface and provides the information in a format that is easy for the user to understand, including an interface where the user can view treatment options and a screen that displays a summary of the diagnosis.

[1037] The server then searches for appropriate nearby medical facilities based on the diagnosis and treatment options. Using internal and external databases, it identifies medical facilities based on the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[1038] For example, if a user enters a diagnosis in text format such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing to extract key information about blood pressure and diabetes. The AI ​​model then interprets the diagnosis and generates treatment options such as "use of antihypertensive medication for high blood pressure" or "dietary therapy to prevent diabetes." This information is then organized and displayed in an easy-to-understand format on the user's device. Furthermore, the system recommends the nearest internal medicine or specialist medical facility based on the user's location.

[1039] An example of a prompt is as follows:

[1040] "Analyze the following diagnostic data and provide possible diagnoses and treatment options: 'A patient has been diagnosed with high blood pressure and possible diabetes.'"

[1041] This invention is a system that enables users to easily understand diagnostic results and quickly obtain satisfactory treatment options, thereby eliminating information asymmetry and making it easier for users to make decisions regarding appropriate treatment.

[1042] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1043] Step 1: Data entry

[1044] Users enter diagnostic data using a smartphone or computer by entering text, recording a voice message, or uploading a scanned image.

[1045] Specifically, users can manually enter the text of the medical certificate, record it as audio and upload it, or take an image of the medical certificate and upload it to the system.

[1046] Input: Diagnostic data (text, audio, images)

[1047] Output: Raw diagnostic data sent to the server

[1048] Step 2: Preprocessing the data

[1049] The server preprocesses the diagnostic data received from the user: if it is text data, it uses natural language processing (NLP) techniques to extract key medical information; if it is audio data, it uses speech recognition technology to convert speech to text; and if it is image data, it uses optical character recognition (OCR) technology to extract text information.

[1050] Input: Raw diagnostic data

[1051] Output: Preprocessed diagnostic data (text with key medical information extracted)

[1052] Step 3: AI-based diagnostic interpretation

[1053] The server inputs the pre-processed data into a generative AI model, which interprets the data based on medical databases and past diagnostic results to generate a summary of the diagnosis and multiple treatment options.

[1054] For example, it identifies the risk of high blood pressure and diabetes and suggests specific treatments for each.

[1055] Input: Preprocessed diagnostic data

[1056] Output: Diagnostic interpretation and treatment options from the AI ​​model

[1057] Step 4: Presenting the results

[1058] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the terminal.

[1059] The device displays the results through a user interface, providing information in a format that is easy for the user to understand, including an interface that allows the user to easily browse treatment options and a screen that displays a summary of the diagnosis.

[1060] Input: Diagnostic interpretation results and treatment options from the AI ​​model

[1061] Output: The results displayed on the user's terminal

[1062] Step 5: Find a recommended medical institution

[1063] The server searches for nearby appropriate medical facilities based on the diagnosis and treatment options, using internal and external databases to identify and present a list of recommended medical facilities based on the user's location.

[1064] Input: Diagnosis, treatment options, user location

[1065] Output: List of recommended medical institutions

[1066] (Application example 1)

[1067] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1068] In modern medicine, it is important for patients to seek accurate and prompt second opinions, but in many cases, obtaining an appropriate opinion is difficult due to a lack of medical expertise or physical distance. Efficiently processing diagnostic data in different formats (text, audio, images) and providing optimal treatment options is also a major challenge. Furthermore, there are limited ways to provide information to patients in a visually understandable manner, resulting in delayed access to appropriate treatment.

[1069] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1070] In this invention, the server includes a means for a user to input diagnostic data, a means for the server to preprocess the diagnostic data and extract important information, and a means for the server to use a generative AI model to interpret the diagnostic results and generate prompts for generating treatment options. This enables the diagnostic data to be analyzed quickly and accurately, and optimal treatment options to be provided to the user. Furthermore, since the user can easily input diagnostic data in text, voice, or image formats, the system can accommodate different data formats and improve information accessibility. Furthermore, by using the generated prompts, the generative AI model can propose highly accurate treatment options, resulting in the rapid recommendation of the most appropriate medical institution, thereby shortening the time it takes for patients to receive appropriate treatment.

[1071] "Diagnostic Data" means information about a patient's medical condition, medical history, test results, and other information provided in text, audio, or image format.

[1072] "Server" refers to a computer system that preprocesses diagnostic data collected from users and analyzes and diagnoses it using a generative AI model.

[1073] A "generative AI model" refers to an artificial intelligence algorithm that uses large amounts of medical data to generate diagnostic results and treatment options based on input data.

[1074] A "prompt sentence" is an input sentence that prompts the generative AI model to generate an appropriate response, and is generated based on diagnostic data.

[1075] "Text" refers to information in a format consisting of strings of letters, numbers, symbols, etc.

[1076] "Voice" refers to information in the form of data that is a recording of what the user is saying.

[1077] An "image" is a data format that contains visual information and is obtained by scanning or photographing.

[1078] "Natural language processing" refers to the technology of processing and analyzing the information and structure contained in text data.

[1079] "Speech recognition" refers to the technology of converting voice data into text data.

[1080] "Optical character recognition" refers to the technology of extracting character information from image data.

[1081] "Treatment options" refer to the treatment methods and specific medical procedures proposed based on the diagnostic results.

[1082] "User interface" refers to an interactive screen or operating means for a user to operate a system.

[1083] The present invention relates to a system that quickly and accurately analyzes diagnostic data and provides appropriate treatment options. The system works through a series of processes: a user inputs diagnostic data using an application installed on a smartphone, smart glasses, a head-mounted display, or a robot, the data is preprocessed on a server, treatment options are generated using a generative AI model, and the results are finally displayed on a terminal.

[1084] Explanation of the process

[1085] 1. User data entry

[1086] Users use devices such as smartphones to input diagnostic data into the system. This data can be input as text, voice, or images. For example, a user might input information such as "I've been diagnosed with high blood pressure" in text format into an application.

[1087] 2. Data preprocessing on the server

[1088] The server preprocesses the diagnostic data received from the user. For text data, it uses natural language processing (NLP) techniques to extract key medical information. For audio data, it uses speech recognition technology to convert the speech to text, which is then processed in the same way as text data. For image data, it uses optical character recognition (OCR) technology to extract text information.

[1089] 3. Diagnostic interpretation using generative AI models

[1090] The server uses a generative AI model based on the preprocessed data to interpret the diagnosis results. At this time, a prompt is generated and input into the AI ​​model to generate the diagnosis results and treatment options. For example, a prompt might be generated such as, "You have been diagnosed with high blood pressure. You have been diagnosed as being at risk for lifestyle-related diseases."

[1091] 4. Displaying the results

[1092] The server organizes the diagnostic interpretation results and treatment options generated by the generative AI model and sends them to the device, which displays the results through a user interface and provides the information in a format that is easy for the user to understand.

[1093] 5. Search for recommended medical institutions

[1094] The server searches for appropriate nearby medical facilities based on the diagnosis and treatment options, using internal and external databases to identify medical facilities that match the user's location and present them as a recommended list.

[1095] Hardware and software used

[1096] The main technologies used in this system include natural language processing (NLP), speech recognition, optical character recognition (OCR), and generative AI models. Specifically, it uses Spacy for NLP, Google Cloud Speech API for speech recognition, Tesseract for OCR, and a Transformer-based model (e.g., rinna / japanese-gpt2-medium) for the generative AI model.

[1097] Specific examples

[1098] When a user enters text data such as "I've been diagnosed with high blood pressure," the server preprocesses the data using natural language processing technology to extract important information. Next, based on this information, the server generates a prompt statement: "I've been diagnosed with high blood pressure. I've been told I'm at risk for lifestyle-related diseases." This prompt statement is then input into a generative AI model, which retrieves optimal treatment options and hospital recommendations. Finally, the results are displayed in an easy-to-understand format on the user's device, allowing the user to quickly and appropriately select treatment.

[1099] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1100] Step 1:

[1101] The user enters diagnostic data.

[1102] Specifically, a user inputs diagnostic data into the system using a device such as a smartphone or PC. This diagnostic data can be provided in the form of text, voice, or images. For example, a user may input text data such as "I have been diagnosed with high blood pressure." The input data is then sent to the server.

[1103] Step 2:

[1104] The server pre-processes the diagnostic data.

[1105] The server pre-processes the diagnostic data received from the user, specifically using natural language processing (NLP), speech recognition, and optical character recognition (OCR) technologies.

[1106] For text data, NLP techniques (such as Spacy) are used to extract important medical information. For example, if the input is "diagnosed with high blood pressure," the keywords "blood pressure" and "diagnosis" are extracted.

[1107] For voice data, the Google Cloud Speech API is used to convert the speech to text, which is then processed in the same way as text data. If the voice input is "I've been diagnosed with high blood pressure," speech recognition will produce the same text.

[1108] For image data, Tesseract is used to extract text information from the image. If you upload an image of a medical certificate, the IMAGE TO TEXT technique will retrieve the text as well.

[1109] Step 3:

[1110] The server uses a generative AI model to interpret the diagnosis and generate prompts to generate treatment options.

[1111] Based on the preprocessed data, the server uses a generative AI model (e.g., rinna / japanese-gpt2-medium) to interpret the diagnosis results and generate prompts to generate treatment options. Specifically, the preprocessed text data is used to generate the prompt, "You have been diagnosed with high blood pressure. You have been deemed to be at risk for lifestyle-related diseases."

[1112] Step 4:

[1113] The server inputs the generated prompts into the AI ​​to generate treatment options.

[1114] The generated prompt sentences are input into a generative AI model, which then generates a diagnosis and treatment options based on the prompt sentence. For example, in response to the prompt sentence "You have been diagnosed with high blood pressure and have been told you are at risk for lifestyle-related diseases," treatment options such as "use of antihypertensive medication" and "improvement of diet" are output.

[1115] Step 5:

[1116] The server presents the interpretation results and treatment options to the user.

[1117] The server organizes the generated diagnostic interpretation results and treatment options and sends them to the user's device. The device displays them through a user interface. For example, the interpretation result may be "Your blood pressure is high. You are at risk of lifestyle-related diseases," with treatment options such as "Use antihypertensive medication and improve your diet."

[1118] Step 6:

[1119] The server searches for recommended medical institutions based on the diagnosis and treatment options.

[1120] Based on the diagnosis and treatment options, the server searches for appropriate nearby medical institutions taking into account the user's location information, using internal and external medical institution databases to identify medical institutions that meet the user's criteria.

[1121] Step 7:

[1122] The terminal displays recommended medical institutions to the user.

[1123] Information about recommended medical institutions is sent to the device, which then displays it through a user interface. For example, "Nearby Internal Medicine Clinics: XX Internal Medicine Clinic" is displayed in list format.

[1124] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1125] This invention combines an emotion engine with a system that allows users to quickly obtain a second opinion based on a doctor's diagnosis using AI. This system functions through a series of processes: the user inputs diagnostic data, the server analyzes the data, provides treatment options, and finally the device displays the results. Furthermore, the emotion engine recognizes the user's emotional state and reflects it in the analysis results and information presentation.

[1126] Specific system functions

[1127] Data Entry Method

[1128] Users can input diagnostic data into the system using devices such as smartphones and PCs. Diagnostic data can be input by text input, voice recording, or uploading a scanned image. For example, if a user takes a photo of a medical certificate and uploads it to the system, the image will be sent.

[1129] Data preprocessing methods

[1130] The server preprocesses the diagnostic data received from the user. Specifically, if it is text data, it uses natural language processing (NLP) technology to extract important medical information. If it is audio data, it uses speech recognition technology to convert the speech into text, which is then further analyzed. If it is image data, it uses optical character recognition (OCR) technology to extract text information. For example, if it is image data, OCR technology extracts text information from the image.

[1131] emotion recognition means

[1132] The server uses an emotion engine to recognize the user's emotions. It analyzes emotions from text and voice data entered by the user and obtains their emotional state. For example, it can recognize whether the user is feeling anxious from the tone of their voice and the words they use.

[1133] AI-based diagnostic interpretation methods

[1134] The server inputs the preprocessed data into the AI ​​model. The AI ​​model analyzes the diagnostic data based on a medical database and past diagnostic results. This analysis includes a summary of the diagnostic results, an interpretation of the current condition, and multiple treatment options. Furthermore, the presentation of the interpretation results and treatment options can be adjusted based on the user's emotional state. For example, if a user is feeling anxious, the AI ​​model will present information in a more understandable and reassuring manner.

[1135] Presentation of results

[1136] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends them to the device in a format that is adjusted based on the user's emotional state. The device then displays the results through a user interface and provides information in a format that is easy for the user to understand. For example, if the user is feeling anxious, the device will explain the interpretation results more carefully and display a reassuring message.

[1137] Search methods for recommended medical institutions

[1138] The server searches for nearby appropriate medical facilities based on the diagnosis and treatment options. It uses internal and external databases to identify medical facilities that match the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[1139] Specific examples

[1140] For example, if a user inputs a diagnosis in text format, such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing to extract key information about blood pressure and diabetes. The emotion engine then evaluates the user's emotional state. Next, an AI model interprets the diagnosis and generates treatment options, such as "antihypertensive medication for high blood pressure" or "dietary therapy for diabetes prevention." If the system recognizes that the user is feeling anxious, specific and detailed explanations are added to the results to provide reassurance. This information is then organized and displayed on the user's device in an easy-to-understand format. Furthermore, the system recommends the nearest internal medicine or specialist clinic based on the user's location. This format allows users to more easily understand the diagnosis and consider treatment options with peace of mind.

[1141] The processing flow will be explained below.

[1142] Step 1:

[1143] The user inputs the diagnostic data. The user inputs the contents of the diagnostic report using a smartphone or PC. This input method includes text input, voice recording, or uploading an image of the diagnostic report. For example, if the user takes a photo of the diagnostic report and uploads it to the system, the image data is sent to the system.

[1144] Step 2:

[1145] The server preprocesses the diagnostic data received from the user. For text data, natural language processing (NLP) techniques are used to extract key medical information. For audio data, speech recognition technology is used to convert the audio into text and then analyze the text. For image data, optical character recognition (OCR) technology is used to extract text information. For example, for image data, OCR technology extracts text information from the image and organizes its content.

[1146] Step 3:

[1147] The server recognizes the user's emotions using an emotion engine. The emotion engine analyzes emotions from the text and voice data entered by the user and determines the user's emotional state. For example, it evaluates whether the user is feeling anxious based on the content of the text, choice of words, and tone of voice.

[1148] Step 4:

[1149] The server inputs the preprocessed data into the AI ​​model to generate a diagnostic interpretation. The AI ​​model analyzes the diagnostic data based on a medical database and past diagnostic results. This analysis includes a summary of the diagnostic results, an interpretation of the current condition, and suggested treatment options. Furthermore, the presentation of the interpretation results and treatment options is adjusted based on the user's emotional state. For example, if the system recognizes that the user is feeling anxious, it will change the presentation format to provide reassurance.

[1150] Step 5:

[1151] The server organizes the interpretation results and treatment options from the AI ​​model and sends them to the device. The server then formats the interpretation results to present them to the user in an easy-to-understand manner. The interpretation results and treatment options are displayed in a format suitable for the user interface. For example, a summary of the diagnosis results and multiple treatment options are organized in a list format, and a message to reassure the user is also displayed.

[1152] Step 6:

[1153] The device receives the information sent from the server and displays the results to the user. The user interface clearly displays a summary of the diagnosis, treatment options, and recommended medical institutions. For example, treatment options are presented in a format that is easy for the user to understand, and additional information can be added to provide special reassurance if the user is feeling anxious.

[1154] Step 7:

[1155] The server searches for recommended medical institutions based on the diagnosis and treatment options. It uses internal and external databases to find appropriate medical institutions that match the user's location. For example, based on the user's location, it may list nearby internal medicine clinics and specialists.

[1156] Step 8:

[1157] The terminal receives the information on recommended medical institutions sent from the server and displays it to the user. A list of recommended medical institutions is displayed through the user interface, including the medical institution's location, medical specialty, contact information, etc. It is designed to make it easy for users to check the information on recommended medical institutions and make appointments if necessary, for example.

[1158] Example 2

[1159] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1160] The present invention aims to provide a system that allows users to obtain a quick and accurate second opinion without having to go to a medical institution. Another objective of the present invention is to realize a system that takes into consideration the emotional state of the user when presenting diagnostic results and provides a sense of security.

[1161] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1162] In this invention, the server includes a means for preprocessing the diagnostic data and extracting important information, a means for recognizing the user's emotions, and a means for interpreting the diagnostic results and generating treatment options using an AI model, thereby enabling the user to obtain a quick and accurate second opinion without having to go to a medical institution.

[1163] "User" refers to an individual who uses the system to input and utilize diagnostic data.

[1164] "Terminal" refers to a device, such as a smartphone or PC, through which a user inputs diagnostic data and receives the results.

[1165] "Server" refers to the central processing unit that processes data, analyzes, and transmits results for the entire system.

[1166] "Diagnostic Data" means any form of information, including text, audio, and images, entered by a user as a medical diagnosis.

[1167] "Pre-processing" refers to the initial data processing performed by the server to analyze the diagnostic data and extract important information.

[1168] "Natural language processing (NLP)" refers to the technology for analyzing text data and extracting important information.

[1169] "Speech recognition" refers to the technology for converting voice data into text.

[1170] "Optical character recognition (OCR)" refers to a technology for extracting text information from image data.

[1171] An "emotion engine" refers to technology that analyzes emotions from user input data and recognizes that emotional state.

[1172] "AI model" refers to artificial intelligence technology that analyzes diagnostic results and generates treatment options based on medical data.

[1173] "Diagnosis result" refers to a medical interpretation based on the user's diagnostic data analyzed by the AI ​​model.

[1174] "Treatment options" refer to multiple treatment methods suggested based on diagnostic results.

[1175] "Recommended medical institution" refers to an appropriate medical institution that the server searches for and suggests to the user based on the diagnosis results and treatment options.

[1176] This invention combines emotion recognition technology with a system that utilizes AI technology to enable users to quickly obtain a second opinion based on a doctor's diagnosis. The system works through a series of processes: the user inputs diagnostic data, the server analyzes the data, provides treatment options, and finally the device displays the results. Furthermore, an emotion engine is used to recognize the user's emotional state and reflect this in the analysis results and information presentation.

[1177] First, the user enters diagnostic data into the system using a device such as a smartphone or PC. This diagnostic data can be provided by text entry, voice recording, or uploading a scanned image. Specifically, the user can also take a photo of the medical certificate and upload it to the system.

[1178] The server then preprocesses the diagnostic data received from the user. For text data, natural language processing (NLP) techniques are used to extract key medical information. For audio data, speech recognition techniques are used to convert the speech to text, which can then be further analyzed. For image data, optical character recognition (OCR) techniques are used to extract text information. This converts the diagnostic data into an analyzable format.

[1179] Furthermore, the server uses an emotion engine to recognize the user's emotions. It analyzes emotions from text and voice data entered by the user and obtains their emotional state. For example, it can recognize whether the user is feeling anxious from the tone of their voice and the words they use.

[1180] The server then inputs the preprocessed data and the user's emotional state into an AI model. The AI ​​model analyzes the diagnostic data based on a medical database and past diagnostic results. This analysis includes a summary of the diagnostic results, an interpretation of the user's current condition, and multiple treatment options. The AI ​​model also adjusts the presentation of the interpretation results and treatment options based on the user's emotional state. For example, if a user is feeling anxious, the AI ​​model can present information in a more understandable and reassuring way.

[1181] The server organizes the diagnostic interpretation results and treatment options generated by the AI ​​model and sends the adjusted results to the user's device based on the user's emotional state. The device then displays the results through a user interface, providing information in a format that is easy for the user to understand. For example, if the user is feeling anxious, the device will explain the interpretation results more carefully and display a reassuring message.

[1182] The server then searches for nearby appropriate medical facilities based on the diagnosis and treatment options. Using internal and external databases, it identifies medical facilities based on the user's location and presents them as a recommended list. This process allows users to quickly find the right medical facility.

[1183] Specific examples

[1184] For example, if a user enters a diagnosis in text format such as "The patient has high blood pressure and has been diagnosed with possible diabetes," the server uses natural language processing technology to extract important information about blood pressure and diabetes. The emotion engine evaluates the user's emotional state. Next, the AI ​​model interprets the diagnosis and generates treatment options such as "use of antihypertensive medication for high blood pressure" or "dietary therapy to prevent diabetes." If the system recognizes that the user is feeling anxious, it provides a detailed and detailed explanation when presenting the results. This information is organized and displayed in an easy-to-understand format on the user's device. Furthermore, the system recommends the nearest internal medicine or specialist medical institution based on the user's location information.

[1185] Prompt Sentence Examples

[1186] An example of a prompt to be input to the generative AI model is as follows:

[1187] A user has entered the following text data: "The patient has been diagnosed with high blood pressure and possible diabetes." Please provide the following information:

[1188] 1. Summary of diagnostic results

[1189] 2. Diagnostic interpretation

[1190] 3. Suggest treatment options, including reassuring explanations (assuming the user is feeling anxious).

[1191] In this way, the system can comprehensively analyze the user's input data and respond in a way that takes into account the necessary information and emotions.

[1192] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1193] Step 1:

[1194] Users access the system using devices such as smartphones or PCs and enter diagnostic data by either text entry, voice recording, or uploading scanned images.

[1195] Input: Text data, audio data, or image data of the medical certificate.

[1196] Output: Diagnostic data sent to the server.

[1197] Step 2:

[1198] The server preprocesses the diagnostic data it receives. Here, it distinguishes the type of data (text, image, audio) and performs appropriate preprocessing for each.

[1199] For text data, natural language processing (NLP) techniques are used to extract key medical information.

[1200] In the case of voice data, speech recognition technology is used to convert the voice into text.

[1201] For image data, optical character recognition (OCR) techniques are used to extract text information.

[1202] Input: Diagnostic data (text, audio, images).

[1203] Output: Preprocessed text data.

[1204] Step 3:

[1205] The server sends the pre-processed data to the emotion engine to analyze the user's emotional state.

[1206] For example, it recognizes whether a user is feeling anxious from the tone of their voice and text expressions.

[1207] Input: Preprocessed text data.

[1208] Output: The user's emotional state.

[1209] Step 4:

[1210] The server inputs the preprocessed data and emotional state into a generative AI model, which analyzes the data based on medical databases and past diagnostic results to generate a summary of the diagnosis, an interpretation of the patient's current condition, and multiple treatment options. It also adjusts the way the information is presented based on the patient's emotional state.

[1211] Input: Preprocessed text data, user emotional state.

[1212] Output: Diagnostic results, treatment options.

[1213] Step 5:

[1214] The server organizes the generated diagnostic results and treatment options, adjusts them according to the user's emotional state, and sends the organized information to the user's device.

[1215] Input: diagnosis, treatment options, and the user's emotional state.

[1216] Output: Tailored diagnostic results and treatment options.

[1217] Step 6:

[1218] The terminal displays the adjusted diagnosis results and treatment options sent from the server through a user interface.

[1219] For example, if a user is feeling anxious, a reassuring message or detailed explanation can be added.

[1220] Input: Adjusted diagnostic results and treatment options.

[1221] Output: Diagnostic results and treatment options displayed to the user.

[1222] Step 7:

[1223] Based on the diagnosis and treatment options, the server uses the user's location and internal and external databases to search for appropriate medical facilities nearby.

[1224] Input: Diagnosis results, treatment options, user location.

[1225] Output: A list of recommended medical institutions.

[1226] Step 8:

[1227] The terminal displays to the user a list of recommended medical institutions sent from the server.

[1228] For example, it displays the location of medical institutions on a map and provides detailed information.

[1229] Input: A list of recommended medical institutions.

[1230] Output: A list of recommended medical institutions that is displayed to the user.

[1231] (Application example 2)

[1232] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1233] Systems that allow users to quickly obtain a second opinion based on a doctor's diagnosis often have problems, such as a poor user interface, an irrational presentation of the diagnosis, or a lack of consideration for the user's emotional state. These problems make it difficult for users to understand the diagnosis information and take appropriate action while feeling sufficiently reassured.

[1234] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input diagnostic data in voice, text, or image format, means for the server to preprocess the diagnostic data and extract important information using natural language processing, speech recognition, or optical character recognition technology, means for the server to recognize the user's emotional state using an emotion engine, means for the server to interpret the diagnostic data using an artificial intelligence model and generate treatment options, means for the server to adjust and present the interpretation results and treatment options according to the user's emotional state, means for the server to search for recommended medical institutions based on the diagnostic results and treatment options, and means for the terminal to display recommended medical institutions to the user and present the results in an easy-to-understand and reassuring manner. This allows the user to quickly and safely obtain diagnostic information and treatment options according to their emotional state.

[1235] "Diagnostic Data" refers to information entered by a user in text, audio, or image format regarding health conditions and diagnostic results.

[1236] "Natural language processing" is a technique for extracting important medical information from diagnostic data in text format.

[1237] "Speech recognition" is a technology that converts diagnostic data in voice format into text and analyzes it.

[1238] "Optical character recognition" is a technology that extracts text information from diagnostic data in image format.

[1239] An "emotion engine" is software or a system that recognizes the emotional state from the user's input data and reflects that state.

[1240] An "artificial intelligence model" is an algorithm or technology that uses medical data to interpret diagnostic data and generate appropriate treatment options.

[1241] "Treatment options" are medical procedures or treatment options offered to the user.

[1242] A "recommended medical institution" is a medical facility identified based on diagnostic results and treatment options.

[1243] "Terminal" refers to a device such as a smartphone, PC, or smart glasses that allows users to display and operate diagnostic results and information.

[1244] The "emotional state of the user" refers to the psychological state of the user when diagnostic data is input or when the diagnostic results are presented.

[1245] This invention is a system that combines an emotion engine with a system that allows users to quickly obtain a second opinion based on a doctor's diagnosis using AI. This system uses a device such as smart glasses and functions through a series of processes including inputting diagnostic data, preprocessing, analysis, presenting results, and recommending medical institutions.

[1246] First, the user uses the smart glasses' voice recognition and camera functions to input diagnostic data in voice, text, or image format. For example, the user can provide information such as "I have high blood pressure and have been diagnosed with possible diabetes" through voice input, and also take an image of the diagnosis certificate with the camera and upload it.

[1247] The server pre-processes the input diagnostic data, converting speech to text using speech recognition and extracting key medical information using natural language processing (NLP) techniques, and extracting text information from image data using optical character recognition (OCR) techniques.

[1248] Next, the server uses an emotion engine to recognize the user's emotional state. It analyzes the user's emotions from voice input and text data to detect anxiety or relief. For example, if the user includes an expression such as "I'm worried," the emotion engine can detect anxiety.

[1249] Based on the analysis results, the server uses a generative AI model to interpret the diagnostic data and generate treatment options. This analysis includes a summary of the diagnostic results, an interpretation of the current condition, and multiple treatment options. The server adjusts the way the interpretation results and treatment options are displayed based on the user's emotional state. For example, if a user is feeling anxious, a more understandable and reassuring message will be displayed.

[1250] The generated diagnostic interpretation results and treatment options are sent from the server to the device. The device displays the results in an easy-to-understand manner through a user interface and, if necessary, provides a voice guide function. For example, the device could visually display the diagnostic results and, for anxious users, display a message such as, "Your condition is manageable. The following treatment options may be considered as next steps."

[1251] Finally, the server searches for appropriate medical institutions in the user's vicinity based on the diagnosis results and treatment options. It uses internal and external databases to identify the most suitable medical institutions and presents them on the device as a list of recommended medical institutions. Based on the information provided, the user can take action to receive appropriate medical treatment more quickly.

[1252] As a concrete example, consider the case where a user voice-inputs "I have high blood pressure and may have diabetes" into the smart glasses and provides a photo of the medical certificate. At this time, the server converts the voice to text using speech recognition technology, extracts important medical information using natural language processing, and then detects the user's anxiety using an emotion engine. The generative AI model generates treatment options such as antihypertensive medications and dietary therapy, and displays them on the smart glasses with reassuring explanations. It also simultaneously provides information recommending the nearest internal medicine doctor or specialist.

[1253] An example of a prompt sentence is, "Create a prompt sentence to provide a diagnosis and treatment options based on the health information provided by the user. For example, if the user enters, 'My blood pressure is high and I may have diabetes,' generate a diagnosis that includes treatment options such as antihypertensive medication and dietary therapy."

[1254] This system allows users to obtain a second opinion quickly and accurately, while still feeling at ease, and to choose appropriate medical action.

[1255] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1256] Step 1:

[1257] Entering diagnostic data

[1258] The user uses the voice recognition and camera functions of the smart glasses to input diagnostic data in the form of voice, text, or images. For example, the user can tell the system through voice input, "I have been diagnosed with high blood pressure and possible diabetes," and then take a picture of the diagnosis with the smart glasses' camera and upload it. The input in this case is voice data or image data.

[1259] Step 2:

[1260] Data Preprocessing

[1261] The server preprocesses the input diagnostic data. For voice data, it converts it into text using voice recognition technology and analyzes it using natural language processing (NLP) technology. Important medical information is extracted from the text-format diagnostic data. For image data, it extracts text information from the image using optical character recognition (OCR) technology. This preprocessing results in text data containing important medical information being output.

[1262] Step 3:

[1263] Recognition of emotional states

[1264] The server uses an emotion engine to recognize the user's emotional state. It analyzes the user's emotions from preprocessed text data and voice data and determines whether the user is in an emotional state such as anxiety or relief. The server generates emotional state data based on this.

[1265] Step 4:

[1266] Analyzing diagnostic data and generating treatment options

[1267] The server inputs the pre-processed diagnostic data and emotional state data into a generative AI model, which analyzes the diagnostic data based on a medical database and past diagnostic results to generate a summary of the diagnostic results, an interpretation of the current state, and treatment options. The generative AI model is used for this analysis, and the diagnostic interpretation and treatment options are obtained as outputs.

[1268] Step 5:

[1269] Emotion-based information presentation adjustment

[1270] The server adjusts the analysis results and treatment options based on the user's emotional state. Users who feel anxious are presented with reassuring messages and explanations, while users who feel at ease are presented with simple information. This results in tailored diagnostic results and treatment options.

[1271] Step 6:

[1272] Sending and displaying results

[1273] The server sends the adjusted diagnosis results and treatment options to the device, which then displays the results through a user interface (UI). Specifically, the diagnosis results and treatment options are visually displayed on the smart glasses' display, and audio guidance is provided as needed. At this time, information reflecting the user's input data is output.

[1274] Step 7:

[1275] Search for recommended medical institutions

[1276] The server searches for recommended medical institutions using internal and external databases based on the diagnosis results and treatment options, and identifies appropriate medical institutions based on the user's location information. The server generates this information as a recommendation list and sends it to the device.

[1277] Step 8:

[1278] Display of recommended medical institutions

[1279] The device will display recommended medical institutions to the user. The smart glasses display allows the user to view the nearest appropriate medical institution and its detailed information. Based on this information, the user can easily access the medical institution.

[1280] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1281] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1282] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1283] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1284] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1285] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1286] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1287] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1288] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1289] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1290] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1291] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1292] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1293] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1294] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1295] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1296] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1297] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1298] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1299] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1300] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1301] The following is further disclosed regarding the above embodiment.

[1302] (Claim 1)

[1303] a means for a user to input diagnostic data;

[1304] a means for the server to pre-process the diagnostic data and extract key information;

[1305] A means for the server to utilize the AI ​​model to interpret the diagnostic results and generate treatment options; and

[1306] means by which the server presents interpretation results and treatment options to the user;

[1307] A means for the server to search for recommended medical institutions based on the diagnosis results and treatment options;

[1308] A system including a means for a terminal to display recommended medical institutions to a user.

[1309] (Claim 2)

[1310] 10. The system of claim 1, wherein the user can input diagnostic data in text, voice, or image format.

[1311] (Claim 3)

[1312] 10. The system of claim 1, wherein the server preprocesses the diagnostic data using natural language processing, speech recognition, or optical character recognition techniques.

[1313] "Example 1"

[1314] (Claim 1)

[1315] a means for a user to input diagnostic data;

[1316] a means for the server to pre-process the diagnostic data and extract key information;

[1317] a means for the server to utilize the generative AI model to interpret the diagnostic results and generate treatment options; and

[1318] means by which the server presents interpretation results and treatment options to the user;

[1319] A means for the server to search for recommended medical institutions based on the diagnosis results and treatment options;

[1320] A means for the terminal to display recommended medical institutions to the user;

[1321] A system that includes a means for a user to view results and select treatment options through an interface.

[1322] (Claim 2)

[1323] 10. The system of claim 1, wherein the user can input diagnostic data in text, voice, or image format.

[1324] (Claim 3)

[1325] 10. The system of claim 1, wherein the server preprocesses the diagnostic data using natural language processing, speech recognition, or optical character recognition techniques.

[1326] "Application Example 1"

[1327] (Claim 1)

[1328] a means for a user to input diagnostic data;

[1329] a means for the server to pre-process the diagnostic data and extract key information;

[1330] a means for the server to utilize the generative AI model to interpret the diagnosis and generate prompt statements for generating treatment options;

[1331] A means for the server to input the generated prompt sentences to the AI ​​to generate treatment options;

[1332] means by which the server presents interpretation results and treatment options to the user;

[1333] A means for the server to search for recommended medical institutions based on the diagnosis results and treatment options;

[1334] A system including a means for a terminal to display recommended medical institutions to a user.

[1335] (Claim 2)

[1336] 10. The system of claim 1, wherein a user can input diagnostic data in text, voice, or image format and review pre-processing and interpretation results using an application installed on a smartphone, smart glasses, a head-mounted display, or a robot.

[1337] (Claim 3)

[1338] 10. The system of claim 1, wherein the server preprocesses the diagnostic data using natural language processing, speech recognition, or optical character recognition technology to generate prompt sentences for the AI ​​model.

[1339] "Example 2: Combining Emotion Engines"

[1340] (Claim 1)

[1341] a means for a user to input diagnostic data;

[1342] a means for the server to pre-process the diagnostic data and extract key information;

[1343] A means for the server to recognize the user's emotion;

[1344] A means for the server to utilize the AI ​​model to interpret the diagnostic results and generate treatment options; and

[1345] means by which the server presents interpretation results and treatment options to the user;

[1346] A means for the server to search for recommended medical institutions based on the diagnosis results and treatment options;

[1347] A system including a means for a terminal to display recommended medical institutions to a user.

[1348] (Claim 2)

[1349] 10. The system of claim 1, wherein the user can input diagnostic data in text, voice, or image format.

[1350] (Claim 3)

[1351] 10. The system of claim 1, wherein the server preprocesses the diagnostic data using natural language processing, speech recognition, or optical character recognition techniques.

[1352] "Application example 2 when combining emotion engines"

[1353] (Claim 1)

[1354] a means for a user to input diagnostic data in voice, text, or image format;

[1355] a means by which the server pre-processes the diagnostic data and extracts key information using natural language processing, speech recognition, or optical character recognition techniques;

[1356] means for the server to recognize the emotional state of the user using an emotion engine;

[1357] a means for the server to utilize an artificial intelligence model to interpret the diagnostic data and generate treatment options;

[1358] means for the server to tailor and present interpretation results and treatment options according to the user's emotional state;

[1359] A means for the server to search for recommended medical institutions based on the diagnosis results and treatment options;

[1360] A system including a means for a terminal to display recommended medical institutions to a user and present the results in an easy-to-understand and reassuring manner.

[1361] (Claim 2)

[1362] 10. The system of claim 1, wherein the smart glasses enable a user to input diagnostic data and recognize emotional states.

[1363] (Claim 3)

[1364] The system of claim 1, wherein the server preprocesses the diagnostic data and the user's emotional state and uses a generative AI model to create prompt sentences for generating diagnostic results and treatment options. [Explanation of symbols]

[1365] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for a user to input diagnostic data; a means for the server to pre-process the diagnostic data and extract key information; A means for the server to utilize the AI ​​model to interpret the diagnostic results and generate treatment options; and means by which the server presents interpretation results and treatment options to the user; A means for the server to search for recommended medical institutions based on the diagnosis results and treatment options; A system including a means for a terminal to display recommended medical institutions to a user.

2. 10. The system of claim 1, wherein the user can input diagnostic data in text, voice, or image format.

3. The system of claim 1 , wherein the server preprocesses the diagnostic data using natural language processing, speech recognition, or optical character recognition techniques.

Images