System

Abstract

To provide a system.SOLUTION: A system includes: means for inputting inquiries in a chat form; means for analyzing a text by using a natural language processing technique according to input contents of the inquiry inputting means; means for extracting important key words from an analysis result; means for analyzing emotions of a user by using an emotion engine; means for diagnosing the user according to an analysis result of the emotion engine; means for issuing a prescription or delivering medicine in cases of mild symptoms and where an effect of the prescription medicine is minor, as a result of the diagnosis of the user; and means for sending the user to online diagnosis in complex cases or if there are any concerns, as a result of the diagnosis of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the current medical system, patients must visit medical institutions in person, which places time and location constraints on them. In addition, they must visit medical institutions even if their symptoms are mild, making it difficult to make effective use of medical resources. [Means for solving the problem]

[0005] This invention provides a chat-style medical interview input method, an AI image diagnostic processing method based on the input content, a method for issuing a prescription or shipping medicine in cases where the symptoms are mild or the effects of prescribed medicine are minor, and a method for sending to an online diagnosis in cases where the symptoms are complicated or there are concerns. This allows patients to receive medical services without being restricted by time or place, and enables the effective use of medical resources. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] An embodiment of the present invention is realized as a medical application. A user inputs their symptoms in a chat format through the application. This input is analyzed using natural language processing technology, and important keywords related to the symptoms are extracted. For example, if a user inputs "I have a headache," "headache" is extracted as a keyword.

[0029] "Example 2"

[0030] Next, the AI ​​image diagnosis processing means operates. The AI ​​analyzes the image data provided by the user (for example, a photo of an abnormal area on the skin) and determines the condition. The results of this analysis are reported to the doctor and serve as reference information for the final diagnosis.

[0031] "Example 3"

[0032] For mild symptoms or minor effects of prescribed medication, the system automatically issues a prescription and sends it to a designated pharmacy, or, if desired, can have the medication delivered directly to the user's home.

[0033] "Example 4"

[0034] If there are any complex symptoms or concerns, the system will refer the user to an online diagnosis, where they can interact directly with a doctor via a real-time video call for a detailed diagnosis.

[0035] The processing flow of each embodiment will be described below.

[0036] "Example 1"

[0037] Step 1: The user opens the medical application and enters their symptoms in a chat format.

[0038] Step 2: The application uses natural language processing techniques to analyze the user's input and extract key symptoms keywords.

[0039] "Example 2"

[0040] Step 1: The application receives user-provided image data (e.g., a photo of an abnormal area of ​​skin).

[0041] Step 2: The AI ​​analyzes the image data and determines the condition. The results of this analysis are reported to the doctor.

[0042] "Example 3"

[0043] Step 1: If the AI ​​determines that the user's symptoms are mild, the system will automatically issue a prescription.

[0044] Step 2: The issued prescription is sent to the designated pharmacy.

[0045] Step 3: If desired, the medication is delivered directly to the user's home.

[0046] "Example 4"

[0047] Step 1: If the AI ​​determines that the user's symptoms are complex or concerning, the system will refer the user to an online diagnosis.

[0048] Step 2: Users can interact directly with a doctor through a real-time video call and receive a detailed diagnosis.

[0049] Example 1

[0050] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0051] In conventional medical systems, when users input their symptoms, it is difficult to extract appropriate keywords and provide prompt and accurate medical advice. Furthermore, there are insufficient means to provide appropriate treatment for mild symptoms or when the effects of prescription medication are minor, and there are also insufficient means to quickly refer users to online diagnoses for complex symptoms or concerns. This reduces user convenience and compromises the quality of medical services.

[0052] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0053] In this invention, the server includes a chat-style medical interview input means, a means for analyzing the input content using natural language processing technology and extracting important keywords, and a means for sending the extracted keywords to a generative AI model to generate appropriate medical advice. This makes it possible to quickly and accurately extract keywords based on the symptoms entered by the user and provide appropriate medical advice.

[0054] The "chat-style medical interview input means" is a means for providing an interface for the user to input their own symptoms in text format.

[0055] "Natural language processing technology" is a technology for analyzing input text data and understanding its meaning.

[0056] The "means for extracting important keywords" is a means for identifying important words and phrases related to symptoms from input text using natural language processing technology.

[0057] A "generative AI model" is an artificial intelligence model that generates appropriate medical advice or next steps based on the input prompt.

[0058] The "means for generating medical advice" is a means for using a generative AI model to generate medical advice to be provided to a user based on the extracted keywords.

[0059] "Means for issuing a prescription or shipping medicine when symptoms are mild or the effects of prescribed medicine are minor" refers to means for issuing a prescription or shipping medicine when the user's symptoms are mild and the effects of prescribed medicine are minor.

[0060] "Online diagnosis" refers to a method for providing a diagnosis through a real-time video call with a doctor if the user's symptoms are complex or of concern.

[0061] This invention is a system realized as a medical application, in which users input their symptoms in a chat format, which is analyzed using natural language processing technology, important keywords are extracted, and appropriate medical advice is provided using a generative AI model.

[0062] A user opens a medical application on a device such as a smartphone or tablet. The application has a chat-style interface, and the user inputs their symptoms in text format. For example, they might input, "I've been having such a bad headache lately that I can't sleep."

[0063] The terminal sends the text data entered by the user to the server using the HTTPS protocol, and the server temporarily stores the received text data in a database.

[0064] The server uses Python's natural language processing libraries, NLTK and spaCy, to tokenize, tag parts of speech, and analyze dependencies on the received text data. This allows important keywords to be extracted from the text data. For example, the keywords "headache" and "can't sleep" are extracted.

[0065] The server sends the extracted keywords to the generative AI model as a prompt sentence. An example of a prompt sentence would be, "User input: I've been having a bad headache lately and can't sleep. Extracted keywords: headache, can't sleep."

[0066] The generative AI model generates appropriate medical advice and next steps based on the input prompt, and the generated medical advice is sent to the device via the server.

[0067] The terminal displays the medical advice received from the server to the user, who can then confirm the displayed medical advice and decide on the next course of action.

[0068] For example, if a user inputs "I've had a bad headache recently and can't sleep," the server extracts the keywords "headache" and "can't sleep" and sends a prompt to the generative AI model. The generative AI model generates medical advice such as "If your headache persists, consult a doctor. We also recommend taking a warm bath to relax," and sends it to the device via the server. The device then displays this advice to the user.

[0069] This system allows users to receive fast and accurate medical advice based on their symptoms, providing appropriate treatment for mild symptoms or minor effects of prescribed medication, and quickly directing users to an online diagnosis for complex symptoms or concerns.

[0070] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0071] Step 1:

[0072] The user enters the symptoms.

[0073] A user opens a medical application and enters their symptoms in text format into a chat-style interface. For example, they might enter, "I've been having a bad headache lately and can't sleep." The input data is in text format.

[0074] Step 2:

[0075] The device sends input to the server.

[0076] The terminal sends the text data entered by the user to the server using the HTTPS protocol. The input data is in text format, and the output is the text data sent to the server.

[0077] Step 3:

[0078] The server receives the input.

[0079] The server receives text data sent from the terminal and temporarily stores it in a database. The input data is in text format, and the output is text data stored in the database.

[0080] Step 4:

[0081] The server performs natural language processing.

[0082] The server uses Python natural language processing libraries NLTK and spaCy to tokenize, tag parts of speech, and analyze dependencies on the received text data. The input data is in text format, and the output is the analysis results.

[0083] Step 5:

[0084] The server extracts the keywords.

[0085] The server extracts important keywords from the results of natural language processing. For example, the keywords "headache" and "can't sleep" are extracted. The input data is the analysis results, and the output is the extracted keywords.

[0086] Step 6:

[0087] The server sends the extraction results to the generative AI model.

[0088] The server sends the extracted keywords as a prompt to the generative AI model. An example of a prompt might be, "User input: I've been having a bad headache lately and can't sleep. Extracted keywords: headache, can't sleep." The input data are the extracted keywords, and the output is the prompt sent to the generative AI model.

[0089] Step 7:

[0090] The server receives the results of the generative AI model.

[0091] The server receives the results returned by the generative AI model, which include appropriate medical advice and next steps. The input data is the prompt sent to the generative AI model, and the output is the medical advice returned by the generative AI model.

[0092] Step 8:

[0093] The server sends the results to the terminal.

[0094] The server sends the results received from the generative AI model to the terminal using the HTTPS protocol. The input data is the medical advice returned by the generative AI model, and the output is the medical advice sent to the terminal.

[0095] Step 9:

[0096] The terminal displays the results to the user.

[0097] The terminal displays the medical advice received from the server to the user, who can then review the displayed medical advice and decide on the next course of action. The input data is the medical advice received from the server, and the output is the medical advice displayed to the user.

[0098] (Application example 1)

[0099] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0100] Conventional security services have had the challenge of efficiently analyzing users' anxieties and problems and providing appropriate responses quickly. In particular, it was difficult to accurately understand the text information entered by users and extract important keywords, making it impossible to provide appropriate countermeasures.

[0101] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0102] In this invention, the server includes a chat-style medical interview input means, a keyword extraction means using natural language processing technology according to the input content, a means for issuing a prescription or shipping medicine if the symptoms are mild or the effects of the prescribed medicine are minor, and a means for sending the patient to an online diagnosis if the symptoms are complicated or there are concerns. This makes it possible to accurately analyze the text information entered by the user, extract important keywords, and quickly provide appropriate countermeasures.

[0103] The "chat-style medical interview input means" is a means for providing an interface for users to input questions and symptoms in text format.

[0104] "Keyword extraction means using natural language processing technology" refers to a means that uses technology to analyze text information entered by the user and extract important keywords.

[0105] "Means for issuing prescriptions or shipping medicines when symptoms are mild or the effects of prescription medicines are minor" refers to means for issuing prescriptions or shipping medicines when the user's symptoms are determined to be mild and the effects of prescription medicines are determined to be minor.

[0106] "Online diagnostic means for complex or concerning cases" refers to a means for users to receive an online diagnosis from a specialist if their symptoms are complex or they have concerns.

[0107] "Means for enabling real-time video calls with experts" means means for enabling a user to have real-time video calls with experts.

[0108] The system for implementing this invention is configured as follows: First, a user inputs security-related concerns or problems in a chat format using a smartphone. The input text information is sent to a server.

[0109] The server uses natural language processing technology to analyze the user's input text and extract important keywords. This analysis is performed using Python and the natural language processing library spaCy. Specifically, if a user types "I've been seeing suspicious people around my house recently," the server will extract the keyword "suspicious person."

[0110] Based on the extracted keywords, the server determines the appropriate course of action: if the symptoms are mild or the effects of a prescription are minor, a prescription or medication delivery is used, while if the symptoms are more complex or there are concerns, the patient is referred for an online diagnosis, which includes a real-time video call with a specialist.

[0111] For example, if a user types in "I've been seeing suspicious people around my house lately," the server will extract the keyword "suspicious people" and suggest a video call with a security expert, allowing the user to communicate with the expert in real time using their smartphone.

[0112] Examples of prompts to input to a generative AI model include:

[0113] A user enters a security issue or concern. For example, if the user enters "I've been seeing suspicious people around my house lately," extract "suspicious people" as an important keyword.

[0114] In this way, it becomes possible to efficiently analyze users' security concerns and problems and quickly provide appropriate countermeasures.

[0115] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0116] Step 1:

[0117] A user uses a smartphone to input security-related concerns or problems in chat format. The input text information is sent from the smartphone to a server. The input data is text information about the user's concerns or problems.

[0118] Step 2:

[0119] The server analyzes the received text information using natural language processing technology. This analysis uses Python and the natural language processing library spaCy. Specifically, the text information is tokenized and important keywords such as nouns, proper nouns, and adjectives are extracted. The input data is the user's text information, and the output data is the extracted keywords.

[0120] Step 3:

[0121] The server determines the appropriate response based on the extracted keywords. If the symptoms are mild or the effects of the prescribed medication are minor, measures such as issuing a prescription or shipping the medication are used. The input data are the extracted keywords, and the output data are instructions to issue a prescription or ship the medication.

[0122] Step 4:

[0123] The server executes a means for routing the extracted keywords to an online diagnostic if they are complex or cause concern, including a means for enabling a real-time video call with an expert, where the input data is the extracted keywords and the output data is an instruction to start the video call.

[0124] Step 5:

[0125] Users can use their smartphones to make real-time video calls with experts, allowing them to receive advice and diagnoses directly from the experts. The input data is an instruction to start the video call, and the output data is the advice and diagnosis results from the experts.

[0126] Example 2

[0127] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0128] In conventional medical systems, it is difficult for patients to accurately communicate their symptoms, which often leads to delayed diagnosis, especially for visual symptoms such as skin abnormalities. Another problem is that even in cases of mild illness, it takes time and effort to receive a diagnosis from a doctor. Furthermore, patients living in remote areas often have difficulty accessing medical institutions. To solve these issues, a system is needed that can efficiently and quickly diagnose illnesses and provide appropriate medical services.

[0129] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0130] In this invention, the server includes a chat-style medical interview input means, an image diagnosis processing means according to the input content, a means for issuing a prescription or shipping medicine if the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending an online diagnosis if the symptoms are complicated or there are concerns, a means for sending image data taken by the user from the terminal to the server, an image analysis means using a generative AI model to analyze the image data received by the server, and a means for reporting the analysis results to a doctor. This allows patients to easily diagnose skin abnormalities from home and receive appropriate medical care promptly.

[0131] The "chat-style medical interview input means" is an interface that allows users to input information such as symptoms and medical history via text messages, and the system receives this information.

[0132] The "image diagnosis processing means" is a technical means for analyzing image data provided by a user and determining the condition of the disease.

[0133] "Means for issuing prescriptions or shipping medicines" refers to a function that enables the system to automatically issue prescriptions or ship medicines to patients when the symptoms or effects of prescribed medicines are minor.

[0134] "Online referral" is a feature that allows the system to refer patients to online consultations, such as a real-time video call with a doctor, in cases of complications or concerns.

[0135] "Means for transmitting image data taken by a user from a terminal to a server" refers to technical means for transmitting image data taken by a user to a server via a terminal.

[0136] "Image analysis means using generative AI models" refers to a technical means for analyzing image data and determining the condition of a disease using an AI model built using a deep learning framework.

[0137] "Means for reporting analysis results to doctors" refers to the technical means for reporting the analysis results obtained by the AI ​​model to doctors.

[0138] MODE FOR CARRYING OUT THE INVENTION

[0139] This invention is a system for analyzing image data captured by a user and determining the condition of a patient. The system includes a chat-style interview input means, an image diagnosis processing means, a means for issuing a prescription or shipping medicine, a means for sending the image data captured by the user from a terminal to a server, an image analysis means using a generative AI model, and a means for reporting the analysis results to a doctor.

[0140] Hardware and software used

[0141] Device: Smartphone, tablet, computer, or other device

[0142] Server: A server with high-performance computing power

[0143] Deep learning frameworks: TENSORFLOW (registered trademark), PyTorch, etc.

[0144] Data processing and calculation

[0145] The user takes a photo of the abnormal area on their skin using a smartphone or digital camera. After taking the photo, they upload the image data to the system using a dedicated application. The device encrypts the image data and sends it to the server using a secure communication protocol (e.g., HTTPS).

[0146] The server analyzes the received image data using a generative AI model. This AI model is built using deep learning frameworks such as TensorFlow and PyTorch. Specifically, it uses a convolutional neural network (CNN) to extract image features and identify patterns of abnormalities. This analysis results in a diagnosis of the condition, such as "suspected skin cancer" or "high probability of eczema."

[0147] The analysis results are reported from the server to the doctor, who then uses this information to make a final diagnosis. For example, if the doctor receives a result that suggests skin cancer, he or she can prescribe further detailed testing.

[0148] Specific examples

[0149] For example, if a user notices an abnormality in their skin, they take a photo of the area with their smartphone and upload the image to the system using a dedicated application. The device encrypts this image and sends it to a server using HTTPS. The server receives the image and analyzes it using a generative AI model. If the analysis results in a "suspected skin cancer," the information is reported to a doctor. The doctor can then use this information to prescribe further tests.

[0150] Prompt Sentence Examples

[0151] "Design a system that analyzes photos of abnormal skin areas taken by users, uses an AI model to determine the condition, and reports the diagnosis to a doctor."

[0152] The system is designed to allow users to easily detect skin abnormalities and seek early medical diagnosis.

[0153] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0154] Program processing flow

[0155] Step 1: User takes and uploads image data

[0156] The user takes a photo of the abnormal area of ​​the skin using a smartphone or digital camera. After taking the photo, the image data is uploaded to the system using a dedicated application. The input is the image data taken by the user, and the output is the image data saved on the device.

[0157] Step 2: The device sends the image data to the server

[0158] The terminal sends image data uploaded by the user to the server. At this time, the image data is encrypted and sent using a secure communication protocol (e.g., HTTPS). The input is the image data stored on the terminal, and the output is the encrypted image data sent to the server.

[0159] Step 3: The server receives the image data and passes it to the AI ​​image diagnosis processing means.

[0160] The server receives the image data sent from the terminal. After receiving the image data, the server passes the image data to the AI ​​image diagnosis processing means. The input is the encrypted image data sent to the server, and the output is the image data passed to the AI ​​image diagnosis processing means.

[0161] Step 4: The server analyzes the image data using the AI ​​model

[0162] The server analyzes the image data using an AI model built using deep learning frameworks such as TensorFlow and PyTorch. Specifically, it uses a convolutional neural network (CNN) to extract image features and identify patterns of abnormalities. The input is the image data passed to the AI ​​image diagnostic processing means, and the output is a diagnosis of the disease state.

[0163] Step 5: The server reports the analysis results to the doctor

[0164] The server reports the analysis results obtained by the AI ​​model to the doctor. This reporting is done through a dedicated medical information system. The input is the diagnosis result of the disease, and the output is the diagnosis information reported to the doctor.

[0165] Adding specific actions

[0166] Step 1: User takes and uploads image data

[0167] If a user feels any abnormality in their skin, they take a photo of the area with their smartphone. They open the dedicated application, select the image they have taken, and press the upload button. The image data is then saved on the device.

[0168] Step 2: The device sends the image data to the server

[0169] The device encrypts the image data uploaded by the user and sends it to the server using HTTPS. When the transmission is complete, the device displays a notification to the user that the transmission is complete.

[0170] Step 3: The server receives the image data and passes it to the AI ​​image diagnosis processing means.

[0171] The server receives the encrypted image data sent from the terminal, decrypts it, and passes the decrypted image data to the AI ​​image diagnosis processing means.

[0172] Step 4: The server analyzes the image data using the AI ​​model

[0173] The server analyzes the image data passed to the AI ​​image diagnostic processing means using a CNN model built using TensorFlow, and obtains a diagnosis of the condition, such as whether there is a suspicion of skin cancer.

[0174] Step 5: The server reports the analysis results to the doctor

[0175] The server reports the diagnosis of the patient's condition to the doctor via a dedicated medical information system, and the doctor uses this information to decide whether to order further detailed tests.

[0176] (Application example 2)

[0177] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0178] Modern medical systems require rapid and accurate diagnosis of patients' conditions. However, conventional systems face challenges in analyzing image data provided by patients and detecting abnormal behavior or suspicious individuals in real time. Furthermore, they lack the ability to quickly notify and respond when an abnormality is detected. This can lead to delayed responses in medical and security settings, potentially threatening the safety of patients and facilities.

[0179] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0180] In this invention, the server includes a chat-style medical interview input means, an AI image diagnosis processing means according to the input content, a means for issuing a prescription or shipping medicine if the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending an online diagnosis if the symptoms are complicated or there are concerns, a means for analyzing images captured by security cameras in real time to detect abnormal behavior or suspicious individuals, and a means for storing the detection results in a database and sending a notification if an abnormality is detected. This makes it possible to quickly and accurately diagnose the patient's condition, and also enables real-time detection of abnormal behavior or suspicious individuals through security camera image analysis and rapid notification.

[0181] The "chat-style medical interview input means" is an interface that allows users to input symptoms and conditions by answering questions in text format.

[0182] An "AI image diagnostic processing means" is a system that uses artificial intelligence to analyze image data and determine the condition or abnormality.

[0183] "Means for issuing prescriptions or shipping medicines" refers to a system for issuing prescriptions or shipping medicines to patients based on a doctor's instructions in cases where the symptoms or effects of prescription medicines are minor.

[0184] "Online referral" is a system that directs patients to a real-time video call with a doctor for complex cases or concerns.

[0185] "Means for analyzing security camera footage in real time to detect abnormal behavior and suspicious individuals" is a system for analyzing security camera footage in real time to automatically detect abnormal behavior and suspicious individuals.

[0186] "Means for saving detection results in a database and sending notifications when abnormalities are detected" refers to a system that saves the results in a database when abnormal behavior or suspicious individuals are detected and sends notifications to responsible personnel as necessary.

[0187] A system for implementing this invention has the following configuration: The server includes a means for inputting medical interviews in chat format, an AI image diagnosis processing means according to the input content, a means for issuing a prescription or shipping medicine in cases where the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending an online diagnosis in cases where the symptoms are complicated or there are concerns, a means for analyzing images captured by security cameras in real time to detect abnormal behavior or suspicious individuals, and a means for storing the detection results in a database and sending a notification if an abnormality is detected.

[0188] Hardware used:

[0189] server

[0190] security cameras

[0191] Smartphone or tablet

[0192] Database Server

[0193] Software used:

[0194] Image analysis AI models (e.g., YOLO, OpenCV)

[0195] Communication protocol (e.g. HTTP, WebSocket)

[0196] Database (e.g., MySQL (registered trademark), MongoDB)

[0197] System behavior:

[0198] 1. Chat-style medical interview input method:

[0199] Users use a smartphone or tablet to input their symptoms and conditions in a text-based chat format, which then sends the medical interview information to the server.

[0200] 2. AI image diagnosis processing methods:

[0201] The server receives image data provided by the user (e.g., a photo of an abnormal area of ​​the skin) and analyzes it using an image analysis AI model (e.g., YOLO). The analysis results are used to determine the condition of the patient.

[0202] 3. Prescription or drug delivery method:

[0203] If the symptoms are mild or the effects of prescribed medication are minor, the server will automatically issue a prescription and ship the medication to the patient.

[0204] 4. Online diagnostics:

[0205] In the case of complex cases or concerns, the server directs the user to a real-time video call with a doctor.

[0206] 5. Security camera footage analysis methods:

[0207] Security cameras capture video in real time and send it to a server, which uses an image analysis AI model to detect abnormal behavior or suspicious individuals in the video.

[0208] 6. Detection result storage and notification method:

[0209] The detection results are stored in a database, and if an anomaly is detected, a notification is sent to a smartphone or tablet, allowing security personnel to take prompt action.

[0210] Examples:

[0211] If a security camera detects a suspicious person at night, the server will send a notification to the smartphone saying, "Suspicious person detected. Location: near the main gate."

[0212] Security personnel open an application on their smartphone, check the details of the analysis results, and decide whether to report the incident to the police.

[0213] Example prompts to input to a generative AI model:

[0214] "Please create an AI model that analyzes footage captured by security cameras and detects abnormal behavior and suspicious individuals. The results of the detection should be sent to a smartphone in real time."

[0215] Specific prompt:

[0216] "Using the YOLO model, please build a system that detects suspicious individuals in real time from security camera footage. Please also implement a function to send the detection results to a server using the HTTP protocol and send notifications to smartphones."

[0217] In this way, a system for implementing the invention allows for a fast and accurate response in both the medical and security fields.

[0218] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0219] Step 1:

[0220] Enter medical questions in chat format

[0221] Users use a smartphone or tablet to input their symptoms and conditions in a text-based chat format, and the input data is sent to a server.

[0222] Input: User-entered text data about symptoms and conditions

[0223] Output: Interview data sent to the server

[0224] Specific behavior: A user opens the application, enters symptoms in the chat box, and presses the send button. The server receives this data and stores it in a database.

[0225] Step 2:

[0226] AI image diagnostic processing

[0227] The server receives image data provided by the user (e.g., a photo of an abnormal area of ​​the skin) and analyzes it using an image analysis AI model (e.g., YOLO).

[0228] Input: User uploaded image data

[0229] Output: Analysis results (diagnosis of the condition)

[0230] How it works: A user uploads an image through the application, and the server receives the image. The server then invokes an image analysis AI model to analyze the image and determine the medical condition. The analysis results are then stored in a database.

[0231] Step 3:

[0232] Prescription issuance or drug delivery

[0233] If the symptoms are mild or the effects of prescribed medication are minor, the server will automatically issue a prescription and ship the medication to the patient.

[0234] Input: Analysis result (if determined to be mild)

[0235] Output: Instructions for issuing a prescription or shipping medicine

[0236] Specific operation: The server evaluates the analysis results and, if it determines that the patient's condition is mild, it automatically generates a prescription and sends instructions to the pharmacy to ship the medication.

[0237] Step 4:

[0238] Guidance to online diagnosis

[0239] In the case of complex cases or concerns, the server directs the user to a real-time video call with a doctor.

[0240] Input: Analysis results (if determined to be a complex case)

[0241] Output: Video call link or invitation

[0242] What it does: The server evaluates the analysis results and, if it determines that the case is complex, it sends the user a link to a video call to set up an online consultation with a doctor.

[0243] Step 5:

[0244] Real-time analysis of security camera footage

[0245] Security cameras capture video in real time and send it to a server, which uses an image analysis AI model to detect abnormal behavior or suspicious individuals in the video.

[0246] Input: Real-time video data from security cameras

[0247] Output: Detection results of abnormal behavior and suspicious individuals

[0248] How it works: A security camera captures video and sends it to a server, which then invokes an image analysis AI model to analyze the video and detect abnormal behavior or suspicious individuals.

[0249] Step 6:

[0250] Storing and notifying detection results

[0251] The detection results are stored in a database, and if an abnormality is detected, a notification is sent to a smartphone or tablet.

[0252] Input: Detection results of abnormal behavior or suspicious individuals

[0253] Output: Save to database and send notification

[0254] How it works: The server stores the detection results in a database, and if an anomaly is detected, a notification is sent to a smartphone or tablet. Security personnel receive the notification and open an application to check the details.

[0255] Example 3

[0256] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0257] In the conventional medical system, users had to visit a medical institution in person even for mild symptoms or minor effects of prescribed medication, which required time and effort. Furthermore, when users had complex symptoms or concerns, numerous procedures were required to receive an appropriate diagnosis, making it difficult to respond quickly. Furthermore, the lack of real-time communication with doctors made it difficult for users to receive appropriate medical services.

[0258] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0259] In this invention, the server includes a chat-style medical interview input means, a health information analysis means using a generative AI model based on the input content, a means for issuing a prescription or shipping medication if the symptoms are mild or the effects of the prescribed medication are minor, a means for sending the patient to an online diagnosis if the symptoms are complicated or there are concerns, and a means for enabling real-time video calls with a doctor. This allows users to easily input their health information from home and receive an appropriate diagnosis and prescription. Furthermore, even if the patient has complex symptoms or concerns, they can receive a quick online diagnosis and a detailed diagnosis through real-time communication with a doctor.

[0260] "Chat-style medical interview input means" is a means for users to input their health conditions and symptoms through a text-based interface.

[0261] A "health information analysis method using a generative AI model" is a method that uses a generative AI model to analyze health information entered by a user and determine the severity of symptoms and appropriate prescriptions.

[0262] "Means for issuing prescriptions or shipping medicines" means means for automatically generating an appropriate prescription and sending it to a designated pharmacy or delivering the medicine to the user's home in the case of a minor illness or minor effects of a prescribed medicine.

[0263] "Online diagnostic referral" refers to a method that guides users to online diagnostics and provides appropriate medical services when they have complex symptoms or concerns.

[0264] "Means enabling real-time video calls with doctors" means means that users can have real-time video calls with doctors and receive detailed diagnoses.

[0265] MODE FOR CARRYING OUT THE INVENTION

[0266] This invention is a system that allows users to easily input health information from home and receive appropriate diagnosis and prescriptions. A specific embodiment of this system will be described below.

[0267] System configuration

[0268] The system consists of the following main components:

[0269] 1. Chat-style medical interview input method

[0270] 2. Health information analysis method using generative AI models

[0271] 3. A means of issuing prescriptions or shipping medications

[0272] 4. Online diagnostics

[0273] 5. A way to enable real-time video calls with doctors

[0274] Hardware and Software Use

[0275] Users access the system using devices such as smartphones or PCs. The system is provided through a dedicated application or website. The server receives input information from users and analyzes it using a generative AI model. This analysis uses natural language processing libraries (e.g., spaCy) and machine learning libraries (e.g., TensorFlow) implemented in Python.

[0276] Data processing and calculation

[0277] The health information entered by the user is received by the server and analyzed using a generative AI model. Based on the analysis results, if the symptom or the effect of the prescribed medication is determined to be minor, the server automatically generates a prescription and sends it to the designated pharmacy. This prescription is sent using a secure communication protocol (e.g., HTTPS). If the user requests home delivery of the medication, the server arranges for the delivery.

[0278] If the user has complex symptoms or concerns, the server will refer them to an online diagnosis, where they will use their device to interact directly with a doctor via a real-time video call, using real-time communication technologies such as WebRTC.

[0279] Specific examples

[0280] For example, if a user inputs "I have a sore throat and a slight cough," the server receives this information and analyzes it using a generative AI model. Based on the analysis results, it is determined to be a mild cold. The server generates a prescription for cold medicine and sends it to the designated pharmacy. If the user requests home delivery of the medicine, the server processes the delivery and the medicine is delivered to the user's home.

[0281] Prompt Sentence Examples

[0282] If a user reports "I have a sore throat and a slight cough," how would the system issue a prescription and deliver the medication?

[0283] In this way, users can easily input their health information from home and receive appropriate diagnosis and prescriptions. Even if users have complex symptoms or concerns, they can receive a quick online diagnosis and receive a detailed diagnosis through real-time communication with a doctor. The flow of the identification process in Example 3 will be described with reference to FIG. 15.

[0284] Step 1:

[0285] The user enters their health information.

[0286] Users use their devices to access the system's dedicated application or website and enter their current health condition and symptoms, for example, specific symptoms such as "I have a sore throat and a slight cough." The entered information is then sent to the server.

[0287] Input: User's health information (e.g., "I have a sore throat and a slight cough")

[0288] Output: Health information sent to the server

[0289] Step 2:

[0290] The server receives and analyzes the health information.

[0291] The server receives health information submitted by the user. This information is analyzed using a generative AI model. The analysis uses natural language processing libraries (e.g., spaCy) and machine learning libraries (e.g., TensorFlow) implemented in Python. The server evaluates the severity of symptoms and the impact of prescribed medications.

[0292] Input: Health information submitted by the user

[0293] Output: Analysis result (e.g., judged to be a mild cold)

[0294] Step 3:

[0295] The server generates a prescription based on the analysis results.

[0296] The server then generates an appropriate prescription based on the analysis results. For example, if it is determined to be a mild cold, a prescription for cold medicine will be generated. This prescription is generated electronically and stored in a secure format.

[0297] Input: Analysis results

[0298] Output: Generated prescription

[0299] Step 4:

[0300] The server sends the prescription to the designated pharmacy.

[0301] The server sends the generated prescription to the designated pharmacy using a secure communication protocol (e.g., HTTPS), and the pharmacy prepares the medication based on the received prescription.

[0302] Input: Generated prescription

[0303] Output: Prescription sent to pharmacy

[0304] Step 5:

[0305] If the user requests delivery of the medicine, the server handles the delivery process.

[0306] If the user requests home delivery of the medicine, the server processes the delivery, confirms the user's address information, and requests a delivery company. The delivery company then delivers the medicine to the user's home.

[0307] Input: User's shipping preferences, address information

[0308] Output: Delivery procedure requested by the delivery company

[0309] Step 6:

[0310] If there are complex symptoms or concerns, the server will direct the user to an online diagnosis.

[0311] If the server determines that the user has complex symptoms or concerns based on the analysis results, it will refer the user to an online diagnosis, and the user will receive a notification from the system and make an appointment for the online diagnosis.

[0312] Input: Analysis results (complex symptoms and concerns)

[0313] Output: Online diagnostic notification, reservation procedure

[0314] Step 7:

[0315] Users have real-time video calls with doctors.

[0316] At the scheduled time, users can use their devices to make a real-time video call using real-time communication technologies such as WebRTC, allowing them to directly communicate with a doctor and receive a detailed diagnosis.

[0317] Input: Scheduled video call time, user device

[0318] Output: Real-time video call with doctor, detailed diagnosis

[0319] (Application example 3)

[0320] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0321] In the conventional medical system, users had to visit a medical institution even for mild symptoms or minor effects of prescribed medication, which required time and effort. Furthermore, when users had complex symptoms or concerns, numerous procedures were required to receive an appropriate diagnosis, making it difficult to respond quickly. Furthermore, when it came to receiving medication, users had to visit a pharmacy, which was inconvenient.

[0322] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0323] In this invention, the server includes a chat-style medical interview input means, a diagnostic processing means using a generative AI model based on the input content, a means for issuing a prescription or shipping medication in cases where the symptoms are mild or the effects of the prescribed medication are minor, a means for referring the patient to an online diagnosis in cases where the symptoms are complicated or there are concerns, a means for delivering medication to the patient's home at the user's request, and a means for conducting a real-time video call with a doctor through the online diagnosis. This allows users to easily receive a diagnosis from home and quickly receive the necessary medication. Furthermore, even in cases where the symptoms are complicated, users can receive prompt and appropriate medical services through the online diagnosis.

[0324] A "chat-style medical interview input means" is a means for users to input symptoms and health conditions through a text-based chat interface.

[0325] A "diagnostic processing means using a generative AI model" is a means for analyzing a medical condition and making a diagnosis using a generative AI model based on information entered by a user.

[0326] "Means for issuing prescriptions or shipping medicines when symptoms are mild or the effects of prescribed medicines are minor" refers to means for the system to automatically issue prescriptions or ship medicines when the user's symptoms are mild and the effects of prescribed medicines are minor.

[0327] "Means for referring users to online diagnosis in complex cases or when there are concerns" refers to the means by which the system directs users to online diagnosis when the user's symptoms are complex or when there are concerns.

[0328] "Means for delivering medicine to the user's home at the user's request" refers to means for delivering prescribed medicine to the user's home if the user so requests.

[0329] "Means for having real-time video calls with doctors through online diagnosis" means a means for users to have real-time video calls with doctors and receive detailed diagnoses.

[0330] A system for implementing the present invention has the following configuration.

[0331] First, the user inputs a medical questionnaire in chat format using a smartphone or smart glasses. The medical questionnaire input method allows users to input symptoms and health conditions through a text-based interface, allowing users to easily communicate their symptoms to the system.

[0332] The input information is then sent to a diagnostic processing means that uses a generative AI model. This means uses the generative AI model to analyze the condition and make a diagnosis based on the information entered by the user. Specifically, the generative AI model analyzes the user's symptoms and outputs an appropriate diagnosis. Based on this diagnosis result, the system proceeds to the next step.

[0333] If the symptoms are mild or the effects of the prescribed medication are minor, the system will automatically issue a prescription or ship the medication. This prescription issuing method automatically generates a prescription and sends it to a designated pharmacy if the user's symptoms are mild and the effects of the prescribed medication are minor. It also includes a method for delivering medication to the user's home, if desired. This allows the user to receive the necessary medication without leaving home.

[0334] On the other hand, if the user has complex symptoms or concerns, the system will refer the user to an online diagnosis. This online diagnosis method allows the user to have a real-time video call with a doctor to receive a detailed diagnosis. The video call is made using the camera and microphone on a smartphone or smart glasses. This allows the user to directly interact with a doctor from the comfort of their own home and receive a proper diagnosis.

[0335] For example, if a user inputs symptoms such as "cough, fever," the generative AI model analyzes them and diagnoses them as a "cold." Based on this diagnosis, the system issues a prescription for "500mg paracetamol" and sends it to the designated pharmacy. If the user requests it, the system can also arrange for the medicine to be delivered to their home.

[0336] Examples of prompts include:

[0337] User symptoms: Cough, fever

[0338] Diagnosis: Cold

[0339] Prescription: Paracetamol 500mg

[0340] In this way, this invention allows users to easily receive a diagnosis from home and quickly receive the necessary medication. In addition, even in cases of complex symptoms, users can receive prompt and appropriate medical services through online diagnosis.

[0341] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0342] Step 1:

[0343] The user inputs the medical questionnaire in chat format using a smartphone or smart glasses.

[0344] Input: Text data about the user's symptoms and health conditions.

[0345] Data processing: Analyze text data using natural language processing technology to extract symptoms.

[0346] Output: Extracted symptom data.

[0347] Step 2:

[0348] The server transmits the input symptom data to a diagnostic processing means using a generative AI model.

[0349] Input: Extracted symptom data.

[0350] Data computation: Generative AI models analyze symptom data and diagnose medical conditions.

[0351] Output: Diagnosis result data (e.g. cold, flu, etc.).

[0352] Step 3:

[0353] Based on the diagnosis, the server issues a prescription if the symptoms are mild or the effects of the prescribed medication are minor.

[0354] Input: Diagnostic result data.

[0355] Data processing: Generate appropriate prescriptions based on diagnostic results.

[0356] Output: Prescription data.

[0357] Step 4:

[0358] The server sends the generated prescription data to the designated pharmacy.

[0359] Input: Prescription data.

[0360] Data Computation: Converts prescription data into a format for transmission to the pharmacy system.

[0361] Output: Prescription data sent to the pharmacy.

[0362] Step 5:

[0363] If desired, the server will arrange for the medicine to be delivered to the user's home.

[0364] Input: User's delivery preference data and prescription data.

[0365] Data processing: Generate data for delivery arrangements and send it to the delivery service.

[0366] Output: Delivery arrangement data sent to the delivery service.

[0367] Step 6:

[0368] If there are complex symptoms or concerns, the server will direct the user to an online diagnosis.

[0369] Input: Diagnostic result data.

[0370] Data calculation: When the diagnostic results are complex, generate an online diagnostic schedule.

[0371] Output: Online diagnostics schedule data.

[0372] Step 7:

[0373] The server arranges for a real-time video call with a doctor through online diagnosis.

[0374] Input: Online diagnostic schedule data and user video call preference data.

[0375] Data processing: Generate connection information for the video call and send it to the user and doctor.

[0376] Output: Video call connection information.

[0377] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0378] "Example 1"

[0379] The invention form combined with the emotion engine recognizes the user's emotions and utilizes them in diagnosis. Specifically, when the user enters a medical questionnaire, emotions are analyzed from the text input and voice. For example, if the user enters "I have a headache," the emotion engine will recognize from the text that the user is feeling pain. Furthermore, emotions are analyzed from the voice based on the tone, strength, and speed of the user's voice. This information can be used for AI diagnosis. "Example Form 2"

[0380] The invention, which incorporates an emotion engine, adjusts the content of the report to the doctor according to the user's emotional state. Specifically, if the emotion engine recognizes that the user is in severe pain, it will report that information to the doctor first. This allows the doctor to make a diagnosis that takes into account not only the user's symptoms but also their emotional state.

[0381] "Example 3"

[0382] The invention, which combines an emotion engine, recognizes the user's emotions and utilizes them for diagnosis. Specifically, when the user enters a medical questionnaire, emotions are analyzed from the text input and voice. For example, if the user enters "I have a headache," the emotion engine recognizes from the text that the user is feeling pain. Emotions are also analyzed from the voice, based on the tone, strength, and speed of the user's voice. This information can be used for AI diagnosis.

[0383] The processing flow of each embodiment will be described below.

[0384] "Example 1"

[0385] Step 1: The user begins entering the medical questionnaire.

[0386] Step 2: The user describes their symptoms by text input or voice.

[0387] Step 3: The emotion engine analyzes emotions from the user's text input or voice.

[0388] Step 4: The emotion engine sends the analyzed emotion information to the AI.

[0389] Step 5: The AI ​​makes a diagnosis taking emotional information into account.

[0390] "Example 2"

[0391] Step 1: The user begins entering the medical questionnaire.

[0392] Step 2: The user describes their symptoms by text input or voice.

[0393] Step 3: The emotion engine analyzes emotions from the user's text input or voice.

[0394] Step 4: The emotion engine reports the analyzed emotional information to the doctor.

[0395] Step 5: The doctor makes a diagnosis, taking emotional information into account.

[0396] "Example 3"

[0397] Step 1: The user begins entering the medical questionnaire.

[0398] Step 2: The user describes their symptoms by text input or voice.

[0399] Step 3: The emotion engine analyzes emotions from the user's text input or voice.

[0400] Step 4: The emotion engine sends the analyzed emotion information to the AI.

[0401] Step 5: The AI ​​makes a diagnosis taking emotional information into account.

[0402] Example 1

[0403] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0404] In conventional medical systems, it is difficult for users to accurately describe their symptoms, and diagnosis does not take into account emotional changes, which can delay appropriate diagnosis and treatment. Furthermore, even in cases where symptoms are mild or the effects of prescribed medication are minor, users must visit a medical institution in person, which takes time and effort.

[0405] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0406] In this invention, the server includes a chat-style medical interview input means, a text analysis means using natural language processing technology according to the input content, a means for extracting important keywords from the analysis results, a means for analyzing the user's emotions using an emotion engine, a means for making a diagnosis based on the analysis results, a means for issuing a prescription or shipping medicine if the symptoms are mild or the effects of the prescribed medicine are minor, and a means for sending the patient to an online diagnosis if the symptoms are complicated or there are concerns. This makes it possible to accurately analyze the user's symptoms and emotions and quickly provide an appropriate diagnosis and treatment.

[0407] The "chat-style medical interview input means" is a means for a user to input their symptoms through a text-based interface.

[0408] "Means for analyzing text using natural language processing technology" refers to means that use technology to analyze text data entered by a user and understand its meaning.

[0409] The "means for extracting important keywords" refers to a means for identifying important words and phrases related to symptoms and medical conditions from the analyzed text data.

[0410] The "means for analyzing user emotions using an emotion engine" refers to a means for using technology to recognize and analyze emotions from user text or voice data.

[0411] The "means for diagnosing based on the analysis results" refers to a means for diagnosing the user's symptoms and condition based on the extracted keywords and the results of emotion analysis.

[0412] "Means for issuing prescriptions or shipping medicines" refers to means for issuing prescriptions to users or shipping medicines directly to users when the symptoms or effects of prescribed medicines are minor.

[0413] "Online diagnostic referral" refers to a method for users with complex symptoms or concerns to be diagnosed, such as through a real-time video call with a doctor.

[0414] This invention is a system realized as a medical application. Users input their symptoms in chat format through the application. This input is analyzed using natural language processing technology, and important keywords for the symptoms are extracted. Furthermore, by combining it with an emotion engine, the system recognizes the user's emotions and uses them for diagnosis.

[0415] Hardware and software used

[0416] Hardware

[0417] Device: The device used by the user, such as a smartphone, tablet, or computer

[0418] Server: Cloud server for data analysis and diagnostic processing

[0419] software

[0420] Natural language processing (NLP) technologies: Python's NLTK library, SpaCy, etc.

[0421] Sentiment analysis library: TextBlob and Librosa, a speech analysis library

[0422] AI models: BERT (Bidirectional Encoder Representations from Transformers), etc.

[0423] Program processing

[0424] When a user enters their symptoms through the application, the device sends the text data to a server. The server then uses NLP technology to analyze the text and extract important keywords from the symptoms. For example, if a user enters "I've had a terrible headache recently, and I can't sleep at night," the server will extract keywords such as "headache" and "can't sleep."

[0425] Furthermore, the server uses an emotion engine to analyze the user's emotions. When the user inputs text, the server recognizes the user's emotions from that text. For example, if the user inputs "I have a headache," the server recognizes from that text that the user is in pain. When there is voice input, the server analyzes the user's emotions from the tone, strength, and speed of the voice.

[0426] This information is used for AI diagnosis. The server provides the user with appropriate advice and a diagnosis based on the analysis results. For example, if a user inputs, "I've had a terrible headache recently and can't sleep at night," the server will use this information to diagnose the possibility of migraines or stress-related insomnia, and send the results back to the user.

[0427] Examples and prompts

[0428] As a concrete example, consider the case where a user types, "I've had a terrible headache recently, and I can't sleep at night." In this case, the device sends this text data to the server. The server uses Python's NLTK library to analyze the text and extract keywords such as "headache" and "can't sleep." It then uses TextBlob to recognize that the user is experiencing pain. Based on this information, the server uses an AI model to diagnose the possibility of migraine or stress-related insomnia, and sends the results back to the user.

[0429] An example of a prompt to input to a generative AI model is as follows:

[0430] A user types, "I've been having terrible headaches lately and can't sleep at night." Analyze this text and extract important keywords and the user's sentiment.

[0431] Using this prompt, the generative AI model can accurately analyze the user's symptoms and emotions and provide appropriate diagnoses and advice.

[0432] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0433] Step 1:

[0434] User enters symptoms

[0435] Users launch the medical application and enter their symptoms through a chat-style interface. For example, they might enter, "I've had a terrible headache recently, and I can't sleep at night." The input data is saved in text format on the device.

[0436] Step 2:

[0437] The device sends the input data to the server

[0438] The terminal transmits the text data entered by the user to the server in real time. The data is encrypted before transmission, ensuring security. The input data is transmitted to the server in text format.

[0439] Step 3:

[0440] The server analyzes the text using natural language processing (NLP) techniques.

[0441] The server analyzes the received text data using natural language processing (NLP) techniques. Specifically, it uses NLP tools such as Python's NLTK library or SpaCy. The input data is in text format, and the output data is the analyzed text data.

[0442] Step 4:

[0443] The server extracts important keywords

[0444] The server extracts important keywords from the results of NLP analysis. For example, keywords such as "headache" and "can't sleep" are extracted. This process uses techniques such as TF-IDF (Term Frequency-Inverse Document Frequency) and BERT (Bidirectional Encoder Representations from Transformers). The input data is the analyzed text data, and the output data is the extracted keywords.

[0445] Step 5:

[0446] The server analyzes the user's emotions using an emotion engine.

[0447] The server uses an emotion engine to analyze the user's emotions. From the text, it is recognized that the user is experiencing pain. If there is voice input, the server also analyzes emotions from the tone, strength, and speed of the user's voice. This process uses the emotion analysis library TextBlob and the voice analysis library Librosa. The input data is text data and voice data, and the output data is the analyzed emotional information.

[0448] Step 6:

[0449] The server performs a diagnosis based on the analysis results.

[0450] The server uses an AI model to make a diagnosis based on the extracted keywords and the results of emotion analysis. For example, if a user inputs "I have a terrible headache and can't sleep at night," the server will use this information to diagnose the possibility of migraines or stress-related insomnia. The input data are the extracted keywords and analyzed emotion information, and the output data are the diagnosis results.

[0451] Step 7:

[0452] The server returns the diagnostic results to the user.

[0453] The server returns the diagnostic results to the user, who can then review them through the application and find out any necessary advice or next steps. The input data is the diagnostic results, and the output data is the diagnostic information that is displayed to the user.

[0454] (Application example 1)

[0455] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0456] Autonomous vehicles require systems that can monitor the health status of passengers in real time and promptly notify medical institutions as necessary. However, conventional systems have had the challenge of being unable to accurately analyze passenger symptoms and emotions and provide appropriate responses. In particular, in emergencies, where a rapid and accurate response is required, more advanced technology is required.

[0457] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0458] In this invention, the server includes a chat-style medical interview input means, an analysis means using natural language processing technology according to the input content, a means for analyzing the user's emotions using an emotion engine, a means for making an initial diagnosis based on the analysis results, a means for notifying a medical institution as necessary, and a means for directing the self-driving vehicle to the nearest medical institution in an emergency. This allows for real-time monitoring of the passenger's health condition and enables prompt and accurate response.

[0459] The "chat-style medical interview input means" is an interface that allows passengers to input their symptoms using text.

[0460] "Analysis means using natural language processing technology" is a technology that analyzes input text and extracts important keywords.

[0461] "Means for analyzing user emotions using an emotion engine" is a technology that analyzes user emotions from text and voice.

[0462] "Means for making an initial diagnosis based on the analysis results" refers to a means by which AI makes an initial diagnosis based on the results of natural language processing technology and sentiment analysis.

[0463] The "means for notifying medical institutions as necessary" refers to a means for notifying medical institutions as necessary based on the analysis results.

[0464] "Means for directing an autonomous vehicle to the nearest medical institution in an emergency" refers to means for an autonomous vehicle to automatically direct itself to the nearest medical institution in an emergency.

[0465] The system for implementing this invention has the following configuration: First, passengers input their symptoms in chat format using their smartphones or the in-car display, which enables real-time monitoring of the passenger's health condition.

[0466] The server analyzes the input text using natural language processing technology to extract important keywords. This analysis uses natural language processing libraries such as spaCy. For example, if a passenger types "I have a headache and I feel sick," the system will extract keywords such as "headache" and "feeling sick."

[0467] Next, the emotion engine analyzes the user's emotions using Hugging Face's Transformers library and TextBlob, which analyzes user emotions from text and speech to determine, for example, whether the passenger is in distress.

[0468] Based on the analysis results, the server makes an initial diagnosis. This initial diagnosis is performed using an AI model, and if necessary, a medical institution is notified. For example, if a passenger inputs "I have a headache and I'm feeling unwell," and the results of the emotion analysis indicate a high level of urgency, the system will automatically notify a medical institution.

[0469] Additionally, in case of an emergency, the self-driving vehicle will be directed to the nearest medical facility, ensuring the safety of passengers.

[0470] As a concrete example, if a passenger inputs "I have a headache and feel unwell," the system will operate as follows. First, natural language processing technology is used to extract keywords such as "headache" and "feeling unwell." Next, an emotion engine is used to analyze whether the passenger is experiencing pain. Based on the analysis results, the AI ​​will make an initial diagnosis and notify a medical institution if necessary. In the event of an emergency, the self-driving vehicle will be instructed to head to the nearest medical institution.

[0471] An example of a prompt is as follows:

[0472] "If a user types, 'I have a headache and I'm feeling sick,' explain how the system will parse that and what action will be taken."

[0473] In this way, the present invention provides a system for monitoring passenger health conditions in real time, enabling rapid and accurate response.

[0474] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0475] Step 1:

[0476] Users input their symptoms in chat format using their smartphone or in-car display.

[0477] Input: Text entered by the user (e.g., "I have a headache and feel sick")

[0478] Output: Input text data

[0479] Step 2:

[0480] The server analyzes the input text using natural language processing technology and extracts important keywords.

[0481] Input: Text data entered by the user

[0482] Data processing: Analyze the text using a natural language processing library such as spaCy to extract important keywords such as nouns and adjectives.

[0483] Output: Extracted keywords (e.g., "headache" or "feeling unwell")

[0484] Step 3:

[0485] The server analyzes the user's emotions using an emotion engine.

[0486] Input: Text data entered by the user

[0487] Data Computation: Analyze emotions from text and audio using Hugging Face's Transformers library and TextBlob.

[0488] Output: Sentiment analysis result (e.g., "feeling distressed")

[0489] Step 4:

[0490] The server performs an initial diagnosis based on the analysis results.

[0491] Input: Extracted keywords and sentiment analysis results

[0492] Data calculation: Using a generative AI model, an initial diagnosis is made based on keywords and sentiment analysis results.

[0493] Output: Initial diagnosis result (e.g. "High Urgency")

[0494] Step 5:

[0495] The server notifies the medical institution as necessary.

[0496] Input: Initial diagnosis result

[0497] Data processing: Based on the initial diagnosis results, medical institutions will be notified if necessary.

[0498] Output: Notification to the medical institution (e.g. "There is an emergency patient")

[0499] Step 6:

[0500] In the event of an emergency, the server will direct the self-driving vehicle to the nearest medical facility.

[0501] Input: Initial diagnosis result

[0502] Data calculation: If the emergency is deemed high, the system will control the navigation system of the autonomous vehicle and direct it to the nearest medical facility.

[0503] Output: Directions for the autonomous vehicle (e.g., "Go to the nearest medical facility")

[0504] Example 2

[0505] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0506] While conventional medical systems can analyze a patient's image data to determine their condition, they do not take into account the patient's emotional state. As a result, even if a patient is in severe pain, this information may not be properly conveyed to the doctor, potentially resulting in a decline in the quality of diagnosis and treatment. Furthermore, there is a lack of means to respond quickly to cases where the patient's symptoms are mild or the effects of prescribed medication are minor.

[0507] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a chat-style medical interview input means, an image analysis means according to the input content, a means for issuing a prescription or shipping medicine in cases where the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending the patient to an online diagnosis in cases where the symptoms are complicated or there are concerns, a means for analyzing the user's emotional state, and a means for adjusting the report content based on the user's emotional state. This enables a diagnosis that takes into account not only the patient's medical condition but also their emotional state, allowing doctors to provide more appropriate treatment. Furthermore, in cases where the symptoms are mild, a quick response can be made, reducing the burden on the patient.

[0508] The "chat-style medical interview input means" is an interface that allows the user to input information such as symptoms and medical history via text messages.

[0509] "Image analysis means" is a technology that analyzes image data provided by users and determines the condition of the patient. Specifically, it uses deep learning models to extract features from the image and identify abnormalities.

[0510] "Means for issuing prescriptions or shipping medicines" refers to a system for issuing prescriptions or shipping medicines to patients based on a doctor's instructions in cases where the symptoms or effects of prescription medicines are minor.

[0511] "Online referral" refers to technology that allows patients to be referred for online evaluation, such as through a real-time video call with a doctor, in complex cases or when there are concerns.

[0512] "Means for analyzing the user's emotional state" refers to technology that analyzes the user's facial expressions and voice data to determine the emotional state the user is feeling. Specifically, it uses algorithms for facial expression recognition and voice analysis.

[0513] The "means for adjusting the report content based on the emotional state" is a technology for adjusting the report content to the doctor based on the emotional state of the user. Specifically, if the user is experiencing severe pain, that information is reported to the doctor as a priority.

[0514] This invention is a system that analyzes image data and emotional states provided by a user and reports the results to a doctor. A specific embodiment for implementing this system is described below.

[0515] The server uses a secure communication protocol (e.g., HTTPS) to receive image data uploaded by users. Users use their own devices (smartphones or PCs) to take photos of abnormal areas of the skin and upload the image data through the system's dedicated application or website.

[0516] The server passes the received image data to the AI ​​image diagnostic processing means, which analyzes the image using a pre-trained deep learning model (e.g., ResNet or Inception) to determine the condition. Specifically, the model extracts features within the image and identifies patterns of abnormalities.

[0517] The analysis results, including a diagnosis estimate and a confidence score, are reported to the doctor via a dedicated dashboard or email.

[0518] Additionally, the server uses an emotion engine to analyze the user's emotional state. The emotion engine uses tools such as OpenCV and Google® Cloud Speech-to-Text API to analyze the user's facial expressions and voice data to determine whether the user is experiencing severe distress.

[0519] The server adjusts the content of the report to the doctor based on the user's emotional state obtained by the emotion engine. Specifically, if the user is experiencing severe pain, that information is reported to the doctor first. This allows the doctor to make a diagnosis that takes into account not only the user's symptoms but also their emotional state.

[0520] For example, when a user wants to upload a photo of an abnormal area of ​​skin, they open a dedicated application and click the "Upload Image" button. The server receives the uploaded image data and passes it to the AI ​​image diagnosis processing means. The AI ​​image diagnosis processing means analyzes the image and determines that there is a "high possibility of skin cancer." The server reports this analysis result to a doctor. At the same time, if the emotion engine recognizes from the user's facial expression that they are in great pain, it will also report this information to the doctor as a priority.

[0521] Example prompts to input to a generative AI model:

[0522] "Analyze photos of abnormal skin areas provided by the user and determine the medical condition. Also, analyze the user's emotional state and report any significant distress to a doctor as a priority."

[0523] In this way, a system is realized in which the server, terminal, and user work together to provide more accurate diagnoses and medical services that take into account the user's emotional state.

[0524] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0525] Step 1:

[0526] The user uploads image data.

[0527] The user takes a photo of the abnormal area of ​​the skin using their own device (smartphone or PC) and uploads the image data through the system's dedicated application or website. The input is the image data taken by the user, and the output is the image data sent to the server.

[0528] Step 2:

[0529] The server receives the image data.

[0530] The server receives image data uploaded by the user. At this time, the image data is transmitted using a secure communication protocol (e.g., HTTPS). The input is the image data transmitted by the user, and the output is the image data stored on the server.

[0531] Step 3:

[0532] The server analyzes the images using AI image diagnostic processing means.

[0533] The server passes the received image data to the AI ​​image diagnostic processing means, which uses a pre-trained deep learning model (e.g., ResNet or Inception) to analyze the image and determine the condition. The input is the image data stored on the server, and the output is an estimated condition and a confidence score. Specifically, the model extracts features from the image and identifies patterns of abnormalities.

[0534] Step 4:

[0535] The server reports the analysis results to the doctor.

[0536] The server reports the analysis results obtained by the AI ​​image diagnosis processing means to the doctor. The report contents include the estimated results of the disease state and a reliability score. The input is the estimated results of the disease state and a reliability score, and the output is the report contents sent to the doctor. The report is sent via a dashboard dedicated to the doctor or via email.

[0537] Step 5:

[0538] The server uses an emotion engine to analyze the user's emotional state.

[0539] The server uses an emotion engine to analyze the user's emotional state. The emotion engine uses tools such as OpenCV and Google Cloud Speech-to-Text API to analyze the user's facial expression and voice data. The input is the user's facial expression and voice data, and the output is the judgment result of the user's emotional state. Specific operations include using algorithms for facial expression recognition and voice analysis.

[0540] Step 6:

[0541] The server adjusts the report to the physician based on the emotional state.

[0542] The server adjusts the content of the report to the doctor based on the user's emotional state obtained by the emotion engine. Specifically, if the user is experiencing severe pain, that information is reported to the doctor first. The input is the result of the user's emotional state assessment, and the output is the adjusted report content. This allows the doctor to make a diagnosis that takes into account not only the user's symptoms but also their emotional state.

[0543] (Application example 2)

[0544] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0545] Conventional security systems have the problem of being unable to take into account the emotional state of security guards when analyzing surveillance camera footage and on-site photos, making it difficult to properly assess high-urgency situations.In addition, because the emotional state of security guards is not reflected in reports, it is difficult for the security center to take prompt and appropriate action.

[0546] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0547] In this invention, the server includes a chat-style medical interview input means, an AI image diagnosis processing means according to the input content, a means for issuing a prescription or shipping medicine if the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending the user to an online diagnosis if the symptoms are complicated or there are concerns, an emotion engine for analyzing the user's emotional state, and a means for adjusting the report content according to the emotional state, which enables anomaly detection and reporting that takes into account the emotional state of the security guard.

[0548] The "chat-style medical interview input means" is an interface that allows a user to input information in a text-based interactive format.

[0549] "AI image diagnostic processing means" refers to a processing means that uses artificial intelligence to analyze image data and determine the condition or abnormality.

[0550] "Means for issuing prescriptions or shipping medicines" refers to means for issuing prescriptions for medicines or shipping medicines to users when the symptoms or effects of prescription medicines are minor.

[0551] "Online referral" refers to a method to direct users to a real-time video call with a doctor for diagnosis in complex cases or where there are concerns.

[0552] The "emotion engine that analyzes the user's emotional state" is an engine that analyzes the user's emotional state from facial expressions, voice, etc.

[0553] The "means for adjusting the report content according to the emotional state" is a means for adjusting the priority and details of the report content based on the emotional state of the user.

[0554] As an embodiment of this invention, we will explain the security diagnostic assistant system as an example. In this system, security guards use smartphones or smart glasses to take surveillance camera footage and photos of the scene, and AI analyzes the images to detect abnormalities. Furthermore, by combining it with an emotion engine, the system is equipped with a function to adjust the content of reports according to the emotional state of the security guard.

[0555] Hardware and software used

[0556] Hardware: Smartphones, smart glasses

[0557] Software: OpenCV (image processing library), Keras (deep learning library), EmotionRecognizer (emotion recognition library)

[0558] System configuration

[0559] 1. Chat-style interview input method: Provides an interface for security guards to input information in a text-based interactive format, allowing them to easily report the situation.

[0560] 2. AI image diagnostic processing method: AI analyzes surveillance camera footage and on-site photos to detect abnormalities. Specifically, images are preprocessed using OpenCV and analyzed using a deep learning model using Keras.

[0561] 3. A means of issuing prescriptions or shipping medicines: This is a means of issuing prescriptions for medicines or shipping medicines to users when the symptoms or effects of prescription medicines are minor. In security systems, it is also used as a means of automatically reporting minor abnormalities.

[0562] 4. Online diagnostics: In case of complications or concerns, this is a way to direct the security guard to a real-time video call with the security center for a diagnostic.

[0563] 5. Emotion engine that analyzes the user's emotional state: This engine analyzes the emotional state of the security guard from their facial expressions and voice. Emotion analysis is performed using EmotionRecognizer.

[0564] 6. Adjusting the report content according to the user's emotional state: This is a means to adjust the priority and details of the report content based on the user's emotional state. For example, if a security guard is feeling very tense or anxious, that information will be reported to the security center as a priority.

[0565] Specific examples

[0566] Example 1: A security guard uses smart glasses to analyze nighttime surveillance camera footage in real time and detects anomalies. The emotion engine detects the guard's state of tension and reports it as a priority.

[0567] Example 2: Using a smartphone, an AI system can detect abnormalities by analyzing the image. The emotion engine can detect the stress level of guards and report it as a priority.

[0568] Prompt Sentence Examples

[0569] "Analyze surveillance camera footage in real time to detect any anomalies. Adjust your reporting accordingly, taking into account the emotional state of the security guards."

[0570] "Take photos of the scene and use AI to detect anomalies. Use an emotion engine to tailor the report based on the emotional state of the security guard."

[0571] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0572] Step 1:

[0573] A user uses a smartphone or smart glasses to capture surveillance camera footage or photos of the scene.

[0574] Input: CCTV footage or photos of the scene

[0575] Output: Captured image data

[0576] Specific operation: The user uses the device's camera function to take video or photos of the monitored object. The captured image data is stored on the device.

[0577] Step 2:

[0578] The device sends the captured image data to an AI image diagnostic processing means, which detects abnormalities.

[0579] Input: Captured image data

[0580] Output: Anomaly detection results

[0581] Specific operation: The device preprocesses the image using OpenCV and analyzes it using a deep learning model using Keras. If an anomaly is detected as a result of the analysis, that information is returned to the device.

[0582] Step 3:

[0583] To analyze the user's emotional state, the device uses an emotion engine to analyze the user's facial expressions and voice.

[0584] Input: User's facial expression or voice data

[0585] Output: Emotional state

[0586] Specific operation: The device uses EmotionRecognizer to analyze the user's emotional state from their facial expressions and voice. The analysis results indicate whether the user is feeling nervous or stressed.

[0587] Step 4:

[0588] The device adjusts the report content based on the anomaly detection results and emotional state.

[0589] Input: Anomaly detection results, emotional state

[0590] Output: Adjusted report

[0591] Specific operation: The device combines the anomaly detection results with the user's emotional state to adjust the priority and details of the report. For example, if the user is feeling strong tension or anxiety, that information will be prioritized in the report.

[0592] Step 5:

[0593] The device sends the adjusted report to the security center.

[0594] Input: Adjusted report

[0595] Output: Report to Security Center

[0596] Specific operation: The terminal sends the adjusted report to the security center via the network. The security center takes appropriate action based on the received report.

[0597] Example 3

[0598] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0599] Conventional medical systems have difficulty comprehensively analyzing a user's symptoms and emotions and providing appropriate medical care. They also lack the functionality to automatically issue prescriptions for mild symptoms or minor effects of prescription medications, and to refer patients with complex symptoms or concerns to online diagnosis. Furthermore, the lack of technology to analyze a user's emotions and utilize them in diagnosis makes it difficult to accurately grasp the user's condition.

[0600] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0601] In this invention, the server includes a chat-style medical interview input means, an artificial intelligence image diagnosis processing means according to the input content, a means for issuing a prescription or shipping medicine if the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending the patient to an online diagnosis if the symptoms are complicated or there are concerns, a means for analyzing the user's emotions from text or voice using an emotion analysis engine, and a means for providing appropriate medical care based on the analysis results. This makes it possible to comprehensively analyze the user's symptoms and emotions and provide the most appropriate medical care.

[0602] The "chat-style medical interview input means" is an interface that allows users to input symptoms and conditions through text messages.

[0603] The "artificial intelligence image diagnosis processing means" is a processing means that uses artificial intelligence to analyze image data provided by a patient and determine the condition of the patient.

[0604] The "means for issuing prescriptions or shipping medicines" refers to a means by which a medical institution automatically issues a prescription or ships medicines to a user when the symptoms or effects of the prescribed medicines are minor.

[0605] "Online referral" refers to a means to direct users to a doctor for a diagnosis via a real-time video call if they have complex symptoms or concerns.

[0606] An "emotion analysis engine" is an engine that analyzes user emotions from text and voice data and uses that information for diagnosis.

[0607] The "means for providing appropriate medical treatment" refers to a means for providing optimal medical treatment to the user based on the analysis results of the emotion analysis engine.

[0608] MODE FOR CARRYING OUT THE INVENTION

[0609] The present invention is a system for analyzing a user's symptoms and emotions and providing appropriate medical treatment. Specific embodiments of this system will be described below.

[0610] System configuration

[0611] The system consists of the following main components:

[0612] 1. Chat-style medical interview input method

[0613] 2. AI image diagnostic processing method

[0614] 3. A means of issuing prescriptions or shipping medications

[0615] 4. Online diagnostics

[0616] 5. Sentiment Analysis Engine

[0617] 6. Means of providing appropriate medical care

[0618] Hardware and software used

[0619] Server: A central server for receiving, analyzing, and processing data.

[0620] Device: The smartphone or computer used by the user

[0621] Sentiment analysis engine: Sentiment analysis software such as IBM Watson (registered trademark) or Google Cloud Natural Language API

[0622] Video calling software: Software for real-time video calling

[0623] System Operation

[0624] 1. The user enters the symptoms

[0625] The user uses a terminal to input text or voice input of symptoms into the system's interface. For example, the user may input "I've been having headaches lately."

[0626] 2. The server receives the input data

[0627] The server receives text and voice data sent by the user, which serves as the basis for analyzing the user's symptoms and emotions.

[0628] 3. The server analyzes the data using the emotion engine

[0629] The server uses an emotion analysis engine to analyze the received text and voice data. The emotion engine recognizes the user's emotions from the text and analyzes the tone, strength, and speed of the voice from the voice data. For example, if a user types "I have a headache," the emotion engine will recognize from the text that the user is feeling pain.

[0630] 4. The server issues a prescription based on the analysis results or sends the patient for online diagnosis.

[0631] Based on the analysis results of the emotion engine, the server determines whether the user's symptoms are mild or complicated or cause concerns. If the symptoms are mild, the server automatically issues a prescription. If the symptoms are complicated, the server sends the user for an online diagnosis.

[0632] 5. The server sends the prescription to the designated pharmacy or arranges for the medication to be delivered to the user's home.

[0633] The server electronically sends the issued prescription to the designated pharmacy and, if the user wishes, arranges for the medication to be delivered directly to the user's home.

[0634] 6. When users receive an online diagnosis, they can have a video call with a doctor via their device.

[0635] Users can use the device to interact directly with a doctor via real-time video call, who will provide a detailed diagnosis of the user's symptoms and provide the necessary prescriptions and treatments.

[0636] Specific examples

[0637] As a concrete example, consider the case where a user inputs "I've been having a headache lately." In this case, the emotion engine recognizes that the user is in pain from the keyword "headache." Furthermore, if there is voice input, it also analyzes emotions from the tone and speed of the voice.

[0638] Example prompt sentence:

[0639] If a user types, "I've been having headaches lately," the emotion engine recognizes from the text that the user is in pain. It also analyzes emotions from voice input, such as tone and speed. Based on the analysis, the server issues an appropriate prescription and sends it to a designated pharmacy or directs the user to an online diagnosis.

[0640] This system aims to comprehensively analyze the symptoms and emotions of the user and provide the most appropriate medical response. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0641] Step 1:

[0642] User enters symptoms

[0643] The user uses a terminal to input text about symptoms into the system's interface. Voice input is also possible. For example, the user might input, "I've been having a headache lately." The input data is sent to the server as text or voice data.

[0644] Step 2:

[0645] The server receives the input data

[0646] The server receives text and voice data sent by the user. The received data becomes the basis for analyzing the user's symptoms and emotions. The input data is saved as is and passed to the next analysis step.

[0647] Step 3:

[0648] The server analyzes the data using an emotion engine

[0649] The server uses an emotion analysis engine to analyze the received text and voice data. The emotion engine recognizes the user's emotions from the text and analyzes the tone, strength, and speed of the voice from the voice data. For example, if a user types "I have a headache," the emotion engine recognizes from the text that the user is feeling pain. The analysis results are output as data indicating the user's emotional state.

[0650] Step 4:

[0651] The server issues a prescription based on the analysis results or sends the patient for online diagnosis.

[0652] Based on the analysis results of the emotion engine, the server determines whether the user's symptoms are mild or whether they are complicated or cause concern. If the symptoms are mild, the server automatically issues a prescription. If the symptoms are complicated, the server sends the user to an online diagnosis. The result of the judgment is output as prescription data or instructions for an online diagnosis.

[0653] Step 5:

[0654] The server sends the prescription to a designated pharmacy or arranges for the medication to be delivered to the user's home.

[0655] The server electronically sends the issued prescription to the designated pharmacy and, if desired, arranges for the medication to be delivered directly to the user's home. Prescription data is sent to the pharmacy, and delivery arrangement data is sent to the delivery company.

[0656] Step 6:

[0657] When a user receives an online diagnosis, they will have a video call with a doctor via their device.

[0658] The user uses the device to directly communicate with the doctor through a real-time video call. The doctor will perform a detailed diagnosis of the user's symptoms and provide the necessary prescriptions and treatment. The instruction to start the video call is sent to the user's device, and the video call data is exchanged in real time.

[0659] (Application example 3)

[0660] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0661] In modern society, there is a demand for appropriate medical services and meal recommendations based on the user's health condition and emotions. However, conventional systems have had challenges in analyzing the user's emotions, proposing optimal meal menus based on those emotions, and then delivering those meals quickly.

[0662] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes a chat-style medical interview input means, an AI image diagnosis processing means according to the input content, a means for issuing a prescription or shipping medicine in cases where the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending the user to an online diagnosis in cases where the symptoms are complicated or there are concerns, an emotion engine means for analyzing the user's emotions, a means for proposing an optimal meal menu based on the analysis results, and a means for delivering the suggested meals to the user's home. This makes it possible to provide optimal medical services and meal suggestions based on the user's health condition and emotions, and to deliver them quickly.

[0663] The "chat-style medical interview input means" is an interface that allows users to input their health condition and symptoms using text or voice.

[0664] "AI image diagnosis processing means according to input content" refers to a processing means that uses artificial intelligence to analyze image data provided by the user and determine the condition of the disease.

[0665] "Means for issuing prescriptions or shipping medicines when symptoms or effects of prescription medicines are minor" means means for automatically issuing appropriate prescriptions or shipping medicines to users when the user's symptoms are mild.

[0666] "Means for referring users to an online diagnosis in complicated cases or when there are concerns" refers to means for directing users to an online diagnosis with a doctor when the user's symptoms are complicated or when there are concerns.

[0667] The "emotion engine means for analyzing user's emotions" is an engine for analyzing emotions from the user's text or voice input and identifying the user's emotional state.

[0668] "Means for suggesting optimal meal menus based on analysis results" refers to means for suggesting optimal meal menus to users based on the analysis results of the emotion engine and health status.

[0669] "Means for delivering suggested meals to the user's home" refers to means for delivering a suggested meal to the user's home when the user selects the meal.

[0670] The system for implementing this invention aims to provide optimal medical services and meal recommendations based on the user's health condition and emotions, and deliver them promptly. Specific embodiments of this system will be described below.

[0671] System configuration

[0672] The system consists of the following main components:

[0673] 1. Chat-style medical interview input method:

[0674] Users enter their health status and symptoms using text or voice via a smartphone app.

[0675] Hardware required: Smartphone (iOS or ANDROID (registered trademark))

[0676] Software used: Speech recognition API (e.g., Google Cloud Speech-to-Text), text analysis API (e.g., Google Cloud Natural Language)

[0677] 2. AI image diagnosis processing method according to input content:

[0678] A processing method using artificial intelligence to analyze image data provided by the user and determine the condition of the patient.

[0679] Software used: AI models (TensorFlow, PyTorch, etc.)

[0680] 3. Means of issuing prescriptions or shipping medications for minor illnesses or minor effects of prescription medications:

[0681] If the user's symptoms are mild, the appropriate prescription will be automatically issued or medication will be shipped to the user.

[0682] Software used: Prescription management system

[0683] 4. Online diagnostics for complex cases or concerns:

[0684] If the user has a complex condition or concerns, they can be directed to an online consultation with a doctor.

[0685] Software used: Video call API (WebRTC, etc.)

[0686] 5. Emotion engine means to analyze user emotions:

[0687] An engine for analyzing emotions from a user's text or voice input and identifying their emotional state.

[0688] Software used: Sentiment analysis API (e.g. IBM Watson Tone Analyzer)

[0689] 6. Means of suggesting optimal meal menus based on analysis results:

[0690] Based on the analysis results of the emotion engine and health status, the system suggests the optimal meal menu for the user.

[0691] Software used: Recommendation engine (e.g., Amazon Personalize)

[0692] 7. How the suggested meals will be delivered to your home:

[0693] If the user selects a suggested meal, the meal will be delivered to the user's home.

[0694] Software used: Order management system (e.g. Uber Eats API)

[0695] System Operation

[0696] 1. Getting user input:

[0697] Users can enter their current health status and emotions via text or voice input through a smartphone app.

[0698] Example: A user says, "I've been feeling tired and stressed lately."

[0699] 2. Analysis by emotion engine:

[0700] The emotion engine analyzes input text and voice data to identify the user's emotional state.

[0701] Example: An emotion engine analyzes a user's emotions as "fatigue" and "stress."

[0702] 3. Health Assessment:

[0703] The AI ​​model analyzes data on the user's health condition and suggests appropriate meal menus.

[0704] Example: An AI model suggests menu items based on a user's health status, including nutritious meals and relaxing ingredients.

[0705] 4. Meal menu suggestions:

[0706] Based on the analysis results, the system suggests the most suitable meal menu for the user.

[0707] For example, the system suggests "a nutritious salad bowl," "relaxing herbal tea," and "high-protein grilled chicken."

[0708] 5. Arrange for meal delivery:

[0709] Once the user selects a suggested menu item, the system sends an order to a partner restaurant or delivery service and delivers the meal to the user's home.

[0710] Example: A user selects a "salad bowl" and the system accepts the order and delivers it within 30 minutes.

[0711] Prompt Sentence Examples

[0712] User: I've been feeling tired and stressed lately.

[0713] System: Emotion engine analyzing user emotion...

[0714] System: The user's emotional state has been determined to be "fatigue" and "stress." The following meal plan is suggested:

[0715] 1. A nutritious salad bowl

[0716] 2. Relaxing herbal tea

[0717] 3. High-protein grilled chicken

[0718] User: Select a salad bowl

[0719] System: Your order has been placed. Your salad bowl will be delivered within 30 minutes.

[0720] In this way, it is possible to provide optimal medical services and meal suggestions based on the user's health condition and emotions, and deliver them quickly.

[0721] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0722] Step 1:

[0723] The user launches the smartphone app and inputs their health status and emotions using text or voice. The input data is sent to the server via the smartphone's voice recognition API (such as Google Cloud Speech-to-Text) or text analysis API (such as Google Cloud Natural Language). The input data is in text format.

[0724] Step 2:

[0725] The server analyzes the received text data using an emotion engine (such as IBM Watson Tone Analyzer) to identify the user's emotional state. The input data is text data, and the output data is the emotional state (e.g., "fatigue," "stress," etc.). The emotion engine analyzes the emotional tone of the text data to identify the user's emotion.

[0726] Step 3:

[0727] The server analyzes the emotion engine's analysis results and the user's health status data using an AI model (such as TensorFlow or PyTorch) to generate an appropriate meal menu. The input data is the user's emotional state and health status data, and the output data is a list of meal menus. The AI ​​model generates a menu that includes ingredients that are nutritious and have a relaxing effect based on the user's health status and emotions.

[0728] Step 4:

[0729] The server proposes the generated meal menu to the user. The user checks and selects the proposed menu through a smartphone app. The input data is a list of meal menus, and the output data is the menu selected by the user. The user selects the desired meal from the proposed menu.

[0730] Step 5:

[0731] The server sends the order with the user's selected menu to the partner restaurant or delivery service. The input data is the user's selected menu, and the output data is the order information. The server orders the selected menu through an order management system (such as Uber Eats API).

[0732] Step 6:

[0733] After receiving the order, the partner restaurant or delivery service prepares the meal and delivers it to the user's home. The input data is the order information, and the output data is the delivered meal. The restaurant or delivery service prepares the meal based on the order and delivers it to the specified address.

[0734] In this way, a system is realized that proposes optimal meal menus based on the user's health condition and emotions and delivers them quickly.

[0735] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0736] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0737] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0738] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0739] [Second embodiment]

[0740] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0741] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0742] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0743] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0744] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0745] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0746] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0747] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0748] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0749] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0750] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0751] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0752] "Example 1"

[0753] An embodiment of the present invention is realized as a medical application. A user inputs their symptoms in a chat format through the application. This input is analyzed using natural language processing technology, and important keywords related to the symptoms are extracted. For example, if a user inputs "I have a headache," "headache" is extracted as a keyword.

[0754] "Example 2"

[0755] Next, the AI ​​image diagnosis processing means operates. The AI ​​analyzes the image data provided by the user (for example, a photo of an abnormal area on the skin) and determines the condition. The results of this analysis are reported to the doctor and serve as reference information for the final diagnosis.

[0756] "Example 3"

[0757] For mild symptoms or minor effects of prescribed medication, the system automatically issues a prescription and sends it to a designated pharmacy, or, if desired, can have the medication delivered directly to the user's home.

[0758] "Example 4"

[0759] If there are any complex symptoms or concerns, the system will refer the user to an online diagnosis, where they can interact directly with a doctor via a real-time video call for a detailed diagnosis.

[0760] The processing flow of each embodiment will be described below.

[0761] "Example 1"

[0762] Step 1: The user opens the medical application and enters their symptoms in a chat format.

[0763] Step 2: The application uses natural language processing techniques to analyze the user's input and extract key symptoms keywords.

[0764] "Example 2"

[0765] Step 1: The application receives user-provided image data (e.g., a photo of an abnormal area of ​​skin).

[0766] Step 2: The AI ​​analyzes the image data and determines the condition. The results of this analysis are reported to the doctor.

[0767] "Example 3"

[0768] Step 1: If the AI ​​determines that the user's symptoms are mild, the system will automatically issue a prescription.

[0769] Step 2: The issued prescription is sent to the designated pharmacy.

[0770] Step 3: If desired, the medication is delivered directly to the user's home.

[0771] "Example 4"

[0772] Step 1: If the AI ​​determines that the user's symptoms are complex or concerning, the system will refer the user to an online diagnosis.

[0773] Step 2: Users can interact directly with a doctor through a real-time video call and receive a detailed diagnosis.

[0774] Example 1

[0775] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0776] In conventional medical systems, when users input their symptoms, it is difficult to extract appropriate keywords and provide prompt and accurate medical advice. Furthermore, there are insufficient means to provide appropriate treatment for mild symptoms or when the effects of prescription medication are minor, and there are also insufficient means to quickly refer users to online diagnoses for complex symptoms or concerns. This reduces user convenience and compromises the quality of medical services.

[0777] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0778] In this invention, the server includes a chat-style medical interview input means, a means for analyzing the input content using natural language processing technology and extracting important keywords, and a means for sending the extracted keywords to a generative AI model to generate appropriate medical advice. This makes it possible to quickly and accurately extract keywords based on the symptoms entered by the user and provide appropriate medical advice.

[0779] The "chat-style medical interview input means" is a means for providing an interface for the user to input their own symptoms in text format.

[0780] "Natural language processing technology" is a technology for analyzing input text data and understanding its meaning.

[0781] The "means for extracting important keywords" is a means for identifying important words and phrases related to symptoms from input text using natural language processing technology.

[0782] A "generative AI model" is an artificial intelligence model that generates appropriate medical advice or next steps based on the input prompt.

[0783] The "means for generating medical advice" is a means for using a generative AI model to generate medical advice to be provided to a user based on the extracted keywords.

[0784] "Means for issuing a prescription or shipping medicine when symptoms are mild or the effects of prescribed medicine are minor" refers to means for issuing a prescription or shipping medicine when the user's symptoms are mild and the effects of prescribed medicine are minor.

[0785] "Online diagnosis" refers to a method for providing a diagnosis through a real-time video call with a doctor if the user's symptoms are complex or of concern.

[0786] This invention is a system realized as a medical application, in which users input their symptoms in a chat format, which is analyzed using natural language processing technology, important keywords are extracted, and appropriate medical advice is provided using a generative AI model.

[0787] A user opens a medical application on a device such as a smartphone or tablet. The application has a chat-style interface, and the user inputs their symptoms in text format. For example, they might input, "I've been having such a bad headache lately that I can't sleep."

[0788] The terminal sends the text data entered by the user to the server using the HTTPS protocol, and the server temporarily stores the received text data in a database.

[0789] The server uses Python's natural language processing libraries, NLTK and spaCy, to tokenize, tag parts of speech, and analyze dependencies on the received text data. This allows important keywords to be extracted from the text data. For example, the keywords "headache" and "can't sleep" are extracted.

[0790] The server sends the extracted keywords to the generative AI model as a prompt sentence. An example of a prompt sentence would be, "User input: I've been having a bad headache lately and can't sleep. Extracted keywords: headache, can't sleep."

[0791] The generative AI model generates appropriate medical advice and next steps based on the input prompt, and the generated medical advice is sent to the device via the server.

[0792] The terminal displays the medical advice received from the server to the user, who can then confirm the displayed medical advice and decide on the next course of action.

[0793] For example, if a user inputs "I've had a bad headache recently and can't sleep," the server extracts the keywords "headache" and "can't sleep" and sends a prompt to the generative AI model. The generative AI model generates medical advice such as "If your headache persists, consult a doctor. We also recommend taking a warm bath to relax," and sends it to the device via the server. The device then displays this advice to the user.

[0794] This system allows users to receive fast and accurate medical advice based on their symptoms, providing appropriate treatment for mild symptoms or minor effects of prescribed medication, and quickly directing users to an online diagnosis for complex symptoms or concerns.

[0795] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0796] Step 1:

[0797] The user enters the symptoms.

[0798] A user opens a medical application and enters their symptoms in text format into a chat-style interface. For example, they might enter, "I've been having a bad headache lately and can't sleep." The input data is in text format.

[0799] Step 2:

[0800] The device sends input to the server.

[0801] The terminal sends the text data entered by the user to the server using the HTTPS protocol. The input data is in text format, and the output is the text data sent to the server.

[0802] Step 3:

[0803] The server receives the input.

[0804] The server receives text data sent from the terminal and temporarily stores it in a database. The input data is in text format, and the output is text data stored in the database.

[0805] Step 4:

[0806] The server performs natural language processing.

[0807] The server uses Python natural language processing libraries NLTK and spaCy to tokenize, tag parts of speech, and analyze dependencies on the received text data. The input data is in text format, and the output is the analysis results.

[0808] Step 5:

[0809] The server extracts the keywords.

[0810] The server extracts important keywords from the results of natural language processing. For example, the keywords "headache" and "can't sleep" are extracted. The input data is the analysis results, and the output is the extracted keywords.

[0811] Step 6:

[0812] The server sends the extraction results to the generative AI model.

[0813] The server sends the extracted keywords as a prompt to the generative AI model. An example of a prompt might be, "User input: I've been having a bad headache lately and can't sleep. Extracted keywords: headache, can't sleep." The input data are the extracted keywords, and the output is the prompt sent to the generative AI model.

[0814] Step 7:

[0815] The server receives the results of the generative AI model.

[0816] The server receives the results returned by the generative AI model, which include appropriate medical advice and next steps. The input data is the prompt sent to the generative AI model, and the output is the medical advice returned by the generative AI model.

[0817] Step 8:

[0818] The server sends the results to the terminal.

[0819] The server sends the results received from the generative AI model to the terminal using the HTTPS protocol. The input data is the medical advice returned by the generative AI model, and the output is the medical advice sent to the terminal.

[0820] Step 9:

[0821] The terminal displays the results to the user.

[0822] The terminal displays the medical advice received from the server to the user, who can then review the displayed medical advice and decide on the next course of action. The input data is the medical advice received from the server, and the output is the medical advice displayed to the user.

[0823] (Application example 1)

[0824] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0825] Conventional security services have had the challenge of efficiently analyzing users' anxieties and problems and providing appropriate responses quickly. In particular, it was difficult to accurately understand the text information entered by users and extract important keywords, making it impossible to provide appropriate countermeasures.

[0826] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0827] In this invention, the server includes a chat-style medical interview input means, a keyword extraction means using natural language processing technology according to the input content, a means for issuing a prescription or shipping medicine if the symptoms are mild or the effects of the prescribed medicine are minor, and a means for sending the patient to an online diagnosis if the symptoms are complicated or there are concerns. This makes it possible to accurately analyze the text information entered by the user, extract important keywords, and quickly provide appropriate countermeasures.

[0828] The "chat-style medical interview input means" is a means for providing an interface for users to input questions and symptoms in text format.

[0829] "Keyword extraction means using natural language processing technology" refers to a means that uses technology to analyze text information entered by the user and extract important keywords.

[0830] "Means for issuing prescriptions or shipping medicines when symptoms are mild or the effects of prescription medicines are minor" refers to means for issuing prescriptions or shipping medicines when the user's symptoms are determined to be mild and the effects of prescription medicines are determined to be minor.

[0831] "Online diagnostic means for complex or concerning cases" refers to a means for users to receive an online diagnosis from a specialist if their symptoms are complex or they have concerns.

[0832] "Means for enabling real-time video calls with experts" means means for enabling a user to have real-time video calls with experts.

[0833] The system for implementing this invention is configured as follows: First, a user inputs security-related concerns or problems in a chat format using a smartphone. The input text information is sent to a server.

[0834] The server uses natural language processing technology to analyze the user's input text and extract important keywords. This analysis is performed using Python and the natural language processing library spaCy. Specifically, if a user types "I've been seeing suspicious people around my house recently," the server will extract the keyword "suspicious person."

[0835] Based on the extracted keywords, the server determines the appropriate course of action: if the symptoms are mild or the effects of a prescription are minor, a prescription or medication delivery is used, while if the symptoms are more complex or there are concerns, the patient is referred for an online diagnosis, which includes a real-time video call with a specialist.

[0836] For example, if a user types in "I've been seeing suspicious people around my house lately," the server will extract the keyword "suspicious people" and suggest a video call with a security expert, allowing the user to communicate with the expert in real time using their smartphone.

[0837] Examples of prompts to input to a generative AI model include:

[0838] A user enters a security issue or concern. For example, if the user enters "I've been seeing suspicious people around my house lately," extract "suspicious people" as an important keyword.

[0839] In this way, it becomes possible to efficiently analyze users' security concerns and problems and quickly provide appropriate countermeasures.

[0840] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0841] Step 1:

[0842] A user uses a smartphone to input security-related concerns or problems in chat format. The input text information is sent from the smartphone to a server. The input data is text information about the user's concerns or problems.

[0843] Step 2:

[0844] The server analyzes the received text information using natural language processing technology. This analysis uses Python and the natural language processing library spaCy. Specifically, the text information is tokenized and important keywords such as nouns, proper nouns, and adjectives are extracted. The input data is the user's text information, and the output data is the extracted keywords.

[0845] Step 3:

[0846] The server determines the appropriate response based on the extracted keywords. If the symptoms are mild or the effects of the prescribed medication are minor, measures such as issuing a prescription or shipping the medication are used. The input data are the extracted keywords, and the output data are instructions to issue a prescription or ship the medication.

[0847] Step 4:

[0848] The server executes a means for routing the extracted keywords to an online diagnostic if they are complex or cause concern, including a means for enabling a real-time video call with an expert, where the input data is the extracted keywords and the output data is an instruction to start the video call.

[0849] Step 5:

[0850] Users can use their smartphones to make real-time video calls with experts, allowing them to receive advice and diagnoses directly from the experts. The input data is an instruction to start the video call, and the output data is the advice and diagnosis results from the experts.

[0851] Example 2

[0852] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0853] In conventional medical systems, it is difficult for patients to accurately communicate their symptoms, which often leads to delayed diagnosis, especially for visual symptoms such as skin abnormalities. Another problem is that even in cases of mild illness, it takes time and effort to receive a diagnosis from a doctor. Furthermore, patients living in remote areas often have difficulty accessing medical institutions. To solve these issues, a system is needed that can efficiently and quickly diagnose illnesses and provide appropriate medical services.

[0854] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0855] In this invention, the server includes a chat-style medical interview input means, an image diagnosis processing means according to the input content, a means for issuing a prescription or shipping medicine if the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending an online diagnosis if the symptoms are complicated or there are concerns, a means for sending image data taken by the user from the terminal to the server, an image analysis means using a generative AI model to analyze the image data received by the server, and a means for reporting the analysis results to a doctor. This allows patients to easily diagnose skin abnormalities from home and receive appropriate medical care promptly.

[0856] The "chat-style medical interview input means" is an interface that allows users to input information such as symptoms and medical history via text messages, and the system receives this information.

[0857] The "image diagnosis processing means" is a technical means for analyzing image data provided by a user and determining the condition of the disease.

[0858] "Means for issuing prescriptions or shipping medicines" refers to a function that enables the system to automatically issue prescriptions or ship medicines to patients when the symptoms or effects of prescribed medicines are minor.

[0859] "Online referral" is a feature that allows the system to refer patients to online consultations, such as a real-time video call with a doctor, in cases of complications or concerns.

[0860] "Means for transmitting image data taken by a user from a terminal to a server" refers to technical means for transmitting image data taken by a user to a server via a terminal.

[0861] "Image analysis means using generative AI models" refers to a technical means for analyzing image data and determining the condition of a disease using an AI model built using a deep learning framework.

[0862] "Means for reporting analysis results to doctors" refers to the technical means for reporting the analysis results obtained by the AI ​​model to doctors.

[0863] MODE FOR CARRYING OUT THE INVENTION

[0864] This invention is a system for analyzing image data captured by a user and determining the condition of a patient. The system includes a chat-style interview input means, an image diagnosis processing means, a means for issuing a prescription or shipping medicine, a means for sending the image data captured by the user from a terminal to a server, an image analysis means using a generative AI model, and a means for reporting the analysis results to a doctor.

[0865] Hardware and software used

[0866] Device: Smartphone, tablet, computer, or other device

[0867] Server: A server with high-performance computing power

[0868] Deep learning frameworks: TensorFlow, PyTorch, etc.

[0869] Data processing and calculation

[0870] The user takes a photo of the abnormal area on their skin using a smartphone or digital camera. After taking the photo, they upload the image data to the system using a dedicated application. The device encrypts the image data and sends it to the server using a secure communication protocol (e.g., HTTPS).

[0871] The server analyzes the received image data using a generative AI model. This AI model is built using deep learning frameworks such as TensorFlow and PyTorch. Specifically, it uses a convolutional neural network (CNN) to extract image features and identify patterns of abnormalities. This analysis results in a diagnosis of the condition, such as "suspected skin cancer" or "high probability of eczema."

[0872] The analysis results are reported from the server to the doctor, who then uses this information to make a final diagnosis. For example, if the doctor receives a result that suggests skin cancer, he or she can prescribe further detailed testing.

[0873] Specific examples

[0874] For example, if a user notices an abnormality in their skin, they take a photo of the area with their smartphone and upload the image to the system using a dedicated application. The device encrypts this image and sends it to a server using HTTPS. The server receives the image and analyzes it using a generative AI model. If the analysis results in a "suspected skin cancer," the information is reported to a doctor. The doctor can then use this information to prescribe further tests.

[0875] Prompt Sentence Examples

[0876] "Design a system that analyzes photos of abnormal skin areas taken by users, uses an AI model to determine the condition, and reports the diagnosis to a doctor."

[0877] The system is designed to allow users to easily detect skin abnormalities and seek early medical diagnosis.

[0878] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0879] Program processing flow

[0880] Step 1: User takes and uploads image data

[0881] The user takes a photo of the abnormal area of ​​the skin using a smartphone or digital camera. After taking the photo, the image data is uploaded to the system using a dedicated application. The input is the image data taken by the user, and the output is the image data saved on the device.

[0882] Step 2: The device sends the image data to the server

[0883] The terminal sends image data uploaded by the user to the server. At this time, the image data is encrypted and sent using a secure communication protocol (e.g., HTTPS). The input is the image data stored on the terminal, and the output is the encrypted image data sent to the server.

[0884] Step 3: The server receives the image data and passes it to the AI ​​image diagnosis processing means.

[0885] The server receives the image data sent from the terminal. After receiving the image data, the server passes the image data to the AI ​​image diagnosis processing means. The input is the encrypted image data sent to the server, and the output is the image data passed to the AI ​​image diagnosis processing means.

[0886] Step 4: The server analyzes the image data using the AI ​​model

[0887] The server analyzes the image data using an AI model built using deep learning frameworks such as TensorFlow and PyTorch. Specifically, it uses a convolutional neural network (CNN) to extract image features and identify patterns of abnormalities. The input is the image data passed to the AI ​​image diagnostic processing means, and the output is a diagnosis of the disease state.

[0888] Step 5: The server reports the analysis results to the doctor

[0889] The server reports the analysis results obtained by the AI ​​model to the doctor. This reporting is done through a dedicated medical information system. The input is the diagnosis result of the disease, and the output is the diagnosis information reported to the doctor.

[0890] Adding specific actions

[0891] Step 1: User takes and uploads image data

[0892] If a user feels any abnormality in their skin, they take a photo of the area with their smartphone. They open the dedicated application, select the image they have taken, and press the upload button. The image data is then saved on the device.

[0893] Step 2: The device sends the image data to the server

[0894] The device encrypts the image data uploaded by the user and sends it to the server using HTTPS. When the transmission is complete, the device displays a notification to the user that the transmission is complete.

[0895] Step 3: The server receives the image data and passes it to the AI ​​image diagnosis processing means.

[0896] The server receives the encrypted image data sent from the terminal, decrypts it, and passes the decrypted image data to the AI ​​image diagnosis processing means.

[0897] Step 4: The server analyzes the image data using the AI ​​model

[0898] The server analyzes the image data passed to the AI ​​image diagnostic processing means using a CNN model built using TensorFlow, and obtains a diagnosis of the condition, such as whether there is a suspicion of skin cancer.

[0899] Step 5: The server reports the analysis results to the doctor

[0900] The server reports the diagnosis of the patient's condition to the doctor via a dedicated medical information system, and the doctor uses this information to decide whether to order further detailed tests.

[0901] (Application example 2)

[0902] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0903] Modern medical systems require rapid and accurate diagnosis of patients' conditions. However, conventional systems face challenges in analyzing image data provided by patients and detecting abnormal behavior or suspicious individuals in real time. Furthermore, they lack the ability to quickly notify and respond when an abnormality is detected. This can lead to delayed responses in medical and security settings, potentially threatening the safety of patients and facilities.

[0904] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0905] In this invention, the server includes a chat-style medical interview input means, an AI image diagnosis processing means according to the input content, a means for issuing a prescription or shipping medicine if the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending an online diagnosis if the symptoms are complicated or there are concerns, a means for analyzing images captured by security cameras in real time to detect abnormal behavior or suspicious individuals, and a means for storing the detection results in a database and sending a notification if an abnormality is detected. This makes it possible to quickly and accurately diagnose the patient's condition, and also enables real-time detection of abnormal behavior or suspicious individuals through security camera image analysis and rapid notification.

[0906] The "chat-style medical interview input means" is an interface that allows users to input symptoms and conditions by answering questions in text format.

[0907] An "AI image diagnostic processing means" is a system that uses artificial intelligence to analyze image data and determine the condition or abnormality.

[0908] "Means for issuing prescriptions or shipping medicines" refers to a system for issuing prescriptions or shipping medicines to patients based on a doctor's instructions in cases where the symptoms or effects of prescription medicines are minor.

[0909] "Online referral" is a system that directs patients to a real-time video call with a doctor for complex cases or concerns.

[0910] "Means for analyzing security camera footage in real time to detect abnormal behavior and suspicious individuals" is a system for analyzing security camera footage in real time to automatically detect abnormal behavior and suspicious individuals.

[0911] "Means for saving detection results in a database and sending notifications when abnormalities are detected" refers to a system that saves the results in a database when abnormal behavior or suspicious individuals are detected and sends notifications to responsible personnel as necessary.

[0912] A system for implementing this invention has the following configuration: The server includes a means for inputting medical interviews in chat format, an AI image diagnosis processing means according to the input content, a means for issuing a prescription or shipping medicine in cases where the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending an online diagnosis in cases where the symptoms are complicated or there are concerns, a means for analyzing images captured by security cameras in real time to detect abnormal behavior or suspicious individuals, and a means for storing the detection results in a database and sending a notification if an abnormality is detected.

[0913] Hardware used:

[0914] server

[0915] security cameras

[0916] Smartphone or tablet

[0917] Database Server

[0918] Software used:

[0919] Image analysis AI models (e.g., YOLO, OpenCV)

[0920] Communication protocol (e.g. HTTP, WebSocket)

[0921] Database (e.g. MySQL, MongoDB)

[0922] System behavior:

[0923] 1. Chat-style medical interview input method:

[0924] Users use a smartphone or tablet to input their symptoms and conditions in a text-based chat format, which then sends the medical interview information to the server.

[0925] 2. AI image diagnosis processing methods:

[0926] The server receives image data provided by the user (e.g., a photo of an abnormal area of ​​the skin) and analyzes it using an image analysis AI model (e.g., YOLO). The analysis results are used to determine the condition of the patient.

[0927] 3. Prescription or drug delivery method:

[0928] If the symptoms are mild or the effects of prescribed medication are minor, the server will automatically issue a prescription and ship the medication to the patient.

[0929] 4. Online diagnostics:

[0930] In the case of complex cases or concerns, the server directs the user to a real-time video call with a doctor.

[0931] 5. Security camera footage analysis methods:

[0932] Security cameras capture video in real time and send it to a server, which uses an image analysis AI model to detect abnormal behavior or suspicious individuals in the video.

[0933] 6. Detection result storage and notification method:

[0934] The detection results are stored in a database, and if an anomaly is detected, a notification is sent to a smartphone or tablet, allowing security personnel to take prompt action.

[0935] Examples:

[0936] If a security camera detects a suspicious person at night, the server will send a notification to the smartphone saying, "Suspicious person detected. Location: near the main gate."

[0937] Security personnel open an application on their smartphone, check the details of the analysis results, and decide whether to report the incident to the police.

[0938] Example prompts to input to a generative AI model:

[0939] "Please create an AI model that analyzes footage captured by security cameras and detects abnormal behavior and suspicious individuals. The results of the detection should be sent to a smartphone in real time."

[0940] Specific prompt:

[0941] "Using the YOLO model, please build a system that detects suspicious individuals in real time from security camera footage. Please also implement a function to send the detection results to a server using the HTTP protocol and send notifications to smartphones."

[0942] In this way, a system for implementing the invention allows for a fast and accurate response in both the medical and security fields.

[0943] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0944] Step 1:

[0945] Enter medical questions in chat format

[0946] Users use a smartphone or tablet to input their symptoms and conditions in a text-based chat format, and the input data is sent to a server.

[0947] Input: User-entered text data about symptoms and conditions

[0948] Output: Interview data sent to the server

[0949] Specific behavior: A user opens the application, enters symptoms in the chat box, and presses the send button. The server receives this data and stores it in a database.

[0950] Step 2:

[0951] AI image diagnostic processing

[0952] The server receives image data provided by the user (e.g., a photo of an abnormal area of ​​the skin) and analyzes it using an image analysis AI model (e.g., YOLO).

[0953] Input: User uploaded image data

[0954] Output: Analysis results (diagnosis of the condition)

[0955] How it works: A user uploads an image through the application, and the server receives the image. The server then invokes an image analysis AI model to analyze the image and determine the medical condition. The analysis results are then stored in a database.

[0956] Step 3:

[0957] Prescription issuance or drug delivery

[0958] If the symptoms are mild or the effects of prescribed medication are minor, the server will automatically issue a prescription and ship the medication to the patient.

[0959] Input: Analysis result (if determined to be mild)

[0960] Output: Instructions for issuing a prescription or shipping medicine

[0961] Specific operation: The server evaluates the analysis results and, if it determines that the patient's condition is mild, it automatically generates a prescription and sends instructions to the pharmacy to ship the medication.

[0962] Step 4:

[0963] Guidance to online diagnosis

[0964] In the case of complex cases or concerns, the server directs the user to a real-time video call with a doctor.

[0965] Input: Analysis results (if determined to be a complex case)

[0966] Output: Video call link or invitation

[0967] What it does: The server evaluates the analysis results and, if it determines that the case is complex, it sends the user a link to a video call to set up an online consultation with a doctor.

[0968] Step 5:

[0969] Real-time analysis of security camera footage

[0970] Security cameras capture video in real time and send it to a server, which uses an image analysis AI model to detect abnormal behavior or suspicious individuals in the video.

[0971] Input: Real-time video data from security cameras

[0972] Output: Detection results of abnormal behavior and suspicious individuals

[0973] How it works: A security camera captures video and sends it to a server, which then invokes an image analysis AI model to analyze the video and detect abnormal behavior or suspicious individuals.

[0974] Step 6:

[0975] Storing and notifying detection results

[0976] The detection results are stored in a database, and if an abnormality is detected, a notification is sent to a smartphone or tablet.

[0977] Input: Detection results of abnormal behavior or suspicious individuals

[0978] Output: Save to database and send notification

[0979] How it works: The server stores the detection results in a database, and if an anomaly is detected, a notification is sent to a smartphone or tablet. Security personnel receive the notification and open an application to check the details.

[0980] Example 3

[0981] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0982] In the conventional medical system, users had to visit a medical institution in person even for mild symptoms or minor effects of prescribed medication, which required time and effort. Furthermore, when users had complex symptoms or concerns, numerous procedures were required to receive an appropriate diagnosis, making it difficult to respond quickly. Furthermore, the lack of real-time communication with doctors made it difficult for users to receive appropriate medical services.

[0983] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0984] In this invention, the server includes a chat-style medical interview input means, a health information analysis means using a generative AI model based on the input content, a means for issuing a prescription or shipping medication if the symptoms are mild or the effects of the prescribed medication are minor, a means for sending the patient to an online diagnosis if the symptoms are complicated or there are concerns, and a means for enabling real-time video calls with a doctor. This allows users to easily input their health information from home and receive an appropriate diagnosis and prescription. Furthermore, even if the patient has complex symptoms or concerns, they can receive a quick online diagnosis and a detailed diagnosis through real-time communication with a doctor.

[0985] "Chat-style medical interview input means" is a means for users to input their health conditions and symptoms through a text-based interface.

[0986] A "health information analysis method using a generative AI model" is a method that uses a generative AI model to analyze health information entered by a user and determine the severity of symptoms and appropriate prescriptions.

[0987] "Means for issuing prescriptions or shipping medicines" means means for automatically generating an appropriate prescription and sending it to a designated pharmacy or delivering the medicine to the user's home in the case of a minor illness or minor effects of a prescribed medicine.

[0988] "Online diagnostic referral" refers to a method that guides users to online diagnostics and provides appropriate medical services when they have complex symptoms or concerns.

[0989] "Means enabling real-time video calls with doctors" means means that users can have real-time video calls with doctors and receive detailed diagnoses.

[0990] MODE FOR CARRYING OUT THE INVENTION

[0991] This invention is a system that allows users to easily input health information from home and receive appropriate diagnosis and prescriptions. A specific embodiment of this system will be described below.

[0992] System configuration

[0993] The system consists of the following main components:

[0994] 1. Chat-style medical interview input method

[0995] 2. Health information analysis method using generative AI models

[0996] 3. A means of issuing prescriptions or shipping medications

[0997] 4. Online diagnostics

[0998] 5. A way to enable real-time video calls with doctors

[0999] Hardware and Software Use

[1000] Users access the system using devices such as smartphones or PCs. The system is provided through a dedicated application or website. The server receives input information from users and analyzes it using a generative AI model. This analysis uses natural language processing libraries (e.g., spaCy) and machine learning libraries (e.g., TensorFlow) implemented in Python.

[1001] Data processing and calculation

[1002] The health information entered by the user is received by the server and analyzed using a generative AI model. Based on the analysis results, if the symptom or the effect of the prescribed medication is determined to be minor, the server automatically generates a prescription and sends it to the designated pharmacy. This prescription is sent using a secure communication protocol (e.g., HTTPS). If the user requests home delivery of the medication, the server arranges for the delivery.

[1003] If the user has complex symptoms or concerns, the server will refer them to an online diagnosis, where they will use their device to interact directly with a doctor via a real-time video call, using real-time communication technologies such as WebRTC.

[1004] Specific examples

[1005] For example, if a user inputs "I have a sore throat and a slight cough," the server receives this information and analyzes it using a generative AI model. Based on the analysis results, it is determined to be a mild cold. The server generates a prescription for cold medicine and sends it to the designated pharmacy. If the user requests home delivery of the medicine, the server processes the delivery and the medicine is delivered to the user's home.

[1006] Prompt Sentence Examples

[1007] If a user reports "I have a sore throat and a slight cough," how would the system issue a prescription and deliver the medication?

[1008] In this way, users can easily input their health information from home and receive appropriate diagnosis and prescriptions. Even if users have complex symptoms or concerns, they can receive a quick online diagnosis and receive a detailed diagnosis through real-time communication with a doctor. The flow of the identification process in Example 3 will be described with reference to FIG. 15.

[1009] Step 1:

[1010] The user enters their health information.

[1011] Users use their devices to access the system's dedicated application or website and enter their current health condition and symptoms, for example, specific symptoms such as "I have a sore throat and a slight cough." The entered information is then sent to the server.

[1012] Input: User's health information (e.g., "I have a sore throat and a slight cough")

[1013] Output: Health information sent to the server

[1014] Step 2:

[1015] The server receives and analyzes the health information.

[1016] The server receives health information submitted by the user. This information is analyzed using a generative AI model. The analysis uses natural language processing libraries (e.g., spaCy) and machine learning libraries (e.g., TensorFlow) implemented in Python. The server evaluates the severity of symptoms and the impact of prescribed medications.

[1017] Input: Health information submitted by the user

[1018] Output: Analysis result (e.g., judged to be a mild cold)

[1019] Step 3:

[1020] The server generates a prescription based on the analysis results.

[1021] The server then generates an appropriate prescription based on the analysis results. For example, if it is determined to be a mild cold, a prescription for cold medicine will be generated. This prescription is generated electronically and stored in a secure format.

[1022] Input: Analysis results

[1023] Output: Generated prescription

[1024] Step 4:

[1025] The server sends the prescription to the designated pharmacy.

[1026] The server sends the generated prescription to the designated pharmacy using a secure communication protocol (e.g., HTTPS), and the pharmacy prepares the medication based on the received prescription.

[1027] Input: Generated prescription

[1028] Output: Prescription sent to pharmacy

[1029] Step 5:

[1030] If the user requests delivery of the medicine, the server handles the delivery process.

[1031] If the user requests home delivery of the medicine, the server processes the delivery, confirms the user's address information, and requests a delivery company. The delivery company then delivers the medicine to the user's home.

[1032] Input: User's shipping preferences, address information

[1033] Output: Delivery procedure requested by the delivery company

[1034] Step 6:

[1035] If there are complex symptoms or concerns, the server will direct the user to an online diagnosis.

[1036] If the server determines that the user has complex symptoms or concerns based on the analysis results, it will refer the user to an online diagnosis, and the user will receive a notification from the system and make an appointment for the online diagnosis.

[1037] Input: Analysis results (complex symptoms and concerns)

[1038] Output: Online diagnostic notification, reservation procedure

[1039] Step 7:

[1040] Users have real-time video calls with doctors.

[1041] At the scheduled time, users can use their devices to make a real-time video call using real-time communication technologies such as WebRTC, allowing them to directly communicate with a doctor and receive a detailed diagnosis.

[1042] Input: Scheduled video call time, user device

[1043] Output: Real-time video call with doctor, detailed diagnosis

[1044] (Application example 3)

[1045] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1046] In the conventional medical system, users had to visit a medical institution even for mild symptoms or minor effects of prescribed medication, which required time and effort. Furthermore, when users had complex symptoms or concerns, numerous procedures were required to receive an appropriate diagnosis, making it difficult to respond quickly. Furthermore, when it came to receiving medication, users had to visit a pharmacy, which was inconvenient.

[1047] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1048] In this invention, the server includes a chat-style medical interview input means, a diagnostic processing means using a generative AI model based on the input content, a means for issuing a prescription or shipping medication in cases where the symptoms are mild or the effects of the prescribed medication are minor, a means for referring the patient to an online diagnosis in cases where the symptoms are complicated or there are concerns, a means for delivering medication to the patient's home at the user's request, and a means for conducting a real-time video call with a doctor through the online diagnosis. This allows users to easily receive a diagnosis from home and quickly receive the necessary medication. Furthermore, even in cases where the symptoms are complicated, users can receive prompt and appropriate medical services through the online diagnosis.

[1049] A "chat-style medical interview input means" is a means for users to input symptoms and health conditions through a text-based chat interface.

[1050] A "diagnostic processing means using a generative AI model" is a means for analyzing a medical condition and making a diagnosis using a generative AI model based on information entered by a user.

[1051] "Means for issuing prescriptions or shipping medicines when symptoms are mild or the effects of prescribed medicines are minor" refers to means for the system to automatically issue prescriptions or ship medicines when the user's symptoms are mild and the effects of prescribed medicines are minor.

[1052] "Means for referring users to online diagnosis in complex cases or when there are concerns" refers to the means by which the system directs users to online diagnosis when the user's symptoms are complex or when there are concerns.

[1053] "Means for delivering medicine to the user's home at the user's request" refers to means for delivering prescribed medicine to the user's home if the user so requests.

[1054] "Means for having real-time video calls with doctors through online diagnosis" means a means for users to have real-time video calls with doctors and receive detailed diagnoses.

[1055] A system for implementing the present invention has the following configuration.

[1056] First, the user inputs a medical questionnaire in chat format using a smartphone or smart glasses. The medical questionnaire input method allows users to input symptoms and health conditions through a text-based interface, allowing users to easily communicate their symptoms to the system.

[1057] The input information is then sent to a diagnostic processing means that uses a generative AI model. This means uses the generative AI model to analyze the condition and make a diagnosis based on the information entered by the user. Specifically, the generative AI model analyzes the user's symptoms and outputs an appropriate diagnosis. Based on this diagnosis result, the system proceeds to the next step.

[1058] If the symptoms are mild or the effects of the prescribed medication are minor, the system will automatically issue a prescription or ship the medication. This prescription issuing method automatically generates a prescription and sends it to a designated pharmacy if the user's symptoms are mild and the effects of the prescribed medication are minor. It also includes a method for delivering medication to the user's home, if desired. This allows the user to receive the necessary medication without leaving home.

[1059] On the other hand, if the user has complex symptoms or concerns, the system will refer the user to an online diagnosis. This online diagnosis method allows the user to have a real-time video call with a doctor to receive a detailed diagnosis. The video call is made using the camera and microphone on a smartphone or smart glasses. This allows the user to directly interact with a doctor from the comfort of their own home and receive a proper diagnosis.

[1060] For example, if a user inputs symptoms such as "cough, fever," the generative AI model analyzes them and diagnoses them as a "cold." Based on this diagnosis, the system issues a prescription for "500mg paracetamol" and sends it to the designated pharmacy. If the user requests it, the system can also arrange for the medicine to be delivered to their home.

[1061] Examples of prompts include:

[1062] User symptoms: Cough, fever

[1063] Diagnosis: Cold

[1064] Prescription: Paracetamol 500mg

[1065] In this way, this invention allows users to easily receive a diagnosis from home and quickly receive the necessary medication. In addition, even in cases of complex symptoms, users can receive prompt and appropriate medical services through online diagnosis.

[1066] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1067] Step 1:

[1068] The user inputs the medical questionnaire in chat format using a smartphone or smart glasses.

[1069] Input: Text data about the user's symptoms and health conditions.

[1070] Data processing: Analyze text data using natural language processing technology to extract symptoms.

[1071] Output: Extracted symptom data.

[1072] Step 2:

[1073] The server transmits the input symptom data to a diagnostic processing means using a generative AI model.

[1074] Input: Extracted symptom data.

[1075] Data computation: Generative AI models analyze symptom data and diagnose medical conditions.

[1076] Output: Diagnosis result data (e.g. cold, flu, etc.).

[1077] Step 3:

[1078] Based on the diagnosis, the server issues a prescription if the symptoms are mild or the effects of the prescribed medication are minor.

[1079] Input: Diagnostic result data.

[1080] Data processing: Generate appropriate prescriptions based on diagnostic results.

[1081] Output: Prescription data.

[1082] Step 4:

[1083] The server sends the generated prescription data to the designated pharmacy.

[1084] Input: Prescription data.

[1085] Data Computation: Converts prescription data into a format for transmission to the pharmacy system.

[1086] Output: Prescription data sent to the pharmacy.

[1087] Step 5:

[1088] If desired, the server will arrange for the medicine to be delivered to the user's home.

[1089] Input: User's delivery preference data and prescription data.

[1090] Data processing: Generate data for delivery arrangements and send it to the delivery service.

[1091] Output: Delivery arrangement data sent to the delivery service.

[1092] Step 6:

[1093] If there are complex symptoms or concerns, the server will direct the user to an online diagnosis.

[1094] Input: Diagnostic result data.

[1095] Data calculation: When the diagnostic results are complex, generate an online diagnostic schedule.

[1096] Output: Online diagnostics schedule data.

[1097] Step 7:

[1098] The server arranges for a real-time video call with a doctor through online diagnosis.

[1099] Input: Online diagnostic schedule data and user video call preference data.

[1100] Data processing: Generate connection information for the video call and send it to the user and doctor.

[1101] Output: Video call connection information.

[1102] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1103] "Example 1"

[1104] The invention, which combines an emotion engine, recognizes the user's emotions and utilizes them for diagnosis. Specifically, when the user enters a medical questionnaire, emotions are analyzed from the text input and voice. For example, if the user enters "I have a headache," the emotion engine recognizes from the text that the user is feeling pain. Emotions are also analyzed from the voice, based on the tone, strength, and speed of the user's voice. This information can be used for AI diagnosis.

[1105] "Example 2"

[1106] The invention, which incorporates an emotion engine, adjusts the content of the report to the doctor according to the user's emotional state. Specifically, if the emotion engine recognizes that the user is in severe pain, it will report that information to the doctor first. This allows the doctor to make a diagnosis that takes into account not only the user's symptoms but also their emotional state.

[1107] "Example 3"

[1108] The invention, which combines an emotion engine, recognizes the user's emotions and utilizes them for diagnosis. Specifically, when the user enters a medical questionnaire, emotions are analyzed from the text input and voice. For example, if the user enters "I have a headache," the emotion engine recognizes from the text that the user is feeling pain. Emotions are also analyzed from the voice, based on the tone, strength, and speed of the user's voice. This information can be used for AI diagnosis.

[1109] The processing flow of each embodiment will be described below.

[1110] "Example 1"

[1111] Step 1: The user begins entering the medical questionnaire.

[1112] Step 2: The user describes their symptoms by text input or voice.

[1113] Step 3: The emotion engine analyzes emotions from the user's text input or voice.

[1114] Step 4: The emotion engine sends the analyzed emotion information to the AI.

[1115] Step 5: The AI ​​makes a diagnosis taking emotional information into account.

[1116] "Example 2"

[1117] Step 1: The user begins entering the medical questionnaire.

[1118] Step 2: The user describes their symptoms by text input or voice.

[1119] Step 3: The emotion engine analyzes emotions from the user's text input or voice.

[1120] Step 4: The emotion engine reports the analyzed emotional information to the doctor.

[1121] Step 5: The doctor makes a diagnosis, taking emotional information into account.

[1122] "Example 3"

[1123] Step 1: The user begins entering the medical questionnaire.

[1124] Step 2: The user describes their symptoms by text input or voice.

[1125] Step 3: The emotion engine analyzes emotions from the user's text input or voice.

[1126] Step 4: The emotion engine sends the analyzed emotion information to the AI.

[1127] Step 5: The AI ​​makes a diagnosis taking emotional information into account.

[1128] Example 1

[1129] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1130] In conventional medical systems, it is difficult for users to accurately describe their symptoms, and diagnosis does not take into account emotional changes, which can delay appropriate diagnosis and treatment. Furthermore, even in cases where symptoms are mild or the effects of prescribed medication are minor, users must visit a medical institution in person, which takes time and effort.

[1131] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1132] In this invention, the server includes a chat-style medical interview input means, a text analysis means using natural language processing technology according to the input content, a means for extracting important keywords from the analysis results, a means for analyzing the user's emotions using an emotion engine, a means for making a diagnosis based on the analysis results, a means for issuing a prescription or shipping medicine if the symptoms are mild or the effects of the prescribed medicine are minor, and a means for sending the patient to an online diagnosis if the symptoms are complicated or there are concerns. This makes it possible to accurately analyze the user's symptoms and emotions and quickly provide an appropriate diagnosis and treatment.

[1133] The "chat-style medical interview input means" is a means for a user to input their symptoms through a text-based interface.

[1134] "Means for analyzing text using natural language processing technology" refers to means that use technology to analyze text data entered by a user and understand its meaning.

[1135] The "means for extracting important keywords" refers to a means for identifying important words and phrases related to symptoms and medical conditions from the analyzed text data.

[1136] The "means for analyzing user emotions using an emotion engine" refers to a means for using technology to recognize and analyze emotions from user text or voice data.

[1137] The "means for diagnosing based on the analysis results" refers to a means for diagnosing the user's symptoms and condition based on the extracted keywords and the results of emotion analysis.

[1138] "Means for issuing prescriptions or shipping medicines" refers to means for issuing prescriptions to users or shipping medicines directly to users when the symptoms or effects of prescribed medicines are minor.

[1139] "Online diagnostic referral" refers to a method for users with complex symptoms or concerns to be diagnosed, such as through a real-time video call with a doctor.

[1140] This invention is a system realized as a medical application. Users input their symptoms in chat format through the application. This input is analyzed using natural language processing technology, and important keywords for the symptoms are extracted. Furthermore, by combining it with an emotion engine, the system recognizes the user's emotions and uses them for diagnosis.

[1141] Hardware and software used

[1142] Hardware

[1143] Device: The device used by the user, such as a smartphone, tablet, or computer

[1144] Server: Cloud server for data analysis and diagnostic processing

[1145] software

[1146] Natural language processing (NLP) technologies: Python's NLTK library, SpaCy, etc.

[1147] Sentiment analysis library: TextBlob and Librosa, a speech analysis library

[1148] AI models: BERT (Bidirectional Encoder Representations from Transformers), etc.

[1149] Program processing

[1150] When a user enters their symptoms through the application, the device sends the text data to a server. The server then uses NLP technology to analyze the text and extract important keywords from the symptoms. For example, if a user enters "I've had a terrible headache recently, and I can't sleep at night," the server will extract keywords such as "headache" and "can't sleep."

[1151] Furthermore, the server uses an emotion engine to analyze the user's emotions. When the user inputs text, the server recognizes the user's emotions from that text. For example, if the user inputs "I have a headache," the server recognizes from that text that the user is in pain. When there is voice input, the server analyzes the user's emotions from the tone, strength, and speed of the voice.

[1152] This information is used for AI diagnosis. The server provides the user with appropriate advice and a diagnosis based on the analysis results. For example, if a user inputs, "I've had a terrible headache recently and can't sleep at night," the server will use this information to diagnose the possibility of migraines or stress-related insomnia, and send the results back to the user.

[1153] Examples and prompts

[1154] As a concrete example, consider the case where a user types, "I've had a terrible headache recently, and I can't sleep at night." In this case, the device sends this text data to the server. The server uses Python's NLTK library to analyze the text and extract keywords such as "headache" and "can't sleep." It then uses TextBlob to recognize that the user is experiencing pain. Based on this information, the server uses an AI model to diagnose the possibility of migraine or stress-related insomnia, and sends the results back to the user.

[1155] An example of a prompt to input to a generative AI model is as follows:

[1156] A user types, "I've been having terrible headaches lately and can't sleep at night." Analyze this text and extract important keywords and the user's sentiment.

[1157] Using this prompt, the generative AI model can accurately analyze the user's symptoms and emotions and provide appropriate diagnoses and advice.

[1158] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1159] Step 1:

[1160] User enters symptoms

[1161] Users launch the medical application and enter their symptoms through a chat-style interface. For example, they might enter, "I've had a terrible headache recently, and I can't sleep at night." The input data is saved in text format on the device.

[1162] Step 2:

[1163] The device sends the input data to the server

[1164] The terminal transmits the text data entered by the user to the server in real time. The data is encrypted before transmission, ensuring security. The input data is transmitted to the server in text format.

[1165] Step 3:

[1166] The server analyzes the text using natural language processing (NLP) techniques.

[1167] The server analyzes the received text data using natural language processing (NLP) techniques. Specifically, it uses NLP tools such as Python's NLTK library or SpaCy. The input data is in text format, and the output data is the analyzed text data.

[1168] Step 4:

[1169] The server extracts important keywords

[1170] The server extracts important keywords from the results of NLP analysis. For example, keywords such as "headache" and "can't sleep" are extracted. This process uses techniques such as TF-IDF (Term Frequency-Inverse Document Frequency) and BERT (Bidirectional Encoder Representations from Transformers). The input data is the analyzed text data, and the output data is the extracted keywords.

[1171] Step 5:

[1172] The server analyzes the user's emotions using an emotion engine.

[1173] The server uses an emotion engine to analyze the user's emotions. From the text, it is recognized that the user is experiencing pain. If there is voice input, the server also analyzes emotions from the tone, strength, and speed of the user's voice. This process uses the emotion analysis library TextBlob and the voice analysis library Librosa. The input data is text data and voice data, and the output data is the analyzed emotional information.

[1174] Step 6:

[1175] The server performs a diagnosis based on the analysis results.

[1176] The server uses an AI model to make a diagnosis based on the extracted keywords and the results of emotion analysis. For example, if a user inputs "I have a terrible headache and can't sleep at night," the server will use this information to diagnose the possibility of migraines or stress-related insomnia. The input data are the extracted keywords and analyzed emotion information, and the output data are the diagnosis results.

[1177] Step 7:

[1178] The server returns the diagnostic results to the user.

[1179] The server returns the diagnostic results to the user, who can then review them through the application and find out any necessary advice or next steps. The input data is the diagnostic results, and the output data is the diagnostic information that is displayed to the user.

[1180] (Application example 1)

[1181] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1182] Autonomous vehicles require systems that can monitor the health status of passengers in real time and promptly notify medical institutions as necessary. However, conventional systems have had the challenge of being unable to accurately analyze passenger symptoms and emotions and provide appropriate responses. In particular, in emergencies, where a rapid and accurate response is required, more advanced technology is required.

[1183] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1184] In this invention, the server includes a chat-style medical interview input means, an analysis means using natural language processing technology according to the input content, a means for analyzing the user's emotions using an emotion engine, a means for making an initial diagnosis based on the analysis results, a means for notifying a medical institution as necessary, and a means for directing the self-driving vehicle to the nearest medical institution in an emergency. This allows for real-time monitoring of the passenger's health condition and enables prompt and accurate response.

[1185] The "chat-style medical interview input means" is an interface that allows passengers to input their symptoms using text.

[1186] "Analysis means using natural language processing technology" is a technology that analyzes input text and extracts important keywords.

[1187] "Means for analyzing user emotions using an emotion engine" is a technology that analyzes user emotions from text and voice.

[1188] "Means for making an initial diagnosis based on the analysis results" refers to a means by which AI makes an initial diagnosis based on the results of natural language processing technology and sentiment analysis.

[1189] The "means for notifying medical institutions as necessary" refers to a means for notifying medical institutions as necessary based on the analysis results.

[1190] "Means for directing an autonomous vehicle to the nearest medical institution in an emergency" refers to means for an autonomous vehicle to automatically direct itself to the nearest medical institution in an emergency.

[1191] The system for implementing this invention has the following configuration: First, passengers input their symptoms in chat format using their smartphones or the in-car display, which enables real-time monitoring of the passenger's health condition.

[1192] The server analyzes the input text using natural language processing technology to extract important keywords. This analysis uses natural language processing libraries such as spaCy. For example, if a passenger types "I have a headache and I feel sick," the system will extract keywords such as "headache" and "feeling sick."

[1193] Next, the emotion engine analyzes the user's emotions using Hugging Face's Transformers library and TextBlob, which analyzes user emotions from text and speech to determine, for example, whether the passenger is in distress.

[1194] Based on the analysis results, the server makes an initial diagnosis. This initial diagnosis is performed using an AI model, and if necessary, a medical institution is notified. For example, if a passenger inputs "I have a headache and I'm feeling unwell," and the results of the emotion analysis indicate a high level of urgency, the system will automatically notify a medical institution.

[1195] Additionally, in case of an emergency, the self-driving vehicle will be directed to the nearest medical facility, ensuring the safety of passengers.

[1196] As a concrete example, if a passenger inputs "I have a headache and feel unwell," the system will operate as follows. First, natural language processing technology is used to extract keywords such as "headache" and "feeling unwell." Next, an emotion engine is used to analyze whether the passenger is experiencing pain. Based on the analysis results, the AI ​​will make an initial diagnosis and notify a medical institution if necessary. In the event of an emergency, the self-driving vehicle will be instructed to head to the nearest medical institution.

[1197] An example of a prompt is as follows:

[1198] "If a user types, 'I have a headache and I'm feeling sick,' explain how the system will parse that and what action will be taken."

[1199] In this way, the present invention provides a system for monitoring passenger health conditions in real time, enabling rapid and accurate response.

[1200] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1201] Step 1:

[1202] Users input their symptoms in chat format using their smartphone or in-car display.

[1203] Input: Text entered by the user (e.g., "I have a headache and feel sick")

[1204] Output: Input text data

[1205] Step 2:

[1206] The server analyzes the input text using natural language processing technology and extracts important keywords.

[1207] Input: Text data entered by the user

[1208] Data processing: Analyze the text using a natural language processing library such as spaCy to extract important keywords such as nouns and adjectives.

[1209] Output: Extracted keywords (e.g., "headache" or "feeling unwell")

[1210] Step 3:

[1211] The server analyzes the user's emotions using an emotion engine.

[1212] Input: Text data entered by the user

[1213] Data Computation: Analyze emotions from text and audio using Hugging Face's Transformers library and TextBlob.

[1214] Output: Sentiment analysis result (e.g., "feeling distressed")

[1215] Step 4:

[1216] The server performs an initial diagnosis based on the analysis results.

[1217] Input: Extracted keywords and sentiment analysis results

[1218] Data calculation: Using a generative AI model, an initial diagnosis is made based on keywords and sentiment analysis results.

[1219] Output: Initial diagnosis result (e.g. "High Urgency")

[1220] Step 5:

[1221] The server notifies the medical institution as necessary.

[1222] Input: Initial diagnosis result

[1223] Data processing: Based on the initial diagnosis results, medical institutions will be notified if necessary.

[1224] Output: Notification to the medical institution (e.g. "There is an emergency patient")

[1225] Step 6:

[1226] In the event of an emergency, the server will direct the self-driving vehicle to the nearest medical facility.

[1227] Input: Initial diagnosis result

[1228] Data calculation: If the emergency is deemed high, the system will control the navigation system of the autonomous vehicle and direct it to the nearest medical facility.

[1229] Output: Directions for the autonomous vehicle (e.g., "Go to the nearest medical facility")

[1230] Example 2

[1231] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1232] While conventional medical systems can analyze a patient's image data to determine their condition, they do not take into account the patient's emotional state. As a result, even if a patient is in severe pain, this information may not be properly conveyed to the doctor, potentially resulting in a decline in the quality of diagnosis and treatment. Furthermore, there is a lack of means to respond quickly to cases where the patient's symptoms are mild or the effects of prescribed medication are minor.

[1233] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a chat-style medical interview input means, an image analysis means according to the input content, a means for issuing a prescription or shipping medicine in cases where the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending the patient to an online diagnosis in cases where the symptoms are complicated or there are concerns, a means for analyzing the user's emotional state, and a means for adjusting the report content based on the user's emotional state. This enables a diagnosis that takes into account not only the patient's medical condition but also their emotional state, allowing doctors to provide more appropriate treatment. Furthermore, in cases where the symptoms are mild, a quick response can be made, reducing the burden on the patient.

[1234] The "chat-style medical interview input means" is an interface that allows the user to input information such as symptoms and medical history via text messages.

[1235] "Image analysis means" is a technology that analyzes image data provided by users and determines the condition of the patient. Specifically, it uses deep learning models to extract features from the image and identify abnormalities.

[1236] "Means for issuing prescriptions or shipping medicines" refers to a system for issuing prescriptions or shipping medicines to patients based on a doctor's instructions in cases where the symptoms or effects of prescription medicines are minor.

[1237] "Online referral" refers to technology that allows patients to be referred for online evaluation, such as through a real-time video call with a doctor, in complex cases or when there are concerns.

[1238] "Means for analyzing the user's emotional state" refers to technology that analyzes the user's facial expressions and voice data to determine the emotional state the user is feeling. Specifically, it uses algorithms for facial expression recognition and voice analysis.

[1239] The "means for adjusting the report content based on the emotional state" is a technology for adjusting the report content to the doctor based on the emotional state of the user. Specifically, if the user is experiencing severe pain, that information is reported to the doctor as a priority.

[1240] This invention is a system that analyzes image data and emotional states provided by a user and reports the results to a doctor. A specific embodiment for implementing this system is described below.

[1241] The server uses a secure communication protocol (e.g., HTTPS) to receive image data uploaded by users. Users use their own devices (smartphones or PCs) to take photos of abnormal areas of the skin and upload the image data through the system's dedicated application or website.

[1242] The server passes the received image data to the AI ​​image diagnostic processing means, which analyzes the image using a pre-trained deep learning model (e.g., ResNet or Inception) to determine the condition. Specifically, the model extracts features within the image and identifies patterns of abnormalities.

[1243] The analysis results, including a diagnosis estimate and a confidence score, are reported to the doctor via a dedicated dashboard or email.

[1244] Additionally, the server uses an emotion engine to analyze the user's emotional state. The emotion engine uses tools such as OpenCV and Google Cloud Speech-to-Text API to analyze the user's facial expressions and voice data to determine whether the user is experiencing severe distress.

[1245] The server adjusts the content of the report to the doctor based on the user's emotional state obtained by the emotion engine. Specifically, if the user is experiencing severe pain, that information is reported to the doctor first. This allows the doctor to make a diagnosis that takes into account not only the user's symptoms but also their emotional state.

[1246] For example, when a user wants to upload a photo of an abnormal area of ​​skin, they open a dedicated application and click the "Upload Image" button. The server receives the uploaded image data and passes it to the AI ​​image diagnosis processing means. The AI ​​image diagnosis processing means analyzes the image and determines that there is a "high possibility of skin cancer." The server reports this analysis result to a doctor. At the same time, if the emotion engine recognizes from the user's facial expression that they are in great pain, it will also report this information to the doctor as a priority.

[1247] Example prompts to input to a generative AI model:

[1248] "Analyze photos of abnormal skin areas provided by the user and determine the medical condition. Also, analyze the user's emotional state and report any significant distress to a doctor as a priority."

[1249] In this way, a system is realized in which the server, terminal, and user work together to provide more accurate diagnoses and medical services that take into account the user's emotional state.

[1250] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1251] Step 1:

[1252] The user uploads image data.

[1253] The user takes a photo of the abnormal area of ​​the skin using their own device (smartphone or PC) and uploads the image data through the system's dedicated application or website. The input is the image data taken by the user, and the output is the image data sent to the server.

[1254] Step 2:

[1255] The server receives the image data.

[1256] The server receives image data uploaded by the user. At this time, the image data is transmitted using a secure communication protocol (e.g., HTTPS). The input is the image data transmitted by the user, and the output is the image data stored on the server.

[1257] Step 3:

[1258] The server analyzes the images using AI image diagnostic processing means.

[1259] The server passes the received image data to the AI ​​image diagnostic processing means, which uses a pre-trained deep learning model (e.g., ResNet or Inception) to analyze the image and determine the condition. The input is the image data stored on the server, and the output is an estimated condition and a confidence score. Specifically, the model extracts features from the image and identifies patterns of abnormalities.

[1260] Step 4:

[1261] The server reports the analysis results to the doctor.

[1262] The server reports the analysis results obtained by the AI ​​image diagnosis processing means to the doctor. The report contents include the estimated results of the disease state and a reliability score. The input is the estimated results of the disease state and a reliability score, and the output is the report contents sent to the doctor. The report is sent via a dashboard dedicated to the doctor or via email.

[1263] Step 5:

[1264] The server uses an emotion engine to analyze the user's emotional state.

[1265] The server uses an emotion engine to analyze the user's emotional state. The emotion engine uses tools such as OpenCV and Google Cloud Speech-to-Text API to analyze the user's facial expression and voice data. The input is the user's facial expression and voice data, and the output is the judgment result of the user's emotional state. Specific operations include using algorithms for facial expression recognition and voice analysis.

[1266] Step 6:

[1267] The server adjusts the report to the physician based on the emotional state.

[1268] The server adjusts the content of the report to the doctor based on the user's emotional state obtained by the emotion engine. Specifically, if the user is experiencing severe pain, that information is reported to the doctor first. The input is the result of the user's emotional state assessment, and the output is the adjusted report content. This allows the doctor to make a diagnosis that takes into account not only the user's symptoms but also their emotional state.

[1269] (Application example 2)

[1270] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1271] Conventional security systems have the problem of being unable to take into account the emotional state of security guards when analyzing surveillance camera footage and on-site photos, making it difficult to properly assess high-urgency situations.In addition, because the emotional state of security guards is not reflected in reports, it is difficult for the security center to take prompt and appropriate action.

[1272] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1273] In this invention, the server includes a chat-style medical interview input means, an AI image diagnosis processing means according to the input content, a means for issuing a prescription or shipping medicine if the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending the user to an online diagnosis if the symptoms are complicated or there are concerns, an emotion engine for analyzing the user's emotional state, and a means for adjusting the report content according to the emotional state, which enables anomaly detection and reporting that takes into account the emotional state of the security guard.

[1274] The "chat-style medical interview input means" is an interface that allows a user to input information in a text-based interactive format.

[1275] "AI image diagnostic processing means" refers to a processing means that uses artificial intelligence to analyze image data and determine the condition or abnormality.

[1276] "Means for issuing prescriptions or shipping medicines" refers to means for issuing prescriptions for medicines or shipping medicines to users when the symptoms or effects of prescription medicines are minor.

[1277] "Online referral" refers to a method to direct users to a real-time video call with a doctor for diagnosis in complex cases or where there are concerns.

[1278] The "emotion engine that analyzes the user's emotional state" is an engine that analyzes the user's emotional state from facial expressions, voice, etc.

[1279] The "means for adjusting the report content according to the emotional state" is a means for adjusting the priority and details of the report content based on the emotional state of the user.

[1280] As an embodiment of this invention, we will explain the security diagnostic assistant system as an example. In this system, security guards use smartphones or smart glasses to take surveillance camera footage and photos of the scene, and AI analyzes the images to detect abnormalities. Furthermore, by combining it with an emotion engine, the system is equipped with a function to adjust the content of reports according to the emotional state of the security guard.

[1281] Hardware and software used

[1282] Hardware: Smartphones, smart glasses

[1283] Software: OpenCV (image processing library), Keras (deep learning library), EmotionRecognizer (emotion recognition library)

[1284] System configuration

[1285] 1. Chat-style interview input method: Provides an interface for security guards to input information in a text-based interactive format, allowing them to easily report the situation.

[1286] 2. AI image diagnostic processing method: AI analyzes surveillance camera footage and on-site photos to detect abnormalities. Specifically, images are preprocessed using OpenCV and analyzed using a deep learning model using Keras.

[1287] 3. A means of issuing prescriptions or shipping medicines: This is a means of issuing prescriptions for medicines or shipping medicines to users when the symptoms or effects of prescription medicines are minor. In security systems, it is also used as a means of automatically reporting minor abnormalities.

[1288] 4. Online diagnostics: In case of complications or concerns, this is a way to direct the security guard to a real-time video call with the security center for a diagnostic.

[1289] 5. Emotion engine that analyzes the user's emotional state: This engine analyzes the emotional state of the security guard from their facial expressions and voice. Emotion analysis is performed using EmotionRecognizer.

[1290] 6. Adjusting the report content according to the user's emotional state: This is a means to adjust the priority and details of the report content based on the user's emotional state. For example, if a security guard is feeling very tense or anxious, that information will be reported to the security center as a priority.

[1291] Specific examples

[1292] Example 1: A security guard uses smart glasses to analyze nighttime surveillance camera footage in real time and detects anomalies. The emotion engine detects the guard's state of tension and reports it as a priority.

[1293] Example 2: Using a smartphone, an AI system can detect abnormalities by analyzing the image. The emotion engine can detect the stress level of guards and report it as a priority.

[1294] Prompt Sentence Examples

[1295] "Analyze surveillance camera footage in real time to detect any anomalies. Adjust your reporting accordingly, taking into account the emotional state of the security guards."

[1296] "Take photos of the scene and use AI to detect anomalies. Use an emotion engine to tailor the report based on the emotional state of the security guard."

[1297] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1298] Step 1:

[1299] A user uses a smartphone or smart glasses to capture surveillance camera footage or photos of the scene.

[1300] Input: CCTV footage or photos of the scene

[1301] Output: Captured image data

[1302] Specific operation: The user uses the device's camera function to take video or photos of the monitored object. The captured image data is stored on the device.

[1303] Step 2:

[1304] The device sends the captured image data to an AI image diagnostic processing means, which detects abnormalities.

[1305] Input: Captured image data

[1306] Output: Anomaly detection results

[1307] Specific operation: The device preprocesses the image using OpenCV and analyzes it using a deep learning model using Keras. If an anomaly is detected as a result of the analysis, that information is returned to the device.

[1308] Step 3:

[1309] To analyze the user's emotional state, the device uses an emotion engine to analyze the user's facial expressions and voice.

[1310] Input: User's facial expression or voice data

[1311] Output: Emotional state

[1312] Specific operation: The device uses EmotionRecognizer to analyze the user's emotional state from their facial expressions and voice. The analysis results indicate whether the user is feeling nervous or stressed.

[1313] Step 4:

[1314] The device adjusts the report content based on the anomaly detection results and emotional state.

[1315] Input: Anomaly detection results, emotional state

[1316] Output: Adjusted report

[1317] Specific operation: The device combines the anomaly detection results with the user's emotional state to adjust the priority and details of the report. For example, if the user is feeling strong tension or anxiety, that information will be prioritized in the report.

[1318] Step 5:

[1319] The device sends the adjusted report to the security center.

[1320] Input: Adjusted report

[1321] Output: Report to Security Center

[1322] Specific operation: The terminal sends the adjusted report to the security center via the network. The security center takes appropriate action based on the received report.

[1323] Example 3

[1324] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1325] Conventional medical systems have difficulty comprehensively analyzing a user's symptoms and emotions and providing appropriate medical care. They also lack the functionality to automatically issue prescriptions for mild symptoms or minor effects of prescription medications, and to refer patients with complex symptoms or concerns to online diagnosis. Furthermore, the lack of technology to analyze a user's emotions and utilize them in diagnosis makes it difficult to accurately grasp the user's condition.

[1326] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1327] In this invention, the server includes a chat-style medical interview input means, an artificial intelligence image diagnosis processing means according to the input content, a means for issuing a prescription or shipping medicine if the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending the patient to an online diagnosis if the symptoms are complicated or there are concerns, a means for analyzing the user's emotions from text or voice using an emotion analysis engine, and a means for providing appropriate medical care based on the analysis results. This makes it possible to comprehensively analyze the user's symptoms and emotions and provide the most appropriate medical care.

[1328] The "chat-style medical interview input means" is an interface that allows users to input symptoms and conditions through text messages.

[1329] The "artificial intelligence image diagnosis processing means" is a processing means that uses artificial intelligence to analyze image data provided by a patient and determine the condition of the patient.

[1330] The "means for issuing prescriptions or shipping medicines" refers to a means by which a medical institution automatically issues a prescription or ships medicines to a user when the symptoms or effects of the prescribed medicines are minor.

[1331] "Online referral" refers to a means to direct users to a doctor for a diagnosis via a real-time video call if they have complex symptoms or concerns.

[1332] An "emotion analysis engine" is an engine that analyzes user emotions from text and voice data and uses that information for diagnosis.

[1333] The "means for providing appropriate medical treatment" refers to a means for providing optimal medical treatment to the user based on the analysis results of the emotion analysis engine.

[1334] MODE FOR CARRYING OUT THE INVENTION

[1335] The present invention is a system for analyzing a user's symptoms and emotions and providing appropriate medical treatment. Specific embodiments of this system will be described below.

[1336] System configuration

[1337] The system consists of the following main components:

[1338] 1. Chat-style medical interview input method

[1339] 2. AI image diagnostic processing method

[1340] 3. A means of issuing prescriptions or shipping medications

[1341] 4. Online diagnostics

[1342] 5. Sentiment Analysis Engine

[1343] 6. Means of providing appropriate medical care

[1344] Hardware and software used

[1345] Server: A central server for receiving, analyzing, and processing data.

[1346] Device: The smartphone or computer used by the user

[1347] Sentiment analysis engine: Sentiment analysis software such as IBM Watson or Google Cloud Natural Language API

[1348] Video calling software: Software for real-time video calling

[1349] System Operation

[1350] 1. The user enters the symptoms

[1351] The user uses a terminal to input text or voice input of symptoms into the system's interface. For example, the user may input "I've been having headaches lately."

[1352] 2. The server receives the input data

[1353] The server receives text and voice data sent by the user, which serves as the basis for analyzing the user's symptoms and emotions.

[1354] 3. The server analyzes the data using the emotion engine

[1355] The server uses an emotion analysis engine to analyze the received text and voice data. The emotion engine recognizes the user's emotions from the text and analyzes the tone, strength, and speed of the voice from the voice data. For example, if a user types "I have a headache," the emotion engine will recognize from the text that the user is feeling pain.

[1356] 4. The server issues a prescription based on the analysis results or sends the patient for online diagnosis.

[1357] Based on the analysis results of the emotion engine, the server determines whether the user's symptoms are mild or complicated or cause concerns. If the symptoms are mild, the server automatically issues a prescription. If the symptoms are complicated, the server sends the user for an online diagnosis.

[1358] 5. The server sends the prescription to the designated pharmacy or arranges for the medication to be delivered to the user's home.

[1359] The server electronically sends the issued prescription to the designated pharmacy and, if the user wishes, arranges for the medication to be delivered directly to the user's home.

[1360] 6. When users receive an online diagnosis, they can have a video call with a doctor via their device.

[1361] Users can use the device to interact directly with a doctor via real-time video call, who will provide a detailed diagnosis of the user's symptoms and provide the necessary prescriptions and treatments.

[1362] Specific examples

[1363] As a concrete example, consider the case where a user inputs "I've been having a headache lately." In this case, the emotion engine recognizes that the user is in pain from the keyword "headache." Furthermore, if there is voice input, it also analyzes emotions from the tone and speed of the voice.

[1364] Example prompt sentence:

[1365] If a user types, "I've been having headaches lately," the emotion engine recognizes from the text that the user is in pain. It also analyzes emotions from voice input, such as tone and speed. Based on the analysis, the server issues an appropriate prescription and sends it to a designated pharmacy or directs the user to an online diagnosis.

[1366] This system aims to comprehensively analyze the symptoms and emotions of the user and provide the most appropriate medical response. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1367] Step 1:

[1368] User enters symptoms

[1369] The user uses a terminal to input text about symptoms into the system's interface. Voice input is also possible. For example, the user might input, "I've been having a headache lately." The input data is sent to the server as text or voice data.

[1370] Step 2:

[1371] The server receives the input data

[1372] The server receives text and voice data sent by the user. The received data becomes the basis for analyzing the user's symptoms and emotions. The input data is saved as is and passed to the next analysis step.

[1373] Step 3:

[1374] The server analyzes the data using an emotion engine

[1375] The server uses an emotion analysis engine to analyze the received text and voice data. The emotion engine recognizes the user's emotions from the text and analyzes the tone, strength, and speed of the voice from the voice data. For example, if a user types "I have a headache," the emotion engine recognizes from the text that the user is feeling pain. The analysis results are output as data indicating the user's emotional state.

[1376] Step 4:

[1377] The server issues a prescription based on the analysis results or sends the patient for online diagnosis.

[1378] Based on the analysis results of the emotion engine, the server determines whether the user's symptoms are mild or whether they are complicated or cause concern. If the symptoms are mild, the server automatically issues a prescription. If the symptoms are complicated, the server sends the user to an online diagnosis. The result of the judgment is output as prescription data or instructions for an online diagnosis.

[1379] Step 5:

[1380] The server sends the prescription to a designated pharmacy or arranges for the medication to be delivered to the user's home.

[1381] The server electronically sends the issued prescription to the designated pharmacy and, if desired, arranges for the medication to be delivered directly to the user's home. Prescription data is sent to the pharmacy, and delivery arrangement data is sent to the delivery company.

[1382] Step 6:

[1383] When a user receives an online diagnosis, they will have a video call with a doctor via their device.

[1384] The user uses the device to directly communicate with the doctor through a real-time video call. The doctor will perform a detailed diagnosis of the user's symptoms and provide the necessary prescriptions and treatment. The instruction to start the video call is sent to the user's device, and the video call data is exchanged in real time.

[1385] (Application example 3)

[1386] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1387] In modern society, there is a demand for appropriate medical services and meal recommendations based on the user's health condition and emotions. However, conventional systems have had challenges in analyzing the user's emotions, proposing optimal meal menus based on those emotions, and then delivering those meals quickly.

[1388] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes a chat-style medical interview input means, an AI image diagnosis processing means according to the input content, a means for issuing a prescription or shipping medicine in cases where the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending the user to an online diagnosis in cases where the symptoms are complicated or there are concerns, an emotion engine means for analyzing the user's emotions, a means for proposing an optimal meal menu based on the analysis results, and a means for delivering the suggested meals to the user's home. This makes it possible to provide optimal medical services and meal suggestions based on the user's health condition and emotions, and to deliver them quickly.

[1389] The "chat-style medical interview input means" is an interface that allows users to input their health condition and symptoms using text or voice.

[1390] "AI image diagnosis processing means according to input content" refers to a processing means that uses artificial intelligence to analyze image data provided by the user and determine the condition of the disease.

[1391] "Means for issuing prescriptions or shipping medicines when symptoms or effects of prescription medicines are minor" means means for automatically issuing appropriate prescriptions or shipping medicines to users when the user's symptoms are mild.

[1392] "Means for referring users to an online diagnosis in complicated cases or when there are concerns" refers to means for directing users to an online diagnosis with a doctor when the user's symptoms are complicated or when there are concerns.

[1393] The "emotion engine means for analyzing user's emotions" is an engine for analyzing emotions from the user's text or voice input and identifying the user's emotional state.

[1394] "Means for suggesting optimal meal menus based on analysis results" refers to means for suggesting optimal meal menus to users based on the analysis results of the emotion engine and health status.

[1395] "Means for delivering suggested meals to the user's home" refers to means for delivering a suggested meal to the user's home when the user selects the meal.

[1396] The system for implementing this invention aims to provide optimal medical services and meal recommendations based on the user's health condition and emotions, and deliver them promptly. Specific embodiments of this system will be described below.

[1397] System configuration

[1398] The system consists of the following main components:

[1399] 1. Chat-style medical interview input method:

[1400] Users enter their health status and symptoms using text or voice via a smartphone app.

[1401] Hardware required: Smartphone (iOS or Android)

[1402] Software used: Speech recognition API (e.g., Google Cloud Speech-to-Text), text analysis API (e.g., Google Cloud Natural Language)

[1403] 2. AI image diagnosis processing method according to input content:

[1404] A processing method using artificial intelligence to analyze image data provided by the user and determine the condition of the patient.

[1405] Software used: AI models (TensorFlow, PyTorch, etc.)

[1406] 3. Means of issuing prescriptions or shipping medications for minor illnesses or minor effects of prescription medications:

[1407] If the user's symptoms are mild, the appropriate prescription will be automatically issued or medication will be shipped to the user.

[1408] Software used: Prescription management system

[1409] 4. Online diagnostics for complex cases or concerns:

[1410] If the user has a complex condition or concerns, they can be directed to an online consultation with a doctor.

[1411] Software used: Video call API (WebRTC, etc.)

[1412] 5. Emotion engine means to analyze user emotions:

[1413] An engine for analyzing emotions from a user's text or voice input and identifying their emotional state.

[1414] Software used: Sentiment analysis API (e.g. IBM Watson Tone Analyzer)

[1415] 6. Means of suggesting optimal meal menus based on analysis results:

[1416] Based on the analysis results of the emotion engine and health status, the system suggests the optimal meal menu for the user.

[1417] Software used: Recommendation engine (e.g., Amazon Personalize)

[1418] 7. How the suggested meals will be delivered to your home:

[1419] If the user selects a suggested meal, the meal will be delivered to the user's home.

[1420] Software used: Order management system (e.g. Uber Eats API)

[1421] System Operation

[1422] 1. Getting user input:

[1423] Users can enter their current health status and emotions via text or voice input through a smartphone app.

[1424] Example: A user says, "I've been feeling tired and stressed lately."

[1425] 2. Analysis by emotion engine:

[1426] The emotion engine analyzes input text and voice data to identify the user's emotional state.

[1427] Example: An emotion engine analyzes a user's emotions as "fatigue" and "stress."

[1428] 3. Health Assessment:

[1429] The AI ​​model analyzes data on the user's health condition and suggests appropriate meal menus.

[1430] Example: An AI model suggests menu items based on a user's health status, including nutritious meals and relaxing ingredients.

[1431] 4. Meal menu suggestions:

[1432] Based on the analysis results, the system suggests the most suitable meal menu for the user.

[1433] For example, the system suggests "a nutritious salad bowl," "relaxing herbal tea," and "high-protein grilled chicken."

[1434] 5. Arrange for meal delivery:

[1435] Once the user selects a suggested menu item, the system sends an order to a partner restaurant or delivery service and delivers the meal to the user's home.

[1436] Example: A user selects a "salad bowl" and the system accepts the order and delivers it within 30 minutes.

[1437] Prompt Sentence Examples

[1438] User: I've been feeling tired and stressed lately.

[1439] System: Emotion engine analyzing user emotion...

[1440] System: The user's emotional state has been determined to be "fatigue" and "stress." The following meal plan is suggested:

[1441] 1. A nutritious salad bowl

[1442] 2. Relaxing herbal tea

[1443] 3. High-protein grilled chicken

[1444] User: Select a salad bowl

[1445] System: Your order has been placed. Your salad bowl will be delivered within 30 minutes.

[1446] In this way, it is possible to provide optimal medical services and meal suggestions based on the user's health condition and emotions, and deliver them quickly.

[1447] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1448] Step 1:

[1449] The user launches the smartphone app and inputs their health status and emotions using text or voice. The input data is sent to the server via the smartphone's voice recognition API (such as Google Cloud Speech-to-Text) or text analysis API (such as Google Cloud Natural Language). The input data is in text format.

[1450] Step 2:

[1451] The server analyzes the received text data using an emotion engine (such as IBM Watson Tone Analyzer) to identify the user's emotional state. The input data is text data, and the output data is the emotional state (e.g., "fatigue," "stress," etc.). The emotion engine analyzes the emotional tone of the text data to identify the user's emotion.

[1452] Step 3:

[1453] The server analyzes the emotion engine's analysis results and the user's health status data using an AI model (such as TensorFlow or PyTorch) to generate an appropriate meal menu. The input data is the user's emotional state and health status data, and the output data is a list of meal menus. The AI ​​model generates a menu that includes ingredients that are nutritious and have a relaxing effect based on the user's health status and emotions.

[1454] Step 4:

[1455] The server proposes the generated meal menu to the user. The user checks and selects the proposed menu through a smartphone app. The input data is a list of meal menus, and the output data is the menu selected by the user. The user selects the desired meal from the proposed menu.

[1456] Step 5:

[1457] The server sends the order with the user's selected menu to the partner restaurant or delivery service. The input data is the user's selected menu, and the output data is the order information. The server orders the selected menu through an order management system (such as Uber Eats API).

[1458] Step 6:

[1459] After receiving the order, the partner restaurant or delivery service prepares the meal and delivers it to the user's home. The input data is the order information, and the output data is the delivered meal. The restaurant or delivery service prepares the meal based on the order and delivers it to the specified address.

[1460] In this way, a system is realized that proposes optimal meal menus based on the user's health condition and emotions and delivers them quickly.

[1461] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1462] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1463] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1464] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1465] [Third embodiment]

[1466] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1467] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1468] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1469] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1470] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1471] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1472] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1473] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1474] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1475] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1476] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1477] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1478] "Example 1"

[1479] An embodiment of the present invention is realized as a medical application. A user inputs their symptoms in a chat format through the application. This input is analyzed using natural language processing technology, and important keywords related to the symptoms are extracted. For example, if a user inputs "I have a headache," "headache" is extracted as a keyword.

[1480] "Example 2"

[1481] Next, the AI ​​image diagnosis processing means operates. The AI ​​analyzes the image data provided by the user (for example, a photo of an abnormal area on the skin) and determines the condition. The results of this analysis are reported to the doctor and serve as reference information for the final diagnosis.

[1482] "Example 3"

[1483] For mild symptoms or minor effects of prescribed medication, the system automatically issues a prescription and sends it to a designated pharmacy, or, if desired, can have the medication delivered directly to the user's home.

[1484] "Example 4"

[1485] If there are any complex symptoms or concerns, the system will refer the user to an online diagnosis, where they can interact directly with a doctor via a real-time video call for a detailed diagnosis.

[1486] The processing flow of each embodiment will be described below.

[1487] "Example 1"

[1488] Step 1: The user opens the medical application and enters their symptoms in a chat format.

[1489] Step 2: The application uses natural language processing techniques to analyze the user's input and extract key symptoms keywords.

[1490] "Example 2"

[1491] Step 1: The application receives user-provided image data (e.g., a photo of an abnormal area of ​​skin).

[1492] Step 2: The AI ​​analyzes the image data and determines the condition. The results of this analysis are reported to the doctor.

[1493] "Example 3"

[1494] Step 1: If the AI ​​determines that the user's symptoms are mild, the system will automatically issue a prescription.

[1495] Step 2: The issued prescription is sent to the designated pharmacy.

[1496] Step 3: If desired, the medication is delivered directly to the user's home.

[1497] "Example 4"

[1498] Step 1: If the AI ​​determines that the user's symptoms are complex or concerning, the system will refer the user to an online diagnosis.

[1499] Step 2: Users can interact directly with a doctor through a real-time video call and receive a detailed diagnosis.

[1500] Example 1

[1501] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1502] In conventional medical systems, when users input their symptoms, it is difficult to extract appropriate keywords and provide prompt and accurate medical advice. Furthermore, there are insufficient means to provide appropriate treatment for mild symptoms or when the effects of prescription medication are minor, and there are also insufficient means to quickly refer users to online diagnoses for complex symptoms or concerns. This reduces user convenience and compromises the quality of medical services.

[1503] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1504] In this invention, the server includes a chat-style medical interview input means, a means for analyzing the input content using natural language processing technology and extracting important keywords, and a means for sending the extracted keywords to a generative AI model to generate appropriate medical advice. This makes it possible to quickly and accurately extract keywords based on the symptoms entered by the user and provide appropriate medical advice.

[1505] The "chat-style medical interview input means" is a means for providing an interface for the user to input their own symptoms in text format.

[1506] "Natural language processing technology" is a technology for analyzing input text data and understanding its meaning.

[1507] The "means for extracting important keywords" is a means for identifying important words and phrases related to symptoms from input text using natural language processing technology.

[1508] A "generative AI model" is an artificial intelligence model that generates appropriate medical advice or next steps based on the input prompt.

[1509] The "means for generating medical advice" is a means for using a generative AI model to generate medical advice to be provided to a user based on the extracted keywords.

[1510] "Means for issuing a prescription or shipping medicine when symptoms are mild or the effects of prescribed medicine are minor" refers to means for issuing a prescription or shipping medicine when the user's symptoms are mild and the effects of prescribed medicine are minor.

[1511] "Online diagnosis" refers to a method for providing a diagnosis through a real-time video call with a doctor if the user's symptoms are complex or of concern.

[1512] This invention is a system realized as a medical application, in which users input their symptoms in a chat format, which is analyzed using natural language processing technology, important keywords are extracted, and appropriate medical advice is provided using a generative AI model.

[1513] A user opens a medical application on a device such as a smartphone or tablet. The application has a chat-style interface, and the user inputs their symptoms in text format. For example, they might input, "I've been having such a bad headache lately that I can't sleep."

[1514] The terminal sends the text data entered by the user to the server using the HTTPS protocol, and the server temporarily stores the received text data in a database.

[1515] The server uses Python's natural language processing libraries, NLTK and spaCy, to tokenize, tag parts of speech, and analyze dependencies on the received text data. This allows important keywords to be extracted from the text data. For example, the keywords "headache" and "can't sleep" are extracted.

[1516] The server sends the extracted keywords to the generative AI model as a prompt sentence. An example of a prompt sentence would be, "User input: I've been having a bad headache lately and can't sleep. Extracted keywords: headache, can't sleep."

[1517] The generative AI model generates appropriate medical advice and next steps based on the input prompt, and the generated medical advice is sent to the device via the server.

[1518] The terminal displays the medical advice received from the server to the user, who can then confirm the displayed medical advice and decide on the next course of action.

[1519] For example, if a user inputs "I've had a bad headache recently and can't sleep," the server extracts the keywords "headache" and "can't sleep" and sends a prompt to the generative AI model. The generative AI model generates medical advice such as "If your headache persists, consult a doctor. We also recommend taking a warm bath to relax," and sends it to the device via the server. The device then displays this advice to the user.

[1520] This system allows users to receive fast and accurate medical advice based on their symptoms, providing appropriate treatment for mild symptoms or minor effects of prescribed medication, and quickly directing users to an online diagnosis for complex symptoms or concerns.

[1521] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1522] Step 1:

[1523] The user enters the symptoms.

[1524] A user opens a medical application and enters their symptoms in text format into a chat-style interface. For example, they might enter, "I've been having a bad headache lately and can't sleep." The input data is in text format.

[1525] Step 2:

[1526] The device sends input to the server.

[1527] The terminal sends the text data entered by the user to the server using the HTTPS protocol. The input data is in text format, and the output is the text data sent to the server.

[1528] Step 3:

[1529] The server receives the input.

[1530] The server receives text data sent from the terminal and temporarily stores it in a database. The input data is in text format, and the output is text data stored in the database.

[1531] Step 4:

[1532] The server performs natural language processing.

[1533] The server uses Python natural language processing libraries NLTK and spaCy to tokenize, tag parts of speech, and analyze dependencies on the received text data. The input data is in text format, and the output is the analysis results.

[1534] Step 5:

[1535] The server extracts the keywords.

[1536] The server extracts important keywords from the results of natural language processing. For example, the keywords "headache" and "can't sleep" are extracted. The input data is the analysis results, and the output is the extracted keywords.

[1537] Step 6:

[1538] The server sends the extraction results to the generative AI model.

[1539] The server sends the extracted keywords as a prompt to the generative AI model. An example of a prompt might be, "User input: I've been having a bad headache lately and can't sleep. Extracted keywords: headache, can't sleep." The input data are the extracted keywords, and the output is the prompt sent to the generative AI model.

[1540] Step 7:

[1541] The server receives the results of the generative AI model.

[1542] The server receives the results returned by the generative AI model, which include appropriate medical advice and next steps. The input data is the prompt sent to the generative AI model, and the output is the medical advice returned by the generative AI model.

[1543] Step 8:

[1544] The server sends the results to the terminal.

[1545] The server sends the results received from the generative AI model to the terminal using the HTTPS protocol. The input data is the medical advice returned by the generative AI model, and the output is the medical advice sent to the terminal.

[1546] Step 9:

[1547] The terminal displays the results to the user.

[1548] The terminal displays the medical advice received from the server to the user, who can then review the displayed medical advice and decide on the next course of action. The input data is the medical advice received from the server, and the output is the medical advice displayed to the user.

[1549] (Application example 1)

[1550] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1551] Conventional security services have had the challenge of efficiently analyzing users' anxieties and problems and providing appropriate responses quickly. In particular, it was difficult to accurately understand the text information entered by users and extract important keywords, making it impossible to provide appropriate countermeasures.

[1552] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1553] In this invention, the server includes a chat-style medical interview input means, a keyword extraction means using natural language processing technology according to the input content, a means for issuing a prescription or shipping medicine if the symptoms are mild or the effects of the prescribed medicine are minor, and a means for sending the patient to an online diagnosis if the symptoms are complicated or there are concerns. This makes it possible to accurately analyze the text information entered by the user, extract important keywords, and quickly provide appropriate countermeasures.

[1554] The "chat-style medical interview input means" is a means for providing an interface for users to input questions and symptoms in text format.

[1555] "Keyword extraction means using natural language processing technology" refers to a means that uses technology to analyze text information entered by the user and extract important keywords.

[1556] "Means for issuing prescriptions or shipping medicines when symptoms are mild or the effects of prescription medicines are minor" refers to means for issuing prescriptions or shipping medicines when the user's symptoms are determined to be mild and the effects of prescription medicines are determined to be minor.

[1557] "Online diagnostic means for complex or concerning cases" refers to a means for users to receive an online diagnosis from a specialist if their symptoms are complex or they have concerns.

[1558] "Means for enabling real-time video calls with experts" means means for enabling a user to have real-time video calls with experts.

[1559] The system for implementing this invention is configured as follows: First, a user inputs security-related concerns or problems in a chat format using a smartphone. The input text information is sent to a server.

[1560] The server uses natural language processing technology to analyze the user's input text and extract important keywords. This analysis is performed using Python and the natural language processing library spaCy. Specifically, if a user types "I've been seeing suspicious people around my house recently," the server will extract the keyword "suspicious person."

[1561] Based on the extracted keywords, the server determines the appropriate course of action: if the symptoms are mild or the effects of a prescription are minor, a prescription or medication delivery is used, while if the symptoms are more complex or there are concerns, the patient is referred for an online diagnosis, which includes a real-time video call with a specialist.

[1562] For example, if a user types in "I've been seeing suspicious people around my house lately," the server will extract the keyword "suspicious people" and suggest a video call with a security expert, allowing the user to communicate with the expert in real time using their smartphone.

[1563] Examples of prompts to input to a generative AI model include:

[1564] A user enters a security issue or concern. For example, if the user enters "I've been seeing suspicious people around my house lately," extract "suspicious people" as an important keyword.

[1565] In this way, it becomes possible to efficiently analyze users' security concerns and problems and quickly provide appropriate countermeasures.

[1566] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1567] Step 1:

[1568] A user uses a smartphone to input security-related concerns or problems in chat format. The input text information is sent from the smartphone to a server. The input data is text information about the user's concerns or problems.

[1569] Step 2:

[1570] The server analyzes the received text information using natural language processing technology. This analysis uses Python and the natural language processing library spaCy. Specifically, the text information is tokenized and important keywords such as nouns, proper nouns, and adjectives are extracted. The input data is the user's text information, and the output data is the extracted keywords.

[1571] Step 3:

[1572] The server determines the appropriate response based on the extracted keywords. If the symptoms are mild or the effects of the prescribed medication are minor, measures such as issuing a prescription or shipping the medication are used. The input data are the extracted keywords, and the output data are instructions to issue a prescription or ship the medication.

[1573] Step 4:

[1574] The server executes a means for routing the extracted keywords to an online diagnostic if they are complex or cause concern, including a means for enabling a real-time video call with an expert, where the input data is the extracted keywords and the output data is an instruction to start the video call.

[1575] Step 5:

[1576] Users can use their smartphones to make real-time video calls with experts, allowing them to receive advice and diagnoses directly from the experts. The input data is an instruction to start the video call, and the output data is the advice and diagnosis results from the experts.

[1577] Example 2

[1578] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1579] In conventional medical systems, it is difficult for patients to accurately communicate their symptoms, which often leads to delayed diagnosis, especially for visual symptoms such as skin abnormalities. Another problem is that even in cases of mild illness, it takes time and effort to receive a diagnosis from a doctor. Furthermore, patients living in remote areas often have difficulty accessing medical institutions. To solve these issues, a system is needed that can efficiently and quickly diagnose illnesses and provide appropriate medical services.

[1580] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1581] In this invention, the server includes a chat-style medical interview input means, an image diagnosis processing means according to the input content, a means for issuing a prescription or shipping medicine if the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending an online diagnosis if the symptoms are complicated or there are concerns, a means for sending image data taken by the user from the terminal to the server, an image analysis means using a generative AI model to analyze the image data received by the server, and a means for reporting the analysis results to a doctor. This allows patients to easily diagnose skin abnormalities from home and receive appropriate medical care promptly.

[1582] The "chat-style medical interview input means" is an interface that allows users to input information such as symptoms and medical history via text messages, and the system receives this information.

[1583] The "image diagnosis processing means" is a technical means for analyzing image data provided by a user and determining the condition of the disease.

[1584] "Means for issuing prescriptions or shipping medicines" refers to a function that enables the system to automatically issue prescriptions or ship medicines to patients when the symptoms or effects of prescribed medicines are minor.

[1585] "Online referral" is a feature that allows the system to refer patients to online consultations, such as a real-time video call with a doctor, in cases of complications or concerns.

[1586] "Means for transmitting image data taken by a user from a terminal to a server" refers to technical means for transmitting image data taken by a user to a server via a terminal.

[1587] "Image analysis means using generative AI models" refers to a technical means for analyzing image data and determining the condition of a disease using an AI model built using a deep learning framework.

[1588] "Means for reporting analysis results to doctors" refers to the technical means for reporting the analysis results obtained by the AI ​​model to doctors.

[1589] MODE FOR CARRYING OUT THE INVENTION

[1590] This invention is a system for analyzing image data captured by a user and determining the condition of a patient. The system includes a chat-style interview input means, an image diagnosis processing means, a means for issuing a prescription or shipping medicine, a means for sending the image data captured by the user from a terminal to a server, an image analysis means using a generative AI model, and a means for reporting the analysis results to a doctor.

[1591] Hardware and software used

[1592] Device: Smartphone, tablet, computer, or other device

[1593] Server: A server with high-performance computing power

[1594] Deep learning frameworks: TensorFlow, PyTorch, etc.

[1595] Data processing and calculation

[1596] The user takes a photo of the abnormal area on their skin using a smartphone or digital camera. After taking the photo, they upload the image data to the system using a dedicated application. The device encrypts the image data and sends it to the server using a secure communication protocol (e.g., HTTPS).

[1597] The server analyzes the received image data using a generative AI model. This AI model is built using deep learning frameworks such as TensorFlow and PyTorch. Specifically, it uses a convolutional neural network (CNN) to extract image features and identify patterns of abnormalities. This analysis results in a diagnosis of the condition, such as "suspected skin cancer" or "high probability of eczema."

[1598] The analysis results are reported from the server to the doctor, who then uses this information to make a final diagnosis. For example, if the doctor receives a result that suggests skin cancer, he or she can prescribe further detailed testing.

[1599] Specific examples

[1600] For example, if a user notices an abnormality in their skin, they take a photo of the area with their smartphone and upload the image to the system using a dedicated application. The device encrypts this image and sends it to a server using HTTPS. The server receives the image and analyzes it using a generative AI model. If the analysis results in a "suspected skin cancer," the information is reported to a doctor. The doctor can then use this information to prescribe further tests.

[1601] Prompt Sentence Examples

[1602] "Design a system that analyzes photos of abnormal skin areas taken by users, uses an AI model to determine the condition, and reports the diagnosis to a doctor."

[1603] The system is designed to allow users to easily detect skin abnormalities and seek early medical diagnosis.

[1604] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1605] Program processing flow

[1606] Step 1: User takes and uploads image data

[1607] The user takes a photo of the abnormal area of ​​the skin using a smartphone or digital camera. After taking the photo, the image data is uploaded to the system using a dedicated application. The input is the image data taken by the user, and the output is the image data saved on the device.

[1608] Step 2: The device sends the image data to the server

[1609] The terminal sends image data uploaded by the user to the server. At this time, the image data is encrypted and sent using a secure communication protocol (e.g., HTTPS). The input is the image data stored on the terminal, and the output is the encrypted image data sent to the server.

[1610] Step 3: The server receives the image data and passes it to the AI ​​image diagnosis processing means.

[1611] The server receives the image data sent from the terminal. After receiving the image data, the server passes the image data to the AI ​​image diagnosis processing means. The input is the encrypted image data sent to the server, and the output is the image data passed to the AI ​​image diagnosis processing means.

[1612] Step 4: The server analyzes the image data using the AI ​​model

[1613] The server analyzes the image data using an AI model built using deep learning frameworks such as TensorFlow and PyTorch. Specifically, it uses a convolutional neural network (CNN) to extract image features and identify patterns of abnormalities. The input is the image data passed to the AI ​​image diagnostic processing means, and the output is a diagnosis of the disease state.

[1614] Step 5: The server reports the analysis results to the doctor

[1615] The server reports the analysis results obtained by the AI ​​model to the doctor. This reporting is done through a dedicated medical information system. The input is the diagnosis result of the disease, and the output is the diagnosis information reported to the doctor.

[1616] Adding specific actions

[1617] Step 1: User takes and uploads image data

[1618] If a user feels any abnormality in their skin, they take a photo of the area with their smartphone. They open the dedicated application, select the image they have taken, and press the upload button. The image data is then saved on the device.

[1619] Step 2: The device sends the image data to the server

[1620] The device encrypts the image data uploaded by the user and sends it to the server using HTTPS. When the transmission is complete, the device displays a notification to the user that the transmission is complete.

[1621] Step 3: The server receives the image data and passes it to the AI ​​image diagnosis processing means.

[1622] The server receives the encrypted image data sent from the terminal, decrypts it, and passes the decrypted image data to the AI ​​image diagnosis processing means.

[1623] Step 4: The server analyzes the image data using the AI ​​model

[1624] The server analyzes the image data passed to the AI ​​image diagnostic processing means using a CNN model built using TensorFlow, and obtains a diagnosis of the condition, such as whether there is a suspicion of skin cancer.

[1625] Step 5: The server reports the analysis results to the doctor

[1626] The server reports the diagnosis of the patient's condition to the doctor via a dedicated medical information system, and the doctor uses this information to decide whether to order further detailed tests.

[1627] (Application example 2)

[1628] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1629] Modern medical systems require rapid and accurate diagnosis of patients' conditions. However, conventional systems face challenges in analyzing image data provided by patients and detecting abnormal behavior or suspicious individuals in real time. Furthermore, they lack the ability to quickly notify and respond when an abnormality is detected. This can lead to delayed responses in medical and security settings, potentially threatening the safety of patients and facilities.

[1630] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1631] In this invention, the server includes a chat-style medical interview input means, an AI image diagnosis processing means according to the input content, a means for issuing a prescription or shipping medicine if the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending an online diagnosis if the symptoms are complicated or there are concerns, a means for analyzing images captured by security cameras in real time to detect abnormal behavior or suspicious individuals, and a means for storing the detection results in a database and sending a notification if an abnormality is detected. This makes it possible to quickly and accurately diagnose the patient's condition, and also enables real-time detection of abnormal behavior or suspicious individuals through security camera image analysis and rapid notification.

[1632] The "chat-style medical interview input means" is an interface that allows users to input symptoms and conditions by answering questions in text format.

[1633] An "AI image diagnostic processing means" is a system that uses artificial intelligence to analyze image data and determine the condition or abnormality.

[1634] "Means for issuing prescriptions or shipping medicines" refers to a system for issuing prescriptions or shipping medicines to patients based on a doctor's instructions in cases where the symptoms or effects of prescription medicines are minor.

[1635] "Online referral" is a system that directs patients to a real-time video call with a doctor for complex cases or concerns.

[1636] "Means for analyzing security camera footage in real time to detect abnormal behavior and suspicious individuals" is a system for analyzing security camera footage in real time to automatically detect abnormal behavior and suspicious individuals.

[1637] "Means for saving detection results in a database and sending notifications when abnormalities are detected" refers to a system that saves the results in a database when abnormal behavior or suspicious individuals are detected and sends notifications to responsible personnel as necessary.

[1638] A system for implementing this invention has the following configuration: The server includes a means for inputting medical interviews in chat format, an AI image diagnosis processing means according to the input content, a means for issuing a prescription or shipping medicine in cases where the symptoms are mild or the effects of the prescribed medicine are minor, a means for sending an online diagnosis in cases where the symptoms are complicated or there are concerns, a means for analyzing images captured by security cameras in real time to detect abnormal behavior or suspicious individuals, and a means for storing the detection results in a database and sending a notification if an abnormality is detected.

[1639] Hardware used:

[1640] server

[1641] security cameras

[1642] Smartphone or tablet

[1643] Database Server

[1644] Software used:

[1645] Image analysis AI models (e.g., YOLO, OpenCV)

[1646] Communication protocol (e.g. HTTP, WebSocket)

[1647] Database (e.g. MySQL, MongoDB)

[1648] System behavior:

[1649] 1. Chat-style medical interview input method:

[1650] Users use a smartphone or tablet to input their symptoms and conditions in a text-based chat format, which then sends the medical interview information to the server.

[1651] 2. AI image diagnosis processing methods:

[1652] The server receives image data provided by the user (e.g., a photo of an abnormal area of ​​the skin) and analyzes it using an image analysis AI model (e.g., YOLO). The analysis results are used to determine the condition of the patient.

[1653] 3. Prescription or drug delivery method:

[1654] If the symptoms are mild or the effects of prescribed medication are minor, the server will automatically issue a prescription and ship the medication to the patient.

[1655] 4. Online diagnostics:

[1656] In the case of complex cases or concerns, the server directs the user to a real-time video call with a doctor.

[1657] 5. Security camera footage analysis methods:

[1658] Security cameras capture video in real time and send it to a server, which uses an image analysis AI model to detect abnormal behavior or suspicious individuals in the video.

[1659] 6. Detection result storage and notification method:

[1660] The detection results are stored in a database, and if an anomaly is detected, a notification is sent to a smartphone or tablet, allowing security personnel to take prompt action.

[1661] Examples:

[1662] If a security camera detects a suspicious person at night, the server will send a notification to the smartphone saying, "Suspicious person detected. Location: near the main gate."

[1663] Security personnel open an application on their smartphone, check the details of the analysis results, and decide whether to report the incident to the police.

[1664] Example prompts to input to a generative AI model:

[1665] "Please create an AI model that analyzes footage captured by security cameras and detects abnormal behavior and suspicious individuals. The results of the detection should be sent to a smartphone in real time."

[1666] Specific prompt:

[1667] "Using the YOLO model, please build a system that detects suspicious individuals in real time from security camera footage. Please also implement a function to send the detection results to a server using the HTTP protocol and send notifications to smartphones."

[1668] In this way, a system for implementing the invention allows for a fast and accurate response in both the medical and security fields.

[1669] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1670] Step 1:

[1671] Enter medical questions in chat format

[1672] Users use a smartphone or tablet to input their symptoms and conditions in a text-based chat format, and the input data is sent to a server.

[1673] Input: User-entered text data about symptoms and conditions

[1674] Output: Interview data sent to the server

[1675] Specific behavior: A user opens the application, enters symptoms in the chat box, and presses the send button. The server receives this data and stores it in a database.

[1676] Step 2:

[1677] AI image diagnostic processing

[1678] The server receives image data provided by the user (e.g., a photo of an abnormal area of ​​the skin) and analyzes it using an image analysis AI model (e.g., YOLO).

[1679] Input: User uploaded image data

[1680] Output: Analysis results (diagnosis of the condition)

[1681] How it works: A user uploads an image through the application, and the server receives the image. The server then invokes an image analysis AI model to analyze the image and determine the medical condition. The analysis results are then stored in a database.

[1682] Step 3:

[1683] Prescription issuance or drug delivery

[1684] If the symptoms are mild or the effects of prescribed medication are minor, the server will automatically issue a prescription and ship the medication to the patient.

[1685] Input: Analysis result (if determined to be mild)

[1686] Output: Instructions for issuing a prescription or shipping medicine

[1687] Specific operation: The server evaluates the analysis results and, if it determines that the patient's condition is mild, it automatically generates a prescription and sends instructions to the pharmacy to ship the medication.

[1688] Step 4:

[1689] Guidance to online diagnosis

[1690] In the case of complex cases or concerns, the server directs the user to a real-time video call with a doctor.

[1691] Input: Analysis results (if determined to be a complex case)

[1692] Output: Video call link or invitation

[1693] What it does: The server evaluates the analysis results and, if it determines that the case is complex, it sends the user a link to a video call to set up an online consultation with a doctor.

[1694] Step 5:

[1695] Real-time analysis of security camera footage

[1696] Security cameras capture video in real time and send it to a server, which uses an image analysis AI model to detect abnormal behavior or suspicious individuals in the video.

[1697] Input: Real-time video data from security cameras

[1698] Output: Detection results of abnormal behavior and suspicious individuals

[1699] How it works: A security camera captures video and sends it to a server, which then invokes an image analysis AI model to analyze the video and detect abnormal behavior or suspicious individuals.

[1700] Step 6:

[1701] Storing and notifying detection results

[1702] The detection results are stored in a database, and if an abnormality is detected, a notification is sent to a smartphone or tablet.

[1703] Input: Detection results of abnormal behavior or suspicious individuals

[1704] Output: Save to database and send notification

[1705] How it works: The server stores the detection results in a database, and if an anomaly is detected, a notification is sent to a smartphone or tablet. Security personnel receive the notification and open an application to check the details.

[1706] Example 3

[1707] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1708] In the conventional medical system, users had to visit a medical institution in person even for mild symptoms or minor effects of prescribed medication, which required time and effort. Furthermore, when users had complex symptoms or concerns, numerous procedures were required to receive an appropriate diagnosis, making it difficult to respond quickly. Furthermore, the lack of real-time communication with doctors made it difficult for users to receive appropriate medical services.

[1709] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1710] In this invention, the server includes a chat-style medical interview input means, a health information analysis means using a generative AI model based on the input content, a means for issuing a prescription or shipping medication if the symptoms are mild or the effects of the prescribed medication are minor, a means for sending the patient to an online diagnosis if the symptoms are complicated or there are concerns, and a means for enabling real-time video calls with a doctor. This allows users to easily input their health information from home and receive an appropriate diagnosis and prescription. Furthermore, even if the patient has complex symptoms or concerns, they can receive a quick online diagnosis and a detailed diagnosis through real-time communication with a doctor.

[1711] "Chat-style medical interview input means" is a means for users to input their health conditions and symptoms through a text-based interface.

[1712] A "health information analysis method using a generative AI model" is a method that uses a generative AI model to analyze health information entered by a user and determine the severity of symptoms and appropriate prescriptions.

[1713] "Means for issuing prescriptions or shipping medicines" means means for automatically generating an appropriate prescription and sending it to a designated pharmacy or delivering the medicine to the user's home in the case of a minor illness or minor effects of a prescribed medicine.

[1714] "Online diagnostic referral" refers to a method that guides users to online diagnostics and provides appropriate medical services when they have complex symptoms or concerns.

[1715] "Means enabling real-time video calls with doctors" means means that users can have real-time video calls with doctors and receive detailed diagnoses.

[1716] MODE FOR CARRYING OUT THE INVENTION

[1717] This invention is a system that allows users to easily input health information from home and receive appropriate diagnosis and prescriptions. A specific embodiment of this system will be described below.

[1718] System configuration

[1719] The system consists of the following main components:

[1720] 1. Chat-style medical interview input method

[1721] 2. Health information analysis method using generative AI models

[1722] 3. A means of issuing prescriptions or shipping medications

[1723] 4. Online diagnostics

[1724] 5. A way to enable real-time video calls with doctors

[1725] Hardware and Software Use

[1726] Users access the system using devices such as smartphones or PCs. The system is provided through a dedicated application or website. The server receives input information from users and analyzes it using a generative AI model. This analysis uses natural language processing libraries (e.g., spaCy) and machine learning libraries (e.g., TensorFlow) implemented in Python.

[1727] Data processing and calculation

[1728] The health information entered by the user is received by the server and analyzed using a generative AI model. Based on the analysis results, if the symptom or the effect of the prescribed medication is determined to be minor, the server automatically generates a prescription and sends it to the designated pharmacy. This prescription is sent using a secure communication protocol (e.g., HTTPS). If the user requests home delivery of the medication, the server arranges for the delivery.

[1729] If the user has complex symptoms or concerns, the server will refer them to an online diagnosis, where they will use their device to interact directly with a doctor via a real-time video call, using real-time communication technologies such as WebRTC.

[1730] Specific examples

[1731] For example, if a user inputs "I have a sore throat and a slight cough," the server receives this information and analyzes it using a generative AI model. Based on the analysis results, it is determined to be a mild cold. The server generates a prescription for cold medicine and sends it to the designated pharmacy. If the user requests home delivery of the medicine, the server processes the delivery and the medicine is delivered to the user's home.

[1732] Prompt Sentence Examples

[1733] If a user reports "I have a sore throat and a slight cough," how would the system issue a prescription and deliver the medication?

[1734] In this way, users can easily input their health information from home and receive appropriate diagnosis and prescriptions. Even if users have complex symptoms or concerns, they can receive a quick online diagnosis and receive a detailed diagnosis through real-time communication with a doctor. The flow of the identification process in Example 3 will be described with reference to FIG. 15.

[1735] Step 1:

[1736] The user enters their health information.

[1737] Users use their devices to access the system's dedicated application or website and enter their current health condition and symptoms, for example, specific symptoms such as "I have a sore throat and a slight cough." The entered information is then sent to the server.

[1738] Input: User's health information (e.g., "I have a sore throat and a slight cough")

[1739] Output: Health information sent to the server

[1740] Step 2:

[1741] The server receives and analyzes the health information.

[1742] The server receives health information submitted by the user. This information is analyzed using a generative AI model. The analysis uses natural language processing libraries (e.g., spaCy) and machine learning libraries (e.g., TensorFlow) implemented in Python. The server evaluates the severity of symptoms and the impact of prescribed medications.

[1743] Input: Health information submitted by the user

[1744] Output: Analysis result (e.g., judged to be a mild cold)

[1745] Step 3:

[1746] The server generates a prescription based on the analysis results.

[1747] The server then generates an appropriate prescription based on the analysis results. For example, if it is determined to be a mild cold, a prescription for cold medicine will be generated. This prescription is generated electronically and stored in a secure format.

[1748] Input: Analysis results

[1749] Output: Generated prescription

[1750] Step 4:

[1751] The server sends the prescription to the designated pharmacy.

[1752] The server sends the generated prescription to the designated pharmacy using a secure communication protocol (e.g., HTTPS), and the pharmacy prepares the medication based on the received prescription.

[1753] Input: Generated prescription

[1754] Output: Prescription sent to pharmacy

[1755] Step 5:

[1756] If the user requests delivery of the medicine, the server handles the delivery process.

[1757] If the user requests home delivery of the medicine, the server processes the delivery, confirms the user's address information, and requests a delivery company. The delivery company then delivers the medicine to the user's home.

[1758] Input: User's shipping preferences, address information

[1759] Output: Delivery procedure requested by the delivery company

[1760] Step 6:

[1761] If there are complex symptoms or concerns, the server will direct the user to an online diagnosis.

[1762] If the server determines that the user has complex symptoms or concerns based on the analysis results, it will refer the user to an online diagnosis, and the user will receive a notification from the system and make an appointment for the online diagnosis.

[1763] Input: Analysis results (complex symptoms and concerns)

[1764] Output: Online diagnostic notification, reservation procedure

[1765] Step 7:

[1766] Users have real-time video calls with doctors.

[1767] At the scheduled time, users can use their devices to make a real-time video call using real-time communication technologies such as WebRTC, allowing them to directly communicate with a doctor and receive a detailed diagnosis.

[1768] Input: Scheduled video call time, user device

[1769] Output: Real-time video call with doctor, detailed diagnosis

[1770] (Application example 3)

[1771] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1772] In the conventional medical system, users had to visit a medical institution even for mild symptoms or minor effects of prescribed medication, which required time and effort. Furthermore, when users had complex symptoms or concerns, numerous procedures were required to receive an appropriate diagnosis, making it difficult to respond quickly. Furthermore, when it came to receiving medication, users had to visit a pharmacy, which was inconvenient.

[1773] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1774] In this invention, the server includes a chat-style medical interview input means, a diagnostic processing means using a generative AI model based on the input content, a means for issuing a prescription or shipping medication in cases where the symptoms are mild or the effects of the prescribed medication are minor, a means for referring the patient to an online diagnosis in cases where the symptoms are complicated or there are concerns, a means for delivering medication to the patient's home at the user's request, and a means for conducting a real-time video call with a doctor through the online diagnosis. This allows users to easily receive a diagnosis from home and quickly receive the necessary medication. Furthermore, even in cases where the symptoms are complicated, users can receive prompt and appropriate medical services through the online diagnosis.

[1775] A "chat-style medical interview input means" is a means for users to input symptoms and health conditions through a text-based chat interface.

[1776] A "diagnostic processing means using a generative AI model" is a means for analyzing a medical condition and making a diagnosis using a generative AI model based on information entered by a user.

[1777] "Means for issuing prescriptions or shipping medicines when symptoms are mild or the effects of prescribed medicines are minor" refers to means for the system to automatically issue prescriptions or ship medicines when the user's symptoms are mild and the effects of prescribed medicines are minor.

[1778] "Means for referring users to online diagnosis in complex cases or when there are concerns" refers to the means by which the system directs users to online diagnosis when the user's symptoms are complex or when there are concerns.

[1779] "Means for delivering medicine to the user's home at the user's request" refers to means for delivering prescribed medicine to the user's home if the user so requests.

[1780] "Means for having real-time video calls with doctors through online diagnosis" means a means for users to have real-time video calls with doctors and receive detailed diagnoses.

[1781] A system for implementing the present invention has the following configuration.

[1782] First, the user inputs a medical questionnaire in chat format using a smartphone or smart glasses. The medical questionnaire input method allows users to input symptoms and health conditions through a text-based interface, allowing users to easily communicate their symptoms to the system.

[1783] The input information is then sent to a diagnostic processing means that uses a generative AI model. This means uses the generative AI model to analyze the condition and make a diagnosis based on the information entered by the user. Specifically, the generative AI model analyzes the user's symptoms and outputs an appropriate diagnosis. Based on this diagnosis result, the system proceeds to the next step.

[1784] If the symptoms are mild or the effects of the prescribed medication are minor, the system will automatically issue a prescription or ship the medication. This prescription issuing method automatically generates a prescription and sends it to a designated pharmacy if the user's symptoms are mild and the effects of the prescribed medication are minor. It also includes a method for delivering medication to the user's home, if desired. This allows the user to receive the necessary medication without leaving home.

[1785] On the other hand, if the user has complex symptoms or concerns, the system will refer the user to an online diagnosis. This online diagnosis method allows the user to have a real-time video call with a doctor to receive a detailed diagnosis. The video call is made using the camera and microphone on a smartphone or smart glasses. This allows the user to directly interact with a doctor from the comfort of their own home and receive a proper diagnosis.

[1786] For example, if a user inputs symptoms such as "cough, fever," the generative AI model analyzes them and diagnoses them as a "cold." Based on this diagnosis, the system issues a prescription for "500mg paracetamol" and sends it to the designated pharmacy. If the user requests it, the system can also arrange for the medicine to be delivered to their home.

[1787] Examples of prompts include:

[1788] User symptoms: Cough, fever

[1789] Diagnosis: Cold

[1790] Prescription: Paracetamol 500mg

[1791] In this way, this invention allows users to easily receive a diagnosis from home and quickly receive the necessary medication. In addition, even in cases of complex symptoms, users can receive prompt and appropriate medical services through online diagnosis.

[1792] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1793] Step 1:

[1794] The user inputs the medical questionnaire in chat format using a smartphone or smart glasses.

[1795] Input: Text data about the user's symptoms and health conditions.

[1796] Data processing: Analyze text data using natural language processing technology to extract symptoms.

[1797] Output: Extracted symptom data.

[1798] Step 2:

[1799] The server transmits the input symptom data to a diagnostic processing means using a generative AI model.

[1800] Input: Extracted symptom data.

[1801] Data computation: Generative AI models analyze symptom data and diagnose medical conditions.

[1802] Output: Diagnosis result data (e.g. cold, flu, etc.).

[1803] Step 3:

[1804] Based on the diagnosis, the server issues a prescription if the symptoms are mild or the effects of the prescribed medication are minor.

[1805] Input: Diagnostic result data.

[1806] Data processing: Generate appropriate prescriptions based on diagnostic results.

[1807] Output: Prescription data.

[1808] Step 4:

[1809] The server sends the generated prescription data to the designated pharmacy.

[1810] Input: Prescription data.

[1811] Data Computation: Converts prescription data into a format for transmission to the pharmacy system.

[1812] Output: Prescription data sent to the pharmacy.

[1813] Step 5:

[1814] If desired, the server will arrange for the medicine to be delivered to the user...

Claims

1. A chat-style questionnaire input method; a means for analyzing text using a natural language processing technique according to the input contents of the inquiry input means; A means of extracting important keywords from the analysis results; A means for analyzing user emotions using an emotion engine; A means for diagnosing the user based on an analysis result of the emotion engine; A means for issuing a prescription or shipping medicine when the diagnosis of the user indicates that the symptoms are mild or the effects of the prescribed medicine are minor; A means for sending the user to an online diagnosis if the diagnosis result is complicated or of concern; A system including:

2. The system according to claim 1 , wherein the means for analyzing text using natural language processing techniques analyzes text data provided by a user to determine a medical condition.

3. 2. The system of claim 1, wherein the means for directing the patient to online diagnosis enables a real-time video call with a doctor.

Images