System

Abstract

A system is provided.SOLUTION: A system includes means for receiving symptom data of a child input by a user, means for receiving a symptom image of the child captured by the user, means for executing artificial intelligence for analyzing the received symptom data and image data, means for estimating a disease name, necessity of hospital visit, and an expected number of days until full recovery based on data analyzed by the artificial intelligence, means for providing user-friendly advice together with an estimation result, means for receiving feedback data from the user, and means for storing the received feedback data in a database and using the feedback data for relearning of the artificial intelligence.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a need to reduce the anxiety and distress parents feel when their children become unwell. In particular, parents raising their first child often find themselves in a situation where they do not know what to do when their child becomes ill, and are forced to visit medical institutions frequently. Furthermore, parents' anxiety does not subside because reliable information cannot be found even when searching for information on the Internet. To solve this problem, a system is needed that can quickly and accurately diagnose children's symptoms and instruct parents on appropriate measures. [Means for solving the problem]

[0005] This invention is a system that receives user-entered symptom data and photographed image data from children and uses artificial intelligence to analyze them to estimate the diagnosis of the child's illness, whether medical visits are necessary, and the expected number of days until recovery. Specifically, the user enters their child's symptoms in text and uploads images of the symptoms using the app's camera function. The device then encrypts this data and sends it to a server, where it is securely stored and analyzed. The AI ​​on the server analyzes the data and provides possible diagnosis and treatment options. It also provides friendly advice to the user based on the estimation results, and receives and analyzes feedback data such as subsequent progress and medical diagnosis results, thereby improving the AI's diagnostic accuracy. This system significantly reduces the anxiety and distress parents experience when their children are unwell and quickly provides appropriate medical information.

[0006] "User" means any person, including a parent or guardian, who uses the system.

[0007] "Symptom data" is textual information about a child's physical condition and state.

[0008] "Image data" refers to photographs and image files of children's symptoms and injuries.

[0009] A "terminal" is a device used by a user, such as a smartphone, tablet, or computer.

[0010] "Server" means the central processing unit through which the system receives, stores, and analyzes data.

[0011] "Artificial intelligence" refers to a program or algorithm that analyzes input data and estimates the name of the disease, the severity of symptoms, whether or not medical treatment is required, etc.

[0012] "Gentle advice" refers to friendly words and guidance offered to parents to ease their anxiety and provide reassurance.

[0013] "Feedback data" refers to data that a user provides to the system, such as information on future progress and diagnosis results from medical institutions.

[0014] A "database" is a storage device within the system that stores and manages data received from users and feedback data.

[0015] "Relearning" is the process of updating an artificial intelligence analysis model using new data to improve diagnostic accuracy.

[0016] The "estimated results" are information obtained as a result of analysis by artificial intelligence, such as the name of the disease, whether or not outpatient treatment is required, and the expected number of days until recovery. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention relates to a system that receives symptom data and photographed image data of a child entered by a user, and analyzes them using artificial intelligence to estimate the name of the disease, whether or not the child needs to visit a hospital, and the expected number of days until recovery. A specific embodiment of this system will be described below.

[0039] Overall system configuration

[0040] The system consists of a device owned by the user, a server for analyzing data, and a communication network for linking these. Users can begin using the system by downloading the application and creating an account.

[0041] 1. User Actions

[0042] Users launch the app and enter their child's symptoms as text. They also take photos of the symptoms and injuries and upload them as image data.

[0043] The user enters details of the symptoms and the date and time of onset and submits the data.

[0044] 2. Device Operation

[0045] The terminal encrypts the text data entered by the user and the captured image data.

[0046] The terminal transmits the encrypted data to the server via a communication network.

[0047] 3. Server Operation

[0048] The server receives the data sent from the terminal, decodes the data, and then prepares the received symptom data and image data for analysis.

[0049] Artificial intelligence on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[0050] The server generates gentle advice to reassure parents along with the inference results.

[0051] The server transmits this information to the terminal.

[0052] 4. Notice to Users

[0053] The device displays the diagnostic results and advice received from the server to the user, and uses the push notification function to notify the user when the diagnostic results are ready.

[0054] Users can check the diagnosis results and take necessary measures. They can also send feedback data by entering the progress of their symptoms and the results of their hospital diagnosis into the app.

[0055] 5. Processing of Feedback Data

[0056] The server receives the feedback data sent by the user and stores it in a database.

[0057] The server retrains the artificial intelligence based on the received feedback data, thereby improving the system's diagnostic accuracy.

[0058] Specific examples

[0059] Example 1: The common cold

[0060] A user launches the app and enters text that their 3-year-old child is coughing. They also take a photo of the child coughing and upload it to the app.

[0061] The device encrypts this data and sends it to the server.

[0062] The server analyzes the received data and determines that the illness is likely a mild cold. It also predicts that the time until recovery will be between 3 and 5 days, and notifies the user via push notification.

[0063] Along with the diagnosis, the user receives gentle advice: "Keep warm and give your child plenty of fluids. If symptoms persist for more than a few days, we recommend that you see a doctor."

[0064] The user later inputs into the system as feedback that the symptoms have improved.

[0065] Example 2: Minor injury

[0066] A user launches the app and enters text that their child fell and scraped their knee, taking a photo of the injury and uploading it to the app.

[0067] The device encrypts this data and sends it to the server.

[0068] The server analyzes the received data and determines that the injury is a minor scratch. No further hospital visits are required, and the estimated time until recovery is about five days.

[0069] Along with the diagnosis, the user receives gentle advice: "Disinfect the wound and apply gauze. If the wound becomes infected, we recommend that you go to the hospital."

[0070] The above is an embodiment of the present invention.

[0071] The processing flow will be explained below.

[0072] Step 1:

[0073] The user launches the app and logs in or creates an account.

[0074] Step 2:

[0075] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[0076] Step 3:

[0077] The user checks the text data entered and the captured image data within the app and taps the "Send" button.

[0078] Step 4:

[0079] The device detects that the send button has been pressed and encrypts the text and image data using a common key encryption method such as AES.

[0080] Step 5:

[0081] The terminal transmits the encrypted data to the server via the communication network.

[0082] Step 6:

[0083] The server decrypts the received encrypted data and prepares it for analysis. The data is temporarily stored in secure storage.

[0084] Step 7:

[0085] The text data received by the server is input into a natural language processing (NLP) engine to extract keywords related to the symptoms.

[0086] Step 8:

[0087] The server preprocesses the received image data and converts it into a format for analysis, adjusting the image resolution and removing noise.

[0088] Step 9:

[0089] The server inputs the preprocessed text and image data into an artificial intelligence (AI) model to perform symptom analysis.

[0090] Step 10:

[0091] The server's AI model analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[0092] Step 11:

[0093] Based on the estimation results of the AI ​​model, the server generates reassuring advice for the user in friendly language.

[0094] Step 12:

[0095] The server composes the generated diagnostics and advice and prepares them for transmission to the device.

[0096] Step 13:

[0097] The device displays the diagnostic results and advice received from the server to the user. Notifications can also be sent using the push notification function.

[0098] Step 14:

[0099] The user can check the diagnosis results and take measures as necessary. They can also enter the progress of symptoms and the results of the diagnosis at the hospital into the app at a later date and send it as feedback data.

[0100] Step 15:

[0101] The server stores the feedback data received from the user in a database.

[0102] Step 16:

[0103] The server uses the new feedback data to retrain the artificial intelligence, helping to improve the accuracy of the diagnostic model.

[0104] The above is a specific processing flow in this system.

[0105] Example 1

[0106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0107] Currently, observing a child's symptoms and making appropriate medical decisions is difficult, especially for parents with limited medical knowledge. In many cases, anxiety leads parents to visit medical institutions even when symptoms are minor, resulting in a waste of medical resources and an increased burden on parents. It is also difficult to properly record symptom information and provide it to medical institutions as needed, often resulting in a lack of consistency in the information. To solve these problems, a system is needed that allows parents to easily record and analyze their child's symptoms at home and receive appropriate advice. Secure handling of data is also required from the perspective of privacy protection.

[0108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0109] In this invention, the server includes: means for receiving symptom data of the child entered by the user; means for receiving symptom images of the child taken by the user; means for encrypting the received symptom data and image data; means for transmitting the encrypted data to the server via a communication network; means for decrypting the received data; means for executing an AI to analyze the decrypted data; means for estimating the disease name, the need for hospital visits, and the expected number of days until recovery based on the data analyzed by the AI; means for providing user-friendly advice along with the estimation results; means for transmitting the estimation results and advice to the user's terminal; means for receiving feedback data from the user; means for storing the received feedback data in a database and using it for retraining the AI; and means for notifying the user's terminal that the diagnosis results are ready. This allows users to easily record their child's symptoms from home, securely transmit the data, and receive prompt and appropriate advice. Furthermore, retraining the AI ​​using the feedback data continues to improve diagnostic accuracy.

[0110] "User" refers to an individual who creates an account to use the system and enters their child's symptom data and image data.

[0111] "Symptom data" is textual information about the child's symptoms entered by the user.

[0112] "Image data" refers to image files of children's symptoms or injuries that users take and upload to the system.

[0113] "Encryption" is the process of transforming data using cryptographic techniques to make it unreadable to third parties.

[0114] A "communications network" is a mesh-like connection system, such as the Internet, for transmitting and receiving data between terminals and servers.

[0115] A "server" is a central processing unit that receives and analyzes data and returns diagnostic results and advice to the user.

[0116] "Decryption" is the process of returning encrypted data to its original form.

[0117] "Artificial intelligence" refers to a computer system that uses machine learning algorithms to analyze received symptom and image data and generate a diagnosis and advice.

[0118] The "disease name" is the name that describes the child's condition, estimated by the artificial intelligence based on the analysis results.

[0119] "Necessity of hospital visit" is information indicating whether or not it is necessary to go to the hospital based on the analysis results.

[0120] "Expected days to recovery" is the estimated number of days it will take for the child to recover, based on the analysis results.

[0121] "Feedback data" refers to information such as the progress of symptoms and the results of hospital diagnoses that are entered by the user at a later date.

[0122] A "database" is an information accumulation device that stores received feedback data and other information for later analysis and relearning.

[0123] "Relearning" is the process by which artificial intelligence improves itself based on new data.

[0124] "Push notification" is a system that automatically sends information from a server to a user's device and notifies them.

[0125] This invention relates to a system that receives symptom data and photographed image data of a child entered by a user, and analyzes them using artificial intelligence to estimate the name of the disease, whether or not the child needs to visit a hospital, and the expected number of days until recovery. A specific embodiment of this system will be described below.

[0126] Overall system configuration

[0127] The system consists of a device owned by the user, a server for analyzing data, and a communications network for linking these. Users can use the system by downloading the application to their smartphone or tablet and creating an account.

[0128] User Actions

[0129] Users launch the app and, when using it for the first time, create an account by entering basic information such as their name, email address, and password. After creating an account, they enter their child's symptoms as text. They also take photos of the symptoms and injury and upload them as image data.

[0130] Users enter details of symptoms and the date and time of onset, and submit the data. For example, they enter specific information such as "My 3-year-old child is coughing."

[0131] Device behavior

[0132] The device encrypts the text data entered by the user and the captured image data using AES encryption technology, specifically the OpenSSL library.

[0133] The terminal sends the encrypted data to the server using SSL / TLS communication via the HTTPS protocol.

[0134] Server Operation

[0135] The server uses RSA encryption to decrypt the encrypted data sent from the terminal. This process is performed using the openssl library on the server side.

[0136] The server prepares the received symptom data and image data for analysis. As processing for analysis, natural language processing is performed on the text data, and resolution standardization and noise removal are performed on the image data.

[0137] AI using TensorFlow on the server analyzes the data and predicts the possible diagnosis, whether or not a hospital visit is necessary, and the expected number of days until recovery. For example, if the diagnosis is a "mild cold," the expected number of days until recovery is "3 to 5 days."

[0138] The server uses natural language generation (NLG) technology to generate reassuring advice for parents along with the prediction results, such as "Keep the child warm and give them plenty of fluids."

[0139] The server sends this information to the terminal using SSL / TLS communication.

[0140] User Notification

[0141] The device decodes the diagnostic results and advice received from the server and notifies the user when the diagnostic results are ready using a push notification function, specifically, Firebase Cloud Messaging (FCM) or Apple Push Notification Service (APNs).

[0142] Users can open the app to check the diagnosis results and take measures as necessary. They can also send feedback data by entering the progress of symptoms and the results of hospital diagnosis into the app.

[0143] Processing of feedback data

[0144] The server receives the feedback data sent by the user and stores it in a database such as MongoDB.

[0145] The server retrains the AI ​​model based on the received feedback data, thereby continuously improving the system's diagnostic accuracy.

[0146] Specific examples

[0147] Example 1: The common cold

[0148] A user launches the app and enters the text "My 3-year-old child is coughing." They also take a photo of their child coughing and upload it to the app.

[0149] The device encrypts this data using AES encryption technology and sends it to the server.

[0150] The server analyzes the received data, diagnoses it as a "mild cold," estimates the expected recovery time to be "3 to 5 days," and notifies the user via push notification. It also provides friendly advice to "keep warm and drink plenty of fluids."

[0151] The user later inputs into the system as feedback that the symptoms have improved.

[0152] Specific prompt examples:

[0153] "My 3-year-old child has a cough. I'm sending a photo. Please tell me your diagnosis and advice based on these symptoms."

[0154] Example 2: Minor injury

[0155] A user launches the app and enters the text "My child fell and scraped his knee," taking a photo of the injury and uploading it to the app.

[0156] The device encrypts this data using AES encryption technology and sends it to the server.

[0157] The server analyzes the received data, diagnoses it as a "minor abrasion," determines that no hospital visit is necessary, and predicts that it will take "about 5 days" to heal. It then generates advice to the user: "Disinfect the wound and apply gauze. If the wound becomes infected, we recommend that you go to the hospital." and sends it to the user.

[0158] The user receives the diagnosis, follows the advice, and enters the progress of the symptoms into the system for later feedback.

[0159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0160] Step 1:

[0161] A user downloads the app onto their smartphone or tablet, launches it, and creates an account. The user enters basic information such as name, email address, and password, and presses the Create Account button. The input data is sent to the account management server, and the user information is registered in the database.

[0162] Input: Basic information entered by the user (name, email address, password)

[0163] Output: Account creation completion notification

[0164] Step 2:

[0165] Users enter symptom information in text format on the app's main screen. They can also take photos of symptoms and injuries with a camera and upload them to the app as image data. Users can also enter the date and time of onset as additional information about their condition, and when they press the send button, the entered data is saved on their device.

[0166] Input: Text data (symptom information), image data (images of symptoms and injuries), date and time information

[0167] Output: Saved symptom data and image data

[0168] Step 3:

[0169] The terminal encrypts the text data entered by the user and the uploaded image data using AES encryption technology. The encryption process is performed using the OpenSSL library.

[0170] Input: Text data, image data

[0171] Output: Encrypted data

[0172] Step 4:

[0173] The device sends encrypted data to the server using SSL / TLS communication, ensuring a secure communication channel using the HTTPS protocol.

[0174] Input: Encrypted data

[0175] Output: Send data to the server

[0176] Step 5:

[0177] The server receives the encrypted data sent from the terminal, decrypts it using RSA encryption, and restores the original text and image data.

[0178] Input: Encrypted data

[0179] Output: Decoded data (text data, image data)

[0180] Step 6:

[0181] The server prepares the decoded data for analysis: text data is tokenized for natural language processing and stop words are removed, and image data is normalized for resolution and noise is removed.

[0182] Input: Decoded data (text data, image data)

[0183] Output: Pre-processed data

[0184] Step 7:

[0185] An artificial intelligence system using TensorFlow on the server analyzes the pre-treatment data and estimates the disease name, whether or not the patient needs to visit the hospital, and the expected number of days until recovery. The analysis results are saved in JSON format.

[0186] Input: Pre-processed data

[0187] Output: Estimated results (disease name, need for hospital visits, expected number of days until recovery)

[0188] Step 8:

[0189] Based on the inference results, the server uses natural language generation (NLG) technology to generate a diagnosis and advice to display to the user. For example, if the diagnosis is a "mild cold," the server generates the advice "Keep warm and drink plenty of fluids."

[0190] Input: Estimation result

[0191] Output: Diagnostic results and advice

[0192] Step 9:

[0193] The diagnostic results and advice generated by the server are sent to the terminal again using SSL / TLS communication.

[0194] Input: Diagnostic results and advice

[0195] Output: Sending data to the terminal

[0196] Step 10:

[0197] The device decodes the diagnostic results and advice received from the server and notifies the user via push notification using Firebase Cloud Messaging (FCM) or Apple Push Notification Service (APNs) when the diagnostic results are ready.

[0198] Input: Diagnostic results and advice (encrypted data)

[0199] Output: User notification

[0200] Step 11:

[0201] The user opens the app, checks the diagnosis results, and takes measures if necessary. Later, they enter the progress of symptoms and the diagnosis results from the hospital into the app as feedback and press the send button.

[0202] Input: Feedback data (progression of symptoms, hospital diagnosis results)

[0203] Output: Feedback data transmission

[0204] Step 12:

[0205] The server receives the feedback data sent by the user and stores it in a database such as MongoDB.

[0206] Input: Feedback data

[0207] Output: Save to database

[0208] Step 13:

[0209] The server retrains the AI ​​model based on newly received feedback data. To improve the diagnostic accuracy of the AI, the retraining process is periodically executed using a scheduler.

[0210] Input: Feedback data

[0211] Output: Retrained AI model

[0212] (Application example 1)

[0213] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0214] Conventional systems for diagnosing children's health simply analyze the symptom data and image data entered by the user and provide a diagnosis. This leaves parents with no way to quickly obtain appropriate medicines or childcare products, and the system lacks the ability to use feedback data to improve the accuracy of the artificial intelligence. Therefore, a system is needed that suggests appropriate countermeasures linked to the diagnosis results and allows users to purchase them smoothly.

[0215] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0216] In this invention, the server includes means for analyzing symptom data entered by the user, means for analyzing symptom images, means for estimating the name of the disease, the need for hospital visits, and the expected number of days until recovery based on the analysis results, means for recommending appropriate medicines and childcare products based on the estimation results, and communication means for allowing the user to purchase the recommended products. This allows appropriate countermeasure products to be suggested in conjunction with the diagnosis results, enabling the user to purchase the necessary products quickly and smoothly.

[0217] "Symptom data entered by the user" is information describing the child's health condition and symptoms in text format.

[0218] A "symptom image" is still image data taken by a user that shows the child's health condition or symptoms.

[0219] "Analytical artificial intelligence" is a software system that has the ability to estimate the name of the disease and the severity of the symptoms based on the received symptom data and image data.

[0220] The "means of estimation" is the process of calculating the name of the disease, the need for outpatient treatment, and the expected number of days until recovery based on data analyzed by artificial intelligence.

[0221] The "means for providing friendly advice" is a function that displays appropriate measures and advice along with the diagnostic results received by the user.

[0222] "Feedback data" refers to data such as follow-up information on the diagnosis results provided by the user, or actual diagnosis results from the hospital.

[0223] The "means used for relearning" refers to the process of improving the algorithm to improve the diagnostic accuracy of the artificial intelligence based on the received feedback data.

[0224] The "means for providing recommended products" is a function that presents appropriate medicines and childcare products to the user based on the diagnostic results.

[0225] "Communication means for purchase" is a function that links with online shopping sites and sales platforms so that users can directly purchase recommended products.

[0226] This invention is a system that receives symptom data and photographed image data entered by the user, analyzes them to estimate the name of the disease, whether or not hospital visits are necessary, and the expected number of days until recovery, and suggests appropriate countermeasure products in conjunction with the diagnosis results, allowing the user to purchase the products quickly and smoothly. Specific embodiments of this system are described below.

[0227] Overall system configuration

[0228] The system consists of a user's device, a server for analyzing data, and a communications network that connects these. Users can begin using the system by downloading the application onto their smartphone, tablet, or other device and creating an account.

[0229] User Actions

[0230] First, the user launches the app and enters their child's symptoms as text. They also take photos of the symptoms and injuries and upload them as image data. Next, they enter details of the symptoms and the date and time of onset, and then send the data.

[0231] Device behavior

[0232] The terminal encrypts the text data entered by the user and the captured image data, and transmits them securely to the server via a communication network.

[0233] Server Operation

[0234] The server receives and decrypts the data sent from the device. It then uses artificial intelligence to analyze the received symptom data and image data, predicting the likely diagnosis, whether or not a hospital visit is necessary, and the expected number of days until recovery. Based on the predictions, it then recommends appropriate medicines and childcare products and provides the user with a link to purchase them.

[0235] Push notifications and checkout

[0236] The server sends the diagnosis results along with recommended product information to the device, which then notifies the user via push notification. The user can then check the recommended products along with the diagnosis results and quickly complete the purchase process using the appropriate communication means.

[0237] Feedback Processing

[0238] Users input feedback data into the app, such as follow-up information on diagnostic results and actual diagnostic results from hospitals, and then submit the data. The server stores the received feedback data in a database and uses it to retrain the AI ​​and improve diagnostic accuracy.

[0239] Techniques used and examples

[0240] The server uses an artificial intelligence model using Python and Flask, and an image processing library using PIL. Data is sent using the HTTP request library, requests.

[0241] Specific examples

[0242] A user launches the app, enters the text "My 3-year-old child has a cough," and uploads an image of the child coughing. The encoded and encrypted data is sent to the server, where it is decoded and analyzed. The server diagnoses the child as having a "mild cold," predicts that the child will be cured in 3 to 5 days, and provides a link to recommended medication. An example prompt might be, "I took a photo of my child coughing. From this image and the following text data, please estimate the name of the illness, whether or not the child needs to visit a doctor, and the expected number of days until recovery: My 3-year-old child has had a cough for several days and has a red throat. He does not have a fever."

[0243] This will enable parents to quickly obtain appropriate advice regarding their child's symptoms and necessary medications.

[0244] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0245] Step 1:

[0246] The user launches the app and logs in. The user enters their child's symptoms in text, describing specific symptoms such as "My 3-year-old child has a cough." They also take an image of the symptoms and upload it to the app. The input data is both text and image data.

[0247] Step 2:

[0248] The terminal receives text data entered by the user and captured image data. It then encrypts the data and transmits it to the server via a communication network. The input data is the encrypted text and image data, and the output data is the encrypted data transmitted to the server.

[0249] Step 3:

[0250] The server receives the encrypted data sent from the terminal and first decrypts it. The decrypted data is then prepared for analysis. The input data is the encrypted symptom data and image data, and the output data is the decrypted data.

[0251] Step 4:

[0252] The server uses an artificial intelligence model to analyze the received symptom data and image data. A generative AI model is used for the analysis, and the name of the disease, the need for medical visits, and the expected number of days until recovery are estimated from the data. An example of a prompt might be, "I took a picture of a child coughing. From this image and the following text data, please estimate the name of the disease, whether medical visits are necessary, and the expected number of days until recovery: A 3-year-old child has been coughing for several days and has a red throat. He does not have a fever." The input data is the decoded symptom data and image data, and the output data is the estimated diagnosis result.

[0253] Step 5:

[0254] The server generates information on recommended medicines and childcare products based on the diagnosis results. The recommended product information includes product descriptions and purchase links. The input data is the estimated diagnosis results, and the output data is the recommended product information and purchase links.

[0255] Step 6:

[0256] The server generates diagnostic results and recommended product information and sends them to the device. At the same time, the push notification function is used to notify the user that the diagnostic results are ready. The input data is the diagnostic results and recommended product information generated by the server, and the output data is the data sent to the device.

[0257] Step 7:

[0258] The terminal displays the diagnosis results and recommended product information received from the server to the user. The user checks the diagnosis results and takes necessary measures. If the user wishes to purchase a recommended product, he or she accesses the purchase link directly through the terminal. The input data is the diagnosis results and recommended product information received from the server, and the output data is the data displayed to the user.

[0259] Step 8:

[0260] The user then inputs the progress of symptoms and the results of the hospital diagnosis into the app as feedback and sends it. The input data is the progress of symptoms and the actual diagnosis results, and the output data is the feedback data entered into the app.

[0261] Step 9:

[0262] The server receives the feedback data sent by the user and stores it in a database. The data is used for retraining to improve the diagnostic accuracy of the AI ​​model. The input data is the feedback data, and the output data is the improved diagnostic model.

[0263] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0264] This invention relates to a system that receives symptom data and photographed image data entered by a user and uses artificial intelligence to analyze the data to estimate the name of the child's illness, whether or not hospital visits are necessary, and the expected number of days until recovery. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system has the function of providing more personalized and considerate advice. A specific embodiment of this system is described below.

[0265] Overall system configuration

[0266] The system consists of a device held by the user, a server for analyzing data, and a communication network for linking these components. The emotion engine also analyzes emotions from the user's text and voice data and reflects them in the diagnosis results.

[0267] 1. User Actions

[0268] The user launches the app and logs in or creates an account.

[0269] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[0270] Users can add text descriptions or recorded audio.

[0271] Check the various data entered by the user and tap the "Send" button in the app.

[0272] 2. Device Operation

[0273] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using a common key encryption method such as AES.

[0274] The terminal transmits the encrypted data to the server via the communication network.

[0275] 3. Server Operation

[0276] The server decrypts the encrypted data it receives and prepares it for analysis.

[0277] Before the server analyzes the data, it runs an emotion engine to recognize the user's emotions.

[0278] + Natural language processing (NLP) of text data extracts emotional keywords using an emotion engine.

[0279] + Voice data is used by a voice recognition and emotion analysis engine to assess the user's emotional state.

[0280] The server preprocesses the received text and image data and converts them into a format for analysis, adjusting image resolution and removing noise.

[0281] An artificial intelligence (AI) model on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is required, and the expected number of days until recovery.

[0282] The server generates reassuring advice in kind words for the user based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is optimized according to the user's emotional state.

[0283] 4. Notice to Users

[0284] The device displays the diagnosis results and advice received from the server to the user, and notifies the user using the push notification function when the diagnosis results are ready.

[0285] The user can check the diagnosis results and take measures as necessary. They can also enter the progress of symptoms and the results of the diagnosis at the hospital into the app at a later date and send it as feedback data.

[0286] 5. Processing of Feedback Data

[0287] The server stores the feedback data received from the user in a database.

[0288] The server retrains the artificial intelligence based on the feedback data received, thereby improving the system's diagnostic accuracy.

[0289] Specific examples

[0290] Example 1: The common cold

[0291] A user launches the app and enters text that their 3-year-old child is coughing. They also take a photo of the child coughing and upload it to the app.

[0292] Along with this data, the device records the user's anxiety as audio, encrypts it, and sends it to the server.

[0293] The server analyzes the received data and determines that the illness is likely a mild cold. It also predicts that the time until recovery will be between 3 and 5 days, and notifies the user via push notification.

[0294] Along with the diagnosis, the user receives gentle advice such as, "Keep warm and give plenty of fluids. If symptoms persist for more than a few days, we recommend seeing a doctor." The advice is optimized to take into account the user's emotional state.

[0295] The user later inputs into the system as feedback that the symptoms have improved.

[0296] Example 2: Minor injury

[0297] A user launches the app and enters text that their child fell and scraped their knee, taking a photo of the injury and uploading it to the app.

[0298] The device encrypts this data along with text expressing the user's concerns and sends it to the server.

[0299] The server analyzes the received data and determines that the injury is a minor scratch. No further hospital visits are required, and the estimated time until recovery is about five days.

[0300] Along with the diagnosis, the user receives gentle advice: "Disinfect and apply gauze. If the wound becomes infected, we recommend going to the hospital." The emotion engine also provides additional advice to ease the worry.

[0301] The above is an embodiment of the present invention.

[0302] The processing flow will be explained below.

[0303] Step 1:

[0304] The user launches the app and logs in or creates an account.

[0305] Step 2:

[0306] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[0307] Step 3:

[0308] Users can add text descriptions or recorded audio.

[0309] Step 4:

[0310] Check the various data entered by the user and tap the "Send" button in the app.

[0311] Step 5:

[0312] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using a common key encryption method such as AES.

[0313] Step 6:

[0314] The terminal transmits the encrypted data to the server via the communication network.

[0315] Step 7:

[0316] The server decrypts the received encrypted data and prepares it for analysis. The data is temporarily stored in secure storage.

[0317] Step 8:

[0318] The server runs an emotion engine to analyze the received text and voice data of the user. For the text data, a natural language processing (NLP) engine is used to extract emotion keywords. For the voice data, a speech recognition and emotion analysis engine is used to evaluate the user's emotional state.

[0319] Step 9:

[0320] The server preprocesses the received image data and converts it into a format for analysis, adjusting the image resolution and removing noise.

[0321] Step 10:

[0322] The server inputs the preprocessed text and image data into an artificial intelligence (AI) model to perform symptom analysis.

[0323] Step 11:

[0324] The server's AI model analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[0325] Step 12:

[0326] The server generates reassuring advice for users based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is personalized according to the user's emotional state.

[0327] Step 13:

[0328] The server prepares to send generated diagnostic results and advice to the device.

[0329] Step 14:

[0330] The device displays the diagnostic results and advice received from the server to the user, and uses the push notification function to notify the user that the diagnostic results are ready.

[0331] Step 15:

[0332] The user can check the diagnosis results and take measures as necessary. At a later date, the user can also enter the progress of symptoms and the results of the diagnosis at the hospital into the app and send it as feedback data.

[0333] Step 16:

[0334] The server stores the feedback data received from the user in a database.

[0335] Step 17:

[0336] The server uses the feedback data to retrain the artificial intelligence, helping to improve the accuracy of the diagnostic model.

[0337] The above is the specific processing flow of this system.

[0338] Example 2

[0339] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0340] Conventional disease diagnosis systems only use symptom data and image data entered by users, and lack emotional state and detailed explanations, making diagnosis results and advice general and making it difficult to provide personalized responses to individual users. Furthermore, the security of transmitted data and appropriate preprocessing of received data are insufficient, making it difficult to provide reliable diagnosis results.

[0341] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving symptom data of the child input by the user, means for receiving symptom images of the child taken by the user, means for receiving voice data input by the user, means for encrypting the received symptom data, image data, and voice data, means for transmitting the encrypted data to the server, means for decrypting the received data, means for preprocessing the decrypted data, means for preparing for data analysis, means for executing an emotion analysis engine, means for executing artificial intelligence using the preprocessed data as input, means for estimating the name of the disease, the need for hospital visits, and the expected number of days until recovery based on the data analyzed by the artificial intelligence, means for generating the estimation results and optimized advice according to the user's emotional state, means for transmitting the generated advice from the server to the terminal, means for the terminal to notify the user of the advice, means for receiving feedback data from the user, and means for storing the received feedback data in a database and using it for relearning the artificial intelligence. This makes it possible to provide personalized diagnosis results and advice that take the user's emotional state into consideration.

[0342] "User" refers to the person who uses this system and inputs the child's symptom data, image data, and voice data.

[0343] "Child symptom data" is text-format information entered by the user, and is data describing the child's physical condition and state.

[0344] "Symptom images" refer to images of a child's symptoms or injuries taken by the user, providing a visual record of the symptoms.

[0345] "Audio data" is data in the form of audio recorded by the user, and is a recording of the user's emotions and symptoms in detail as audio.

[0346] "Encryption" is a process performed to securely transmit received data, and is an operation to convert the data into a format that cannot be read by a third party.

[0347] "Decryption" is the process of returning encrypted data to its original form, making the data ready for analysis by the receiving party.

[0348] "Preprocessing" refers to processing performed to convert received data into a format suitable for analysis, and includes tokenizing text data and adjusting the resolution of image data.

[0349] "Data analysis" is the process of interpreting information based on received data, and involves using an artificial intelligence model to estimate the name of the disease, whether or not outpatient treatment is required, and the expected number of days until recovery.

[0350] An "emotion analysis engine" is software that recognizes and analyzes emotions from text data and voice data entered by the user.

[0351] Artificial intelligence (AI) is a computer program that analyzes received data and estimates the name of the disease, whether or not outpatient treatment is required, and the expected number of days until recovery.

[0352] "Advice" refers to advice or suggestions provided to users based on the results of analysis by artificial intelligence, and is information optimized according to the user's emotional state.

[0353] "Feedback data" refers to data that includes subsequent reports and progress information from users, and is information that the system receives and uses for relearning the artificial intelligence.

[0354] "Database" is an information management system for storing received feedback data, which is used for re-learning and data analysis.

[0355] "Relearning" is the process of updating an artificial intelligence model based on newly received data to improve diagnostic accuracy.

[0356] System Overview

[0357] This invention is a system that receives symptom data and photographed image data entered by the user and uses artificial intelligence to analyze the data to estimate the name of the illness, whether or not the child needs to visit a doctor, and the estimated number of days until recovery. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it has the function of providing more personalized and friendly advice. The overall system configuration consists of a device held by the user, a server for analyzing the data, and a communication network for linking these.

[0358] System configuration and operation

[0359] User Actions

[0360] The user launches the app and logs in or creates an account.

[0361] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[0362] Users can add text descriptions or recorded audio.

[0363] Check the various data entered by the user and tap the "Send" button in the app.

[0364] Device behavior

[0365] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using AES (a common key encryption method).

[0366] The terminal transmits the encrypted data to the server via the communication network.

[0367] Server Operation

[0368] The server decrypts the encrypted data it receives and prepares it for analysis.

[0369] The server runs an emotion engine to recognize the user's emotions.

[0370] The server extracts emotion keywords from the text data using natural language processing (NLP).

[0371] The server analyzes the voice data using voice recognition and emotion analysis to assess the user's emotional state.

[0372] The server preprocesses the received text and image data and converts them into a format for analysis, adjusting image resolution and removing noise.

[0373] An artificial intelligence (AI) model on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is required, and the expected number of days until recovery.

[0374] The server generates reassuring advice in kind words for the user based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is optimized according to the user's emotional state.

[0375] Specific examples

[0376] Example 1: A case of the common cold

[0377] A user launches the app, enters text that their 3-year-old child is coughing, takes a photo of the child coughing, and uploads it to the app.

[0378] The device encrypts text data, image data, and recorded audio data using AES and sends them to the server.

[0379] The server decrypts the received data and prepares it for analysis.

[0380] The server's emotion engine detects anxiety from the user's voice data.

[0381] The server uses an AI model to determine that the condition is likely a "mild cold" and predicts that the time until recovery will be "3 to 5 days."

[0382] The server generates gentle advice such as, "Keep warm and give yourself plenty of fluids. If symptoms persist for more than a few days, we recommend that you see a doctor," and sends it to the device.

[0383] The user checks the diagnostic results and advice and takes appropriate measures.

[0384] Example 2: Minor injury case

[0385] A user launches the app, enters in text that their child fell and scraped their knee, takes a picture of the injury, and uploads it to the app.

[0386] The terminal encrypts the text data, image data, and voice data expressing anxiety and transmits them to the server.

[0387] The server decrypts the received data and prepares it for analysis.

[0388] The server's emotion engine detects worry from the user's voice data.

[0389] The server uses an AI model to determine that the injury is a "minor abrasion," and predicts that no hospital visit is necessary and that the estimated time until recovery is "about 5 days."

[0390] The server generates gentle advice such as "Disinfect the wound and apply gauze. If the wound becomes infected, we recommend that you go to the hospital," and sends it to the device.

[0391] The user checks the diagnostic results and advice and takes appropriate measures.

[0392] Prompt Sentence Examples

[0393] My child has been coughing since this morning. Should I take him to the hospital? I've attached a photo.

[0394] My 3 year old fell and scraped his knee. Should I disinfect it?

[0395] In this way, the system is designed to effectively link the functions of the user, terminal, and server, and to ensure that each process is executed efficiently, thereby providing users with highly reliable diagnostic results and appropriate advice.

[0396] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0397] Step 1: User Enters Data

[0398] The user launches the app and logs in or creates an account.

[0399] The user enters the child's symptom data (e.g., fever, cough, etc.) in text format.

[0400] Users use the app's camera function to take pictures of their child's symptoms or injuries.

[0401] If necessary, the user can record a detailed description of their symptoms and their feelings as audio data.

[0402] Check the various data entered by the user and tap the "Send" button in the app.

[0403] Input: Symptom data, image data, audio data.

[0404] Output: User data stored within the app.

[0405] Step 2: The device encrypts and transmits the data

[0406] The device detects that the user has tapped the "Send" button.

[0407] The terminal encrypts text data, image data, and audio data using the AES encryption method.

[0408] The terminal transmits the encrypted data to the server via the communication network.

[0409] Input: text data, image data, audio data.

[0410] Output: The encrypted data.

[0411] Step 3: The server receives and decrypts the data

[0412] The server receives the encrypted data sent from the terminal.

[0413] The server decrypts the received data and prepares it for analysis.

[0414] Input: Encrypted data.

[0415] Output: The decrypted data.

[0416] Step 4: The server runs the sentiment analysis engine

[0417] The server uses NLP to extract emotional keywords from the text data.

[0418] The server converts the voice data into text using voice recognition technology and performs emotion analysis.

[0419] Input: Decoded text and audio data.

[0420] Output: Emotion analysis results.

[0421] Step 5: The server performs data preprocessing

[0422] The server tokenizes the received text data and standardizes the format.

[0423] The server adjusts the resolution of the image data and removes noise.

[0424] Input: Decoded text and image data.

[0425] Output: Preprocessed text and image data.

[0426] Step 6: The server analyzes the data with the AI ​​model

[0427] The server inputs the pre-processed data into an artificial intelligence (AI) model.

[0428] The server's AI model analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[0429] Input: Preprocessed data.

[0430] Output: Disease name, whether or not outpatient treatment is required, and estimated number of days until recovery.

[0431] Step 7: Server generates diagnostics and advice

[0432] The server generates friendly advice for the user based on the results of the AI ​​model's analysis and emotion analysis.

[0433] The server generates advice that is optimized according to the user's emotional state.

[0434] Input: disease name, whether or not hospital visits are required, expected number of days until recovery, and emotion analysis results.

[0435] Output: Optimized advice.

[0436] Step 8: The server sends the results to the device

[0437] The server transmits the generated diagnostic results and advice to the terminal.

[0438] The terminal notifies the user of the result received from the server.

[0439] Input: Diagnostic results and advice.

[0440] Output: Notification to the user.

[0441] Step 9: User confirms diagnosis results and advice

[0442] The user checks the diagnosis results and advice within the app.

[0443] The user will take appropriate measures as necessary.

[0444] Input: Diagnostic results and advice.

[0445] Output: User response and actions.

[0446] Step 10: User submits feedback

[0447] The user will then enter progress reports and hospital diagnosis results into the app at a later date.

[0448] The data entered by the user is sent as feedback data.

[0449] Input: Progress reports and diagnostic results.

[0450] Output: Feedback data.

[0451] Step 11: Server stores feedback data and retrains

[0452] The server stores the feedback data received from the user in a database.

[0453] The server retrains the artificial intelligence model based on the feedback data received, improving diagnostic accuracy.

[0454] Input: Feedback data.

[0455] Output: A retrained artificial intelligence model.

[0456] (Application example 2)

[0457] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0458] Previous systems had the problem of not being able to fully consider the anxiety and worries of parents when assessing their child's condition. Furthermore, parents had to take the time and effort to check on their child's safety one by one, placing a heavy burden on them. Furthermore, the lack of emotion recognition capabilities made it difficult to provide personalized advice to parents.

[0459] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving user location information and tracking it in real time, means for receiving parental emotion data in text and voice, and means for analyzing the received emotion data with an emotion engine. This allows parents to easily check the safety of their children and provides personalized, kind advice that takes the parent's emotions into consideration.

[0460] The "means for receiving the child's symptom data entered by the user" refers to a method and device for the terminal or server to receive text data relating to the child's symptoms that the user has entered into the application.

[0461] The "means for receiving an image of a child's symptoms taken by a user" refers to a method and device for a terminal or a server to receive image data relating to a child's symptoms taken by a user.

[0462] The "means for executing artificial intelligence" refers to an artificial intelligence algorithm used to analyze the received symptom data and image data, and a computer device for executing the algorithm.

[0463] "Means for estimating the name of the disease, whether or not outpatient treatment is required, and the expected number of days until complete recovery" refers to a method and device for identifying the name of the disease based on data analyzed by artificial intelligence, and predicting the need for outpatient treatment and the expected number of days until complete recovery.

[0464] The "means for providing user-friendly advice" refers to a method and device for notifying the user of advice generated based on the estimation results in easy-to-understand and friendly language.

[0465] "Means for receiving user location information and tracking it in real time" refers to a method and device for obtaining the user's current location using technology such as GPS and tracking that information in real time.

[0466] The "means for receiving parental emotion data in text and voice" refers to a method and device for a terminal or a server to receive emotion-related data input by a parent in text or voice.

[0467] "Means for analyzing received emotion data with an emotion engine" refers to software and algorithms for analyzing received text and voice data and assessing the parent's emotional state.

[0468] The "means for notifying the user of the generated advice" refers to a method and device for providing the user with advice generated based on the analysis results using push notification or the like.

[0469] "Means for receiving feedback data" refers to methods and apparatus for receiving feedback from a user.

[0470] "Means for storing feedback data in a database and using it for retraining artificial intelligence" refers to a method and apparatus for storing received feedback data in a database and retraining the artificial intelligence model based on that data to improve it.

[0471] This invention relates to a system that receives symptom data, photographed image data, and location information entered by parents, and analyzes them using artificial intelligence to estimate the name of the illness, whether hospital visits are necessary, and the expected number of days until recovery. Furthermore, by combining it with an emotion engine that recognizes the emotions of parents, it has the function of providing more personalized and considerate advice. A specific embodiment of this system is described below.

[0472] Overall system configuration

[0473] The system consists of a device held by the parent, a server for analyzing data, and a communication network for linking these. The emotion engine also analyzes emotions from the parent's text and voice data and reflects them in the diagnosis results.

[0474] 1. Parent behavior

[0475] Parents launch the app and log in or create an account.

[0476] Parents enter their child's symptoms as text and take an image of the symptoms using the in-app camera function.

[0477] Parents can add text instructions or audio recordings.

[0478] Check the various data entered by the parent and tap the "Send" button in the app.

[0479] 2. Device Operation

[0480] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using a common key encryption method such as AES.

[0481] The terminal transmits the encrypted data to the server via the communication network.

[0482] 3. Server Operation

[0483] The server decrypts the encrypted data it receives and prepares it for analysis.

[0484] Before the server analyzes the data, it runs an emotion engine to recognize the parent's emotions.

[0485] Natural language processing (NLP) of text data uses an emotion engine to extract emotion keywords. Libraries such as TextBlob are used.

[0486] The voice data is then passed through a speech recognition and emotion analysis engine to assess the parent's emotional state, using the Google Cloud Speech-to-Text API.

[0487] The server preprocesses the received text and image data and converts them into a format for analysis, adjusting image resolution and removing noise.

[0488] An artificial intelligence (AI) model on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is required, and the expected number of days until recovery.

[0489] The server generates reassuring advice to parents in gentle language based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is optimized according to the parent's emotional state.

[0490] 4. Parental Notification

[0491] The device displays the diagnosis results and advice received from the server to the parent, and notifies the parent using the push notification function when the diagnosis results are ready.

[0492] Parents can check the diagnosis results and take measures as necessary. They can also enter the progress of symptoms and the results of medical examinations at medical institutions into the app at a later date and send it as feedback data.

[0493] 5. Processing of Feedback Data

[0494] The server stores the feedback data received from the parent in a database.

[0495] The server retrains the artificial intelligence based on the feedback data received, thereby improving the system's diagnostic accuracy.

[0496] Specific examples

[0497] Example 1: A mild cold

[0498] A parent can enter text that their 3-year-old child is coughing and take a photo of the coughing and upload it. The device then records this data, along with the parent's anxiety, as audio, which is then encrypted and sent to a server. The server analyzes this data, determines that the child has a mild cold, and estimates the expected recovery time at 3-5 days. The parent is advised to keep the child warm and provide plenty of fluids, and is advised to seek medical advice if symptoms persist for more than a few days.

[0499] Prompt Sentence Examples

[0500] When a parent launches a child safety check app and wants to check their child's location, a location request is sent to the server. Along with the current location, the parent's emotional state is also analyzed. To ease the parent's anxiety, the app provides gentle advice such as, "Your child is safe. There's nothing to worry about."

[0501] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0502] Step 1:

[0503] The parent launches the app and logs in or creates an account. The input for this step is the parent's credentials (username, password), and the output is a notification of successful or failed login.

[0504] Step 2:

[0505] Parents enter their child's symptoms as text and take images of the symptoms using the in-app camera function. The input for this step is text data and image data, and the output is these data stored in the app.

[0506] Step 3:

[0507] The parent adds textual instructions and recorded audio data. The input of this step is textual data and audio data, and the output is these data stored on the device.

[0508] Step 4:

[0509] The parent confirms the various data entered (text data, image data, audio data) and taps the "Send" button in the app. The input of this step is the confirmed data, and the output is the trigger that causes the device to start sending data.

[0510] Step 5:

[0511] The terminal detects that the send button has been pressed and encrypts the text data, image data, and audio data. This uses a common key encryption method such as AES. The input is unencrypted data, and the output is encrypted data.

[0512] Step 6:

[0513] The terminal transmits the encrypted data to the server via the communication network. The input of this step is the encrypted data, and the output is a transmission confirmation to the server.

[0514] Step 7:

[0515] The server decrypts the received encrypted data and prepares it for analysis. The input is the encrypted data and the output is the decrypted data.

[0516] Step 8:

[0517] Before the server analyzes the received data, it runs an emotion engine to recognize the parent's emotion. The input of this step is text data and audio data, and the output is the analyzed emotion data.

[0518] The server performs natural language processing (NLP) on the text data to extract emotional keywords. The input is the text data, and the output is the extracted emotional keywords.

[0519] The server sends the voice data to an engine that performs speech recognition and emotion analysis to evaluate the parent's emotional state. The input is the voice data, and the output is the emotion evaluation result.

[0520] Step 9:

[0521] The server preprocesses the text and image data it receives and converts it into a format for analysis. The input is text and image data, and the output is the data converted into the format for analysis. During this process, image resolution is adjusted and noise is removed.

[0522] Step 10:

[0523] An artificial intelligence (AI) model on the server analyzes the data and predicts the likely diagnosis, whether or not the patient will need to visit the hospital, and the expected number of days until recovery. The input is preprocessed data, and the output is the predicted result.

[0524] Step 11:

[0525] The server generates reassuring advice in parent-friendly language based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The input is the estimation results and emotion evaluation data, and the output is the generated advice.

[0526] Step 12:

[0527] The device displays the diagnosis results and advice received from the server to the parent. The input is the data received from the server, and the output is a notification to the parent. The push notification function is used to notify the parent that the diagnosis results are ready.

[0528] Step 13:

[0529] The parent checks the diagnosis results and takes measures as necessary. The parent also inputs the progress of symptoms and the results of the diagnosis at the medical institution into the app at a later date and sends it as feedback data. The input is the feedback data for the later date, and the output is a notification that the feedback has been sent.

[0530] Step 14:

[0531] The server saves the feedback data received from the parent to a database. The input is the received feedback data, and the output is confirmation of the save completion.

[0532] Step 15:

[0533] The server retrains the artificial intelligence based on the feedback data it receives. The input is the feedback data stored in the database, and the output is the retrained AI model. This improves the system's diagnostic accuracy.

[0534] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0535] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0536] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0537] [Second embodiment]

[0538] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0539] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0540] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0541] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0542] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0543] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0544] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0545] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0546] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0547] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0548] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0549] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0550] This invention relates to a system that receives symptom data and photographed image data of a child entered by a user, and analyzes them using artificial intelligence to estimate the name of the disease, whether or not the child needs to visit a hospital, and the expected number of days until recovery. A specific embodiment of this system will be described below.

[0551] Overall system configuration

[0552] The system consists of a device owned by the user, a server for analyzing data, and a communication network for linking these. Users can begin using the system by downloading the application and creating an account.

[0553] 1. User Actions

[0554] Users launch the app and enter their child's symptoms as text. They also take photos of the symptoms and injuries and upload them as image data.

[0555] The user enters details of the symptoms and the date and time of onset and submits the data.

[0556] 2. Device Operation

[0557] The terminal encrypts the text data entered by the user and the captured image data.

[0558] The terminal transmits the encrypted data to the server via a communication network.

[0559] 3. Server Operation

[0560] The server receives the data sent from the terminal, decodes the data, and then prepares the received symptom data and image data for analysis.

[0561] Artificial intelligence on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[0562] The server generates gentle advice to reassure parents along with the inference results.

[0563] The server transmits this information to the terminal.

[0564] 4. Notice to Users

[0565] The device displays the diagnostic results and advice received from the server to the user, and uses the push notification function to notify the user when the diagnostic results are ready.

[0566] Users can check the diagnosis results and take necessary measures. They can also send feedback data by entering the progress of their symptoms and the results of their hospital diagnosis into the app.

[0567] 5. Processing of Feedback Data

[0568] The server receives the feedback data sent by the user and stores it in a database.

[0569] The server retrains the artificial intelligence based on the received feedback data, thereby improving the system's diagnostic accuracy.

[0570] Specific examples

[0571] Example 1: The common cold

[0572] A user launches the app and enters text that their 3-year-old child is coughing. They also take a photo of the child coughing and upload it to the app.

[0573] The device encrypts this data and sends it to the server.

[0574] The server analyzes the received data and determines that the illness is likely a mild cold. It also predicts that the time until recovery will be between 3 and 5 days, and notifies the user via push notification.

[0575] Along with the diagnosis, the user receives gentle advice: "Keep warm and give your child plenty of fluids. If symptoms persist for more than a few days, we recommend that you see a doctor."

[0576] The user later inputs into the system as feedback that the symptoms have improved.

[0577] Example 2: Minor injury

[0578] A user launches the app and enters text that their child fell and scraped their knee, taking a photo of the injury and uploading it to the app.

[0579] The device encrypts this data and sends it to the server.

[0580] The server analyzes the received data and determines that the injury is a minor scratch. No further hospital visits are required, and the estimated time until recovery is about five days.

[0581] Along with the diagnosis, the user receives gentle advice: "Disinfect the wound and apply gauze. If the wound becomes infected, we recommend that you go to the hospital."

[0582] The above is an embodiment of the present invention.

[0583] The processing flow will be explained below.

[0584] Step 1:

[0585] The user launches the app and logs in or creates an account.

[0586] Step 2:

[0587] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[0588] Step 3:

[0589] The user checks the text data entered and the captured image data within the app and taps the "Send" button.

[0590] Step 4:

[0591] The device detects that the send button has been pressed and encrypts the text and image data using a common key encryption method such as AES.

[0592] Step 5:

[0593] The terminal transmits the encrypted data to the server via the communication network.

[0594] Step 6:

[0595] The server decrypts the received encrypted data and prepares it for analysis. The data is temporarily stored in secure storage.

[0596] Step 7:

[0597] The text data received by the server is input into a natural language processing (NLP) engine to extract keywords related to the symptoms.

[0598] Step 8:

[0599] The server preprocesses the received image data and converts it into a format for analysis, adjusting the image resolution and removing noise.

[0600] Step 9:

[0601] The server inputs the preprocessed text and image data into an artificial intelligence (AI) model to perform symptom analysis.

[0602] Step 10:

[0603] The server's AI model analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[0604] Step 11:

[0605] Based on the estimation results of the AI ​​model, the server generates reassuring advice for the user in friendly language.

[0606] Step 12:

[0607] The server composes the generated diagnostics and advice and prepares them for transmission to the device.

[0608] Step 13:

[0609] The device displays the diagnostic results and advice received from the server to the user. Notifications can also be sent using the push notification function.

[0610] Step 14:

[0611] The user can check the diagnosis results and take measures as necessary. They can also enter the progress of symptoms and the results of the diagnosis at the hospital into the app at a later date and send it as feedback data.

[0612] Step 15:

[0613] The server stores the feedback data received from the user in a database.

[0614] Step 16:

[0615] The server uses the new feedback data to retrain the artificial intelligence, helping to improve the accuracy of the diagnostic model.

[0616] The above is a specific processing flow in this system.

[0617] Example 1

[0618] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0619] Currently, observing a child's symptoms and making appropriate medical decisions is difficult, especially for parents with limited medical knowledge. In many cases, anxiety leads parents to visit medical institutions even when symptoms are minor, resulting in a waste of medical resources and an increased burden on parents. It is also difficult to properly record symptom information and provide it to medical institutions as needed, often resulting in a lack of consistency in the information. To solve these problems, a system is needed that allows parents to easily record and analyze their child's symptoms at home and receive appropriate advice. Secure handling of data is also required from the perspective of privacy protection.

[0620] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0621] In this invention, the server includes: means for receiving symptom data of the child entered by the user; means for receiving symptom images of the child taken by the user; means for encrypting the received symptom data and image data; means for transmitting the encrypted data to the server via a communication network; means for decrypting the received data; means for executing an AI to analyze the decrypted data; means for estimating the disease name, the need for hospital visits, and the expected number of days until recovery based on the data analyzed by the AI; means for providing user-friendly advice along with the estimation results; means for transmitting the estimation results and advice to the user's terminal; means for receiving feedback data from the user; means for storing the received feedback data in a database and using it for retraining the AI; and means for notifying the user's terminal that the diagnosis results are ready. This allows users to easily record their child's symptoms from home, securely transmit the data, and receive prompt and appropriate advice. Furthermore, retraining the AI ​​using the feedback data continues to improve diagnostic accuracy.

[0622] "User" refers to an individual who creates an account to use the system and enters their child's symptom data and image data.

[0623] "Symptom data" is textual information about the child's symptoms entered by the user.

[0624] "Image data" refers to image files of children's symptoms or injuries that users take and upload to the system.

[0625] "Encryption" is the process of transforming data using cryptographic techniques to make it unreadable to third parties.

[0626] A "communications network" is a mesh-like connection system, such as the Internet, for transmitting and receiving data between terminals and servers.

[0627] A "server" is a central processing unit that receives and analyzes data and returns diagnostic results and advice to the user.

[0628] "Decryption" is the process of returning encrypted data to its original form.

[0629] "Artificial intelligence" refers to a computer system that uses machine learning algorithms to analyze received symptom and image data and generate a diagnosis and advice.

[0630] The "disease name" is the name that describes the child's condition, estimated by the artificial intelligence based on the analysis results.

[0631] "Necessity of hospital visit" is information indicating whether or not it is necessary to go to the hospital based on the analysis results.

[0632] "Expected days to recovery" is the estimated number of days it will take for the child to recover, based on the analysis results.

[0633] "Feedback data" refers to information such as the progress of symptoms and the results of hospital diagnoses that are entered by the user at a later date.

[0634] A "database" is an information accumulation device that stores received feedback data and other information for later analysis and relearning.

[0635] "Relearning" is the process by which artificial intelligence improves itself based on new data.

[0636] "Push notification" is a system that automatically sends information from a server to a user's device and notifies them.

[0637] This invention relates to a system that receives symptom data and photographed image data of a child entered by a user, and analyzes them using artificial intelligence to estimate the name of the disease, whether or not the child needs to visit a hospital, and the expected number of days until recovery. A specific embodiment of this system will be described below.

[0638] Overall system configuration

[0639] The system consists of a device owned by the user, a server for analyzing data, and a communications network for linking these. Users can use the system by downloading the application to their smartphone or tablet and creating an account.

[0640] User Actions

[0641] Users launch the app and, when using it for the first time, create an account by entering basic information such as their name, email address, and password. After creating an account, they enter their child's symptoms as text. They also take photos of the symptoms and injury and upload them as image data.

[0642] Users enter details of symptoms and the date and time of onset, and submit the data. For example, they enter specific information such as "My 3-year-old child is coughing."

[0643] Device behavior

[0644] The device encrypts the text data entered by the user and the captured image data using AES encryption technology, specifically the OpenSSL library.

[0645] The terminal sends the encrypted data to the server using SSL / TLS communication via the HTTPS protocol.

[0646] Server Operation

[0647] The server uses RSA encryption to decrypt the encrypted data sent from the terminal. This process is performed using the openssl library on the server side.

[0648] The server prepares the received symptom data and image data for analysis. As processing for analysis, natural language processing is performed on the text data, and resolution standardization and noise removal are performed on the image data.

[0649] AI using TensorFlow on the server analyzes the data and predicts the possible diagnosis, whether or not a hospital visit is necessary, and the expected number of days until recovery. For example, if the diagnosis is a "mild cold," the expected number of days until recovery is "3 to 5 days."

[0650] The server uses natural language generation (NLG) technology to generate reassuring advice for parents along with the prediction results, such as "Keep the child warm and give them plenty of fluids."

[0651] The server sends this information to the terminal using SSL / TLS communication.

[0652] User Notification

[0653] The device decodes the diagnostic results and advice received from the server and notifies the user when the diagnostic results are ready using a push notification function, specifically, Firebase Cloud Messaging (FCM) or Apple Push Notification Service (APNs).

[0654] Users can open the app to check the diagnosis results and take measures as necessary. They can also send feedback data by entering the progress of symptoms and the results of hospital diagnosis into the app.

[0655] Processing of feedback data

[0656] The server receives the feedback data sent by the user and stores it in a database such as MongoDB.

[0657] The server retrains the AI ​​model based on the received feedback data, thereby continuously improving the system's diagnostic accuracy.

[0658] Specific examples

[0659] Example 1: The common cold

[0660] A user launches the app and enters the text "My 3-year-old child is coughing." They also take a photo of their child coughing and upload it to the app.

[0661] The device encrypts this data using AES encryption technology and sends it to the server.

[0662] The server analyzes the received data, diagnoses it as a "mild cold," estimates the expected recovery time to be "3 to 5 days," and notifies the user via push notification. It also provides friendly advice to "keep warm and drink plenty of fluids."

[0663] The user later inputs into the system as feedback that the symptoms have improved.

[0664] Specific prompt examples:

[0665] "My 3-year-old child has a cough. I'm sending a photo. Please tell me your diagnosis and advice based on these symptoms."

[0666] Example 2: Minor injury

[0667] A user launches the app and enters the text "My child fell and scraped his knee," taking a photo of the injury and uploading it to the app.

[0668] The device encrypts this data using AES encryption technology and sends it to the server.

[0669] The server analyzes the received data, diagnoses it as a "minor abrasion," determines that no hospital visit is necessary, and predicts that it will take "about 5 days" to heal. It then generates advice to the user: "Disinfect the wound and apply gauze. If the wound becomes infected, we recommend that you go to the hospital." and sends it to the user.

[0670] The user receives the diagnosis, follows the advice, and enters the progress of the symptoms into the system for later feedback.

[0671] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0672] Step 1:

[0673] A user downloads the app onto their smartphone or tablet, launches it, and creates an account. The user enters basic information such as name, email address, and password, and presses the Create Account button. The input data is sent to the account management server, and the user information is registered in the database.

[0674] Input: Basic information entered by the user (name, email address, password)

[0675] Output: Account creation completion notification

[0676] Step 2:

[0677] Users enter symptom information in text format on the app's main screen. They can also take photos of symptoms and injuries with a camera and upload them to the app as image data. Users can also enter the date and time of onset as additional information about their condition, and when they press the send button, the entered data is saved on their device.

[0678] Input: Text data (symptom information), image data (images of symptoms and injuries), date and time information

[0679] Output: Saved symptom data and image data

[0680] Step 3:

[0681] The terminal encrypts the text data entered by the user and the uploaded image data using AES encryption technology. The encryption process is performed using the OpenSSL library.

[0682] Input: Text data, image data

[0683] Output: Encrypted data

[0684] Step 4:

[0685] The device sends encrypted data to the server using SSL / TLS communication, ensuring a secure communication channel using the HTTPS protocol.

[0686] Input: Encrypted data

[0687] Output: Send data to the server

[0688] Step 5:

[0689] The server receives the encrypted data sent from the terminal, decrypts it using RSA encryption, and restores the original text and image data.

[0690] Input: Encrypted data

[0691] Output: Decoded data (text data, image data)

[0692] Step 6:

[0693] The server prepares the decoded data for analysis: text data is tokenized for natural language processing and stop words are removed, and image data is normalized for resolution and noise is removed.

[0694] Input: Decoded data (text data, image data)

[0695] Output: Pre-processed data

[0696] Step 7:

[0697] An artificial intelligence system using TensorFlow on the server analyzes the pre-treatment data and estimates the disease name, whether or not the patient needs to visit the hospital, and the expected number of days until recovery. The analysis results are saved in JSON format.

[0698] Input: Pre-processed data

[0699] Output: Estimated results (disease name, need for hospital visits, expected number of days until recovery)

[0700] Step 8:

[0701] Based on the inference results, the server uses natural language generation (NLG) technology to generate a diagnosis and advice to display to the user. For example, if the diagnosis is a "mild cold," the server generates the advice "Keep warm and drink plenty of fluids."

[0702] Input: Estimation result

[0703] Output: Diagnostic results and advice

[0704] Step 9:

[0705] The diagnostic results and advice generated by the server are sent to the terminal again using SSL / TLS communication.

[0706] Input: Diagnostic results and advice

[0707] Output: Sending data to the terminal

[0708] Step 10:

[0709] The device decodes the diagnostic results and advice received from the server and notifies the user via push notification using Firebase Cloud Messaging (FCM) or Apple Push Notification Service (APNs) when the diagnostic results are ready.

[0710] Input: Diagnostic results and advice (encrypted data)

[0711] Output: User notification

[0712] Step 11:

[0713] The user opens the app, checks the diagnosis results, and takes measures if necessary. Later, they enter the progress of symptoms and the diagnosis results from the hospital into the app as feedback and press the send button.

[0714] Input: Feedback data (progression of symptoms, hospital diagnosis results)

[0715] Output: Feedback data transmission

[0716] Step 12:

[0717] The server receives the feedback data sent by the user and stores it in a database such as MongoDB.

[0718] Input: Feedback data

[0719] Output: Save to database

[0720] Step 13:

[0721] The server retrains the AI ​​model based on newly received feedback data. To improve the diagnostic accuracy of the AI, the retraining process is periodically executed using a scheduler.

[0722] Input: Feedback data

[0723] Output: Retrained AI model

[0724] (Application example 1)

[0725] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0726] Conventional systems for diagnosing children's health simply analyze the symptom data and image data entered by the user and provide a diagnosis. This leaves parents with no way to quickly obtain appropriate medicines or childcare products, and the system lacks the ability to use feedback data to improve the accuracy of the artificial intelligence. Therefore, a system is needed that suggests appropriate countermeasures linked to the diagnosis results and allows users to purchase them smoothly.

[0727] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0728] In this invention, the server includes means for analyzing symptom data entered by the user, means for analyzing symptom images, means for estimating the name of the disease, the need for hospital visits, and the expected number of days until recovery based on the analysis results, means for recommending appropriate medicines and childcare products based on the estimation results, and communication means for allowing the user to purchase the recommended products. This allows appropriate countermeasure products to be suggested in conjunction with the diagnosis results, enabling the user to purchase the necessary products quickly and smoothly.

[0729] "Symptom data entered by the user" is information describing the child's health condition and symptoms in text format.

[0730] A "symptom image" is still image data taken by a user that shows the child's health condition or symptoms.

[0731] "Analytical artificial intelligence" is a software system that has the ability to estimate the name of the disease and the severity of the symptoms based on the received symptom data and image data.

[0732] The "means of estimation" is the process of calculating the name of the disease, the need for outpatient treatment, and the expected number of days until recovery based on data analyzed by artificial intelligence.

[0733] The "means for providing friendly advice" is a function that displays appropriate measures and advice along with the diagnostic results received by the user.

[0734] "Feedback data" refers to data such as follow-up information on the diagnosis results provided by the user, or actual diagnosis results from the hospital.

[0735] The "means used for relearning" refers to the process of improving the algorithm to improve the diagnostic accuracy of the artificial intelligence based on the received feedback data.

[0736] The "means for providing recommended products" is a function that presents appropriate medicines and childcare products to the user based on the diagnostic results.

[0737] "Communication means for purchase" is a function that links with online shopping sites and sales platforms so that users can directly purchase recommended products.

[0738] This invention is a system that receives symptom data and photographed image data entered by the user, analyzes them to estimate the name of the disease, whether or not hospital visits are necessary, and the expected number of days until recovery, and suggests appropriate countermeasure products in conjunction with the diagnosis results, allowing the user to purchase the products quickly and smoothly. Specific embodiments of this system are described below.

[0739] Overall system configuration

[0740] The system consists of a user's device, a server for analyzing data, and a communications network that connects these. Users can begin using the system by downloading the application onto their smartphone, tablet, or other device and creating an account.

[0741] User Actions

[0742] First, the user launches the app and enters their child's symptoms as text. They also take photos of the symptoms and injuries and upload them as image data. Next, they enter details of the symptoms and the date and time of onset, and then send the data.

[0743] Device behavior

[0744] The terminal encrypts the text data entered by the user and the captured image data, and transmits them securely to the server via a communication network.

[0745] Server Operation

[0746] The server receives and decrypts the data sent from the device. It then uses artificial intelligence to analyze the received symptom data and image data, predicting the likely diagnosis, whether or not a hospital visit is necessary, and the expected number of days until recovery. Based on the predictions, it then recommends appropriate medicines and childcare products and provides the user with a link to purchase them.

[0747] Push notifications and checkout

[0748] The server sends the diagnosis results along with recommended product information to the device, which then notifies the user via push notification. The user can then check the recommended products along with the diagnosis results and quickly complete the purchase process using the appropriate communication means.

[0749] Feedback Processing

[0750] Users input feedback data into the app, such as follow-up information on diagnostic results and actual diagnostic results from hospitals, and then submit the data. The server stores the received feedback data in a database and uses it to retrain the AI ​​and improve diagnostic accuracy.

[0751] Techniques used and examples

[0752] The server uses an artificial intelligence model using Python and Flask, and an image processing library using PIL. Data is sent using the HTTP request library, requests.

[0753] Specific examples

[0754] A user launches the app, enters the text "My 3-year-old child has a cough," and uploads an image of the child coughing. The encoded and encrypted data is sent to the server, where it is decoded and analyzed. The server diagnoses the child as having a "mild cold," predicts that the child will be cured in 3 to 5 days, and provides a link to recommended medication. An example prompt might be, "I took a photo of my child coughing. From this image and the following text data, please estimate the name of the illness, whether or not the child needs to visit a doctor, and the expected number of days until recovery: My 3-year-old child has had a cough for several days and has a red throat. He does not have a fever."

[0755] This will enable parents to quickly obtain appropriate advice regarding their child's symptoms and necessary medications.

[0756] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0757] Step 1:

[0758] The user launches the app and logs in. The user enters their child's symptoms in text, describing specific symptoms such as "My 3-year-old child has a cough." They also take an image of the symptoms and upload it to the app. The input data is both text and image data.

[0759] Step 2:

[0760] The terminal receives text data entered by the user and captured image data. It then encrypts the data and transmits it to the server via a communication network. The input data is the encrypted text and image data, and the output data is the encrypted data transmitted to the server.

[0761] Step 3:

[0762] The server receives the encrypted data sent from the terminal and first decrypts it. The decrypted data is then prepared for analysis. The input data is the encrypted symptom data and image data, and the output data is the decrypted data.

[0763] Step 4:

[0764] The server uses an artificial intelligence model to analyze the received symptom data and image data. A generative AI model is used for the analysis, and the name of the disease, the need for medical visits, and the expected number of days until recovery are estimated from the data. An example of a prompt might be, "I took a picture of a child coughing. From this image and the following text data, please estimate the name of the disease, whether medical visits are necessary, and the expected number of days until recovery: A 3-year-old child has been coughing for several days and has a red throat. He does not have a fever." The input data is the decoded symptom data and image data, and the output data is the estimated diagnosis result.

[0765] Step 5:

[0766] The server generates information on recommended medicines and childcare products based on the diagnosis results. The recommended product information includes product descriptions and purchase links. The input data is the estimated diagnosis results, and the output data is the recommended product information and purchase links.

[0767] Step 6:

[0768] The server generates diagnostic results and recommended product information and sends them to the device. At the same time, the push notification function is used to notify the user that the diagnostic results are ready. The input data is the diagnostic results and recommended product information generated by the server, and the output data is the data sent to the device.

[0769] Step 7:

[0770] The terminal displays the diagnosis results and recommended product information received from the server to the user. The user checks the diagnosis results and takes necessary measures. If the user wishes to purchase a recommended product, he or she accesses the purchase link directly through the terminal. The input data is the diagnosis results and recommended product information received from the server, and the output data is the data displayed to the user.

[0771] Step 8:

[0772] The user then inputs the progress of symptoms and the results of the hospital diagnosis into the app as feedback and sends it. The input data is the progress of symptoms and the actual diagnosis results, and the output data is the feedback data entered into the app.

[0773] Step 9:

[0774] The server receives the feedback data sent by the user and stores it in a database. The data is used for retraining to improve the diagnostic accuracy of the AI ​​model. The input data is the feedback data, and the output data is the improved diagnostic model.

[0775] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0776] This invention relates to a system that receives symptom data and photographed image data entered by a user and uses artificial intelligence to analyze the data to estimate the name of the child's illness, whether or not hospital visits are necessary, and the expected number of days until recovery. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system has the function of providing more personalized and considerate advice. A specific embodiment of this system is described below.

[0777] Overall system configuration

[0778] The system consists of a device held by the user, a server for analyzing data, and a communication network for linking these components. The emotion engine also analyzes emotions from the user's text and voice data and reflects them in the diagnosis results.

[0779] 1. User Actions

[0780] The user launches the app and logs in or creates an account.

[0781] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[0782] Users can add text descriptions or recorded audio.

[0783] Check the various data entered by the user and tap the "Send" button in the app.

[0784] 2. Device Operation

[0785] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using a common key encryption method such as AES.

[0786] The terminal transmits the encrypted data to the server via the communication network.

[0787] 3. Server Operation

[0788] The server decrypts the encrypted data it receives and prepares it for analysis.

[0789] Before the server analyzes the data, it runs an emotion engine to recognize the user's emotions.

[0790] + Natural language processing (NLP) of text data extracts emotional keywords using an emotion engine.

[0791] + Voice data is used by a voice recognition and emotion analysis engine to assess the user's emotional state.

[0792] The server preprocesses the received text and image data and converts them into a format for analysis, adjusting image resolution and removing noise.

[0793] An artificial intelligence (AI) model on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is required, and the expected number of days until recovery.

[0794] The server generates reassuring advice in kind words for the user based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is optimized according to the user's emotional state.

[0795] 4. Notice to Users

[0796] The device displays the diagnosis results and advice received from the server to the user, and notifies the user using the push notification function when the diagnosis results are ready.

[0797] The user can check the diagnosis results and take measures as necessary. They can also enter the progress of symptoms and the results of the diagnosis at the hospital into the app at a later date and send it as feedback data.

[0798] 5. Processing of Feedback Data

[0799] The server stores the feedback data received from the user in a database.

[0800] The server retrains the artificial intelligence based on the feedback data received, thereby improving the system's diagnostic accuracy.

[0801] Specific examples

[0802] Example 1: The common cold

[0803] A user launches the app and enters text that their 3-year-old child is coughing. They also take a photo of the child coughing and upload it to the app.

[0804] Along with this data, the device records the user's anxiety as audio, encrypts it, and sends it to the server.

[0805] The server analyzes the received data and determines that the illness is likely a mild cold. It also predicts that the time until recovery will be between 3 and 5 days, and notifies the user via push notification.

[0806] Along with the diagnosis, the user receives gentle advice such as, "Keep warm and give plenty of fluids. If symptoms persist for more than a few days, we recommend seeing a doctor." The advice is optimized to take into account the user's emotional state.

[0807] The user later inputs into the system as feedback that the symptoms have improved.

[0808] Example 2: Minor injury

[0809] A user launches the app and enters text that their child fell and scraped their knee, taking a photo of the injury and uploading it to the app.

[0810] The device encrypts this data along with text expressing the user's concerns and sends it to the server.

[0811] The server analyzes the received data and determines that the injury is a minor scratch. No further hospital visits are required, and the estimated time until recovery is about five days.

[0812] Along with the diagnosis, the user receives gentle advice: "Disinfect and apply gauze. If the wound becomes infected, we recommend going to the hospital." The emotion engine also provides additional advice to ease the worry.

[0813] The above is an embodiment of the present invention.

[0814] The processing flow will be explained below.

[0815] Step 1:

[0816] The user launches the app and logs in or creates an account.

[0817] Step 2:

[0818] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[0819] Step 3:

[0820] Users can add text descriptions or recorded audio.

[0821] Step 4:

[0822] Check the various data entered by the user and tap the "Send" button in the app.

[0823] Step 5:

[0824] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using a common key encryption method such as AES.

[0825] Step 6:

[0826] The terminal transmits the encrypted data to the server via the communication network.

[0827] Step 7:

[0828] The server decrypts the received encrypted data and prepares it for analysis. The data is temporarily stored in secure storage.

[0829] Step 8:

[0830] The server runs an emotion engine to analyze the received text and voice data of the user. For the text data, a natural language processing (NLP) engine is used to extract emotion keywords. For the voice data, a speech recognition and emotion analysis engine is used to evaluate the user's emotional state.

[0831] Step 9:

[0832] The server preprocesses the received image data and converts it into a format for analysis, adjusting the image resolution and removing noise.

[0833] Step 10:

[0834] The server inputs the preprocessed text and image data into an artificial intelligence (AI) model to perform symptom analysis.

[0835] Step 11:

[0836] The server's AI model analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[0837] Step 12:

[0838] The server generates reassuring advice for users based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is personalized according to the user's emotional state.

[0839] Step 13:

[0840] The server prepares to send generated diagnostic results and advice to the device.

[0841] Step 14:

[0842] The device displays the diagnostic results and advice received from the server to the user, and uses the push notification function to notify the user that the diagnostic results are ready.

[0843] Step 15:

[0844] The user can check the diagnosis results and take measures as necessary. At a later date, the user can also enter the progress of symptoms and the results of the diagnosis at the hospital into the app and send it as feedback data.

[0845] Step 16:

[0846] The server stores the feedback data received from the user in a database.

[0847] Step 17:

[0848] The server uses the feedback data to retrain the artificial intelligence, helping to improve the accuracy of the diagnostic model.

[0849] The above is the specific processing flow of this system.

[0850] Example 2

[0851] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0852] Conventional disease diagnosis systems only use symptom data and image data entered by users, and lack emotional state and detailed explanations, making diagnosis results and advice general and making it difficult to provide personalized responses to individual users. Furthermore, the security of transmitted data and appropriate preprocessing of received data are insufficient, making it difficult to provide reliable diagnosis results.

[0853] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving symptom data of the child input by the user, means for receiving symptom images of the child taken by the user, means for receiving voice data input by the user, means for encrypting the received symptom data, image data, and voice data, means for transmitting the encrypted data to the server, means for decrypting the received data, means for preprocessing the decrypted data, means for preparing for data analysis, means for executing an emotion analysis engine, means for executing artificial intelligence using the preprocessed data as input, means for estimating the name of the disease, the need for hospital visits, and the expected number of days until recovery based on the data analyzed by the artificial intelligence, means for generating the estimation results and optimized advice according to the user's emotional state, means for transmitting the generated advice from the server to the terminal, means for the terminal to notify the user of the advice, means for receiving feedback data from the user, and means for storing the received feedback data in a database and using it for relearning the artificial intelligence. This makes it possible to provide personalized diagnosis results and advice that take the user's emotional state into consideration.

[0854] "User" refers to the person who uses this system and inputs the child's symptom data, image data, and voice data.

[0855] "Child symptom data" is text-format information entered by the user, and is data describing the child's physical condition and state.

[0856] "Symptom images" refer to images of a child's symptoms or injuries taken by the user, providing a visual record of the symptoms.

[0857] "Audio data" is data in the form of audio recorded by the user, and is a recording of the user's emotions and symptoms in detail as audio.

[0858] "Encryption" is a process performed to securely transmit received data, and is an operation to convert the data into a format that cannot be read by a third party.

[0859] "Decryption" is the process of returning encrypted data to its original form, making the data ready for analysis by the receiving party.

[0860] "Preprocessing" refers to processing performed to convert received data into a format suitable for analysis, and includes tokenizing text data and adjusting the resolution of image data.

[0861] "Data analysis" is the process of interpreting information based on received data, and involves using an artificial intelligence model to estimate the name of the disease, whether or not outpatient treatment is required, and the expected number of days until recovery.

[0862] An "emotion analysis engine" is software that recognizes and analyzes emotions from text data and voice data entered by the user.

[0863] Artificial intelligence (AI) is a computer program that analyzes received data and estimates the name of the disease, whether or not outpatient treatment is required, and the expected number of days until recovery.

[0864] "Advice" refers to advice or suggestions provided to users based on the results of analysis by artificial intelligence, and is information optimized according to the user's emotional state.

[0865] "Feedback data" refers to data that includes subsequent reports and progress information from users, and is information that the system receives and uses for relearning the artificial intelligence.

[0866] "Database" is an information management system for storing received feedback data, which is used for re-learning and data analysis.

[0867] "Relearning" is the process of updating an artificial intelligence model based on newly received data to improve diagnostic accuracy.

[0868] System Overview

[0869] This invention is a system that receives symptom data and photographed image data entered by the user and uses artificial intelligence to analyze the data to estimate the name of the illness, whether or not the child needs to visit a doctor, and the estimated number of days until recovery. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it has the function of providing more personalized and friendly advice. The overall system configuration consists of a device held by the user, a server for analyzing the data, and a communication network for linking these.

[0870] System configuration and operation

[0871] User Actions

[0872] The user launches the app and logs in or creates an account.

[0873] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[0874] Users can add text descriptions or recorded audio.

[0875] Check the various data entered by the user and tap the "Send" button in the app.

[0876] Device behavior

[0877] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using AES (a common key encryption method).

[0878] The terminal transmits the encrypted data to the server via the communication network.

[0879] Server Operation

[0880] The server decrypts the encrypted data it receives and prepares it for analysis.

[0881] The server runs an emotion engine to recognize the user's emotions.

[0882] The server extracts emotion keywords from the text data using natural language processing (NLP).

[0883] The server analyzes the voice data using voice recognition and emotion analysis to assess the user's emotional state.

[0884] The server preprocesses the received text and image data and converts them into a format for analysis, adjusting image resolution and removing noise.

[0885] An artificial intelligence (AI) model on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is required, and the expected number of days until recovery.

[0886] The server generates reassuring advice in kind words for the user based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is optimized according to the user's emotional state.

[0887] Specific examples

[0888] Example 1: A case of the common cold

[0889] A user launches the app, enters text that their 3-year-old child is coughing, takes a photo of the child coughing, and uploads it to the app.

[0890] The device encrypts text data, image data, and recorded audio data using AES and sends them to the server.

[0891] The server decrypts the received data and prepares it for analysis.

[0892] The server's emotion engine detects anxiety from the user's voice data.

[0893] The server uses an AI model to determine that the condition is likely a "mild cold" and predicts that the time until recovery will be "3 to 5 days."

[0894] The server generates gentle advice such as, "Keep warm and give yourself plenty of fluids. If symptoms persist for more than a few days, we recommend that you see a doctor," and sends it to the device.

[0895] The user checks the diagnostic results and advice and takes appropriate measures.

[0896] Example 2: Minor injury case

[0897] A user launches the app, enters in text that their child fell and scraped their knee, takes a picture of the injury, and uploads it to the app.

[0898] The terminal encrypts the text data, image data, and voice data expressing anxiety and transmits them to the server.

[0899] The server decrypts the received data and prepares it for analysis.

[0900] The server's emotion engine detects worry from the user's voice data.

[0901] The server uses an AI model to determine that the injury is a "minor abrasion," and predicts that no hospital visit is necessary and that the estimated time until recovery is "about 5 days."

[0902] The server generates gentle advice such as "Disinfect the wound and apply gauze. If the wound becomes infected, we recommend that you go to the hospital," and sends it to the device.

[0903] The user checks the diagnostic results and advice and takes appropriate measures.

[0904] Prompt Sentence Examples

[0905] My child has been coughing since this morning. Should I take him to the hospital? I've attached a photo.

[0906] My 3 year old fell and scraped his knee. Should I disinfect it?

[0907] In this way, the system is designed to effectively link the functions of the user, terminal, and server, and to ensure that each process is executed efficiently, thereby providing users with highly reliable diagnostic results and appropriate advice.

[0908] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0909] Step 1: User Enters Data

[0910] The user launches the app and logs in or creates an account.

[0911] The user enters the child's symptom data (e.g., fever, cough, etc.) in text format.

[0912] Users use the app's camera function to take pictures of their child's symptoms or injuries.

[0913] If necessary, the user can record a detailed description of their symptoms and their feelings as audio data.

[0914] Check the various data entered by the user and tap the "Send" button in the app.

[0915] Input: Symptom data, image data, audio data.

[0916] Output: User data stored within the app.

[0917] Step 2: The device encrypts and transmits the data

[0918] The device detects that the user has tapped the "Send" button.

[0919] The terminal encrypts text data, image data, and audio data using the AES encryption method.

[0920] The terminal transmits the encrypted data to the server via the communication network.

[0921] Input: text data, image data, audio data.

[0922] Output: The encrypted data.

[0923] Step 3: The server receives and decrypts the data

[0924] The server receives the encrypted data sent from the terminal.

[0925] The server decrypts the received data and prepares it for analysis.

[0926] Input: Encrypted data.

[0927] Output: The decrypted data.

[0928] Step 4: The server runs the sentiment analysis engine

[0929] The server uses NLP to extract emotional keywords from the text data.

[0930] The server converts the voice data into text using voice recognition technology and performs emotion analysis.

[0931] Input: Decoded text and audio data.

[0932] Output: Emotion analysis results.

[0933] Step 5: The server performs data preprocessing

[0934] The server tokenizes the received text data and standardizes the format.

[0935] The server adjusts the resolution of the image data and removes noise.

[0936] Input: Decoded text and image data.

[0937] Output: Preprocessed text and image data.

[0938] Step 6: The server analyzes the data with the AI ​​model

[0939] The server inputs the pre-processed data into an artificial intelligence (AI) model.

[0940] The server's AI model analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[0941] Input: Preprocessed data.

[0942] Output: Disease name, whether or not outpatient treatment is required, and estimated number of days until recovery.

[0943] Step 7: Server generates diagnostics and advice

[0944] The server generates friendly advice for the user based on the results of the AI ​​model's analysis and emotion analysis.

[0945] The server generates advice that is optimized according to the user's emotional state.

[0946] Input: disease name, whether or not hospital visits are required, expected number of days until recovery, and emotion analysis results.

[0947] Output: Optimized advice.

[0948] Step 8: The server sends the results to the device

[0949] The server transmits the generated diagnostic results and advice to the terminal.

[0950] The terminal notifies the user of the result received from the server.

[0951] Input: Diagnostic results and advice.

[0952] Output: Notification to the user.

[0953] Step 9: User confirms diagnosis results and advice

[0954] The user checks the diagnosis results and advice within the app.

[0955] The user will take appropriate measures as necessary.

[0956] Input: Diagnostic results and advice.

[0957] Output: User response and actions.

[0958] Step 10: User submits feedback

[0959] The user will then enter progress reports and hospital diagnosis results into the app at a later date.

[0960] The data entered by the user is sent as feedback data.

[0961] Input: Progress reports and diagnostic results.

[0962] Output: Feedback data.

[0963] Step 11: Server stores feedback data and retrains

[0964] The server stores the feedback data received from the user in a database.

[0965] The server retrains the artificial intelligence model based on the feedback data received, improving diagnostic accuracy.

[0966] Input: Feedback data.

[0967] Output: A retrained artificial intelligence model.

[0968] (Application example 2)

[0969] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0970] Previous systems had the problem of not being able to fully consider the anxiety and worries of parents when assessing their child's condition. Furthermore, parents had to take the time and effort to check on their child's safety one by one, placing a heavy burden on them. Furthermore, the lack of emotion recognition capabilities made it difficult to provide personalized advice to parents.

[0971] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving user location information and tracking it in real time, means for receiving parental emotion data in text and voice, and means for analyzing the received emotion data with an emotion engine. This allows parents to easily check the safety of their children and provides personalized, kind advice that takes the parent's emotions into consideration.

[0972] The "means for receiving the child's symptom data entered by the user" refers to a method and device for the terminal or server to receive text data relating to the child's symptoms that the user has entered into the application.

[0973] The "means for receiving an image of a child's symptoms taken by a user" refers to a method and device for a terminal or a server to receive image data relating to a child's symptoms taken by a user.

[0974] The "means for executing artificial intelligence" refers to an artificial intelligence algorithm used to analyze the received symptom data and image data, and a computer device for executing the algorithm.

[0975] "Means for estimating the name of the disease, whether or not outpatient treatment is required, and the expected number of days until complete recovery" refers to a method and device for identifying the name of the disease based on data analyzed by artificial intelligence, and predicting the need for outpatient treatment and the expected number of days until complete recovery.

[0976] The "means for providing user-friendly advice" refers to a method and device for notifying the user of advice generated based on the estimation results in easy-to-understand and friendly language.

[0977] "Means for receiving user location information and tracking it in real time" refers to a method and device for obtaining the user's current location using technology such as GPS and tracking that information in real time.

[0978] The "means for receiving parental emotion data in text and voice" refers to a method and device for a terminal or a server to receive emotion-related data input by a parent in text or voice.

[0979] "Means for analyzing received emotion data with an emotion engine" refers to software and algorithms for analyzing received text and voice data and assessing the parent's emotional state.

[0980] The "means for notifying the user of the generated advice" refers to a method and device for providing the user with advice generated based on the analysis results using push notification or the like.

[0981] "Means for receiving feedback data" refers to methods and apparatus for receiving feedback from a user.

[0982] "Means for storing feedback data in a database and using it for retraining artificial intelligence" refers to a method and apparatus for storing received feedback data in a database and retraining the artificial intelligence model based on that data to improve it.

[0983] This invention relates to a system that receives symptom data, photographed image data, and location information entered by parents, and analyzes them using artificial intelligence to estimate the name of the illness, whether hospital visits are necessary, and the expected number of days until recovery. Furthermore, by combining it with an emotion engine that recognizes the emotions of parents, it has the function of providing more personalized and considerate advice. A specific embodiment of this system is described below.

[0984] Overall system configuration

[0985] The system consists of a device held by the parent, a server for analyzing data, and a communication network for linking these. The emotion engine also analyzes emotions from the parent's text and voice data and reflects them in the diagnosis results.

[0986] 1. Parent behavior

[0987] Parents launch the app and log in or create an account.

[0988] Parents enter their child's symptoms as text and take an image of the symptoms using the in-app camera function.

[0989] Parents can add text instructions or audio recordings.

[0990] Check the various data entered by the parent and tap the "Send" button in the app.

[0991] 2. Device Operation

[0992] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using a common key encryption method such as AES.

[0993] The terminal transmits the encrypted data to the server via the communication network.

[0994] 3. Server Operation

[0995] The server decrypts the encrypted data it receives and prepares it for analysis.

[0996] Before the server analyzes the data, it runs an emotion engine to recognize the parent's emotions.

[0997] Natural language processing (NLP) of text data uses an emotion engine to extract emotion keywords. Libraries such as TextBlob are used.

[0998] The voice data is then passed through a speech recognition and emotion analysis engine to assess the parent's emotional state, using the Google Cloud Speech-to-Text API.

[0999] The server preprocesses the received text and image data and converts them into a format for analysis, adjusting image resolution and removing noise.

[1000] An artificial intelligence (AI) model on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is required, and the expected number of days until recovery.

[1001] The server generates reassuring advice to parents in gentle language based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is optimized according to the parent's emotional state.

[1002] 4. Parental Notification

[1003] The device displays the diagnosis results and advice received from the server to the parent, and notifies the parent using the push notification function when the diagnosis results are ready.

[1004] Parents can check the diagnosis results and take measures as necessary. They can also enter the progress of symptoms and the results of medical examinations at medical institutions into the app at a later date and send it as feedback data.

[1005] 5. Processing of Feedback Data

[1006] The server stores the feedback data received from the parent in a database.

[1007] The server retrains the artificial intelligence based on the feedback data received, thereby improving the system's diagnostic accuracy.

[1008] Specific examples

[1009] Example 1: A mild cold

[1010] A parent can enter text that their 3-year-old child is coughing and take a photo of the coughing and upload it. The device then records this data, along with the parent's anxiety, as audio, which is then encrypted and sent to a server. The server analyzes this data, determines that the child has a mild cold, and estimates the expected recovery time at 3-5 days. The parent is advised to keep the child warm and provide plenty of fluids, and is advised to seek medical advice if symptoms persist for more than a few days.

[1011] Prompt Sentence Examples

[1012] When a parent launches a child safety check app and wants to check their child's location, a location request is sent to the server. Along with the current location, the parent's emotional state is also analyzed. To ease the parent's anxiety, the app provides gentle advice such as, "Your child is safe. There's nothing to worry about."

[1013] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1014] Step 1:

[1015] The parent launches the app and logs in or creates an account. The input for this step is the parent's credentials (username, password), and the output is a notification of successful or failed login.

[1016] Step 2:

[1017] Parents enter their child's symptoms as text and take images of the symptoms using the in-app camera function. The input for this step is text data and image data, and the output is these data stored in the app.

[1018] Step 3:

[1019] The parent adds textual instructions and recorded audio data. The input of this step is textual data and audio data, and the output is these data stored on the device.

[1020] Step 4:

[1021] The parent confirms the various data entered (text data, image data, audio data) and taps the "Send" button in the app. The input of this step is the confirmed data, and the output is the trigger that causes the device to start sending data.

[1022] Step 5:

[1023] The terminal detects that the send button has been pressed and encrypts the text data, image data, and audio data. This uses a common key encryption method such as AES. The input is unencrypted data, and the output is encrypted data.

[1024] Step 6:

[1025] The terminal transmits the encrypted data to the server via the communication network. The input of this step is the encrypted data, and the output is a transmission confirmation to the server.

[1026] Step 7:

[1027] The server decrypts the received encrypted data and prepares it for analysis. The input is the encrypted data and the output is the decrypted data.

[1028] Step 8:

[1029] Before the server analyzes the received data, it runs an emotion engine to recognize the parent's emotion. The input of this step is text data and audio data, and the output is the analyzed emotion data.

[1030] The server performs natural language processing (NLP) on the text data to extract emotional keywords. The input is the text data, and the output is the extracted emotional keywords.

[1031] The server sends the voice data to an engine that performs speech recognition and emotion analysis to evaluate the parent's emotional state. The input is the voice data, and the output is the emotion evaluation result.

[1032] Step 9:

[1033] The server preprocesses the text and image data it receives and converts it into a format for analysis. The input is text and image data, and the output is the data converted into the format for analysis. During this process, image resolution is adjusted and noise is removed.

[1034] Step 10:

[1035] An artificial intelligence (AI) model on the server analyzes the data and predicts the likely diagnosis, whether or not the patient will need to visit the hospital, and the expected number of days until recovery. The input is preprocessed data, and the output is the predicted result.

[1036] Step 11:

[1037] The server generates reassuring advice in parent-friendly language based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The input is the estimation results and emotion evaluation data, and the output is the generated advice.

[1038] Step 12:

[1039] The device displays the diagnosis results and advice received from the server to the parent. The input is the data received from the server, and the output is a notification to the parent. The push notification function is used to notify the parent that the diagnosis results are ready.

[1040] Step 13:

[1041] The parent checks the diagnosis results and takes measures as necessary. The parent also inputs the progress of symptoms and the results of the diagnosis at the medical institution into the app at a later date and sends it as feedback data. The input is the feedback data for the later date, and the output is a notification that the feedback has been sent.

[1042] Step 14:

[1043] The server saves the feedback data received from the parent to a database. The input is the received feedback data, and the output is confirmation of the save completion.

[1044] Step 15:

[1045] The server retrains the artificial intelligence based on the feedback data it receives. The input is the feedback data stored in the database, and the output is the retrained AI model. This improves the system's diagnostic accuracy.

[1046] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1047] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1048] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1049] [Third embodiment]

[1050] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1051] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1052] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1053] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1054] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1055] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1056] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1057] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1058] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1059] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1060] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1061] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1062] This invention relates to a system that receives symptom data and photographed image data of a child entered by a user, and analyzes them using artificial intelligence to estimate the name of the disease, whether or not the child needs to visit a hospital, and the expected number of days until recovery. A specific embodiment of this system will be described below.

[1063] Overall system configuration

[1064] The system consists of a device owned by the user, a server for analyzing data, and a communication network for linking these. Users can begin using the system by downloading the application and creating an account.

[1065] 1. User Actions

[1066] Users launch the app and enter their child's symptoms as text. They also take photos of the symptoms and injuries and upload them as image data.

[1067] The user enters details of the symptoms and the date and time of onset and submits the data.

[1068] 2. Device Operation

[1069] The terminal encrypts the text data entered by the user and the captured image data.

[1070] The terminal transmits the encrypted data to the server via a communication network.

[1071] 3. Server Operation

[1072] The server receives the data sent from the terminal, decodes the data, and then prepares the received symptom data and image data for analysis.

[1073] Artificial intelligence on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[1074] The server generates gentle advice to reassure parents along with the inference results.

[1075] The server transmits this information to the terminal.

[1076] 4. Notice to Users

[1077] The device displays the diagnostic results and advice received from the server to the user, and uses the push notification function to notify the user when the diagnostic results are ready.

[1078] Users can check the diagnosis results and take necessary measures. They can also send feedback data by entering the progress of their symptoms and the results of their hospital diagnosis into the app.

[1079] 5. Processing of Feedback Data

[1080] The server receives the feedback data sent by the user and stores it in a database.

[1081] The server retrains the artificial intelligence based on the received feedback data, thereby improving the system's diagnostic accuracy.

[1082] Specific examples

[1083] Example 1: The common cold

[1084] A user launches the app and enters text that their 3-year-old child is coughing. They also take a photo of the child coughing and upload it to the app.

[1085] The device encrypts this data and sends it to the server.

[1086] The server analyzes the received data and determines that the illness is likely a mild cold. It also predicts that the time until recovery will be between 3 and 5 days, and notifies the user via push notification.

[1087] Along with the diagnosis, the user receives gentle advice: "Keep warm and give your child plenty of fluids. If symptoms persist for more than a few days, we recommend that you see a doctor."

[1088] The user later inputs into the system as feedback that the symptoms have improved.

[1089] Example 2: Minor injury

[1090] A user launches the app and enters text that their child fell and scraped their knee, taking a photo of the injury and uploading it to the app.

[1091] The device encrypts this data and sends it to the server.

[1092] The server analyzes the received data and determines that the injury is a minor scratch. No further hospital visits are required, and the estimated time until recovery is about five days.

[1093] Along with the diagnosis, the user receives gentle advice: "Disinfect the wound and apply gauze. If the wound becomes infected, we recommend that you go to the hospital."

[1094] The above is an embodiment of the present invention.

[1095] The processing flow will be explained below.

[1096] Step 1:

[1097] The user launches the app and logs in or creates an account.

[1098] Step 2:

[1099] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[1100] Step 3:

[1101] The user checks the text data entered and the captured image data within the app and taps the "Send" button.

[1102] Step 4:

[1103] The device detects that the send button has been pressed and encrypts the text and image data using a common key encryption method such as AES.

[1104] Step 5:

[1105] The terminal transmits the encrypted data to the server via the communication network.

[1106] Step 6:

[1107] The server decrypts the received encrypted data and prepares it for analysis. The data is temporarily stored in secure storage.

[1108] Step 7:

[1109] The text data received by the server is input into a natural language processing (NLP) engine to extract keywords related to the symptoms.

[1110] Step 8:

[1111] The server preprocesses the received image data and converts it into a format for analysis, adjusting the image resolution and removing noise.

[1112] Step 9:

[1113] The server inputs the preprocessed text and image data into an artificial intelligence (AI) model to perform symptom analysis.

[1114] Step 10:

[1115] The server's AI model analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[1116] Step 11:

[1117] Based on the estimation results of the AI ​​model, the server generates reassuring advice for the user in friendly language.

[1118] Step 12:

[1119] The server composes the generated diagnostics and advice and prepares them for transmission to the device.

[1120] Step 13:

[1121] The device displays the diagnostic results and advice received from the server to the user. Notifications can also be sent using the push notification function.

[1122] Step 14:

[1123] The user can check the diagnosis results and take measures as necessary. They can also enter the progress of symptoms and the results of the diagnosis at the hospital into the app at a later date and send it as feedback data.

[1124] Step 15:

[1125] The server stores the feedback data received from the user in a database.

[1126] Step 16:

[1127] The server uses the new feedback data to retrain the artificial intelligence, helping to improve the accuracy of the diagnostic model.

[1128] The above is a specific processing flow in this system.

[1129] Example 1

[1130] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1131] Currently, observing a child's symptoms and making appropriate medical decisions is difficult, especially for parents with limited medical knowledge. In many cases, anxiety leads parents to visit medical institutions even when symptoms are minor, resulting in a waste of medical resources and an increased burden on parents. It is also difficult to properly record symptom information and provide it to medical institutions as needed, often resulting in a lack of consistency in the information. To solve these problems, a system is needed that allows parents to easily record and analyze their child's symptoms at home and receive appropriate advice. Secure handling of data is also required from the perspective of privacy protection.

[1132] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1133] In this invention, the server includes: means for receiving symptom data of the child entered by the user; means for receiving symptom images of the child taken by the user; means for encrypting the received symptom data and image data; means for transmitting the encrypted data to the server via a communication network; means for decrypting the received data; means for executing an AI to analyze the decrypted data; means for estimating the disease name, the need for hospital visits, and the expected number of days until recovery based on the data analyzed by the AI; means for providing user-friendly advice along with the estimation results; means for transmitting the estimation results and advice to the user's terminal; means for receiving feedback data from the user; means for storing the received feedback data in a database and using it for retraining the AI; and means for notifying the user's terminal that the diagnosis results are ready. This allows users to easily record their child's symptoms from home, securely transmit the data, and receive prompt and appropriate advice. Furthermore, retraining the AI ​​using the feedback data continues to improve diagnostic accuracy.

[1134] "User" refers to an individual who creates an account to use the system and enters their child's symptom data and image data.

[1135] "Symptom data" is textual information about the child's symptoms entered by the user.

[1136] "Image data" refers to image files of children's symptoms or injuries that users take and upload to the system.

[1137] "Encryption" is the process of transforming data using cryptographic techniques to make it unreadable to third parties.

[1138] A "communications network" is a mesh-like connection system, such as the Internet, for transmitting and receiving data between terminals and servers.

[1139] A "server" is a central processing unit that receives and analyzes data and returns diagnostic results and advice to the user.

[1140] "Decryption" is the process of returning encrypted data to its original form.

[1141] "Artificial intelligence" refers to a computer system that uses machine learning algorithms to analyze received symptom and image data and generate a diagnosis and advice.

[1142] The "disease name" is the name that describes the child's condition, estimated by the artificial intelligence based on the analysis results.

[1143] "Necessity of hospital visit" is information indicating whether or not it is necessary to go to the hospital based on the analysis results.

[1144] "Expected days to recovery" is the estimated number of days it will take for the child to recover, based on the analysis results.

[1145] "Feedback data" refers to information such as the progress of symptoms and the results of hospital diagnoses that are entered by the user at a later date.

[1146] A "database" is an information accumulation device that stores received feedback data and other information for later analysis and relearning.

[1147] "Relearning" is the process by which artificial intelligence improves itself based on new data.

[1148] "Push notification" is a system that automatically sends information from a server to a user's device and notifies them.

[1149] This invention relates to a system that receives symptom data and photographed image data of a child entered by a user, and analyzes them using artificial intelligence to estimate the name of the disease, whether or not the child needs to visit a hospital, and the expected number of days until recovery. A specific embodiment of this system will be described below.

[1150] Overall system configuration

[1151] The system consists of a device owned by the user, a server for analyzing data, and a communications network for linking these. Users can use the system by downloading the application to their smartphone or tablet and creating an account.

[1152] User Actions

[1153] Users launch the app and, when using it for the first time, create an account by entering basic information such as their name, email address, and password. After creating an account, they enter their child's symptoms as text. They also take photos of the symptoms and injury and upload them as image data.

[1154] Users enter details of symptoms and the date and time of onset, and submit the data. For example, they enter specific information such as "My 3-year-old child is coughing."

[1155] Device behavior

[1156] The device encrypts the text data entered by the user and the captured image data using AES encryption technology, specifically the OpenSSL library.

[1157] The terminal sends the encrypted data to the server using SSL / TLS communication via the HTTPS protocol.

[1158] Server Operation

[1159] The server uses RSA encryption to decrypt the encrypted data sent from the terminal. This process is performed using the openssl library on the server side.

[1160] The server prepares the received symptom data and image data for analysis. As processing for analysis, natural language processing is performed on the text data, and resolution standardization and noise removal are performed on the image data.

[1161] AI using TensorFlow on the server analyzes the data and predicts the possible diagnosis, whether or not a hospital visit is necessary, and the expected number of days until recovery. For example, if the diagnosis is a "mild cold," the expected number of days until recovery is "3 to 5 days."

[1162] The server uses natural language generation (NLG) technology to generate reassuring advice for parents along with the prediction results, such as "Keep the child warm and give them plenty of fluids."

[1163] The server sends this information to the terminal using SSL / TLS communication.

[1164] User Notification

[1165] The device decodes the diagnostic results and advice received from the server and notifies the user when the diagnostic results are ready using a push notification function, specifically, Firebase Cloud Messaging (FCM) or Apple Push Notification Service (APNs).

[1166] Users can open the app to check the diagnosis results and take measures as necessary. They can also send feedback data by entering the progress of symptoms and the results of hospital diagnosis into the app.

[1167] Processing of feedback data

[1168] The server receives the feedback data sent by the user and stores it in a database such as MongoDB.

[1169] The server retrains the AI ​​model based on the received feedback data, thereby continuously improving the system's diagnostic accuracy.

[1170] Specific examples

[1171] Example 1: The common cold

[1172] A user launches the app and enters the text "My 3-year-old child is coughing." They also take a photo of their child coughing and upload it to the app.

[1173] The device encrypts this data using AES encryption technology and sends it to the server.

[1174] The server analyzes the received data, diagnoses it as a "mild cold," estimates the expected recovery time to be "3 to 5 days," and notifies the user via push notification. It also provides friendly advice to "keep warm and drink plenty of fluids."

[1175] The user later inputs into the system as feedback that the symptoms have improved.

[1176] Specific prompt examples:

[1177] "My 3-year-old child has a cough. I'm sending a photo. Please tell me your diagnosis and advice based on these symptoms."

[1178] Example 2: Minor injury

[1179] A user launches the app and enters the text "My child fell and scraped his knee," taking a photo of the injury and uploading it to the app.

[1180] The device encrypts this data using AES encryption technology and sends it to the server.

[1181] The server analyzes the received data, diagnoses it as a "minor abrasion," determines that no hospital visit is necessary, and predicts that it will take "about 5 days" to heal. It then generates advice to the user: "Disinfect the wound and apply gauze. If the wound becomes infected, we recommend that you go to the hospital." and sends it to the user.

[1182] The user receives the diagnosis, follows the advice, and enters the progress of the symptoms into the system for later feedback.

[1183] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1184] Step 1:

[1185] A user downloads the app onto their smartphone or tablet, launches it, and creates an account. The user enters basic information such as name, email address, and password, and presses the Create Account button. The input data is sent to the account management server, and the user information is registered in the database.

[1186] Input: Basic information entered by the user (name, email address, password)

[1187] Output: Account creation completion notification

[1188] Step 2:

[1189] Users enter symptom information in text format on the app's main screen. They can also take photos of symptoms and injuries with a camera and upload them to the app as image data. Users can also enter the date and time of onset as additional information about their condition, and when they press the send button, the entered data is saved on their device.

[1190] Input: Text data (symptom information), image data (images of symptoms and injuries), date and time information

[1191] Output: Saved symptom data and image data

[1192] Step 3:

[1193] The terminal encrypts the text data entered by the user and the uploaded image data using AES encryption technology. The encryption process is performed using the OpenSSL library.

[1194] Input: Text data, image data

[1195] Output: Encrypted data

[1196] Step 4:

[1197] The device sends encrypted data to the server using SSL / TLS communication, ensuring a secure communication channel using the HTTPS protocol.

[1198] Input: Encrypted data

[1199] Output: Send data to the server

[1200] Step 5:

[1201] The server receives the encrypted data sent from the terminal, decrypts it using RSA encryption, and restores the original text and image data.

[1202] Input: Encrypted data

[1203] Output: Decoded data (text data, image data)

[1204] Step 6:

[1205] The server prepares the decoded data for analysis: text data is tokenized for natural language processing and stop words are removed, and image data is normalized for resolution and noise is removed.

[1206] Input: Decoded data (text data, image data)

[1207] Output: Pre-processed data

[1208] Step 7:

[1209] An artificial intelligence system using TensorFlow on the server analyzes the pre-treatment data and estimates the disease name, whether or not the patient needs to visit the hospital, and the expected number of days until recovery. The analysis results are saved in JSON format.

[1210] Input: Pre-processed data

[1211] Output: Estimated results (disease name, need for hospital visits, expected number of days until recovery)

[1212] Step 8:

[1213] Based on the inference results, the server uses natural language generation (NLG) technology to generate a diagnosis and advice to display to the user. For example, if the diagnosis is a "mild cold," the server generates the advice "Keep warm and drink plenty of fluids."

[1214] Input: Estimation result

[1215] Output: Diagnostic results and advice

[1216] Step 9:

[1217] The diagnostic results and advice generated by the server are sent to the terminal again using SSL / TLS communication.

[1218] Input: Diagnostic results and advice

[1219] Output: Sending data to the terminal

[1220] Step 10:

[1221] The device decodes the diagnostic results and advice received from the server and notifies the user via push notification using Firebase Cloud Messaging (FCM) or Apple Push Notification Service (APNs) when the diagnostic results are ready.

[1222] Input: Diagnostic results and advice (encrypted data)

[1223] Output: User notification

[1224] Step 11:

[1225] The user opens the app, checks the diagnosis results, and takes measures if necessary. Later, they enter the progress of symptoms and the diagnosis results from the hospital into the app as feedback and press the send button.

[1226] Input: Feedback data (progression of symptoms, hospital diagnosis results)

[1227] Output: Feedback data transmission

[1228] Step 12:

[1229] The server receives the feedback data sent by the user and stores it in a database such as MongoDB.

[1230] Input: Feedback data

[1231] Output: Save to database

[1232] Step 13:

[1233] The server retrains the AI ​​model based on newly received feedback data. To improve the diagnostic accuracy of the AI, the retraining process is periodically executed using a scheduler.

[1234] Input: Feedback data

[1235] Output: Retrained AI model

[1236] (Application example 1)

[1237] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1238] Conventional systems for diagnosing children's health simply analyze the symptom data and image data entered by the user and provide a diagnosis. This leaves parents with no way to quickly obtain appropriate medicines or childcare products, and the system lacks the ability to use feedback data to improve the accuracy of the artificial intelligence. Therefore, a system is needed that suggests appropriate countermeasures linked to the diagnosis results and allows users to purchase them smoothly.

[1239] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1240] In this invention, the server includes means for analyzing symptom data entered by the user, means for analyzing symptom images, means for estimating the name of the disease, the need for hospital visits, and the expected number of days until recovery based on the analysis results, means for recommending appropriate medicines and childcare products based on the estimation results, and communication means for allowing the user to purchase the recommended products. This allows appropriate countermeasure products to be suggested in conjunction with the diagnosis results, enabling the user to purchase the necessary products quickly and smoothly.

[1241] "Symptom data entered by the user" is information describing the child's health condition and symptoms in text format.

[1242] A "symptom image" is still image data taken by a user that shows the child's health condition or symptoms.

[1243] "Analytical artificial intelligence" is a software system that has the ability to estimate the name of the disease and the severity of the symptoms based on the received symptom data and image data.

[1244] The "means of estimation" is the process of calculating the name of the disease, the need for outpatient treatment, and the expected number of days until recovery based on data analyzed by artificial intelligence.

[1245] The "means for providing friendly advice" is a function that displays appropriate measures and advice along with the diagnostic results received by the user.

[1246] "Feedback data" refers to data such as follow-up information on the diagnosis results provided by the user, or actual diagnosis results from the hospital.

[1247] The "means used for relearning" refers to the process of improving the algorithm to improve the diagnostic accuracy of the artificial intelligence based on the received feedback data.

[1248] The "means for providing recommended products" is a function that presents appropriate medicines and childcare products to the user based on the diagnostic results.

[1249] "Communication means for purchase" is a function that links with online shopping sites and sales platforms so that users can directly purchase recommended products.

[1250] This invention is a system that receives symptom data and photographed image data entered by the user, analyzes them to estimate the name of the disease, whether or not hospital visits are necessary, and the expected number of days until recovery, and suggests appropriate countermeasure products in conjunction with the diagnosis results, allowing the user to purchase the products quickly and smoothly. Specific embodiments of this system are described below.

[1251] Overall system configuration

[1252] The system consists of a user's device, a server for analyzing data, and a communications network that connects these. Users can begin using the system by downloading the application onto their smartphone, tablet, or other device and creating an account.

[1253] User Actions

[1254] First, the user launches the app and enters their child's symptoms as text. They also take photos of the symptoms and injuries and upload them as image data. Next, they enter details of the symptoms and the date and time of onset, and then send the data.

[1255] Device behavior

[1256] The terminal encrypts the text data entered by the user and the captured image data, and transmits them securely to the server via a communication network.

[1257] Server Operation

[1258] The server receives and decrypts the data sent from the device. It then uses artificial intelligence to analyze the received symptom data and image data, predicting the likely diagnosis, whether or not a hospital visit is necessary, and the expected number of days until recovery. Based on the predictions, it then recommends appropriate medicines and childcare products and provides the user with a link to purchase them.

[1259] Push notifications and checkout

[1260] The server sends the diagnosis results along with recommended product information to the device, which then notifies the user via push notification. The user can then check the recommended products along with the diagnosis results and quickly complete the purchase process using the appropriate communication means.

[1261] Feedback Processing

[1262] Users input feedback data into the app, such as follow-up information on diagnostic results and actual diagnostic results from hospitals, and then submit the data. The server stores the received feedback data in a database and uses it to retrain the AI ​​and improve diagnostic accuracy.

[1263] Techniques used and examples

[1264] The server uses an artificial intelligence model using Python and Flask, and an image processing library using PIL. Data is sent using the HTTP request library, requests.

[1265] Specific examples

[1266] A user launches the app, enters the text "My 3-year-old child has a cough," and uploads an image of the child coughing. The encoded and encrypted data is sent to the server, where it is decoded and analyzed. The server diagnoses the child as having a "mild cold," predicts that the child will be cured in 3 to 5 days, and provides a link to recommended medication. An example prompt might be, "I took a photo of my child coughing. From this image and the following text data, please estimate the name of the illness, whether or not the child needs to visit a doctor, and the expected number of days until recovery: My 3-year-old child has had a cough for several days and has a red throat. He does not have a fever."

[1267] This will enable parents to quickly obtain appropriate advice regarding their child's symptoms and necessary medications.

[1268] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1269] Step 1:

[1270] The user launches the app and logs in. The user enters their child's symptoms in text, describing specific symptoms such as "My 3-year-old child has a cough." They also take an image of the symptoms and upload it to the app. The input data is both text and image data.

[1271] Step 2:

[1272] The terminal receives text data entered by the user and captured image data. It then encrypts the data and transmits it to the server via a communication network. The input data is the encrypted text and image data, and the output data is the encrypted data transmitted to the server.

[1273] Step 3:

[1274] The server receives the encrypted data sent from the terminal and first decrypts it. The decrypted data is then prepared for analysis. The input data is the encrypted symptom data and image data, and the output data is the decrypted data.

[1275] Step 4:

[1276] The server uses an artificial intelligence model to analyze the received symptom data and image data. A generative AI model is used for the analysis, and the name of the disease, the need for medical visits, and the expected number of days until recovery are estimated from the data. An example of a prompt might be, "I took a picture of a child coughing. From this image and the following text data, please estimate the name of the disease, whether medical visits are necessary, and the expected number of days until recovery: A 3-year-old child has been coughing for several days and has a red throat. He does not have a fever." The input data is the decoded symptom data and image data, and the output data is the estimated diagnosis result.

[1277] Step 5:

[1278] The server generates information on recommended medicines and childcare products based on the diagnosis results. The recommended product information includes product descriptions and purchase links. The input data is the estimated diagnosis results, and the output data is the recommended product information and purchase links.

[1279] Step 6:

[1280] The server generates diagnostic results and recommended product information and sends them to the device. At the same time, the push notification function is used to notify the user that the diagnostic results are ready. The input data is the diagnostic results and recommended product information generated by the server, and the output data is the data sent to the device.

[1281] Step 7:

[1282] The terminal displays the diagnosis results and recommended product information received from the server to the user. The user checks the diagnosis results and takes necessary measures. If the user wishes to purchase a recommended product, he or she accesses the purchase link directly through the terminal. The input data is the diagnosis results and recommended product information received from the server, and the output data is the data displayed to the user.

[1283] Step 8:

[1284] The user then inputs the progress of symptoms and the results of the hospital diagnosis into the app as feedback and sends it. The input data is the progress of symptoms and the actual diagnosis results, and the output data is the feedback data entered into the app.

[1285] Step 9:

[1286] The server receives the feedback data sent by the user and stores it in a database. The data is used for retraining to improve the diagnostic accuracy of the AI ​​model. The input data is the feedback data, and the output data is the improved diagnostic model.

[1287] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1288] This invention relates to a system that receives symptom data and photographed image data entered by a user and uses artificial intelligence to analyze the data to estimate the name of the child's illness, whether or not hospital visits are necessary, and the expected number of days until recovery. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system has the function of providing more personalized and considerate advice. A specific embodiment of this system is described below.

[1289] Overall system configuration

[1290] The system consists of a device held by the user, a server for analyzing data, and a communication network for linking these components. The emotion engine also analyzes emotions from the user's text and voice data and reflects them in the diagnosis results.

[1291] 1. User Actions

[1292] The user launches the app and logs in or creates an account.

[1293] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[1294] Users can add text descriptions or recorded audio.

[1295] Check the various data entered by the user and tap the "Send" button in the app.

[1296] 2. Device Operation

[1297] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using a common key encryption method such as AES.

[1298] The terminal transmits the encrypted data to the server via the communication network.

[1299] 3. Server Operation

[1300] The server decrypts the encrypted data it receives and prepares it for analysis.

[1301] Before the server analyzes the data, it runs an emotion engine to recognize the user's emotions.

[1302] + Natural language processing (NLP) of text data extracts emotional keywords using an emotion engine.

[1303] + Voice data is used by a voice recognition and emotion analysis engine to assess the user's emotional state.

[1304] The server preprocesses the received text and image data and converts them into a format for analysis, adjusting image resolution and removing noise.

[1305] An artificial intelligence (AI) model on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is required, and the expected number of days until recovery.

[1306] The server generates reassuring advice in kind words for the user based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is optimized according to the user's emotional state.

[1307] 4. Notice to Users

[1308] The device displays the diagnosis results and advice received from the server to the user, and notifies the user using the push notification function when the diagnosis results are ready.

[1309] The user can check the diagnosis results and take measures as necessary. They can also enter the progress of symptoms and the results of the diagnosis at the hospital into the app at a later date and send it as feedback data.

[1310] 5. Processing of Feedback Data

[1311] The server stores the feedback data received from the user in a database.

[1312] The server retrains the artificial intelligence based on the feedback data received, thereby improving the system's diagnostic accuracy.

[1313] Specific examples

[1314] Example 1: The common cold

[1315] A user launches the app and enters text that their 3-year-old child is coughing. They also take a photo of the child coughing and upload it to the app.

[1316] Along with this data, the device records the user's anxiety as audio, encrypts it, and sends it to the server.

[1317] The server analyzes the received data and determines that the illness is likely a mild cold. It also predicts that the time until recovery will be between 3 and 5 days, and notifies the user via push notification.

[1318] Along with the diagnosis, the user receives gentle advice such as, "Keep warm and give plenty of fluids. If symptoms persist for more than a few days, we recommend seeing a doctor." The advice is optimized to take into account the user's emotional state.

[1319] The user later inputs into the system as feedback that the symptoms have improved.

[1320] Example 2: Minor injury

[1321] A user launches the app and enters text that their child fell and scraped their knee, taking a photo of the injury and uploading it to the app.

[1322] The device encrypts this data along with text expressing the user's concerns and sends it to the server.

[1323] The server analyzes the received data and determines that the injury is a minor scratch. No further hospital visits are required, and the estimated time until recovery is about five days.

[1324] Along with the diagnosis, the user receives gentle advice: "Disinfect and apply gauze. If the wound becomes infected, we recommend going to the hospital." The emotion engine also provides additional advice to ease the worry.

[1325] The above is an embodiment of the present invention.

[1326] The processing flow will be explained below.

[1327] Step 1:

[1328] The user launches the app and logs in or creates an account.

[1329] Step 2:

[1330] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[1331] Step 3:

[1332] Users can add text descriptions or recorded audio.

[1333] Step 4:

[1334] Check the various data entered by the user and tap the "Send" button in the app.

[1335] Step 5:

[1336] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using a common key encryption method such as AES.

[1337] Step 6:

[1338] The terminal transmits the encrypted data to the server via the communication network.

[1339] Step 7:

[1340] The server decrypts the received encrypted data and prepares it for analysis. The data is temporarily stored in secure storage.

[1341] Step 8:

[1342] The server runs an emotion engine to analyze the received text and voice data of the user. For the text data, a natural language processing (NLP) engine is used to extract emotion keywords. For the voice data, a speech recognition and emotion analysis engine is used to evaluate the user's emotional state.

[1343] Step 9:

[1344] The server preprocesses the received image data and converts it into a format for analysis, adjusting the image resolution and removing noise.

[1345] Step 10:

[1346] The server inputs the preprocessed text and image data into an artificial intelligence (AI) model to perform symptom analysis.

[1347] Step 11:

[1348] The server's AI model analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[1349] Step 12:

[1350] The server generates reassuring advice for users based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is personalized according to the user's emotional state.

[1351] Step 13:

[1352] The server prepares to send generated diagnostic results and advice to the device.

[1353] Step 14:

[1354] The device displays the diagnostic results and advice received from the server to the user, and uses the push notification function to notify the user that the diagnostic results are ready.

[1355] Step 15:

[1356] The user can check the diagnosis results and take measures as necessary. At a later date, the user can also enter the progress of symptoms and the results of the diagnosis at the hospital into the app and send it as feedback data.

[1357] Step 16:

[1358] The server stores the feedback data received from the user in a database.

[1359] Step 17:

[1360] The server uses the feedback data to retrain the artificial intelligence, helping to improve the accuracy of the diagnostic model.

[1361] The above is the specific processing flow of this system.

[1362] Example 2

[1363] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1364] Conventional disease diagnosis systems only use symptom data and image data entered by users, and lack emotional state and detailed explanations, making diagnosis results and advice general and making it difficult to provide personalized responses to individual users. Furthermore, the security of transmitted data and appropriate preprocessing of received data are insufficient, making it difficult to provide reliable diagnosis results.

[1365] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving symptom data of the child input by the user, means for receiving symptom images of the child taken by the user, means for receiving voice data input by the user, means for encrypting the received symptom data, image data, and voice data, means for transmitting the encrypted data to the server, means for decrypting the received data, means for preprocessing the decrypted data, means for preparing for data analysis, means for executing an emotion analysis engine, means for executing artificial intelligence using the preprocessed data as input, means for estimating the name of the disease, the need for hospital visits, and the expected number of days until recovery based on the data analyzed by the artificial intelligence, means for generating the estimation results and optimized advice according to the user's emotional state, means for transmitting the generated advice from the server to the terminal, means for the terminal to notify the user of the advice, means for receiving feedback data from the user, and means for storing the received feedback data in a database and using it for relearning the artificial intelligence. This makes it possible to provide personalized diagnosis results and advice that take the user's emotional state into consideration.

[1366] "User" refers to the person who uses this system and inputs the child's symptom data, image data, and voice data.

[1367] "Child symptom data" is text-format information entered by the user, and is data describing the child's physical condition and state.

[1368] "Symptom images" refer to images of a child's symptoms or injuries taken by the user, providing a visual record of the symptoms.

[1369] "Audio data" is data in the form of audio recorded by the user, and is a recording of the user's emotions and symptoms in detail as audio.

[1370] "Encryption" is a process performed to securely transmit received data, and is an operation to convert the data into a format that cannot be read by a third party.

[1371] "Decryption" is the process of returning encrypted data to its original form, making the data ready for analysis by the receiving party.

[1372] "Preprocessing" refers to processing performed to convert received data into a format suitable for analysis, and includes tokenizing text data and adjusting the resolution of image data.

[1373] "Data analysis" is the process of interpreting information based on received data, and involves using an artificial intelligence model to estimate the name of the disease, whether or not outpatient treatment is required, and the expected number of days until recovery.

[1374] An "emotion analysis engine" is software that recognizes and analyzes emotions from text data and voice data entered by the user.

[1375] Artificial intelligence (AI) is a computer program that analyzes received data and estimates the name of the disease, whether or not outpatient treatment is required, and the expected number of days until recovery.

[1376] "Advice" refers to advice or suggestions provided to users based on the results of analysis by artificial intelligence, and is information optimized according to the user's emotional state.

[1377] "Feedback data" refers to data that includes subsequent reports and progress information from users, and is information that the system receives and uses for relearning the artificial intelligence.

[1378] "Database" is an information management system for storing received feedback data, which is used for re-learning and data analysis.

[1379] "Relearning" is the process of updating an artificial intelligence model based on newly received data to improve diagnostic accuracy.

[1380] System Overview

[1381] This invention is a system that receives symptom data and photographed image data entered by the user and uses artificial intelligence to analyze the data to estimate the name of the illness, whether or not the child needs to visit a doctor, and the estimated number of days until recovery. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it has the function of providing more personalized and friendly advice. The overall system configuration consists of a device held by the user, a server for analyzing the data, and a communication network for linking these.

[1382] System configuration and operation

[1383] User Actions

[1384] The user launches the app and logs in or creates an account.

[1385] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[1386] Users can add text descriptions or recorded audio.

[1387] Check the various data entered by the user and tap the "Send" button in the app.

[1388] Device behavior

[1389] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using AES (a common key encryption method).

[1390] The terminal transmits the encrypted data to the server via the communication network.

[1391] Server Operation

[1392] The server decrypts the encrypted data it receives and prepares it for analysis.

[1393] The server runs an emotion engine to recognize the user's emotions.

[1394] The server extracts emotion keywords from the text data using natural language processing (NLP).

[1395] The server analyzes the voice data using voice recognition and emotion analysis to assess the user's emotional state.

[1396] The server preprocesses the received text and image data and converts them into a format for analysis, adjusting image resolution and removing noise.

[1397] An artificial intelligence (AI) model on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is required, and the expected number of days until recovery.

[1398] The server generates reassuring advice in kind words for the user based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is optimized according to the user's emotional state.

[1399] Specific examples

[1400] Example 1: A case of the common cold

[1401] A user launches the app, enters text that their 3-year-old child is coughing, takes a photo of the child coughing, and uploads it to the app.

[1402] The device encrypts text data, image data, and recorded audio data using AES and sends them to the server.

[1403] The server decrypts the received data and prepares it for analysis.

[1404] The server's emotion engine detects anxiety from the user's voice data.

[1405] The server uses an AI model to determine that the condition is likely a "mild cold" and predicts that the time until recovery will be "3 to 5 days."

[1406] The server generates gentle advice such as, "Keep warm and give yourself plenty of fluids. If symptoms persist for more than a few days, we recommend that you see a doctor," and sends it to the device.

[1407] The user checks the diagnostic results and advice and takes appropriate measures.

[1408] Example 2: Minor injury case

[1409] A user launches the app, enters in text that their child fell and scraped their knee, takes a picture of the injury, and uploads it to the app.

[1410] The terminal encrypts the text data, image data, and voice data expressing anxiety and transmits them to the server.

[1411] The server decrypts the received data and prepares it for analysis.

[1412] The server's emotion engine detects worry from the user's voice data.

[1413] The server uses an AI model to determine that the injury is a "minor abrasion," and predicts that no hospital visit is necessary and that the estimated time until recovery is "about 5 days."

[1414] The server generates gentle advice such as "Disinfect the wound and apply gauze. If the wound becomes infected, we recommend that you go to the hospital," and sends it to the device.

[1415] The user checks the diagnostic results and advice and takes appropriate measures.

[1416] Prompt Sentence Examples

[1417] My child has been coughing since this morning. Should I take him to the hospital? I've attached a photo.

[1418] My 3 year old fell and scraped his knee. Should I disinfect it?

[1419] In this way, the system is designed to effectively link the functions of the user, terminal, and server, and to ensure that each process is executed efficiently, thereby providing users with highly reliable diagnostic results and appropriate advice.

[1420] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1421] Step 1: User Enters Data

[1422] The user launches the app and logs in or creates an account.

[1423] The user enters the child's symptom data (e.g., fever, cough, etc.) in text format.

[1424] Users use the app's camera function to take pictures of their child's symptoms or injuries.

[1425] If necessary, the user can record a detailed description of their symptoms and their feelings as audio data.

[1426] Check the various data entered by the user and tap the "Send" button in the app.

[1427] Input: Symptom data, image data, audio data.

[1428] Output: User data stored within the app.

[1429] Step 2: The device encrypts and transmits the data

[1430] The device detects that the user has tapped the "Send" button.

[1431] The terminal encrypts text data, image data, and audio data using the AES encryption method.

[1432] The terminal transmits the encrypted data to the server via the communication network.

[1433] Input: text data, image data, audio data.

[1434] Output: The encrypted data.

[1435] Step 3: The server receives and decrypts the data

[1436] The server receives the encrypted data sent from the terminal.

[1437] The server decrypts the received data and prepares it for analysis.

[1438] Input: Encrypted data.

[1439] Output: The decrypted data.

[1440] Step 4: The server runs the sentiment analysis engine

[1441] The server uses NLP to extract emotional keywords from the text data.

[1442] The server converts the voice data into text using voice recognition technology and performs emotion analysis.

[1443] Input: Decoded text and audio data.

[1444] Output: Emotion analysis results.

[1445] Step 5: The server performs data preprocessing

[1446] The server tokenizes the received text data and standardizes the format.

[1447] The server adjusts the resolution of the image data and removes noise.

[1448] Input: Decoded text and image data.

[1449] Output: Preprocessed text and image data.

[1450] Step 6: The server analyzes the data with the AI ​​model

[1451] The server inputs the pre-processed data into an artificial intelligence (AI) model.

[1452] The server's AI model analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[1453] Input: Preprocessed data.

[1454] Output: Disease name, whether or not outpatient treatment is required, and estimated number of days until recovery.

[1455] Step 7: Server generates diagnostics and advice

[1456] The server generates friendly advice for the user based on the results of the AI ​​model's analysis and emotion analysis.

[1457] The server generates advice that is optimized according to the user's emotional state.

[1458] Input: disease name, whether or not hospital visits are required, expected number of days until recovery, and emotion analysis results.

[1459] Output: Optimized advice.

[1460] Step 8: The server sends the results to the device

[1461] The server transmits the generated diagnostic results and advice to the terminal.

[1462] The terminal notifies the user of the result received from the server.

[1463] Input: Diagnostic results and advice.

[1464] Output: Notification to the user.

[1465] Step 9: User confirms diagnosis results and advice

[1466] The user checks the diagnosis results and advice within the app.

[1467] The user will take appropriate measures as necessary.

[1468] Input: Diagnostic results and advice.

[1469] Output: User response and actions.

[1470] Step 10: User submits feedback

[1471] The user will then enter progress reports and hospital diagnosis results into the app at a later date.

[1472] The data entered by the user is sent as feedback data.

[1473] Input: Progress reports and diagnostic results.

[1474] Output: Feedback data.

[1475] Step 11: Server stores feedback data and retrains

[1476] The server stores the feedback data received from the user in a database.

[1477] The server retrains the artificial intelligence model based on the feedback data received, improving diagnostic accuracy.

[1478] Input: Feedback data.

[1479] Output: A retrained artificial intelligence model.

[1480] (Application example 2)

[1481] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1482] Previous systems had the problem of not being able to fully consider the anxiety and worries of parents when assessing their child's condition. Furthermore, parents had to take the time and effort to check on their child's safety one by one, placing a heavy burden on them. Furthermore, the lack of emotion recognition capabilities made it difficult to provide personalized advice to parents.

[1483] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving user location information and tracking it in real time, means for receiving parental emotion data in text and voice, and means for analyzing the received emotion data with an emotion engine. This allows parents to easily check the safety of their children and provides personalized, kind advice that takes the parent's emotions into consideration.

[1484] The "means for receiving the child's symptom data entered by the user" refers to a method and device for the terminal or server to receive text data relating to the child's symptoms that the user has entered into the application.

[1485] The "means for receiving an image of a child's symptoms taken by a user" refers to a method and device for a terminal or a server to receive image data relating to a child's symptoms taken by a user.

[1486] The "means for executing artificial intelligence" refers to an artificial intelligence algorithm used to analyze the received symptom data and image data, and a computer device for executing the algorithm.

[1487] "Means for estimating the name of the disease, whether or not outpatient treatment is required, and the expected number of days until complete recovery" refers to a method and device for identifying the name of the disease based on data analyzed by artificial intelligence, and predicting the need for outpatient treatment and the expected number of days until complete recovery.

[1488] The "means for providing user-friendly advice" refers to a method and device for notifying the user of advice generated based on the estimation results in easy-to-understand and friendly language.

[1489] "Means for receiving user location information and tracking it in real time" refers to a method and device for obtaining the user's current location using technology such as GPS and tracking that information in real time.

[1490] The "means for receiving parental emotion data in text and voice" refers to a method and device for a terminal or a server to receive emotion-related data input by a parent in text or voice.

[1491] "Means for analyzing received emotion data with an emotion engine" refers to software and algorithms for analyzing received text and voice data and assessing the parent's emotional state.

[1492] The "means for notifying the user of the generated advice" refers to a method and device for providing the user with advice generated based on the analysis results using push notification or the like.

[1493] "Means for receiving feedback data" refers to methods and apparatus for receiving feedback from a user.

[1494] "Means for storing feedback data in a database and using it for retraining artificial intelligence" refers to a method and apparatus for storing received feedback data in a database and retraining the artificial intelligence model based on that data to improve it.

[1495] This invention relates to a system that receives symptom data, photographed image data, and location information entered by parents, and analyzes them using artificial intelligence to estimate the name of the illness, whether hospital visits are necessary, and the expected number of days until recovery. Furthermore, by combining it with an emotion engine that recognizes the emotions of parents, it has the function of providing more personalized and considerate advice. A specific embodiment of this system is described below.

[1496] Overall system configuration

[1497] The system consists of a device held by the parent, a server for analyzing data, and a communication network for linking these. The emotion engine also analyzes emotions from the parent's text and voice data and reflects them in the diagnosis results.

[1498] 1. Parent behavior

[1499] Parents launch the app and log in or create an account.

[1500] Parents enter their child's symptoms as text and take an image of the symptoms using the in-app camera function.

[1501] Parents can add text instructions or audio recordings.

[1502] Check the various data entered by the parent and tap the "Send" button in the app.

[1503] 2. Device Operation

[1504] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using a common key encryption method such as AES.

[1505] The terminal transmits the encrypted data to the server via the communication network.

[1506] 3. Server Operation

[1507] The server decrypts the encrypted data it receives and prepares it for analysis.

[1508] Before the server analyzes the data, it runs an emotion engine to recognize the parent's emotions.

[1509] Natural language processing (NLP) of text data uses an emotion engine to extract emotion keywords. Libraries such as TextBlob are used.

[1510] The voice data is then passed through a speech recognition and emotion analysis engine to assess the parent's emotional state, using the Google Cloud Speech-to-Text API.

[1511] The server preprocesses the received text and image data and converts them into a format for analysis, adjusting image resolution and removing noise.

[1512] An artificial intelligence (AI) model on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is required, and the expected number of days until recovery.

[1513] The server generates reassuring advice to parents in gentle language based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is optimized according to the parent's emotional state.

[1514] 4. Parental Notification

[1515] The device displays the diagnosis results and advice received from the server to the parent, and notifies the parent using the push notification function when the diagnosis results are ready.

[1516] Parents can check the diagnosis results and take measures as necessary. They can also enter the progress of symptoms and the results of medical examinations at medical institutions into the app at a later date and send it as feedback data.

[1517] 5. Processing of Feedback Data

[1518] The server stores the feedback data received from the parent in a database.

[1519] The server retrains the artificial intelligence based on the feedback data received, thereby improving the system's diagnostic accuracy.

[1520] Specific examples

[1521] Example 1: A mild cold

[1522] A parent can enter text that their 3-year-old child is coughing and take a photo of the coughing and upload it. The device then records this data, along with the parent's anxiety, as audio, which is then encrypted and sent to a server. The server analyzes this data, determines that the child has a mild cold, and estimates the expected recovery time at 3-5 days. The parent is advised to keep the child warm and provide plenty of fluids, and is advised to seek medical advice if symptoms persist for more than a few days.

[1523] Prompt Sentence Examples

[1524] When a parent launches a child safety check app and wants to check their child's location, a location request is sent to the server. Along with the current location, the parent's emotional state is also analyzed. To ease the parent's anxiety, the app provides gentle advice such as, "Your child is safe. There's nothing to worry about."

[1525] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1526] Step 1:

[1527] The parent launches the app and logs in or creates an account. The input for this step is the parent's credentials (username, password), and the output is a notification of successful or failed login.

[1528] Step 2:

[1529] Parents enter their child's symptoms as text and take images of the symptoms using the in-app camera function. The input for this step is text data and image data, and the output is these data stored in the app.

[1530] Step 3:

[1531] The parent adds textual instructions and recorded audio data. The input of this step is textual data and audio data, and the output is these data stored on the device.

[1532] Step 4:

[1533] The parent confirms the various data entered (text data, image data, audio data) and taps the "Send" button in the app. The input of this step is the confirmed data, and the output is the trigger that causes the device to start sending data.

[1534] Step 5:

[1535] The terminal detects that the send button has been pressed and encrypts the text data, image data, and audio data. This uses a common key encryption method such as AES. The input is unencrypted data, and the output is encrypted data.

[1536] Step 6:

[1537] The terminal transmits the encrypted data to the server via the communication network. The input of this step is the encrypted data, and the output is a transmission confirmation to the server.

[1538] Step 7:

[1539] The server decrypts the received encrypted data and prepares it for analysis. The input is the encrypted data and the output is the decrypted data.

[1540] Step 8:

[1541] Before the server analyzes the received data, it runs an emotion engine to recognize the parent's emotion. The input of this step is text data and audio data, and the output is the analyzed emotion data.

[1542] The server performs natural language processing (NLP) on the text data to extract emotional keywords. The input is the text data, and the output is the extracted emotional keywords.

[1543] The server sends the voice data to an engine that performs speech recognition and emotion analysis to evaluate the parent's emotional state. The input is the voice data, and the output is the emotion evaluation result.

[1544] Step 9:

[1545] The server preprocesses the text and image data it receives and converts it into a format for analysis. The input is text and image data, and the output is the data converted into the format for analysis. During this process, image resolution is adjusted and noise is removed.

[1546] Step 10:

[1547] An artificial intelligence (AI) model on the server analyzes the data and predicts the likely diagnosis, whether or not the patient will need to visit the hospital, and the expected number of days until recovery. The input is preprocessed data, and the output is the predicted result.

[1548] Step 11:

[1549] The server generates reassuring advice in parent-friendly language based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The input is the estimation results and emotion evaluation data, and the output is the generated advice.

[1550] Step 12:

[1551] The device displays the diagnosis results and advice received from the server to the parent. The input is the data received from the server, and the output is a notification to the parent. The push notification function is used to notify the parent that the diagnosis results are ready.

[1552] Step 13:

[1553] The parent checks the diagnosis results and takes measures as necessary. The parent also inputs the progress of symptoms and the results of the diagnosis at the medical institution into the app at a later date and sends it as feedback data. The input is the feedback data for the later date, and the output is a notification that the feedback has been sent.

[1554] Step 14:

[1555] The server saves the feedback data received from the parent to a database. The input is the received feedback data, and the output is confirmation of the save completion.

[1556] Step 15:

[1557] The server retrains the artificial intelligence based on the feedback data it receives. The input is the feedback data stored in the database, and the output is the retrained AI model. This improves the system's diagnostic accuracy.

[1558] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1559] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1560] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1561] [Fourth embodiment]

[1562] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1563] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1564] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1565] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1566] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1567] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1568] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1569] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1570] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1571] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1572] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1573] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1574] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1575] This invention relates to a system that receives symptom data and photographed image data of a child entered by a user, and analyzes them using artificial intelligence to estimate the name of the disease, whether or not the child needs to visit a hospital, and the expected number of days until recovery. A specific embodiment of this system will be described below.

[1576] Overall system configuration

[1577] The system consists of a device owned by the user, a server for analyzing data, and a communication network for linking these. Users can begin using the system by downloading the application and creating an account.

[1578] 1. User Actions

[1579] Users launch the app and enter their child's symptoms as text. They also take photos of the symptoms and injuries and upload them as image data.

[1580] The user enters details of the symptoms and the date and time of onset and submits the data.

[1581] 2. Device Operation

[1582] The terminal encrypts the text data entered by the user and the captured image data.

[1583] The terminal transmits the encrypted data to the server via a communication network.

[1584] 3. Server Operation

[1585] The server receives the data sent from the terminal, decodes the data, and then prepares the received symptom data and image data for analysis.

[1586] Artificial intelligence on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[1587] The server generates gentle advice to reassure parents along with the inference results.

[1588] The server transmits this information to the terminal.

[1589] 4. Notice to Users

[1590] The device displays the diagnostic results and advice received from the server to the user, and uses the push notification function to notify the user when the diagnostic results are ready.

[1591] Users can check the diagnosis results and take necessary measures. They can also send feedback data by entering the progress of their symptoms and the results of their hospital diagnosis into the app.

[1592] 5. Processing of Feedback Data

[1593] The server receives the feedback data sent by the user and stores it in a database.

[1594] The server retrains the artificial intelligence based on the received feedback data, thereby improving the system's diagnostic accuracy.

[1595] Specific examples

[1596] Example 1: The common cold

[1597] A user launches the app and enters text that their 3-year-old child is coughing. They also take a photo of the child coughing and upload it to the app.

[1598] The device encrypts this data and sends it to the server.

[1599] The server analyzes the received data and determines that the illness is likely a mild cold. It also predicts that the time until recovery will be between 3 and 5 days, and notifies the user via push notification.

[1600] Along with the diagnosis, the user receives gentle advice: "Keep warm and give your child plenty of fluids. If symptoms persist for more than a few days, we recommend that you see a doctor."

[1601] The user later inputs into the system as feedback that the symptoms have improved.

[1602] Example 2: Minor injury

[1603] A user launches the app and enters text that their child fell and scraped their knee, taking a photo of the injury and uploading it to the app.

[1604] The device encrypts this data and sends it to the server.

[1605] The server analyzes the received data and determines that the injury is a minor scratch. No further hospital visits are required, and the estimated time until recovery is about five days.

[1606] Along with the diagnosis, the user receives gentle advice: "Disinfect the wound and apply gauze. If the wound becomes infected, we recommend that you go to the hospital."

[1607] The above is an embodiment of the present invention.

[1608] The processing flow will be explained below.

[1609] Step 1:

[1610] The user launches the app and logs in or creates an account.

[1611] Step 2:

[1612] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[1613] Step 3:

[1614] The user checks the text data entered and the captured image data within the app and taps the "Send" button.

[1615] Step 4:

[1616] The device detects that the send button has been pressed and encrypts the text and image data using a common key encryption method such as AES.

[1617] Step 5:

[1618] The terminal transmits the encrypted data to the server via the communication network.

[1619] Step 6:

[1620] The server decrypts the received encrypted data and prepares it for analysis. The data is temporarily stored in secure storage.

[1621] Step 7:

[1622] The text data received by the server is input into a natural language processing (NLP) engine to extract keywords related to the symptoms.

[1623] Step 8:

[1624] The server preprocesses the received image data and converts it into a format for analysis, adjusting the image resolution and removing noise.

[1625] Step 9:

[1626] The server inputs the preprocessed text and image data into an artificial intelligence (AI) model to perform symptom analysis.

[1627] Step 10:

[1628] The server's AI model analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[1629] Step 11:

[1630] Based on the estimation results of the AI ​​model, the server generates reassuring advice for the user in friendly language.

[1631] Step 12:

[1632] The server composes the generated diagnostics and advice and prepares them for transmission to the device.

[1633] Step 13:

[1634] The device displays the diagnostic results and advice received from the server to the user. Notifications can also be sent using the push notification function.

[1635] Step 14:

[1636] The user can check the diagnosis results and take measures as necessary. They can also enter the progress of symptoms and the results of the diagnosis at the hospital into the app at a later date and send it as feedback data.

[1637] Step 15:

[1638] The server stores the feedback data received from the user in a database.

[1639] Step 16:

[1640] The server uses the new feedback data to retrain the artificial intelligence, helping to improve the accuracy of the diagnostic model.

[1641] The above is a specific processing flow in this system.

[1642] Example 1

[1643] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1644] Currently, observing a child's symptoms and making appropriate medical decisions is difficult, especially for parents with limited medical knowledge. In many cases, anxiety leads parents to visit medical institutions even when symptoms are minor, resulting in a waste of medical resources and an increased burden on parents. It is also difficult to properly record symptom information and provide it to medical institutions as needed, often resulting in a lack of consistency in the information. To solve these problems, a system is needed that allows parents to easily record and analyze their child's symptoms at home and receive appropriate advice. Secure handling of data is also required from the perspective of privacy protection.

[1645] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1646] In this invention, the server includes: means for receiving symptom data of the child entered by the user; means for receiving symptom images of the child taken by the user; means for encrypting the received symptom data and image data; means for transmitting the encrypted data to the server via a communication network; means for decrypting the received data; means for executing an AI to analyze the decrypted data; means for estimating the disease name, the need for hospital visits, and the expected number of days until recovery based on the data analyzed by the AI; means for providing user-friendly advice along with the estimation results; means for transmitting the estimation results and advice to the user's terminal; means for receiving feedback data from the user; means for storing the received feedback data in a database and using it for retraining the AI; and means for notifying the user's terminal that the diagnosis results are ready. This allows users to easily record their child's symptoms from home, securely transmit the data, and receive prompt and appropriate advice. Furthermore, retraining the AI ​​using the feedback data continues to improve diagnostic accuracy.

[1647] "User" refers to an individual who creates an account to use the system and enters their child's symptom data and image data.

[1648] "Symptom data" is textual information about the child's symptoms entered by the user.

[1649] "Image data" refers to image files of children's symptoms or injuries that users take and upload to the system.

[1650] "Encryption" is the process of transforming data using cryptographic techniques to make it unreadable to third parties.

[1651] A "communications network" is a mesh-like connection system, such as the Internet, for transmitting and receiving data between terminals and servers.

[1652] A "server" is a central processing unit that receives and analyzes data and returns diagnostic results and advice to the user.

[1653] "Decryption" is the process of returning encrypted data to its original form.

[1654] "Artificial intelligence" refers to a computer system that uses machine learning algorithms to analyze received symptom and image data and generate a diagnosis and advice.

[1655] The "disease name" is the name that describes the child's condition, estimated by the artificial intelligence based on the analysis results.

[1656] "Necessity of hospital visit" is information indicating whether or not it is necessary to go to the hospital based on the analysis results.

[1657] "Expected days to recovery" is the estimated number of days it will take for the child to recover, based on the analysis results.

[1658] "Feedback data" refers to information such as the progress of symptoms and the results of hospital diagnoses that are entered by the user at a later date.

[1659] A "database" is an information accumulation device that stores received feedback data and other information for later analysis and relearning.

[1660] "Relearning" is the process by which artificial intelligence improves itself based on new data.

[1661] "Push notification" is a system that automatically sends information from a server to a user's device and notifies them.

[1662] This invention relates to a system that receives symptom data and photographed image data of a child entered by a user, and analyzes them using artificial intelligence to estimate the name of the disease, whether or not the child needs to visit a hospital, and the expected number of days until recovery. A specific embodiment of this system will be described below.

[1663] Overall system configuration

[1664] The system consists of a device owned by the user, a server for analyzing data, and a communications network for linking these. Users can use the system by downloading the application to their smartphone or tablet and creating an account.

[1665] User Actions

[1666] Users launch the app and, when using it for the first time, create an account by entering basic information such as their name, email address, and password. After creating an account, they enter their child's symptoms as text. They also take photos of the symptoms and injury and upload them as image data.

[1667] Users enter details of symptoms and the date and time of onset, and submit the data. For example, they enter specific information such as "My 3-year-old child is coughing."

[1668] Device behavior

[1669] The device encrypts the text data entered by the user and the captured image data using AES encryption technology, specifically the OpenSSL library.

[1670] The terminal sends the encrypted data to the server using SSL / TLS communication via the HTTPS protocol.

[1671] Server Operation

[1672] The server uses RSA encryption to decrypt the encrypted data sent from the terminal. This process is performed using the openssl library on the server side.

[1673] The server prepares the received symptom data and image data for analysis. As processing for analysis, natural language processing is performed on the text data, and resolution standardization and noise removal are performed on the image data.

[1674] AI using TensorFlow on the server analyzes the data and predicts the possible diagnosis, whether or not a hospital visit is necessary, and the expected number of days until recovery. For example, if the diagnosis is a "mild cold," the expected number of days until recovery is "3 to 5 days."

[1675] The server uses natural language generation (NLG) technology to generate reassuring advice for parents along with the prediction results, such as "Keep the child warm and give them plenty of fluids."

[1676] The server sends this information to the terminal using SSL / TLS communication.

[1677] User Notification

[1678] The device decodes the diagnostic results and advice received from the server and notifies the user when the diagnostic results are ready using a push notification function, specifically, Firebase Cloud Messaging (FCM) or Apple Push Notification Service (APNs).

[1679] Users can open the app to check the diagnosis results and take measures as necessary. They can also send feedback data by entering the progress of symptoms and the results of hospital diagnosis into the app.

[1680] Processing of feedback data

[1681] The server receives the feedback data sent by the user and stores it in a database such as MongoDB.

[1682] The server retrains the AI ​​model based on the received feedback data, thereby continuously improving the system's diagnostic accuracy.

[1683] Specific examples

[1684] Example 1: The common cold

[1685] A user launches the app and enters the text "My 3-year-old child is coughing." They also take a photo of their child coughing and upload it to the app.

[1686] The device encrypts this data using AES encryption technology and sends it to the server.

[1687] The server analyzes the received data, diagnoses it as a "mild cold," estimates the expected recovery time to be "3 to 5 days," and notifies the user via push notification. It also provides friendly advice to "keep warm and drink plenty of fluids."

[1688] The user later inputs into the system as feedback that the symptoms have improved.

[1689] Specific prompt examples:

[1690] "My 3-year-old child has a cough. I'm sending a photo. Please tell me your diagnosis and advice based on these symptoms."

[1691] Example 2: Minor injury

[1692] A user launches the app and enters the text "My child fell and scraped his knee," taking a photo of the injury and uploading it to the app.

[1693] The device encrypts this data using AES encryption technology and sends it to the server.

[1694] The server analyzes the received data, diagnoses it as a "minor abrasion," determines that no hospital visit is necessary, and predicts that it will take "about 5 days" to heal. It then generates advice to the user: "Disinfect the wound and apply gauze. If the wound becomes infected, we recommend that you go to the hospital." and sends it to the user.

[1695] The user receives the diagnosis, follows the advice, and enters the progress of the symptoms into the system for later feedback.

[1696] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1697] Step 1:

[1698] A user downloads the app onto their smartphone or tablet, launches it, and creates an account. The user enters basic information such as name, email address, and password, and presses the Create Account button. The input data is sent to the account management server, and the user information is registered in the database.

[1699] Input: Basic information entered by the user (name, email address, password)

[1700] Output: Account creation completion notification

[1701] Step 2:

[1702] Users enter symptom information in text format on the app's main screen. They can also take photos of symptoms and injuries with a camera and upload them to the app as image data. Users can also enter the date and time of onset as additional information about their condition, and when they press the send button, the entered data is saved on their device.

[1703] Input: Text data (symptom information), image data (images of symptoms and injuries), date and time information

[1704] Output: Saved symptom data and image data

[1705] Step 3:

[1706] The terminal encrypts the text data entered by the user and the uploaded image data using AES encryption technology. The encryption process is performed using the OpenSSL library.

[1707] Input: Text data, image data

[1708] Output: Encrypted data

[1709] Step 4:

[1710] The device sends encrypted data to the server using SSL / TLS communication, ensuring a secure communication channel using the HTTPS protocol.

[1711] Input: Encrypted data

[1712] Output: Send data to the server

[1713] Step 5:

[1714] The server receives the encrypted data sent from the terminal, decrypts it using RSA encryption, and restores the original text and image data.

[1715] Input: Encrypted data

[1716] Output: Decoded data (text data, image data)

[1717] Step 6:

[1718] The server prepares the decoded data for analysis: text data is tokenized for natural language processing and stop words are removed, and image data is normalized for resolution and noise is removed.

[1719] Input: Decoded data (text data, image data)

[1720] Output: Pre-processed data

[1721] Step 7:

[1722] An artificial intelligence system using TensorFlow on the server analyzes the pre-treatment data and estimates the disease name, whether or not the patient needs to visit the hospital, and the expected number of days until recovery. The analysis results are saved in JSON format.

[1723] Input: Pre-processed data

[1724] Output: Estimated results (disease name, need for hospital visits, expected number of days until recovery)

[1725] Step 8:

[1726] Based on the inference results, the server uses natural language generation (NLG) technology to generate a diagnosis and advice to display to the user. For example, if the diagnosis is a "mild cold," the server generates the advice "Keep warm and drink plenty of fluids."

[1727] Input: Estimation result

[1728] Output: Diagnostic results and advice

[1729] Step 9:

[1730] The diagnostic results and advice generated by the server are sent to the terminal again using SSL / TLS communication.

[1731] Input: Diagnostic results and advice

[1732] Output: Sending data to the terminal

[1733] Step 10:

[1734] The device decodes the diagnostic results and advice received from the server and notifies the user via push notification using Firebase Cloud Messaging (FCM) or Apple Push Notification Service (APNs) when the diagnostic results are ready.

[1735] Input: Diagnostic results and advice (encrypted data)

[1736] Output: User notification

[1737] Step 11:

[1738] The user opens the app, checks the diagnosis results, and takes measures if necessary. Later, they enter the progress of symptoms and the diagnosis results from the hospital into the app as feedback and press the send button.

[1739] Input: Feedback data (progression of symptoms, hospital diagnosis results)

[1740] Output: Feedback data transmission

[1741] Step 12:

[1742] The server receives the feedback data sent by the user and stores it in a database such as MongoDB.

[1743] Input: Feedback data

[1744] Output: Save to database

[1745] Step 13:

[1746] The server retrains the AI ​​model based on newly received feedback data. To improve the diagnostic accuracy of the AI, the retraining process is periodically executed using a scheduler.

[1747] Input: Feedback data

[1748] Output: Retrained AI model

[1749] (Application example 1)

[1750] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1751] Conventional systems for diagnosing children's health simply analyze the symptom data and image data entered by the user and provide a diagnosis. This leaves parents with no way to quickly obtain appropriate medicines or childcare products, and the system lacks the ability to use feedback data to improve the accuracy of the artificial intelligence. Therefore, a system is needed that suggests appropriate countermeasures linked to the diagnosis results and allows users to purchase them smoothly.

[1752] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1753] In this invention, the server includes means for analyzing symptom data entered by the user, means for analyzing symptom images, means for estimating the name of the disease, the need for hospital visits, and the expected number of days until recovery based on the analysis results, means for recommending appropriate medicines and childcare products based on the estimation results, and communication means for allowing the user to purchase the recommended products. This allows appropriate countermeasure products to be suggested in conjunction with the diagnosis results, enabling the user to purchase the necessary products quickly and smoothly.

[1754] "Symptom data entered by the user" is information describing the child's health condition and symptoms in text format.

[1755] A "symptom image" is still image data taken by a user that shows the child's health condition or symptoms.

[1756] "Analytical artificial intelligence" is a software system that has the ability to estimate the name of the disease and the severity of the symptoms based on the received symptom data and image data.

[1757] The "means of estimation" is the process of calculating the name of the disease, the need for outpatient treatment, and the expected number of days until recovery based on data analyzed by artificial intelligence.

[1758] The "means for providing friendly advice" is a function that displays appropriate measures and advice along with the diagnostic results received by the user.

[1759] "Feedback data" refers to data such as follow-up information on the diagnosis results provided by the user, or actual diagnosis results from the hospital.

[1760] The "means used for relearning" refers to the process of improving the algorithm to improve the diagnostic accuracy of the artificial intelligence based on the received feedback data.

[1761] The "means for providing recommended products" is a function that presents appropriate medicines and childcare products to the user based on the diagnostic results.

[1762] "Communication means for purchase" is a function that links with online shopping sites and sales platforms so that users can directly purchase recommended products.

[1763] This invention is a system that receives symptom data and photographed image data entered by the user, analyzes them to estimate the name of the disease, whether or not hospital visits are necessary, and the expected number of days until recovery, and suggests appropriate countermeasure products in conjunction with the diagnosis results, allowing the user to purchase the products quickly and smoothly. Specific embodiments of this system are described below.

[1764] Overall system configuration

[1765] The system consists of a user's device, a server for analyzing data, and a communications network that connects these. Users can begin using the system by downloading the application onto their smartphone, tablet, or other device and creating an account.

[1766] User Actions

[1767] First, the user launches the app and enters their child's symptoms as text. They also take photos of the symptoms and injuries and upload them as image data. Next, they enter details of the symptoms and the date and time of onset, and then send the data.

[1768] Device behavior

[1769] The terminal encrypts the text data entered by the user and the captured image data, and transmits them securely to the server via a communication network.

[1770] Server Operation

[1771] The server receives and decrypts the data sent from the device. It then uses artificial intelligence to analyze the received symptom data and image data, predicting the likely diagnosis, whether or not a hospital visit is necessary, and the expected number of days until recovery. Based on the predictions, it then recommends appropriate medicines and childcare products and provides the user with a link to purchase them.

[1772] Push notifications and checkout

[1773] The server sends the diagnosis results along with recommended product information to the device, which then notifies the user via push notification. The user can then check the recommended products along with the diagnosis results and quickly complete the purchase process using the appropriate communication means.

[1774] Feedback Processing

[1775] Users input feedback data into the app, such as follow-up information on diagnostic results and actual diagnostic results from hospitals, and then submit the data. The server stores the received feedback data in a database and uses it to retrain the AI ​​and improve diagnostic accuracy.

[1776] Techniques used and examples

[1777] The server uses an artificial intelligence model using Python and Flask, and an image processing library using PIL. Data is sent using the HTTP request library, requests.

[1778] Specific examples

[1779] A user launches the app, enters the text "My 3-year-old child has a cough," and uploads an image of the child coughing. The encoded and encrypted data is sent to the server, where it is decoded and analyzed. The server diagnoses the child as having a "mild cold," predicts that the child will be cured in 3 to 5 days, and provides a link to recommended medication. An example prompt might be, "I took a photo of my child coughing. From this image and the following text data, please estimate the name of the illness, whether or not the child needs to visit a doctor, and the expected number of days until recovery: My 3-year-old child has had a cough for several days and has a red throat. He does not have a fever."

[1780] This will enable parents to quickly obtain appropriate advice regarding their child's symptoms and necessary medications.

[1781] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1782] Step 1:

[1783] The user launches the app and logs in. The user enters their child's symptoms in text, describing specific symptoms such as "My 3-year-old child has a cough." They also take an image of the symptoms and upload it to the app. The input data is both text and image data.

[1784] Step 2:

[1785] The terminal receives text data entered by the user and captured image data. It then encrypts the data and transmits it to the server via a communication network. The input data is the encrypted text and image data, and the output data is the encrypted data transmitted to the server.

[1786] Step 3:

[1787] The server receives the encrypted data sent from the terminal and first decrypts it. The decrypted data is then prepared for analysis. The input data is the encrypted symptom data and image data, and the output data is the decrypted data.

[1788] Step 4:

[1789] The server uses an artificial intelligence model to analyze the received symptom data and image data. A generative AI model is used for the analysis, and the name of the disease, the need for medical visits, and the expected number of days until recovery are estimated from the data. An example of a prompt might be, "I took a picture of a child coughing. From this image and the following text data, please estimate the name of the disease, whether medical visits are necessary, and the expected number of days until recovery: A 3-year-old child has been coughing for several days and has a red throat. He does not have a fever." The input data is the decoded symptom data and image data, and the output data is the estimated diagnosis result.

[1790] Step 5:

[1791] The server generates information on recommended medicines and childcare products based on the diagnosis results. The recommended product information includes product descriptions and purchase links. The input data is the estimated diagnosis results, and the output data is the recommended product information and purchase links.

[1792] Step 6:

[1793] The server generates diagnostic results and recommended product information and sends them to the device. At the same time, the push notification function is used to notify the user that the diagnostic results are ready. The input data is the diagnostic results and recommended product information generated by the server, and the output data is the data sent to the device.

[1794] Step 7:

[1795] The terminal displays the diagnosis results and recommended product information received from the server to the user. The user checks the diagnosis results and takes necessary measures. If the user wishes to purchase a recommended product, he or she accesses the purchase link directly through the terminal. The input data is the diagnosis results and recommended product information received from the server, and the output data is the data displayed to the user.

[1796] Step 8:

[1797] The user then inputs the progress of symptoms and the results of the hospital diagnosis into the app as feedback and sends it. The input data is the progress of symptoms and the actual diagnosis results, and the output data is the feedback data entered into the app.

[1798] Step 9:

[1799] The server receives the feedback data sent by the user and stores it in a database. The data is used for retraining to improve the diagnostic accuracy of the AI ​​model. The input data is the feedback data, and the output data is the improved diagnostic model.

[1800] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1801] This invention relates to a system that receives symptom data and photographed image data entered by a user and uses artificial intelligence to analyze the data to estimate the name of the child's illness, whether or not hospital visits are necessary, and the expected number of days until recovery. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system has the function of providing more personalized and considerate advice. A specific embodiment of this system is described below.

[1802] Overall system configuration

[1803] The system consists of a device held by the user, a server for analyzing data, and a communication network for linking these components. The emotion engine also analyzes emotions from the user's text and voice data and reflects them in the diagnosis results.

[1804] 1. User Actions

[1805] The user launches the app and logs in or creates an account.

[1806] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[1807] Users can add text descriptions or recorded audio.

[1808] Check the various data entered by the user and tap the "Send" button in the app.

[1809] 2. Device Operation

[1810] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using a common key encryption method such as AES.

[1811] The terminal transmits the encrypted data to the server via the communication network.

[1812] 3. Server Operation

[1813] The server decrypts the encrypted data it receives and prepares it for analysis.

[1814] Before the server analyzes the data, it runs an emotion engine to recognize the user's emotions.

[1815] + Natural language processing (NLP) of text data extracts emotional keywords using an emotion engine.

[1816] + Voice data is used by a voice recognition and emotion analysis engine to assess the user's emotional state.

[1817] The server preprocesses the received text and image data and converts them into a format for analysis, adjusting image resolution and removing noise.

[1818] An artificial intelligence (AI) model on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is required, and the expected number of days until recovery.

[1819] The server generates reassuring advice in kind words for the user based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is optimized according to the user's emotional state.

[1820] 4. Notice to Users

[1821] The device displays the diagnosis results and advice received from the server to the user, and notifies the user using the push notification function when the diagnosis results are ready.

[1822] The user can check the diagnosis results and take measures as necessary. They can also enter the progress of symptoms and the results of the diagnosis at the hospital into the app at a later date and send it as feedback data.

[1823] 5. Processing of Feedback Data

[1824] The server stores the feedback data received from the user in a database.

[1825] The server retrains the artificial intelligence based on the feedback data received, thereby improving the system's diagnostic accuracy.

[1826] Specific examples

[1827] Example 1: The common cold

[1828] A user launches the app and enters text that their 3-year-old child is coughing. They also take a photo of the child coughing and upload it to the app.

[1829] Along with this data, the device records the user's anxiety as audio, encrypts it, and sends it to the server.

[1830] The server analyzes the received data and determines that the illness is likely a mild cold. It also predicts that the time until recovery will be between 3 and 5 days, and notifies the user via push notification.

[1831] Along with the diagnosis, the user receives gentle advice such as, "Keep warm and give plenty of fluids. If symptoms persist for more than a few days, we recommend seeing a doctor." The advice is optimized to take into account the user's emotional state.

[1832] The user later inputs into the system as feedback that the symptoms have improved.

[1833] Example 2: Minor injury

[1834] A user launches the app and enters text that their child fell and scraped their knee, taking a photo of the injury and uploading it to the app.

[1835] The device encrypts this data along with text expressing the user's concerns and sends it to the server.

[1836] The server analyzes the received data and determines that the injury is a minor scratch. No further hospital visits are required, and the estimated time until recovery is about five days.

[1837] Along with the diagnosis, the user receives gentle advice: "Disinfect and apply gauze. If the wound becomes infected, we recommend going to the hospital." The emotion engine also provides additional advice to ease the worry.

[1838] The above is an embodiment of the present invention.

[1839] The processing flow will be explained below.

[1840] Step 1:

[1841] The user launches the app and logs in or creates an account.

[1842] Step 2:

[1843] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[1844] Step 3:

[1845] Users can add text descriptions or recorded audio.

[1846] Step 4:

[1847] Check the various data entered by the user and tap the "Send" button in the app.

[1848] Step 5:

[1849] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using a common key encryption method such as AES.

[1850] Step 6:

[1851] The terminal transmits the encrypted data to the server via the communication network.

[1852] Step 7:

[1853] The server decrypts the received encrypted data and prepares it for analysis. The data is temporarily stored in secure storage.

[1854] Step 8:

[1855] The server runs an emotion engine to analyze the received text and voice data of the user. For the text data, a natural language processing (NLP) engine is used to extract emotion keywords. For the voice data, a speech recognition and emotion analysis engine is used to evaluate the user's emotional state.

[1856] Step 9:

[1857] The server preprocesses the received image data and converts it into a format for analysis, adjusting the image resolution and removing noise.

[1858] Step 10:

[1859] The server inputs the preprocessed text and image data into an artificial intelligence (AI) model to perform symptom analysis.

[1860] Step 11:

[1861] The server's AI model analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[1862] Step 12:

[1863] The server generates reassuring advice for users based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is personalized according to the user's emotional state.

[1864] Step 13:

[1865] The server prepares to send generated diagnostic results and advice to the device.

[1866] Step 14:

[1867] The device displays the diagnostic results and advice received from the server to the user, and uses the push notification function to notify the user that the diagnostic results are ready.

[1868] Step 15:

[1869] The user can check the diagnosis results and take measures as necessary. At a later date, the user can also enter the progress of symptoms and the results of the diagnosis at the hospital into the app and send it as feedback data.

[1870] Step 16:

[1871] The server stores the feedback data received from the user in a database.

[1872] Step 17:

[1873] The server uses the feedback data to retrain the artificial intelligence, helping to improve the accuracy of the diagnostic model.

[1874] The above is the specific processing flow of this system.

[1875] Example 2

[1876] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1877] Conventional disease diagnosis systems only use symptom data and image data entered by users, and lack emotional state and detailed explanations, making diagnosis results and advice general and making it difficult to provide personalized responses to individual users. Furthermore, the security of transmitted data and appropriate preprocessing of received data are insufficient, making it difficult to provide reliable diagnosis results.

[1878] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving symptom data of the child input by the user, means for receiving symptom images of the child taken by the user, means for receiving voice data input by the user, means for encrypting the received symptom data, image data, and voice data, means for transmitting the encrypted data to the server, means for decrypting the received data, means for preprocessing the decrypted data, means for preparing for data analysis, means for executing an emotion analysis engine, means for executing artificial intelligence using the preprocessed data as input, means for estimating the name of the disease, the need for hospital visits, and the expected number of days until recovery based on the data analyzed by the artificial intelligence, means for generating the estimation results and optimized advice according to the user's emotional state, means for transmitting the generated advice from the server to the terminal, means for the terminal to notify the user of the advice, means for receiving feedback data from the user, and means for storing the received feedback data in a database and using it for relearning the artificial intelligence. This makes it possible to provide personalized diagnosis results and advice that take the user's emotional state into consideration.

[1879] "User" refers to the person who uses this system and inputs the child's symptom data, image data, and voice data.

[1880] "Child symptom data" is text-format information entered by the user, and is data describing the child's physical condition and state.

[1881] "Symptom images" refer to images of a child's symptoms or injuries taken by the user, providing a visual record of the symptoms.

[1882] "Audio data" is data in the form of audio recorded by the user, and is a recording of the user's emotions and symptoms in detail as audio.

[1883] "Encryption" is a process performed to securely transmit received data, and is an operation to convert the data into a format that cannot be read by a third party.

[1884] "Decryption" is the process of returning encrypted data to its original form, making the data ready for analysis by the receiving party.

[1885] "Preprocessing" refers to processing performed to convert received data into a format suitable for analysis, and includes tokenizing text data and adjusting the resolution of image data.

[1886] "Data analysis" is the process of interpreting information based on received data, and involves using an artificial intelligence model to estimate the name of the disease, whether or not outpatient treatment is required, and the expected number of days until recovery.

[1887] An "emotion analysis engine" is software that recognizes and analyzes emotions from text data and voice data entered by the user.

[1888] Artificial intelligence (AI) is a computer program that analyzes received data and estimates the name of the disease, whether or not outpatient treatment is required, and the expected number of days until recovery.

[1889] "Advice" refers to advice or suggestions provided to users based on the results of analysis by artificial intelligence, and is information optimized according to the user's emotional state.

[1890] "Feedback data" refers to data that includes subsequent reports and progress information from users, and is information that the system receives and uses for relearning the artificial intelligence.

[1891] "Database" is an information management system for storing received feedback data, which is used for re-learning and data analysis.

[1892] "Relearning" is the process of updating an artificial intelligence model based on newly received data to improve diagnostic accuracy.

[1893] System Overview

[1894] This invention is a system that receives symptom data and photographed image data entered by the user and uses artificial intelligence to analyze the data to estimate the name of the illness, whether or not the child needs to visit a doctor, and the estimated number of days until recovery. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it has the function of providing more personalized and friendly advice. The overall system configuration consists of a device held by the user, a server for analyzing the data, and a communication network for linking these.

[1895] System configuration and operation

[1896] User Actions

[1897] The user launches the app and logs in or creates an account.

[1898] Users enter their child's symptoms as text and use the in-app camera feature to take pictures of the symptoms or injuries.

[1899] Users can add text descriptions or recorded audio.

[1900] Check the various data entered by the user and tap the "Send" button in the app.

[1901] Device behavior

[1902] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using AES (a common key encryption method).

[1903] The terminal transmits the encrypted data to the server via the communication network.

[1904] Server Operation

[1905] The server decrypts the encrypted data it receives and prepares it for analysis.

[1906] The server runs an emotion engine to recognize the user's emotions.

[1907] The server extracts emotion keywords from the text data using natural language processing (NLP).

[1908] The server analyzes the voice data using voice recognition and emotion analysis to assess the user's emotional state.

[1909] The server preprocesses the received text and image data and converts them into a format for analysis, adjusting image resolution and removing noise.

[1910] An artificial intelligence (AI) model on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is required, and the expected number of days until recovery.

[1911] The server generates reassuring advice in kind words for the user based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is optimized according to the user's emotional state.

[1912] Specific examples

[1913] Example 1: A case of the common cold

[1914] A user launches the app, enters text that their 3-year-old child is coughing, takes a photo of the child coughing, and uploads it to the app.

[1915] The device encrypts text data, image data, and recorded audio data using AES and sends them to the server.

[1916] The server decrypts the received data and prepares it for analysis.

[1917] The server's emotion engine detects anxiety from the user's voice data.

[1918] The server uses an AI model to determine that the condition is likely a "mild cold" and predicts that the time until recovery will be "3 to 5 days."

[1919] The server generates gentle advice such as, "Keep warm and give yourself plenty of fluids. If symptoms persist for more than a few days, we recommend that you see a doctor," and sends it to the device.

[1920] The user checks the diagnostic results and advice and takes appropriate measures.

[1921] Example 2: Minor injury case

[1922] A user launches the app, enters in text that their child fell and scraped their knee, takes a picture of the injury, and uploads it to the app.

[1923] The terminal encrypts the text data, image data, and voice data expressing anxiety and transmits them to the server.

[1924] The server decrypts the received data and prepares it for analysis.

[1925] The server's emotion engine detects worry from the user's voice data.

[1926] The server uses an AI model to determine that the injury is a "minor abrasion," and predicts that no hospital visit is necessary and that the estimated time until recovery is "about 5 days."

[1927] The server generates gentle advice such as "Disinfect the wound and apply gauze. If the wound becomes infected, we recommend that you go to the hospital," and sends it to the device.

[1928] The user checks the diagnostic results and advice and takes appropriate measures.

[1929] Prompt Sentence Examples

[1930] My child has been coughing since this morning. Should I take him to the hospital? I've attached a photo.

[1931] My 3 year old fell and scraped his knee. Should I disinfect it?

[1932] In this way, the system is designed to effectively link the functions of the user, terminal, and server, and to ensure that each process is executed efficiently, thereby providing users with highly reliable diagnostic results and appropriate advice.

[1933] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1934] Step 1: User Enters Data

[1935] The user launches the app and logs in or creates an account.

[1936] The user enters the child's symptom data (e.g., fever, cough, etc.) in text format.

[1937] Users use the app's camera function to take pictures of their child's symptoms or injuries.

[1938] If necessary, the user can record a detailed description of their symptoms and their feelings as audio data.

[1939] Check the various data entered by the user and tap the "Send" button in the app.

[1940] Input: Symptom data, image data, audio data.

[1941] Output: User data stored within the app.

[1942] Step 2: The device encrypts and transmits the data

[1943] The device detects that the user has tapped the "Send" button.

[1944] The terminal encrypts text data, image data, and audio data using the AES encryption method.

[1945] The terminal transmits the encrypted data to the server via the communication network.

[1946] Input: text data, image data, audio data.

[1947] Output: The encrypted data.

[1948] Step 3: The server receives and decrypts the data

[1949] The server receives the encrypted data sent from the terminal.

[1950] The server decrypts the received data and prepares it for analysis.

[1951] Input: Encrypted data.

[1952] Output: The decrypted data.

[1953] Step 4: The server runs the sentiment analysis engine

[1954] The server uses NLP to extract emotional keywords from the text data.

[1955] The server converts the voice data into text using voice recognition technology and performs emotion analysis.

[1956] Input: Decoded text and audio data.

[1957] Output: Emotion analysis results.

[1958] Step 5: The server performs data preprocessing

[1959] The server tokenizes the received text data and standardizes the format.

[1960] The server adjusts the resolution of the image data and removes noise.

[1961] Input: Decoded text and image data.

[1962] Output: Preprocessed text and image data.

[1963] Step 6: The server analyzes the data with the AI ​​model

[1964] The server inputs the pre-processed data into an artificial intelligence (AI) model.

[1965] The server's AI model analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is necessary, and the expected number of days until recovery.

[1966] Input: Preprocessed data.

[1967] Output: Disease name, whether or not outpatient treatment is required, and estimated number of days until recovery.

[1968] Step 7: Server generates diagnostics and advice

[1969] The server generates friendly advice for the user based on the results of the AI ​​model's analysis and emotion analysis.

[1970] The server generates advice that is optimized according to the user's emotional state.

[1971] Input: disease name, whether or not hospital visits are required, expected number of days until recovery, and emotion analysis results.

[1972] Output: Optimized advice.

[1973] Step 8: The server sends the results to the device

[1974] The server transmits the generated diagnostic results and advice to the terminal.

[1975] The terminal notifies the user of the result received from the server.

[1976] Input: Diagnostic results and advice.

[1977] Output: Notification to the user.

[1978] Step 9: User confirms diagnosis results and advice

[1979] The user checks the diagnosis results and advice within the app.

[1980] The user will take appropriate measures as necessary.

[1981] Input: Diagnostic results and advice.

[1982] Output: User response and actions.

[1983] Step 10: User submits feedback

[1984] The user will then enter progress reports and hospital diagnosis results into the app at a later date.

[1985] The data entered by the user is sent as feedback data.

[1986] Input: Progress reports and diagnostic results.

[1987] Output: Feedback data.

[1988] Step 11: Server stores feedback data and retrains

[1989] The server stores the feedback data received from the user in a database.

[1990] The server retrains the artificial intelligence model based on the feedback data received, improving diagnostic accuracy.

[1991] Input: Feedback data.

[1992] Output: A retrained artificial intelligence model.

[1993] (Application example 2)

[1994] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1995] Previous systems had the problem of not being able to fully consider the anxiety and worries of parents when assessing their child's condition. Furthermore, parents had to take the time and effort to check on their child's safety one by one, placing a heavy burden on them. Furthermore, the lack of emotion recognition capabilities made it difficult to provide personalized advice to parents.

[1996] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving user location information and tracking it in real time, means for receiving parental emotion data in text and voice, and means for analyzing the received emotion data with an emotion engine. This allows parents to easily check the safety of their children and provides personalized, kind advice that takes the parent's emotions into consideration.

[1997] The "means for receiving the child's symptom data entered by the user" refers to a method and device for the terminal or server to receive text data relating to the child's symptoms that the user has entered into the application.

[1998] The "means for receiving an image of a child's symptoms taken by a user" refers to a method and device for a terminal or a server to receive image data relating to a child's symptoms taken by a user.

[1999] The "means for executing artificial intelligence" refers to an artificial intelligence algorithm used to analyze the received symptom data and image data, and a computer device for executing the algorithm.

[2000] "Means for estimating the name of the disease, whether or not outpatient treatment is required, and the expected number of days until complete recovery" refers to a method and device for identifying the name of the disease based on data analyzed by artificial intelligence, and predicting the need for outpatient treatment and the expected number of days until complete recovery.

[2001] The "means for providing user-friendly advice" refers to a method and device for notifying the user of advice generated based on the estimation results in easy-to-understand and friendly language.

[2002] "Means for receiving user location information and tracking it in real time" refers to a method and device for obtaining the user's current location using technology such as GPS and tracking that information in real time.

[2003] The "means for receiving parental emotion data in text and voice" refers to a method and device for a terminal or a server to receive emotion-related data input by a parent in text or voice.

[2004] "Means for analyzing received emotion data with an emotion engine" refers to software and algorithms for analyzing received text and voice data and assessing the parent's emotional state.

[2005] The "means for notifying the user of the generated advice" refers to a method and device for providing the user with advice generated based on the analysis results using push notification or the like.

[2006] "Means for receiving feedback data" refers to methods and apparatus for receiving feedback from a user.

[2007] "Means for storing feedback data in a database and using it for retraining artificial intelligence" refers to a method and apparatus for storing received feedback data in a database and retraining the artificial intelligence model based on that data to improve it.

[2008] This invention relates to a system that receives symptom data, photographed image data, and location information entered by parents, and analyzes them using artificial intelligence to estimate the name of the illness, whether hospital visits are necessary, and the expected number of days until recovery. Furthermore, by combining it with an emotion engine that recognizes the emotions of parents, it has the function of providing more personalized and considerate advice. A specific embodiment of this system is described below.

[2009] Overall system configuration

[2010] The system consists of a device held by the parent, a server for analyzing data, and a communication network for linking these. The emotion engine also analyzes emotions from the parent's text and voice data and reflects them in the diagnosis results.

[2011] 1. Parent behavior

[2012] Parents launch the app and log in or create an account.

[2013] Parents enter their child's symptoms as text and take an image of the symptoms using the in-app camera function.

[2014] Parents can add text instructions or audio recordings.

[2015] Check the various data entered by the parent and tap the "Send" button in the app.

[2016] 2. Device Operation

[2017] The device detects that the send button has been pressed and encrypts the text data, image data, and audio data using a common key encryption method such as AES.

[2018] The terminal transmits the encrypted data to the server via the communication network.

[2019] 3. Server Operation

[2020] The server decrypts the encrypted data it receives and prepares it for analysis.

[2021] Before the server analyzes the data, it runs an emotion engine to recognize the parent's emotions.

[2022] Natural language processing (NLP) of text data uses an emotion engine to extract emotion keywords. Libraries such as TextBlob are used.

[2023] The voice data is then passed through a speech recognition and emotion analysis engine to assess the parent's emotional state, using the Google Cloud Speech-to-Text API.

[2024] The server preprocesses the received text and image data and converts them into a format for analysis, adjusting image resolution and removing noise.

[2025] An artificial intelligence (AI) model on the server analyzes the data and estimates the possible diagnosis, whether or not outpatient treatment is required, and the expected number of days until recovery.

[2026] The server generates reassuring advice to parents in gentle language based on the estimation results of the AI ​​model and the evaluation results of the emotion engine. The advice is optimized according to the parent's emotional state.

[2027] 4. Parental Notification

[2028] The device displays the diagnosis results and advice received from the server to the parent, and notifies the parent using the push notification function when the diagnosis results are ready.

[2029] Parents can check the diagnosis results and take measures as necessary. They can also enter the progress of symptoms and the results of medical examinations at medical institutions into the app at a later date and send it as feedback data.

[2030] 5. Processing of Feedback Data

[2031] The server stores the feedback data received from the parent in a database.

[2032] The server retrains the artificial intelligence based on the feedback data received, thereby improving the system's diagnostic accuracy.

[2033] Specific examples

[2034] Example 1: A mild cold

[2035] A parent can enter text that their 3-year-old child is coughing and take a photo of the coughing and upload it. The device then records this data, along with the parent's anxiety, as audio, which is then encrypted and sent to a server. The server analyzes this data, determines that the child has a mild cold, and estimates the expected recovery time at 3-5 days. The parent is advised to keep the child warm and provide plenty of fluids, and is advised to seek medical advice if symptoms persist for more than a few days.

[2036] Prompt Sentence Examples

[2037] When a parent launches a child safety check app and wants to check their child's location, a location request is sent to the server. Along with the current location, the parent's emotional state is also analyzed. To ease the parent's anxiety, the app provides gentle advice such as, "Your child is safe. There's nothing to worry about."

[2038] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2039] Step 1:

[2040] The parent launches the app and logs in or creates an account. The input for this step is the parent's credentials (username, password), and the output is a notification of successful or failed login.

[2041] Step 2:

[2042] Parents enter their child's symptoms as text and take images of the symptoms using the in-app camera function. The input for this step is text data and image data, and the output is these data stored in the app.

[2043] Step 3:

[2044] The parent adds textual instructions and recorded audio data. The input of this step is textual data and audio data, and the output is these data stored on the device.

[2045] Step 4:

[2046] The parent confirms the various data entered (text data, image data, audio data) and taps the "Send" button in the app. The input of this step is the confirmed data, and the output is the trigger that causes the device to start sending data.

[2047] Step 5:

[2048] The terminal detects that the send button has been pressed and encrypts the text data, image data, and audio data. This uses a common key encryption method such as AES. The input is unencrypted data, and the output is encrypted data.

[2049] Step 6:

[2050] The terminal transmits the encrypted data to the server via the communication network. The input of this step is the encrypted data, and the output is a transmission confirmation to the server.

[2051] Step 7:

[2052] The server decrypts the received encrypted data and prepares it for analysis. The input is the encrypted data and the output is the decrypted data.

[2053] Step 8:

[2054] Before the server analyzes the received data, it runs an emotion engine to recognize the parent's emotion. The input of this step is text data and audio data, and the output is the analyzed emotion data.

[2055] The server performs natural language processing (NLP) on the text data to extract emotional keywords. The input is the text data, and the output is the extracted emotional keywords.

[2056] The server sends the voice data to an engine that performs speech recognition and emotion analysis to evaluate the parent's emotional state. The input is the voice data, and the output is the emotion evaluation result.

[2057] Step 9:

[2058] The server preprocesses the text and image data it receives and converts it into a format for analysis. The input is text and image data, and the output is the data converted into the format for analysis. During this process, image resolution is adjusted and noise is removed.

[2059] Step 10:

[2060] An artificial intel...

Claims

1. means for receiving user-entered symptom data of the child; means for receiving an image of a child's symptoms taken by a user; means for implementing artificial intelligence to analyze the received symptom data and image data; A means for estimating the name of the disease, the need for outpatient treatment, and the expected number of days until recovery based on data analyzed by artificial intelligence; A means for providing user-friendly advice along with the estimation results; means for receiving feedback data from a user; A means for storing the received feedback data in a database and using it to retrain the artificial intelligence; A system including:

2. 10. The system of claim 1, further comprising means for performing appropriate account registration or login authentication.

3. 10. The system of claim 1, further comprising means for encrypting and securely transmitting received data.

Images