system

Abstract

We provide the system. [Solution] A means of obtaining information about lifestyle habits or health status via a user input section, A means of generating questions for medical professionals by performing analysis based on acquired information, The generated questions are presented to the user, with adjustable means, A means of sending the adjusted questions externally, A means of obtaining external feedback and incorporating it into future analyses, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is a problem that it is difficult for many users to ask appropriate questions in a medical institution. In particular, when lacking knowledge and experience on how to convey one's own lifestyle and health condition and what questions to ask based on that, the quality of medical consultations may decline. Furthermore, as a result, accurate diagnoses and appropriate treatment plans cannot be established, and the opportunity to receive high-quality medical services is restricted. It is required to improve such a situation and create an environment where users can effectively interact with medical experts.

Means for Solving the Problems

[0005] This invention provides a system that acquires information on a user's lifestyle and health status, analyzes it, and generates questions for medical professionals. Specifically, it includes means for performing analysis using data acquired through a user input unit, presenting the generated questions to the user, and allowing the user to adjust the questions. Furthermore, the system transmits the adjusted questions externally, obtains feedback from medical institutions, and incorporates it into the analysis to improve the quality of future medical consultations. By performing this series of processes, users can ask more effective questions at medical institutions, thereby promoting the benefit of high-quality medical services.

[0006] The "user input section" is an interface for users to input information about their lifestyle and health status.

[0007] "Analysis" is the process of processing information obtained from users to extract trends and problems.

[0008] A "means for generating questions" refers to a mechanism for automatically creating specific questions for medical professionals based on analysis results.

[0009] "Means of presentation" refers to technologies for displaying generated questions to the user visually or audibly.

[0010] "Feedback" refers to the answers and advice received from medical institutions, which are used to help improve the user's health.

[0011] "Adjustable means" refers to a feature that allows users to review the questions provided and make corrections or changes as needed.

[0012] "Encryption methods" refer to the process of encrypting information in order to securely send and receive data. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a tagged processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a tagged RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a tagged storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] The system according to the present invention is a support system for users to engage in effective dialogue with medical institutions. This system collects and analyzes information on the user's lifestyle and health status, and generates appropriate questions for medical professionals.

[0035] First, the user enters their health information using a device. This information includes their usual diet, exercise frequency, medical history, and current symptoms. The device sends this information to a server, which uses it as part of the data necessary for analysis.

[0036] The server analyzes the received data using machine learning algorithms. This assesses the user's current health status and identifies potential risks and concerns. Based on this analysis, it generates specific questions that should be asked of healthcare professionals.

[0037] The generated questions are sent back to the device and presented to the user visually. The user can review this list of questions and modify or add questions as needed. The modified questions are saved by the device and used when the user visits a healthcare facility.

[0038] After the consultation at the medical institution is complete, the user re-enters the feedback received into the device. The device sends this feedback to the server, which then undergoes multiple cycles to improve the question generation algorithm. Improved analysis makes it possible to generate more appropriate questions for future medical consultations.

[0039] As a concrete example, consider a case where a user diagnosed with hypertension provides daily salt intake, exercise time, and blood pressure measurement results as input data. Based on this data, the server generates questions such as, "Should I revise my diet to improve my hypertension?" or "Should I consider the side effects of my current medication?" The user then consults with a doctor based on this information and inputs the advice received into the system as feedback data.

[0040] Thus, this system supports users in appropriately and effectively collecting information at medical institutions and receiving high-quality medical consultations.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] Users use a device to input information about their lifestyle and health status. This information includes eating habits, exercise frequency, medical history, and current symptoms. The device verifies the entered data and prepares it in the appropriate format.

[0044] Step 2:

[0045] The terminal transmits the prepared user data to the server using a secure communication method. The information is transmitted according to an encryption protocol, ensuring the security of the data.

[0046] Step 3:

[0047] The server retrieves the received data and performs analysis using a data analysis module. This analysis assesses the user's current health status and identifies specific health risks and points of concern.

[0048] Step 4:

[0049] Based on the analysis results, the server uses a generation AI to generate questions for medical professionals. The generated questions are designed to be specific and tailored to the user's situation.

[0050] Step 5:

[0051] The server sends the generated list of questions to the terminal. The terminal receives the list of questions and displays it in a format that is easy for the user to understand intuitively.

[0052] Step 6:

[0053] The user reviews the question list on their device and modifies or adds questions as needed. This edited question list is then verified to meet the user's needs.

[0054] Step 7:

[0055] Users save the revised list of questions and use it when consulting with healthcare professionals. By presenting questions based on this list to medical professionals, efficient information gathering becomes possible.

[0056] Step 8:

[0057] The user inputs feedback received from the medical institution into the terminal. The terminal organizes the feedback data and prepares it for transmission to the server.

[0058] Step 9:

[0059] The device sends feedback data to the server. The server uses this feedback to update the training data to improve the question generation algorithm. This improves the accuracy of subsequent question generation.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] In communication with medical institutions, users face the challenge of being able to ask appropriate questions to healthcare professionals based on their health status and lifestyle. Furthermore, continuously improving the accuracy of data analysis is necessary to effectively utilize user feedback in future medical consultations. In addition, the secure transmission and reception of data, including personal information, is a crucial issue.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes means for acquiring lifestyle information or health status information via a user input device, means for performing analysis using a machine learning algorithm based on the acquired information to generate questions for experts, and means for presenting the generated questions to the user and making them adjustable. This enables the user to efficiently generate appropriate questions and conduct medical consultations more effectively. Furthermore, by saving the generated questions and outputting them as information that can be brought to external organizations, it is possible to facilitate the presentation of information at medical institutions. In addition, by including means for acquiring feedback information and reflecting it in improving the analysis algorithm, and information protection means for securely sending and receiving data, it is possible to improve the accuracy of question generation while ensuring the security of personal information.

[0065] A "user input device" is a device that provides an interface for users to input information about their lifestyle and health status.

[0066] "Lifestyle information" refers to information about a user's daily activities, such as their diet, exercise, and sleep.

[0067] "Health status information" refers to information about the user's physical or mental health status.

[0068] A "machine learning algorithm" is a computational method that performs predictions and classifications by analyzing data and learning patterns.

[0069] "Questions for Experts" refers to content intended for use by users in healthcare settings, representing questions and answers directed to medical professionals.

[0070] "Information protection measures" refer to technologies and methods used to ensure the secure transmission and reception of data.

[0071] "Feedback information" refers to information about responses received by users from medical institutions, such as advice and diagnostic results.

[0072] "Algorithm improvement" refers to updating or adjusting models or analysis methods to improve the accuracy and efficiency of the analysis.

[0073] Users input their lifestyle and health information using mobile devices or computers. This includes daily dietary habits, exercise levels, medical history, and changes in physical condition. The entered information is transmitted to the server via the device. During this process, the data is encrypted using the HTTPS protocol for secure transmission and reception.

[0074] The server analyzes the received information using machine learning algorithms. The analysis utilizes Python libraries such as TENSORFLOW® and scikit-learn to assess the user's health risks. In particular, it processes data to identify potential health risks that the user may not be aware of, and to address important questions that may arise during medical consultations.

[0075] Based on the analysis results, the server generates questions for experts using a generative AI model. Natural language processing technology is used for generation to produce specific and highly specialized questions. For example, technologies such as GPT-3 (registered trademark) are used as the generative AI model.

[0076] The generated questions are sent back to the terminal and presented to the user. The user can review the displayed questions, add any new questions they need, or modify any questions that have already been generated. The completed question list can be output in PDF format or other formats, and can be printed out and brought along when visiting medical institutions.

[0077] After a consultation at a medical institution is completed, the user enters the feedback received into their device. This feedback is then sent back to the server and used to improve the question generation algorithm. As a new question generation model is built, it becomes possible to provide even more accurate questions during future medical consultations.

[0078] As a concrete example, consider a case where a user diagnosed with hypertension inputs their diet and weekly exercise time, and based on that, the system automatically generates a question asking, "Should I seek specific dietary advice to manage my hypertension?" As part of this process, the prompt would be designed to read, "The user has been diagnosed with hypertension. Based on their daily salt intake, exercise time, and blood pressure readings, please generate questions that they should ask a professional."

[0079] This system allows users to communicate effectively with healthcare providers and improve the quality of medical consultations.

[0080] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0081] Step 1:

[0082] Users input health-related information using a mobile device or computer. This input includes data on diet, exercise levels, medical history, and current symptoms. This input data is saved to the device via the application interface. This allows users to record detailed information about their own health status.

[0083] Step 2:

[0084] The terminal encrypts health-related information entered by the user and sends it to the server using the HTTPS protocol. Encryption of the input data ensures information security during transmission. The server stores the received information in a database for processing. This step ensures secure data transfer and storage.

[0085] Step 3:

[0086] The server applies machine learning algorithms to the received health-related information and analyzes the data. Specific analysis includes assessing the user's health risks and identifying symptoms that require attention. The analysis is performed using libraries such as TensorFlow and scikit-learn, providing outputs to evaluate the user's health status and identify potential risks. The output obtained in this process is then used to generate questions in the next step.

[0087] Step 4:

[0088] The server uses a generative AI model based on the analysis results to generate questions for experts. The generative AI model employs natural language processing technology. Upon input of a prompt, specific and relevant questions tailored to the user's health condition are generated. The generated questions provide important information for the user to use in medical settings.

[0089] Step 5:

[0090] The terminal receives pre-generated questions sent from the server and presents them visually to the user. The user can review the questions on the screen and modify or add to them as needed. This process is important for the user to clearly communicate information in a healthcare setting. The completed questions are listed and output to the terminal.

[0091] Step 6:

[0092] After a user visits a medical institution and completes a consultation with a doctor, they input the resulting feedback into a terminal. The input feedback data is then sent back to the server by the terminal. The feedback obtained in this step will contribute to improving the accuracy of future analyses.

[0093] Step 7:

[0094] The server analyzes the feedback sent by the user and uses it to improve the question generation algorithm. This process involves learning from new data to increase generation accuracy. The questions generated for the user's next medical consultation will be based on the improved algorithm, resulting in more appropriate and helpful questions.

[0095] (Application Example 1)

[0096] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0097] In modern times, individuals often lack sufficient information when selecting products and services tailored to their specific health needs. This makes it difficult for them to find the optimal options based on their own health status and lifestyle. Furthermore, there is a lack of support in generating appropriate questions and facilitating meaningful information exchange during conversations with health professionals.

[0098] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0099] In this invention, the server includes means for acquiring information on lifestyle or health status via a user input unit, means for performing analysis based on the acquired information and generating questions for a third party, and means for recommending products or services based on the information. This makes it possible to generate appropriate questions and suggest products or services tailored to each individual's health condition.

[0100] The "user input section" is an interface for obtaining information about lifestyle habits or health status.

[0101] "Means of analysis" refers to a function that analyzes data based on acquired information and generates questions necessary for dialogue with a third party.

[0102] "Methods for generating questions" refers to the process of creating specific questions that a user should ask a third party, based on the analysis results.

[0103] "Means for presenting generated questions to the user" refers to a function that allows users to review and adjust automatically generated questions.

[0104] "Means of sending questions externally" refers to the process of sending a prepared question to an external service or third party.

[0105] "Means for obtaining feedback" refers to a function that receives responses and evaluations from external sources and uses them to inform future data analysis.

[0106] "Means of recommending products or services based on information" refers to a function that suggests the most suitable products or services according to the individual health data acquired.

[0107] "Encryption methods" are technologies used to encrypt information in order to communicate data securely and enable secure transmission and reception.

[0108] This system begins by utilizing a user input field to acquire data on the user's lifestyle and health status. Users input information such as their diet, exercise frequency, and current health status through a smartphone application. This information is sent from the device to a server for advanced data analysis. The analysis utilizes the Django framework using Python and Scikit-learn, a tool specifically designed for data analysis. This process lays the foundation for comprehensively evaluating the user's health status and recommending health-related services and products.

[0109] Next, the server generates specific questions that the user should ask a third party, such as store staff or a health professional, via a generative AI model. GPT-3 is used for question generation to support effective dialogue. The generated questions and suggested products and services are then displayed again on the smartphone, allowing the user to make product selections based on that information.

[0110] In this process, an example of a prompt might be, "Generate questions for a user who is aiming to improve their hypertension and has a history of high salt intake." Specifically, when a user enters data, the system might generate and present questions such as, "Are health foods rich in omega-3 fatty acids good?" or "Which supplements are best for reducing stress?" This allows users to make more informed decisions when choosing health-related products and services.

[0111] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0112] Step 1:

[0113] Users input information about their lifestyle and health status using a smartphone application. This includes diet, exercise frequency, medical history, and current symptoms. The entered data is immediately stored in the app and prepared for transmission to the server.

[0114] Step 2:

[0115] The terminal sends the information entered by the user to the server. Here, the data is encrypted, and protocols are configured to ensure secure transmission. The input data is stored on the server as foundational data for analysis.

[0116] Step 3:

[0117] The server uses Scikit-learn to analyze the received information. It comprehensively assesses the user's health status and identifies potential health risks and concerns. This assessment result is stored as internal data for use in the next step.

[0118] Step 4:

[0119] The server generates questions using GPT-3 based on the analysis results. This question generation process creates appropriate questions based on the user's health status and risks, and the content follows criteria defined as prompt statements. These generated questions are later presented to the user visually.

[0120] Step 5:

[0121] The generated questions and recommendations for products and services based on the analysis results are sent back to the device. Users receive these questions and recommendations through the app and can select the products and services that best suit their needs.

[0122] Step 6:

[0123] The user interacts with a third party based on the questions and recommendations received. After the interaction, the user inputs the acquired feedback information into the app and sends it back to the server. This feedback is used to improve the accuracy of the question generation algorithm.

[0124] Step 7:

[0125] The server receives feedback and uses it to improve the GPT-3 model. This allows for more appropriate output in subsequent question generation processes, accelerating the evolution of the user experience and improving the overall effectiveness of the system.

[0126] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0127] The system according to the present invention is an advanced system that supports dialogue with medical institutions by considering not only the user's lifestyle and health condition but also their emotional state. This system facilitates smooth communication with medical professionals through the acquisition, analysis, question generation, and adjustment of input and output of user input data.

[0128] First, the user inputs information about their health status, lifestyle, and emotions via the device. This emotional information is obtained through the user's facial expressions, tone of voice, and self-reporting. The device then organizes this data and prepares it for transmission to the server.

[0129] The server uses the received data to perform analysis to understand the user's overall profile. This analysis includes analyzing the user's emotions. The emotion engine recognizes the user's current emotional state and extracts factors such as stress, joy, and anxiety. By considering the user's emotions, the question generation process is personalized, enabling the creation of flexible questions that are tailored to the user's psychological state.

[0130] The generated questions are adjusted to suit the user's emotional state and sent to the device. The device displays the questions in a way that allows the user to easily understand and edit them as needed. By using these adjusted questions in a healthcare setting, users can have more effective consultations.

[0131] After a consultation with a medical institution, the user inputs the feedback received into their device and sends it to the server. The server analyzes this feedback to further improve the question generation algorithm and emotion engine. For example, if a user experiencing stress self-reports "what has been bothering them lately," the system can generate health consultation questions that take that emotional state into account. In this way, this system, which also responds to the user's emotions, can evolve conversations at medical institutions into something deeper and more personal.

[0132] The following describes the processing flow.

[0133] Step 1:

[0134] Users use the device to input lifestyle, health, and emotional information. Emotional information is automatically acquired through facial recognition or voice analysis, or it is entered by the user through self-reporting. The device appropriately formats this information and prepares it for transmission.

[0135] Step 2:

[0136] The device sends the collected data to the server. This transmission process uses encryption to ensure data security.

[0137] Step 3:

[0138] The server analyzes the received data to assess the user's health status and lifestyle, while simultaneously analyzing their emotional state using an emotion engine. The emotion engine detects stress, anxiety, joy, etc., and provides information to understand how these affect their health.

[0139] Step 4:

[0140] Based on the analysis results, the server generates specific and adaptive questions for medical professionals, taking into account the user's emotional state. This question generation process employs an approach that takes into account the user's psychological state.

[0141] Step 5:

[0142] The server sends a generated list of questions to the terminal. The terminal displays the questions so that the user can easily understand them and adjust them as needed.

[0143] Step 6:

[0144] Users can review the questions presented on their device and modify or add questions as needed. The user's edited list of questions is saved and ready for use during consultations at healthcare facilities.

[0145] Step 7:

[0146] After consulting with a medical institution, the user inputs the feedback received into their device and sends the feedback to the server.

[0147] Step 8:

[0148] The server analyzes the feedback and uses it to improve the question generation algorithm and sentiment engine. This allows the system to improve the accuracy and responsiveness of future user inquiries.

[0149] (Example 2)

[0150] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0151] Traditional medical consultation systems focused on information based on the user's lifestyle and health status, making it difficult to consider the user's emotional state. As a result, the user's stress, anxiety, and other emotions were not adequately reflected in the consultation, leading to insufficient communication with medical professionals.

[0152] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0153] In this invention, the server includes means for acquiring information on the user's lifestyle, health status, and emotions; means for analyzing the user's emotional state and generating questions for medical professionals; and means for adjusting the questions to suit the user's emotional state using a generative AI model. This enables deeper and more personal communication with medical professionals by considering the user's emotions and generating personalized questions.

[0154] A "user" refers to an individual who uses the system to input information about their lifestyle, health status, and emotions.

[0155] "Lifestyle habits" refers to information about a user's daily actions and habits. This includes things like diet, exercise, and sleep.

[0156] "Health status" refers to information about the user's physical health. This includes medical history, current illnesses, and overall physical condition.

[0157] "Emotional information" refers to data about the user's emotional state. This includes information obtained through facial expressions, tone of voice, and self-reporting.

[0158] "Analysis" refers to evaluating and analyzing a user's lifestyle, health status, and emotional state based on the information acquired.

[0159] "Question generation" refers to the process of formulating questions for medical professionals using analysis results.

[0160] A "generative AI model" refers to a computer program that uses artificial intelligence technology to extract useful information from data and generate appropriate questions.

[0161] "Adjustment" refers to modifying the generated questions so that they are in a form that is appropriate to the user's emotional state.

[0162] "Feedback" refers to the act of returning the results and answers obtained from the user's opinions and advice received at a medical institution to the system.

[0163] "Encryption" refers to a technology that transforms information in a specific way to securely transfer data and prevent unauthorized access.

[0164] "Medical institutions" refer to external organizations or professionals that provide medical services to users.

[0165] This invention is a system in which a user inputs information about their health status, lifestyle, and emotions via a terminal, and a server analyzes this data to generate questions useful for medical consultation. This system places particular emphasis on emotional information, enabling the generation of questions that take into account the user's emotional state.

[0166] Users input health-related information using devices such as smartphones and personal computers. This information includes not only lifestyle habits and health status, but also real-time emotional information of the user. Emotional information is obtained by analyzing facial expressions and voice tone using the camera and microphone built into the device. Users can also input their emotions through self-reporting.

[0167] The terminal organizes the acquired information and sends it to the server using encryption technology. Data security is ensured by using commonly used encryption protocols (e.g., TLS).

[0168] The server analyzes the user's health and emotional state based on the received data. An emotion analysis engine is used to identify emotional factors such as stress, joy, and anxiety that the user is experiencing. Based on the analysis results, a generative AI model generates questions for medical professionals that are tailored to the user's emotions. This enables flexible responses that are appropriate to the user's psychological state.

[0169] For example, the prompt message to the generating AI model could include instructions such as, "The user is feeling anxious. Please generate questions that address this state." This allows the system to provide consultation content optimized for the user's current situation. Based on this prompt message, the model generates appropriate questions and sends them from the server to the terminal.

[0170] The terminal presents the user with generated questions and allows the user to edit them as needed, creating a set of questions that the user can use when consulting with a medical institution. The feedback received after the consultation is then sent back to the server, and the system continues to improve its question generation algorithm and emotion engine based on this feedback. In this way, the system evolves with each use, enabling more personalized and effective medical consultations.

[0171] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0172] Step 1:

[0173] Users input health status, lifestyle habits, and emotional information using a device. The input data includes information directly entered by the user, as well as information from cameras and microphones that analyze facial expressions and voice tone. This allows for the acquisition of both the user's subjective state and objective emotional information.

[0174] Step 2:

[0175] The terminal organizes the acquired data and prepares it for transmission to the server using encryption technology. The input data is categorized as lifestyle, health status, and emotional information, and sent to the server after ensuring data reliability through encryption protocols such as TLS.

[0176] Step 3:

[0177] The server analyzes the data received from the terminal. Based on the input data, the emotion analysis engine identifies the user's emotional state (e.g., stress, anxiety, joy). As a result of the analysis, an overall health status and emotional profile of the user is generated.

[0178] Step 4:

[0179] The server uses a generative AI model based on the analysis results to generate questions that take into account the user's emotional state. The AI ​​model is given a prompt such as, "The user is feeling anxious. Generate questions that address this state," and an appropriate set of questions is generated.

[0180] Step 5:

[0181] The generated questions are sent from the server to the terminal. The terminal displays the questions to the user, helping them understand the questions and edit them as needed. The output here is a visualized list of questions provided to the user visually.

[0182] Step 6:

[0183] Users consult using questions generated by the medical institution. They then input the feedback they receive into their device. This feedback includes the medical professional's opinion based on the consultation and the user's response.

[0184] Step 7:

[0185] The terminal organizes the feedback data and sends it to the server. The server analyzes the feedback and uses it to improve the question generation algorithm and sentiment analysis engine. This process improves the accuracy and effectiveness of the system and prepares it for future use.

[0186] (Application Example 2)

[0187] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0188] Traditional service delivery systems based on users' lifestyles and health conditions suffer from insufficient personalization. They are unable to generate suggestions or questions that take into account the user's emotional state, thus failing to achieve adequate personalization. In particular, in the in-store shopping experience, product suggestions are not tailored to the user's current emotions or stress levels, making it difficult to improve customer satisfaction.

[0189] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0190] In this invention, the server includes means for acquiring information on lifestyle, health status, or emotional state via user input means, means for performing analysis based on the acquired information and generating questions or suggestions for experts, and means for presenting the generated questions or suggestions to the user and making them adjustable. This makes it possible to provide personalized product suggestions and questions in real time that are tailored to the user's emotions and health status.

[0191] "User input means" refers to a device or interface for receiving information from a user regarding their lifestyle, health status, or emotional state.

[0192] "Lifestyle habits" refer to the actions and habits that users perform on a daily basis, and include things like eating, exercise, and sleep patterns.

[0193] "Health status" is a concept that describes the overall physical and mental health of a user, and is often judged based on medical evaluations.

[0194] "Emotional state" refers to the emotions and moods a user feels at a particular time, and includes states such as stress, joy, and anxiety.

[0195] "Analysis means" refers to software or hardware devices used to analyze acquired information and extract meaning.

[0196] "Means for generating questions or suggestions" refers to a system for creating questions or product suggestions for users based on analyzed information.

[0197] "Adjustable means" refers to an interface or function that allows users to re-edit or modify generated questions or suggestions.

[0198] "Feedback" refers to information such as responses and opinions received from users, which can be used to improve the system.

[0199] The system implementing this invention includes a terminal used by the user, a server that analyzes the data, and an interface for displaying and adjusting the generated suggestions and questions. The user inputs information about their lifestyle, health status, and emotional state using a terminal such as a smartphone or smart glasses. The terminal transmits this information to the server.

[0200] The server uses an emotion analysis engine to analyze the received data. Specific examples include APIs from Microsoft® Azure® and IBM Watson®. It analyzes the user's emotional state, understanding conditions such as stress, joy, and anxiety. Based on the analyzed information, it utilizes an AI model to generate helpful questions or product suggestions for the user. The generating AI model provides personalized suggestions that take into account the user's health and emotional state. The generated questions and suggestions are sent to and displayed on the user's device.

[0201] Users can review the displayed questions or suggestions and make adjustments as needed. This adjusted information is then sent back to the server and stored as data. Based on the feedback, the server improves its generation algorithms and sentiment analysis engine, enabling more accurate suggestions.

[0202] As a concrete example, if stress is detected via smart glasses while a user is shopping in a store, relaxation-related product suggestions will be automatically displayed. An example of a prompt message to the generating AI model might be: "Consider the user's emotional state and generate a list of products best suited for relaxation. The user is experiencing stress, and recent health data includes: insufficient sleep and lack of exercise." This allows for specific and practical suggestions tailored to each user's individual needs.

[0203] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0204] Step 1:

[0205] Users input information about their lifestyle, health status, and emotional state using a device. The input information undergoes initial data formatting within the device and is then prepared for transmission to the server. Input can be in the form of text, audio, or images, and output is organized digital data.

[0206] Step 2:

[0207] The terminal sends the organized data to the server. The transmitted data is securely delivered to the server using a secure communication protocol. The input is formatted digital data, and the output is stored as a dataset on the server.

[0208] Step 3:

[0209] The server analyzes the received data and uses an emotion analysis engine to identify the user's emotional state. The input is a dataset stored on the server, and the output is the emotion analysis result. This process utilizes natural language processing and speech analysis techniques.

[0210] Step 4:

[0211] The server uses a generative AI model based on the analysis results to generate questions and product suggestions tailored to the user. The input is the sentiment analysis results, and the output is the generated questions or product suggestions. Prompt sentences are created in this step, serving as instructions for the AI.

[0212] Step 5:

[0213] The server sends the generated questions and suggestions to the user's terminal. The input is the generated questions and suggestions, and the output is what is displayed on the terminal. Communication is secure as it is performed using encryption technology.

[0214] Step 6:

[0215] The user reviews the questions and suggestions displayed on the terminal and makes adjustments as needed. The input is the user's judgment and adjustments, and the output is the adjusted data. The terminal then sends the adjusted data back to the server.

[0216] Step 7:

[0217] The server receives user feedback and uses it to improve its generation algorithms and sentiment analysis engine. The input is user feedback, and the output is data that contributes to improving the overall accuracy of the system. The improved algorithms are then reflected in subsequent analyses and generation processes.

[0218] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0219] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0220] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0221] [Second Embodiment]

[0222] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0223] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0224] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0225] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0226] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0227] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0228] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0229] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0230] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0231] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0232] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0233] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0234] The system according to the present invention is a support system for users to engage in effective dialogue with medical institutions. This system collects and analyzes information on the user's lifestyle and health status, and generates appropriate questions for medical professionals.

[0235] First, the user enters their health information using a device. This information includes their usual diet, exercise frequency, medical history, and current symptoms. The device sends this information to a server, which uses it as part of the data necessary for analysis.

[0236] The server analyzes the received data using machine learning algorithms. This assesses the user's current health status and identifies potential risks and concerns. Based on this analysis, it generates specific questions that should be asked of healthcare professionals.

[0237] The generated questions are sent back to the device and presented to the user visually. The user can review this list of questions and modify or add questions as needed. The modified questions are saved by the device and used when the user visits a healthcare facility.

[0238] After the consultation at the medical institution is complete, the user re-enters the feedback received into the device. The device sends this feedback to the server, which then undergoes multiple cycles to improve the question generation algorithm. Improved analysis makes it possible to generate more appropriate questions for future medical consultations.

[0239] As a concrete example, consider a case where a user diagnosed with hypertension provides daily salt intake, exercise time, and blood pressure measurement results as input data. Based on this data, the server generates questions such as, "Should I revise my diet to improve my hypertension?" or "Should I consider the side effects of my current medication?" The user then consults with a doctor based on this information and inputs the advice received into the system as feedback data.

[0240] Thus, this system supports users in appropriately and effectively collecting information at medical institutions and receiving high-quality medical consultations.

[0241] The following describes the processing flow.

[0242] Step 1:

[0243] Users use a device to input information about their lifestyle and health status. This information includes eating habits, exercise frequency, medical history, and current symptoms. The device verifies the entered data and prepares it in the appropriate format.

[0244] Step 2:

[0245] The terminal transmits the prepared user data to the server using a secure communication method. The information is transmitted according to an encryption protocol, ensuring the security of the data.

[0246] Step 3:

[0247] The server retrieves the received data and performs analysis using a data analysis module. This analysis assesses the user's current health status and identifies specific health risks and points of concern.

[0248] Step 4:

[0249] Based on the analysis results, the server uses a generation AI to generate questions for medical professionals. The generated questions are designed to be specific and tailored to the user's situation.

[0250] Step 5:

[0251] The server sends the generated list of questions to the terminal. The terminal receives the list of questions and displays it in a format that is easy for the user to understand intuitively.

[0252] Step 6:

[0253] The user reviews the question list on their device and modifies or adds questions as needed. This edited question list is then verified to meet the user's needs.

[0254] Step 7:

[0255] Users save the revised list of questions and use it when consulting with healthcare professionals. By presenting questions based on this list to medical professionals, efficient information gathering becomes possible.

[0256] Step 8:

[0257] The user inputs feedback received from the medical institution into the terminal. The terminal organizes the feedback data and prepares it for transmission to the server.

[0258] Step 9:

[0259] The device sends feedback data to the server. The server uses this feedback to update the training data to improve the question generation algorithm. This improves the accuracy of subsequent question generation.

[0260] (Example 1)

[0261] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0262] In communication with medical institutions, users face the challenge of being able to ask appropriate questions to healthcare professionals based on their health status and lifestyle. Furthermore, continuously improving the accuracy of data analysis is necessary to effectively utilize user feedback in future medical consultations. In addition, the secure transmission and reception of data, including personal information, is a crucial issue.

[0263] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0264] In this invention, the server includes means for acquiring lifestyle information or health status information via a user input device, means for performing analysis using a machine learning algorithm based on the acquired information to generate questions for experts, and means for presenting the generated questions to the user and making them adjustable. This enables the user to efficiently generate appropriate questions and conduct medical consultations more effectively. Furthermore, by saving the generated questions and outputting them as information that can be brought to external organizations, it is possible to facilitate the presentation of information at medical institutions. In addition, by including means for acquiring feedback information and reflecting it in improving the analysis algorithm, and information protection means for securely sending and receiving data, it is possible to improve the accuracy of question generation while ensuring the security of personal information.

[0265] A "user input device" is a device that provides an interface for users to input information about their lifestyle and health status.

[0266] "Lifestyle information" refers to information about a user's daily activities, such as their diet, exercise, and sleep.

[0267] "Health status information" refers to information about the user's physical or mental health status.

[0268] A "machine learning algorithm" is a computational method that performs predictions and classifications by analyzing data and learning patterns.

[0269] "Questions for Experts" refers to content intended for use by users in healthcare settings, representing questions and answers directed to medical professionals.

[0270] "Information protection measures" refer to technologies and methods used to ensure the secure transmission and reception of data.

[0271] "Feedback information" refers to information about responses received by users from medical institutions, such as advice and diagnostic results.

[0272] "Algorithm improvement" refers to updating or adjusting models or analysis methods to improve the accuracy and efficiency of the analysis.

[0273] Users input their lifestyle and health information using mobile devices or computers. This includes daily dietary habits, exercise levels, medical history, and changes in physical condition. The entered information is transmitted to the server via the device. During this process, the data is encrypted using the HTTPS protocol for secure transmission and reception.

[0274] The server analyzes the received information using machine learning algorithms. This analysis utilizes Python libraries such as TensorFlow and scikit-learn to assess the user's health risks. In particular, it processes data to identify potential health risks that the user may not be aware of, and to address important questions that may arise during medical consultations.

[0275] Based on the analysis results, the server generates questions for experts using a generative AI model. Natural language processing techniques are employed to generate specific and highly specialized questions. For example, technologies such as GPT-3 are used as the generative AI model.

[0276] The generated questions are sent back to the terminal and presented to the user. The user can review the displayed questions, add any new questions they need, or modify any questions that have already been generated. The completed question list can be output in PDF format or other formats, and can be printed out and brought along when visiting medical institutions.

[0277] After a consultation at a medical institution is completed, the user enters the feedback received into their device. This feedback is then sent back to the server and used to improve the question generation algorithm. As a new question generation model is built, it becomes possible to provide even more accurate questions during future medical consultations.

[0278] As a specific example, consider a case where a user who has received a hypertension diagnosis inputs their dietary content and weekly exercise time, and based on this, automatically generates a question "Should I seek specific dietary advice for managing hypertension?". As part of this process, the prompt text is designed to be something like "The user has been diagnosed with hypertension. Based on the daily salt intake, exercise time, and blood pressure measurement results, please generate a question to ask an expert."

[0279] With this system, the user can effectively communicate at a medical institution and improve the quality of medical consultations.

[0280] The flow of the specific process in Example 1 will be described using FIG. 11.

[0281] Step 1:

[0282] The user inputs health-related information using a mobile terminal or a computer. The input content includes data related to dietary content, exercise amount, past history, and current symptoms. These input data are stored in the terminal via an application interface. Thereby, the user can record detailed information about their health status.

[0283] Step 2:

[0284] The terminal encrypts the health-related information input by the user and sends it to the server using the HTTPS protocol. By encrypting the input data, the information security during the transmission process is ensured. The server stores the received information in a database for processing. In this step, secure data transfer and storage are achieved.

[0285] Step 3:

[0286] The server applies a machine learning algorithm based on the received health-related information and analyzes the data. Specific analyses include the user's health risk assessment and the identification of symptoms that require attention. The analysis is performed using libraries such as TensorFlow and scikit-learn, providing outputs for evaluating the user's health status and identifying potential risks. The output obtained in this process is utilized for question generation in the next step.

[0287] Step 4:

[0288] Based on the analysis results, the server utilizes a generative AI model to generate questions for experts. As the generative AI model, a model using natural language processing technology is employed. By inputting a prompt sentence, specific and highly relevant questions according to the user's health status are generated. The output questions are important information for the user to use at a medical institution.

[0289] Step 5:

[0290] The terminal receives the generated questions sent from the server and visually presents them to the user. The user can check the question content on the screen and make corrections or additions if necessary. This operation is important for the user to clearly convey information at a medical institution. The completed questions are listed and output within the terminal.

[0291] Step 6:

[0292] After the user visits a medical institution and consults with a doctor, the feedback obtained as a result is input into the terminal. The input feedback data is sent back to the server by the terminal. The feedback obtained in this step leads to an improvement in future analysis accuracy.

[0293] Step 7:

[0294] The server analyzes the feedback sent by the user and uses it to improve the question generation algorithm. This process involves learning from new data to increase generation accuracy. The questions generated for the user's next medical consultation will be based on the improved algorithm, resulting in more appropriate and helpful questions.

[0295] (Application Example 1)

[0296] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0297] In modern times, individuals often lack sufficient information when selecting products and services tailored to their specific health needs. This makes it difficult for them to find the optimal options based on their own health status and lifestyle. Furthermore, there is a lack of support in generating appropriate questions and facilitating meaningful information exchange during conversations with health professionals.

[0298] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0299] In this invention, the server includes means for acquiring information on lifestyle or health status via a user input unit, means for performing analysis based on the acquired information and generating questions for a third party, and means for recommending products or services based on the information. This makes it possible to generate appropriate questions and suggest products or services tailored to each individual's health condition.

[0300] The "user input section" is an interface for obtaining information about lifestyle habits or health status.

[0301] "Means of analysis" refers to a function that analyzes data based on acquired information and generates questions necessary for dialogue with a third party.

[0302] The "means for generating questions" is a process of creating specific questions that a user should ask a third party based on the analysis results.

[0303] The "means for presenting the generated questions to the user" is a function for providing the automatically generated questions so that the user can confirm and adjust them.

[0304] The "means for sending questions externally" is a process for sending the adjusted questions to an external service or a third party.

[0305] The "means for obtaining feedback" is a function for receiving answers and evaluations returned from outside and making use of them in future data analysis.

[0306] The "means for recommending products or services based on information" is a function for proposing optimal products and services according to the acquired individual health data.

[0307] The "encryption means" is a technology for encrypting information to communicate data securely and perform secure transmission and reception.

[0308] This system starts by using the user input part to acquire data on the user's lifestyle and health status. The user inputs their diet content, exercise frequency, current health status, etc. through a smartphone application. This information is sent from the terminal to the server, and advanced data analysis is performed. For the analysis, the Django framework using Python and Scikit-learn specialized in data analysis are used. Through this process, a basis for comprehensively evaluating the user's health status and recommending health-related services and products is built.

[0309] Next, the server generates specific questions that the user should ask a third party, such as store staff or a health professional, via a generative AI model. GPT-3 is used for question generation to support effective dialogue. The generated questions and suggested products and services are then displayed again on the smartphone, allowing the user to make product selections based on that information.

[0310] In this process, an example of a prompt might be, "Generate questions for a user who is aiming to improve their hypertension and has a history of high salt intake." Specifically, when a user enters data, the system might generate and present questions such as, "Are health foods rich in omega-3 fatty acids good?" or "Which supplements are best for reducing stress?" This allows users to make more informed decisions when selecting health-related products and services.

[0311] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0312] Step 1:

[0313] Users input information about their lifestyle and health status using a smartphone application. This includes diet, exercise frequency, medical history, and current symptoms. The entered data is immediately stored in the app and prepared for transmission to the server.

[0314] Step 2:

[0315] The terminal sends the information entered by the user to the server. Here, the data is encrypted, and protocols are configured to ensure secure transmission. The input data is stored on the server as foundational data for analysis.

[0316] Step 3:

[0317] The server uses Scikit-learn to analyze the received information. It comprehensively assesses the user's health status and identifies potential health risks and concerns. This assessment result is stored as internal data for use in the next step.

[0318] Step 4:

[0319] The server generates questions using GPT-3 based on the analysis results. This question generation process creates appropriate questions based on the user's health status and risks, and the content follows criteria defined as prompt statements. These generated questions are later presented to the user visually.

[0320] Step 5:

[0321] The generated questions and recommendations for products and services based on the analysis results are sent back to the device. Users receive these questions and recommendations through the app and can select the products and services that best suit their needs.

[0322] Step 6:

[0323] The user interacts with a third party based on the questions and recommendations received. After the interaction, the user inputs the acquired feedback information into the app and sends it back to the server. This feedback is used to improve the accuracy of the question generation algorithm.

[0324] Step 7:

[0325] The server receives feedback and uses it to improve the GPT-3 model. This allows for more appropriate output in subsequent question generation processes, accelerating the evolution of the user experience and improving the overall effectiveness of the system.

[0326] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0327] The system according to the present invention is an advanced system that supports dialogue with medical institutions by considering not only the user's lifestyle and health condition but also their emotional state. This system facilitates smooth communication with medical professionals through the acquisition, analysis, question generation, and adjustment of input and output of user input data.

[0328] First, the user inputs information about their health status, lifestyle, and emotions via the device. This emotional information is obtained through the user's facial expressions, tone of voice, and self-reporting. The device then organizes this data and prepares it for transmission to the server.

[0329] The server uses the received data to perform analysis to understand the user's overall profile. This analysis includes analyzing the user's emotions. The emotion engine recognizes the user's current emotional state and extracts factors such as stress, joy, and anxiety. By considering the user's emotions, the question generation process is personalized, enabling the creation of flexible questions that are tailored to the user's psychological state.

[0330] The generated questions are adjusted to suit the user's emotional state and sent to the device. The device displays the questions in a way that allows the user to easily understand and edit them as needed. By using these adjusted questions in a healthcare setting, users can have more effective consultations.

[0331] After a consultation with a medical institution, the user inputs the feedback received into their device and sends it to the server. The server analyzes this feedback to further improve the question generation algorithm and emotion engine. For example, if a user experiencing stress self-reports "what has been bothering them lately," the system can generate health consultation questions that take that emotional state into account. In this way, this system, which also responds to the user's emotions, can evolve conversations at medical institutions into something deeper and more personal.

[0332] The following describes the processing flow.

[0333] Step 1:

[0334] Users use the device to input lifestyle, health, and emotional information. Emotional information is automatically acquired through facial recognition or voice analysis, or it is entered by the user through self-reporting. The device appropriately formats this information and prepares it for transmission.

[0335] Step 2:

[0336] The device sends the collected data to the server. This transmission process uses encryption to ensure data security.

[0337] Step 3:

[0338] The server analyzes the received data to assess the user's health status and lifestyle, while simultaneously analyzing their emotional state using an emotion engine. The emotion engine detects stress, anxiety, joy, etc., and provides information to understand how these affect their health.

[0339] Step 4:

[0340] Based on the analysis results, the server generates specific and adaptive questions for medical professionals, taking into account the user's emotional state. This question generation process employs an approach that takes into account the user's psychological state.

[0341] Step 5:

[0342] The server sends a generated list of questions to the terminal. The terminal displays the questions so that the user can easily understand them and adjust them as needed.

[0343] Step 6:

[0344] Users can review the questions presented on their device and modify or add questions as needed. The user's edited list of questions is saved and ready for use during consultations at healthcare facilities.

[0345] Step 7:

[0346] After consulting with a medical institution, the user inputs the feedback received into their device and sends the feedback to the server.

[0347] Step 8:

[0348] The server analyzes the feedback and uses it to improve the question generation algorithm and sentiment engine. This allows the system to improve the accuracy and responsiveness of future user inquiries.

[0349] (Example 2)

[0350] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0351] Traditional medical consultation systems focused on information based on the user's lifestyle and health status, making it difficult to consider the user's emotional state. As a result, the user's stress, anxiety, and other emotions were not adequately reflected in the consultation, leading to insufficient communication with medical professionals.

[0352] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0353] In this invention, the server includes means for acquiring information on the user's lifestyle, health status, and emotions; means for analyzing the user's emotional state and generating questions for medical professionals; and means for adjusting the questions to suit the user's emotional state using a generative AI model. This enables deeper and more personal communication with medical professionals by considering the user's emotions and generating personalized questions.

[0354] A "user" refers to an individual who uses the system to input information about their lifestyle, health status, and emotions.

[0355] "Lifestyle habits" refers to information about a user's daily actions and habits. This includes things like diet, exercise, and sleep.

[0356] "Health status" refers to information about the user's physical health. This includes medical history, current illnesses, and overall physical condition.

[0357] "Emotional information" refers to data about the user's emotional state. This includes information obtained through facial expressions, tone of voice, and self-reporting.

[0358] "Analysis" refers to evaluating and analyzing a user's lifestyle, health status, and emotional state based on the information acquired.

[0359] "Question generation" refers to the process of formulating questions for medical professionals using analysis results.

[0360] A "generative AI model" refers to a computer program that uses artificial intelligence technology to extract useful information from data and generate appropriate questions.

[0361] "Adjustment" refers to modifying the generated questions so that they are in a form that is appropriate to the user's emotional state.

[0362] "Feedback" refers to the act of returning the results and answers obtained from the user's opinions and advice received at a medical institution to the system.

[0363] "Encryption" refers to a technology that transforms information in a specific way to securely transfer data and prevent unauthorized access.

[0364] "Medical institutions" refer to external organizations or professionals that provide medical services to users.

[0365] This invention is a system in which a user inputs information about their health status, lifestyle, and emotions via a terminal, and a server analyzes this data to generate questions useful for medical consultation. This system places particular emphasis on emotional information, enabling the generation of questions that take into account the user's emotional state.

[0366] Users input health-related information using devices such as smartphones and personal computers. This information includes not only lifestyle habits and health status, but also real-time emotional information of the user. Emotional information is obtained by analyzing facial expressions and voice tone using the camera and microphone built into the device. Users can also input their emotions through self-reporting.

[0367] The terminal organizes the acquired information and sends it to the server using encryption technology. Data security is ensured by using commonly used encryption protocols (e.g., TLS).

[0368] The server analyzes the user's health and emotional state based on the received data. An emotion analysis engine is used to identify emotional factors such as stress, joy, and anxiety that the user is experiencing. Based on the analysis results, a generative AI model generates questions for medical professionals that are tailored to the user's emotions. This enables flexible responses that are appropriate to the user's psychological state.

[0369] For example, the prompt message to the generating AI model could include instructions such as, "The user is feeling anxious. Please generate questions that address this state." This allows the system to provide consultation content optimized for the user's current situation. Based on this prompt message, the model generates appropriate questions and sends them from the server to the terminal.

[0370] The terminal presents the user with generated questions and allows the user to edit them as needed, creating a set of questions that the user can use when consulting with a medical institution. The feedback received after the consultation is then sent back to the server, and the system continues to improve its question generation algorithm and emotion engine based on this feedback. In this way, the system evolves with each use, enabling more personalized and effective medical consultations.

[0371] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0372] Step 1:

[0373] Users input health status, lifestyle habits, and emotional information using a device. The input data includes information directly entered by the user, as well as information from cameras and microphones that analyze facial expressions and voice tone. This allows for the acquisition of both the user's subjective state and objective emotional information.

[0374] Step 2:

[0375] The terminal organizes the acquired data and prepares it for transmission to the server using encryption technology. The input data is categorized as lifestyle, health status, and emotional information, and sent to the server after ensuring data reliability through encryption protocols such as TLS.

[0376] Step 3:

[0377] The server analyzes the data received from the terminal. Based on the input data, the emotion analysis engine identifies the user's emotional state (e.g., stress, anxiety, joy). As a result of the analysis, an overall health status and emotional profile of the user is generated.

[0378] Step 4:

[0379] The server uses a generative AI model based on the analysis results to generate questions that take into account the user's emotional state. The AI ​​model is given a prompt such as, "The user is feeling anxious. Generate questions that address this state," and an appropriate set of questions is generated.

[0380] Step 5:

[0381] The generated questions are sent from the server to the terminal. The terminal displays the questions to the user, helping them understand the questions and edit them as needed. The output here is a visualized list of questions provided to the user visually.

[0382] Step 6:

[0383] Users consult using questions generated by the medical institution. They then input the feedback they receive into their device. This feedback includes the medical professional's opinion based on the consultation and the user's response.

[0384] Step 7:

[0385] The terminal organizes the feedback data and sends it to the server. The server analyzes the feedback and uses it to improve the question generation algorithm and sentiment analysis engine. This process improves the accuracy and effectiveness of the system and prepares it for future use.

[0386] (Application Example 2)

[0387] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0388] Traditional service delivery systems based on users' lifestyles and health conditions suffer from insufficient personalization. They are unable to generate suggestions or questions that take into account the user's emotional state, thus failing to achieve adequate personalization. In particular, in the in-store shopping experience, product suggestions are not tailored to the user's current emotions or stress levels, making it difficult to improve customer satisfaction.

[0389] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0390] In this invention, the server includes means for acquiring information on lifestyle, health status, or emotional state via user input means, means for performing analysis based on the acquired information and generating questions or suggestions for experts, and means for presenting the generated questions or suggestions to the user and making them adjustable. This makes it possible to provide personalized product suggestions and questions in real time that are tailored to the user's emotions and health status.

[0391] "User input means" refers to a device or interface for receiving information from a user regarding their lifestyle, health status, or emotional state.

[0392] "Lifestyle habits" refer to the actions and habits that users perform on a daily basis, and include things like eating, exercise, and sleep patterns.

[0393] "Health status" is a concept that describes the overall physical and mental health of a user, and is often judged based on medical evaluations.

[0394] "Emotional state" refers to the emotions and moods a user feels at a particular time, and includes states such as stress, joy, and anxiety.

[0395] "Analysis means" refers to software or hardware devices used to analyze acquired information and extract meaning.

[0396] "Means for generating questions or suggestions" refers to a system for creating questions or product suggestions for users based on analyzed information.

[0397] "Adjustable means" refers to an interface or function that allows users to re-edit or modify generated questions or suggestions.

[0398] "Feedback" refers to information such as responses and opinions received from users, which can be used to improve the system.

[0399] The system implementing this invention includes a terminal used by the user, a server that analyzes the data, and an interface for displaying and adjusting the generated suggestions and questions. The user inputs information about their lifestyle, health status, and emotional state using a terminal such as a smartphone or smart glasses. The terminal transmits this information to the server.

[0400] The server uses an emotion analysis engine to analyze the received data. Specific examples include APIs from Microsoft Azure and IBM Watson. It analyzes the user's emotional state to understand conditions such as stress, joy, and anxiety. Based on the analyzed information, it utilizes an AI model to generate helpful questions or product suggestions for the user. The generating AI model provides personalized suggestions that take into account the user's health and emotional state. The generated questions and suggestions are sent to and displayed on the user's device.

[0401] Users can review the displayed questions or suggestions and make adjustments as needed. This adjusted information is then sent back to the server and stored as data. Based on the feedback, the server improves its generation algorithms and sentiment analysis engine, enabling more accurate suggestions.

[0402] As a concrete example, if stress is detected via smart glasses while a user is shopping in a store, relaxation-related product suggestions will be automatically displayed. An example of a prompt message to the generating AI model might be: "Consider the user's emotional state and generate a list of products best suited for relaxation. The user is experiencing stress, and recent health data includes: insufficient sleep and lack of exercise." This allows for specific and practical suggestions tailored to each user's individual needs.

[0403] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0404] Step 1:

[0405] Users input information about their lifestyle, health status, and emotional state using a device. The input information undergoes initial data formatting within the device and is then prepared for transmission to the server. Input can be in the form of text, audio, or images, and output is organized digital data.

[0406] Step 2:

[0407] The terminal sends the organized data to the server. The transmitted data is securely delivered to the server using a secure communication protocol. The input is formatted digital data, and the output is stored as a dataset on the server.

[0408] Step 3:

[0409] The server analyzes the received data and uses an emotion analysis engine to identify the user's emotional state. The input is a dataset stored on the server, and the output is the emotion analysis result. This process utilizes natural language processing and speech analysis techniques.

[0410] Step 4:

[0411] The server uses a generative AI model based on the analysis results to generate questions and product suggestions tailored to the user. The input is the sentiment analysis results, and the output is the generated questions or product suggestions. Prompt sentences are created in this step, serving as instructions for the AI.

[0412] Step 5:

[0413] The server sends the generated questions and suggestions to the user's terminal. The input is the generated questions and suggestions, and the output is what is displayed on the terminal. Communication is secure as it is performed using encryption technology.

[0414] Step 6:

[0415] The user reviews the questions and suggestions displayed on the terminal and makes adjustments as needed. The input is the user's judgment and adjustments, and the output is the adjusted data. The terminal then sends the adjusted data back to the server.

[0416] Step 7:

[0417] The server receives user feedback and uses it to improve its generation algorithms and sentiment analysis engine. The input is user feedback, and the output is data that contributes to improving the overall accuracy of the system. The improved algorithms are then reflected in subsequent analyses and generation processes.

[0418] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0419] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0420] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0421] [Third Embodiment]

[0422] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0423] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0424] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0425] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0426] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0427] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0428] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0429] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0430] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0431] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0432] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0433] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0434] The system according to the present invention is a support system for users to engage in effective dialogue with medical institutions. This system collects and analyzes information on the user's lifestyle and health status, and generates appropriate questions for medical professionals.

[0435] First, the user enters their health information using a device. This information includes their usual diet, exercise frequency, medical history, and current symptoms. The device sends this information to a server, which uses it as part of the data necessary for analysis.

[0436] The server analyzes the received data using machine learning algorithms. This assesses the user's current health status and identifies potential risks and concerns. Based on this analysis, it generates specific questions that should be asked of healthcare professionals.

[0437] The generated questions are sent back to the device and presented to the user visually. The user can review this list of questions and modify or add questions as needed. The modified questions are saved by the device and used when the user visits a healthcare facility.

[0438] After the consultation at the medical institution is complete, the user re-enters the feedback received into the device. The device sends this feedback to the server, which then undergoes multiple cycles to improve the question generation algorithm. Improved analysis makes it possible to generate more appropriate questions for future medical consultations.

[0439] As a concrete example, consider a case where a user diagnosed with hypertension provides daily salt intake, exercise time, and blood pressure measurement results as input data. Based on this data, the server generates questions such as, "Should I revise my diet to improve my hypertension?" or "Should I consider the side effects of my current medication?" The user then consults with a doctor based on this information and inputs the advice received into the system as feedback data.

[0440] Thus, this system supports users in appropriately and effectively collecting information at medical institutions and receiving high-quality medical consultations.

[0441] The following describes the processing flow.

[0442] Step 1:

[0443] Users use a device to input information about their lifestyle and health status. This information includes eating habits, exercise frequency, medical history, and current symptoms. The device verifies the entered data and prepares it in the appropriate format.

[0444] Step 2:

[0445] The terminal transmits the prepared user data to the server using a secure communication method. The information is transmitted according to an encryption protocol, ensuring the security of the data.

[0446] Step 3:

[0447] The server retrieves the received data and performs analysis using a data analysis module. This analysis assesses the user's current health status and identifies specific health risks and points of concern.

[0448] Step 4:

[0449] Based on the analysis results, the server uses a generation AI to generate questions for medical professionals. The generated questions are designed to be specific and tailored to the user's situation.

[0450] Step 5:

[0451] The server sends the generated list of questions to the terminal. The terminal receives the list of questions and displays it in a format that is easy for the user to understand intuitively.

[0452] Step 6:

[0453] The user reviews the question list on their device and modifies or adds questions as needed. This edited question list is then verified to meet the user's needs.

[0454] Step 7:

[0455] Users save the revised list of questions and use it when consulting with healthcare professionals. By presenting questions based on this list to medical professionals, efficient information gathering becomes possible.

[0456] Step 8:

[0457] The user inputs feedback received from the medical institution into the terminal. The terminal organizes the feedback data and prepares it for transmission to the server.

[0458] Step 9:

[0459] The device sends feedback data to the server. The server uses this feedback to update the training data to improve the question generation algorithm. This improves the accuracy of subsequent question generation.

[0460] (Example 1)

[0461] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0462] In communication with medical institutions, users face the challenge of being able to ask appropriate questions to healthcare professionals based on their health status and lifestyle. Furthermore, continuously improving the accuracy of data analysis is necessary to effectively utilize user feedback in future medical consultations. In addition, the secure transmission and reception of data, including personal information, is a crucial issue.

[0463] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0464] In this invention, the server includes means for acquiring lifestyle information or health status information via a user input device, means for performing analysis using a machine learning algorithm based on the acquired information to generate questions for experts, and means for presenting the generated questions to the user and making them adjustable. This enables the user to efficiently generate appropriate questions and conduct medical consultations more effectively. Furthermore, by saving the generated questions and outputting them as information that can be brought to external organizations, it is possible to facilitate the presentation of information at medical institutions. In addition, by including means for acquiring feedback information and reflecting it in improving the analysis algorithm, and information protection means for securely sending and receiving data, it is possible to improve the accuracy of question generation while ensuring the security of personal information.

[0465] A "user input device" is a device that provides an interface for users to input information about their lifestyle and health status.

[0466] "Lifestyle information" refers to information about a user's daily activities, such as their diet, exercise, and sleep.

[0467] "Health status information" refers to information about the user's physical or mental health status.

[0468] A "machine learning algorithm" is a computational method that performs predictions and classifications by analyzing data and learning patterns.

[0469] "Questions for Experts" refers to content intended for use by users in healthcare settings, representing questions and answers directed to medical professionals.

[0470] "Information protection measures" refer to technologies and methods used to ensure the secure transmission and reception of data.

[0471] "Feedback information" refers to information about responses received by users from medical institutions, such as advice and diagnostic results.

[0472] "Algorithm improvement" refers to updating or adjusting models or analysis methods to improve the accuracy and efficiency of the analysis.

[0473] Users input their lifestyle and health information using mobile devices or computers. This includes daily dietary habits, exercise levels, medical history, and changes in physical condition. The entered information is transmitted to the server via the device. During this process, the data is encrypted using the HTTPS protocol for secure transmission and reception.

[0474] The server analyzes the received information using machine learning algorithms. This analysis utilizes Python libraries such as TensorFlow and scikit-learn to assess the user's health risks. In particular, it processes data to identify potential health risks that the user may not be aware of, and to address important questions that may arise during medical consultations.

[0475] Based on the analysis results, the server generates questions for experts using a generative AI model. Natural language processing techniques are employed to generate specific and highly specialized questions. For example, technologies such as GPT-3 are used as the generative AI model.

[0476] The generated questions are sent back to the terminal and presented to the user. The user can review the displayed questions, add any new questions they need, or modify any questions that have already been generated. The completed question list can be output in PDF format or other formats, and can be printed out and brought along when visiting medical institutions.

[0477] After a consultation at a medical institution is completed, the user enters the feedback received into their device. This feedback is then sent back to the server and used to improve the question generation algorithm. As a new question generation model is built, it becomes possible to provide even more accurate questions during future medical consultations.

[0478] As a concrete example, consider a case where a user diagnosed with hypertension inputs their diet and weekly exercise time, and based on that, the system automatically generates a question asking, "Should I seek specific dietary advice to manage my hypertension?" As part of this process, the prompt would be designed to read, "The user has been diagnosed with hypertension. Based on their daily salt intake, exercise time, and blood pressure readings, please generate questions that they should ask a professional."

[0479] This system allows users to communicate effectively with healthcare providers and improve the quality of medical consultations.

[0480] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0481] Step 1:

[0482] Users input health-related information using a mobile device or computer. This input includes data on diet, exercise levels, medical history, and current symptoms. This input data is saved to the device via the application interface. This allows users to record detailed information about their own health status.

[0483] Step 2:

[0484] The terminal encrypts health-related information entered by the user and sends it to the server using the HTTPS protocol. Encryption of the input data ensures information security during transmission. The server stores the received information in a database for processing. This step ensures secure data transfer and storage.

[0485] Step 3:

[0486] The server applies machine learning algorithms to the received health-related information and analyzes the data. Specific analysis includes assessing the user's health risks and identifying symptoms that require attention. The analysis is performed using libraries such as TensorFlow and scikit-learn, providing outputs to evaluate the user's health status and identify potential risks. The output obtained in this process is then used to generate questions in the next step.

[0487] Step 4:

[0488] The server uses a generative AI model based on the analysis results to generate questions for experts. The generative AI model employs natural language processing technology. Upon input of a prompt, specific and relevant questions tailored to the user's health condition are generated. The generated questions provide important information for the user to use in medical settings.

[0489] Step 5:

[0490] The terminal receives pre-generated questions sent from the server and presents them visually to the user. The user can review the questions on the screen and modify or add to them as needed. This process is important for the user to clearly communicate information in a healthcare setting. The completed questions are listed and output to the terminal.

[0491] Step 6:

[0492] After a user visits a medical institution and completes a consultation with a doctor, they input the resulting feedback into a terminal. The input feedback data is then sent back to the server by the terminal. The feedback obtained in this step will contribute to improving the accuracy of future analyses.

[0493] Step 7:

[0494] The server analyzes the feedback sent by the user and uses it to improve the question generation algorithm. This process involves learning from new data to increase generation accuracy. The questions generated for the user's next medical consultation will be based on the improved algorithm, resulting in more appropriate and helpful questions.

[0495] (Application Example 1)

[0496] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0497] In modern times, individuals often lack sufficient information when selecting products and services tailored to their specific health needs. This makes it difficult for them to find the optimal options based on their own health status and lifestyle. Furthermore, there is a lack of support in generating appropriate questions and facilitating meaningful information exchange during conversations with health professionals.

[0498] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0499] In this invention, the server includes means for acquiring information on lifestyle or health status via a user input unit, means for performing analysis based on the acquired information and generating questions for a third party, and means for recommending products or services based on the information. This makes it possible to generate appropriate questions and suggest products or services tailored to each individual's health condition.

[0500] The "user input section" is an interface for obtaining information about lifestyle habits or health status.

[0501] "Means of analysis" refers to a function that analyzes data based on acquired information and generates questions necessary for dialogue with a third party.

[0502] "Methods for generating questions" refers to the process of creating specific questions that a user should ask a third party, based on the analysis results.

[0503] "Means for presenting generated questions to the user" refers to a function that allows users to review and adjust automatically generated questions.

[0504] "Means of sending questions externally" refers to the process of sending a prepared question to an external service or third party.

[0505] "Means for obtaining feedback" refers to a function that receives responses and evaluations from external sources and uses them to inform future data analysis.

[0506] "Means of recommending products or services based on information" refers to a function that suggests the most suitable products or services according to the individual health data acquired.

[0507] "Encryption methods" are technologies used to encrypt information in order to communicate data securely and enable secure transmission and reception.

[0508] This system begins by utilizing a user input field to acquire data on the user's lifestyle and health status. Users input information such as their diet, exercise frequency, and current health status through a smartphone application. This information is sent from the device to a server for advanced data analysis. The analysis utilizes the Django framework using Python and Scikit-learn, a tool specifically designed for data analysis. This process lays the foundation for comprehensively evaluating the user's health status and recommending health-related services and products.

[0509] Next, the server generates specific questions that the user should ask a third party, such as store staff or a health professional, via a generative AI model. GPT-3 is used for question generation to support effective dialogue. The generated questions and suggested products and services are then displayed again on the smartphone, allowing the user to make product selections based on that information.

[0510] In this process, an example of a prompt might be, "Generate questions for a user who is aiming to improve their hypertension and has a history of high salt intake." Specifically, when a user enters data, the system might generate and present questions such as, "Are health foods rich in omega-3 fatty acids good?" or "Which supplements are best for reducing stress?" This allows users to make more informed decisions when choosing health-related products and services.

[0511] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0512] Step 1:

[0513] Users input information about their lifestyle and health status using a smartphone application. This includes diet, exercise frequency, medical history, and current symptoms. The entered data is immediately stored in the app and prepared for transmission to the server.

[0514] Step 2:

[0515] The terminal sends the information entered by the user to the server. Here, the data is encrypted, and protocols are configured to ensure secure transmission. The input data is stored on the server as foundational data for analysis.

[0516] Step 3:

[0517] The server uses Scikit-learn to analyze the received information. It comprehensively assesses the user's health status and identifies potential health risks and concerns. This assessment result is stored as internal data for use in the next step.

[0518] Step 4:

[0519] The server generates questions using GPT-3 based on the analysis results. This question generation process creates appropriate questions based on the user's health status and risks, and the content follows criteria defined as prompt statements. These generated questions are later presented to the user visually.

[0520] Step 5:

[0521] The generated questions and recommendations for products and services based on the analysis results are sent back to the device. Users receive these questions and recommendations through the app and can select the products and services that best suit their needs.

[0522] Step 6:

[0523] The user interacts with a third party based on the questions and recommendations received. After the interaction, the user inputs the acquired feedback information into the app and sends it back to the server. This feedback is used to improve the accuracy of the question generation algorithm.

[0524] Step 7:

[0525] The server receives feedback and uses it to improve the GPT-3 model. This allows for more appropriate output in subsequent question generation processes, accelerating the evolution of the user experience and improving the overall effectiveness of the system.

[0526] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0527] The system according to the present invention is an advanced system that supports dialogue with medical institutions by considering not only the user's lifestyle and health condition but also their emotional state. This system facilitates smooth communication with medical professionals through the acquisition, analysis, question generation, and adjustment of input and output of user input data.

[0528] First, the user inputs information about their health status, lifestyle, and emotions via the device. This emotional information is obtained through the user's facial expressions, tone of voice, and self-reporting. The device then organizes this data and prepares it for transmission to the server.

[0529] The server uses the received data to perform analysis to understand the user's overall profile. This analysis includes analyzing the user's emotions. The emotion engine recognizes the user's current emotional state and extracts factors such as stress, joy, and anxiety. By considering the user's emotions, the question generation process is personalized, enabling the creation of flexible questions that are tailored to the user's psychological state.

[0530] The generated questions are adjusted to suit the user's emotional state and sent to the device. The device displays the questions in a way that allows the user to easily understand and edit them as needed. By using these adjusted questions in a healthcare setting, users can have more effective consultations.

[0531] After a consultation with a medical institution, the user inputs the feedback received into their device and sends it to the server. The server analyzes this feedback to further improve the question generation algorithm and emotion engine. For example, if a user experiencing stress self-reports "what has been bothering them lately," the system can generate health consultation questions that take that emotional state into account. In this way, this system, which also responds to the user's emotions, can evolve conversations at medical institutions into something deeper and more personal.

[0532] The following describes the processing flow.

[0533] Step 1:

[0534] Users use the device to input lifestyle, health, and emotional information. Emotional information is automatically acquired through facial recognition or voice analysis, or it is entered by the user through self-reporting. The device appropriately formats this information and prepares it for transmission.

[0535] Step 2:

[0536] The device sends the collected data to the server. This transmission process uses encryption to ensure data security.

[0537] Step 3:

[0538] The server analyzes the received data to assess the user's health status and lifestyle, while simultaneously analyzing their emotional state using an emotion engine. The emotion engine detects stress, anxiety, joy, etc., and provides information to understand how these affect their health.

[0539] Step 4:

[0540] Based on the analysis results, the server generates specific and adaptive questions for medical professionals, taking into account the user's emotional state. This question generation process employs an approach that takes into account the user's psychological state.

[0541] Step 5:

[0542] The server sends a generated list of questions to the terminal. The terminal displays the questions so that the user can easily understand them and adjust them as needed.

[0543] Step 6:

[0544] Users can review the questions presented on their device and modify or add questions as needed. The user's edited list of questions is saved and ready for use during consultations at healthcare facilities.

[0545] Step 7:

[0546] After consulting with a medical institution, the user inputs the feedback received into their device and sends the feedback to the server.

[0547] Step 8:

[0548] The server analyzes the feedback and uses it to improve the question generation algorithm and sentiment engine. This allows the system to improve the accuracy and responsiveness of future user inquiries.

[0549] (Example 2)

[0550] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0551] Traditional medical consultation systems focused on information based on the user's lifestyle and health status, making it difficult to consider the user's emotional state. As a result, the user's stress, anxiety, and other emotions were not adequately reflected in the consultation, leading to insufficient communication with medical professionals.

[0552] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0553] In this invention, the server includes means for acquiring information on the user's lifestyle, health status, and emotions; means for analyzing the user's emotional state and generating questions for medical professionals; and means for adjusting the questions to suit the user's emotional state using a generative AI model. This enables deeper and more personal communication with medical professionals by considering the user's emotions and generating personalized questions.

[0554] A "user" refers to an individual who uses the system to input information about their lifestyle, health status, and emotions.

[0555] "Lifestyle habits" refers to information about a user's daily actions and habits. This includes things like diet, exercise, and sleep.

[0556] "Health status" refers to information about the user's physical health. This includes medical history, current illnesses, and overall physical condition.

[0557] "Emotional information" refers to data about the user's emotional state. This includes information obtained through facial expressions, tone of voice, and self-reporting.

[0558] "Analysis" refers to evaluating and analyzing a user's lifestyle, health status, and emotional state based on the information acquired.

[0559] "Question generation" refers to the process of formulating questions for medical professionals using analysis results.

[0560] A "generative AI model" refers to a computer program that uses artificial intelligence technology to extract useful information from data and generate appropriate questions.

[0561] "Adjustment" refers to modifying the generated questions so that they are in a form that is appropriate to the user's emotional state.

[0562] "Feedback" refers to the act of returning the results and answers obtained from the user's opinions and advice received at a medical institution to the system.

[0563] "Encryption" refers to a technology that transforms information in a specific way to securely transfer data and prevent unauthorized access.

[0564] "Medical institutions" refer to external organizations or professionals that provide medical services to users.

[0565] This invention is a system in which a user inputs information about their health status, lifestyle, and emotions via a terminal, and a server analyzes this data to generate questions useful for medical consultation. This system places particular emphasis on emotional information, enabling the generation of questions that take into account the user's emotional state.

[0566] Users input health-related information using devices such as smartphones and personal computers. This information includes not only lifestyle habits and health status, but also real-time emotional information of the user. Emotional information is obtained by analyzing facial expressions and voice tone using the camera and microphone built into the device. Users can also input their emotions through self-reporting.

[0567] The terminal organizes the acquired information and sends it to the server using encryption technology. Data security is ensured by using commonly used encryption protocols (e.g., TLS).

[0568] The server analyzes the user's health and emotional state based on the received data. An emotion analysis engine is used to identify emotional factors such as stress, joy, and anxiety that the user is experiencing. Based on the analysis results, a generative AI model generates questions for medical professionals that are tailored to the user's emotions. This enables flexible responses that are appropriate to the user's psychological state.

[0569] For example, the prompt message to the generating AI model could include instructions such as, "The user is feeling anxious. Please generate questions that address this state." This allows the system to provide consultation content optimized for the user's current situation. Based on this prompt message, the model generates appropriate questions and sends them from the server to the terminal.

[0570] The terminal presents the user with generated questions and allows the user to edit them as needed, creating a set of questions that the user can use when consulting with a medical institution. The feedback received after the consultation is then sent back to the server, and the system continues to improve its question generation algorithm and emotion engine based on this feedback. In this way, the system evolves with each use, enabling more personalized and effective medical consultations.

[0571] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0572] Step 1:

[0573] Users input health status, lifestyle habits, and emotional information using a device. The input data includes information directly entered by the user, as well as information from cameras and microphones that analyze facial expressions and voice tone. This allows for the acquisition of both the user's subjective state and objective emotional information.

[0574] Step 2:

[0575] The terminal organizes the acquired data and prepares it for transmission to the server using encryption technology. The input data is categorized as lifestyle, health status, and emotional information, and sent to the server after ensuring data reliability through encryption protocols such as TLS.

[0576] Step 3:

[0577] The server analyzes the data received from the terminal. Based on the input data, the emotion analysis engine identifies the user's emotional state (e.g., stress, anxiety, joy). As a result of the analysis, an overall health status and emotional profile of the user is generated.

[0578] Step 4:

[0579] The server uses a generative AI model based on the analysis results to generate questions that take into account the user's emotional state. The AI ​​model is given a prompt such as, "The user is feeling anxious. Generate questions that address this state," and an appropriate set of questions is generated.

[0580] Step 5:

[0581] The generated questions are sent from the server to the terminal. The terminal displays the questions to the user, helping them understand the questions and edit them as needed. The output here is a visualized list of questions provided to the user visually.

[0582] Step 6:

[0583] Users consult using questions generated by the medical institution. They then input the feedback they receive into their device. This feedback includes the medical professional's opinion based on the consultation and the user's response.

[0584] Step 7:

[0585] The terminal organizes the feedback data and sends it to the server. The server analyzes the feedback and uses it to improve the question generation algorithm and sentiment analysis engine. This process improves the accuracy and effectiveness of the system and prepares it for future use.

[0586] (Application Example 2)

[0587] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0588] Traditional service delivery systems based on users' lifestyles and health conditions suffer from insufficient personalization. They are unable to generate suggestions or questions that take into account the user's emotional state, thus failing to achieve adequate personalization. In particular, in the in-store shopping experience, product suggestions are not tailored to the user's current emotions or stress levels, making it difficult to improve customer satisfaction.

[0589] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0590] In this invention, the server includes means for acquiring information on lifestyle, health status, or emotional state via user input means, means for performing analysis based on the acquired information and generating questions or suggestions for experts, and means for presenting the generated questions or suggestions to the user and making them adjustable. This makes it possible to provide personalized product suggestions and questions in real time that are tailored to the user's emotions and health status.

[0591] "User input means" refers to a device or interface for receiving information from a user regarding their lifestyle, health status, or emotional state.

[0592] "Lifestyle habits" refer to the actions and habits that users perform on a daily basis, and include things like eating, exercise, and sleep patterns.

[0593] "Health status" is a concept that describes the overall physical and mental health of a user, and is often judged based on medical evaluations.

[0594] "Emotional state" refers to the emotions and moods a user feels at a particular time, and includes states such as stress, joy, and anxiety.

[0595] "Analysis means" refers to software or hardware devices used to analyze acquired information and extract meaning.

[0596] "Means for generating questions or suggestions" refers to a system for creating questions or product suggestions for users based on analyzed information.

[0597] "Adjustable means" refers to an interface or function that allows users to re-edit or modify generated questions or suggestions.

[0598] "Feedback" refers to information such as responses and opinions received from users, which can be used to improve the system.

[0599] The system implementing this invention includes a terminal used by the user, a server that analyzes the data, and an interface for displaying and adjusting the generated suggestions and questions. The user inputs information about their lifestyle, health status, and emotional state using a terminal such as a smartphone or smart glasses. The terminal transmits this information to the server.

[0600] The server uses an emotion analysis engine to analyze the received data. Specific examples include APIs from Microsoft Azure and IBM Watson. It analyzes the user's emotional state to understand conditions such as stress, joy, and anxiety. Based on the analyzed information, it utilizes an AI model to generate helpful questions or product suggestions for the user. The generating AI model provides personalized suggestions that take into account the user's health and emotional state. The generated questions and suggestions are sent to and displayed on the user's device.

[0601] Users can review the displayed questions or suggestions and make adjustments as needed. This adjusted information is then sent back to the server and stored as data. Based on the feedback, the server improves its generation algorithms and sentiment analysis engine, enabling more accurate suggestions.

[0602] As a concrete example, if stress is detected via smart glasses while a user is shopping in a store, relaxation-related product suggestions will be automatically displayed. An example of a prompt message to the generating AI model might be: "Consider the user's emotional state and generate a list of products best suited for relaxation. The user is experiencing stress, and recent health data includes: insufficient sleep and lack of exercise." This allows for specific and practical suggestions tailored to each user's individual needs.

[0603] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0604] Step 1:

[0605] Users input information about their lifestyle, health status, and emotional state using a device. The input information undergoes initial data formatting within the device and is then prepared for transmission to the server. Input can be in the form of text, audio, or images, and output is organized digital data.

[0606] Step 2:

[0607] The terminal sends the organized data to the server. The transmitted data is securely delivered to the server using a secure communication protocol. The input is formatted digital data, and the output is stored as a dataset on the server.

[0608] Step 3:

[0609] The server analyzes the received data and uses an emotion analysis engine to identify the user's emotional state. The input is a dataset stored on the server, and the output is the emotion analysis result. This process utilizes natural language processing and speech analysis techniques.

[0610] Step 4:

[0611] The server uses a generative AI model based on the analysis results to generate questions and product suggestions tailored to the user. The input is the sentiment analysis results, and the output is the generated questions or product suggestions. Prompt sentences are created in this step, serving as instructions for the AI.

[0612] Step 5:

[0613] The server sends the generated questions and suggestions to the user's terminal. The input is the generated questions and suggestions, and the output is what is displayed on the terminal. Communication is secure as it is performed using encryption technology.

[0614] Step 6:

[0615] The user reviews the questions and suggestions displayed on the terminal and makes adjustments as needed. The input is the user's judgment and adjustments, and the output is the adjusted data. The terminal then sends the adjusted data back to the server.

[0616] Step 7:

[0617] The server receives user feedback and uses it to improve its generation algorithms and sentiment analysis engine. The input is user feedback, and the output is data that contributes to improving the overall accuracy of the system. The improved algorithms are then reflected in subsequent analyses and generation processes.

[0618] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0619] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0620] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0621] [Fourth Embodiment]

[0622] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0623] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0624] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0625] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0626] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0627] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0628] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0629] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0630] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0631] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0632] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0633] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0634] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0635] The system according to the present invention is a support system for users to engage in effective dialogue with medical institutions. This system collects and analyzes information on the user's lifestyle and health status, and generates appropriate questions for medical professionals.

[0636] First, the user enters their health information using a device. This information includes their usual diet, exercise frequency, medical history, and current symptoms. The device sends this information to a server, which uses it as part of the data necessary for analysis.

[0637] The server analyzes the received data using machine learning algorithms. This assesses the user's current health status and identifies potential risks and concerns. Based on this analysis, it generates specific questions that should be asked of healthcare professionals.

[0638] The generated questions are sent back to the device and presented to the user visually. The user can review this list of questions and modify or add questions as needed. The modified questions are saved by the device and used when the user visits a healthcare facility.

[0639] After the consultation at the medical institution is complete, the user re-enters the feedback received into the device. The device sends this feedback to the server, which then undergoes multiple cycles to improve the question generation algorithm. Improved analysis makes it possible to generate more appropriate questions for future medical consultations.

[0640] As a concrete example, consider a case where a user diagnosed with hypertension provides daily salt intake, exercise time, and blood pressure measurement results as input data. Based on this data, the server generates questions such as, "Should I revise my diet to improve my hypertension?" or "Should I consider the side effects of my current medication?" The user then consults with a doctor based on this information and inputs the advice received into the system as feedback data.

[0641] Thus, this system supports users in appropriately and effectively collecting information at medical institutions and receiving high-quality medical consultations.

[0642] The following describes the processing flow.

[0643] Step 1:

[0644] Users use a device to input information about their lifestyle and health status. This information includes eating habits, exercise frequency, medical history, and current symptoms. The device verifies the entered data and prepares it in the appropriate format.

[0645] Step 2:

[0646] The terminal transmits the prepared user data to the server using a secure communication method. The information is transmitted according to an encryption protocol, ensuring the security of the data.

[0647] Step 3:

[0648] The server retrieves the received data and performs analysis using a data analysis module. This analysis assesses the user's current health status and identifies specific health risks and points of concern.

[0649] Step 4:

[0650] Based on the analysis results, the server uses a generation AI to generate questions for medical professionals. The generated questions are designed to be specific and tailored to the user's situation.

[0651] Step 5:

[0652] The server sends the generated list of questions to the terminal. The terminal receives the list of questions and displays it in a format that is easy for the user to understand intuitively.

[0653] Step 6:

[0654] The user reviews the question list on their device and modifies or adds questions as needed. This edited question list is then verified to meet the user's needs.

[0655] Step 7:

[0656] Users save the revised list of questions and use it when consulting with healthcare professionals. By presenting questions based on this list to medical professionals, efficient information gathering becomes possible.

[0657] Step 8:

[0658] The user inputs feedback received from the medical institution into the terminal. The terminal organizes the feedback data and prepares it for transmission to the server.

[0659] Step 9:

[0660] The device sends feedback data to the server. The server uses this feedback to update the training data to improve the question generation algorithm. This improves the accuracy of subsequent question generation.

[0661] (Example 1)

[0662] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0663] In communication with medical institutions, users face the challenge of being able to ask appropriate questions to healthcare professionals based on their health status and lifestyle. Furthermore, continuously improving the accuracy of data analysis is necessary to effectively utilize user feedback in future medical consultations. In addition, the secure transmission and reception of data, including personal information, is a crucial issue.

[0664] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0665] In this invention, the server includes means for acquiring lifestyle information or health status information via a user input device, means for performing analysis using a machine learning algorithm based on the acquired information to generate questions for experts, and means for presenting the generated questions to the user and making them adjustable. This enables the user to efficiently generate appropriate questions and conduct medical consultations more effectively. Furthermore, by saving the generated questions and outputting them as information that can be brought to external organizations, it is possible to facilitate the presentation of information at medical institutions. In addition, by including means for acquiring feedback information and reflecting it in improving the analysis algorithm, and information protection means for securely sending and receiving data, it is possible to improve the accuracy of question generation while ensuring the security of personal information.

[0666] A "user input device" is a device that provides an interface for users to input information about their lifestyle and health status.

[0667] "Lifestyle information" refers to information about a user's daily activities, such as their diet, exercise, and sleep.

[0668] "Health status information" refers to information about the user's physical or mental health status.

[0669] A "machine learning algorithm" is a computational method that performs predictions and classifications by analyzing data and learning patterns.

[0670] "Questions for Experts" refers to content intended for use by users in healthcare settings, representing questions and answers directed to medical professionals.

[0671] "Information protection measures" refer to technologies and methods used to ensure the secure transmission and reception of data.

[0672] "Feedback information" refers to information about responses received by users from medical institutions, such as advice and diagnostic results.

[0673] "Algorithm improvement" refers to updating or adjusting models or analysis methods to improve the accuracy and efficiency of the analysis.

[0674] Users input their lifestyle and health information using mobile devices or computers. This includes daily dietary habits, exercise levels, medical history, and changes in physical condition. The entered information is transmitted to the server via the device. During this process, the data is encrypted using the HTTPS protocol for secure transmission and reception.

[0675] The server analyzes the received information using machine learning algorithms. This analysis utilizes Python libraries such as TensorFlow and scikit-learn to assess the user's health risks. In particular, it processes data to identify potential health risks that the user may not be aware of, and to address important questions that may arise during medical consultations.

[0676] Based on the analysis results, the server generates questions for experts using a generative AI model. Natural language processing techniques are employed to generate specific and highly specialized questions. For example, technologies such as GPT-3 are used as the generative AI model.

[0677] The generated questions are sent back to the terminal and presented to the user. The user can review the displayed questions, add any new questions they need, or modify any questions that have already been generated. The completed question list can be output in PDF format or other formats, and can be printed out and brought along when visiting medical institutions.

[0678] After a consultation at a medical institution is completed, the user enters the feedback received into their device. This feedback is then sent back to the server and used to improve the question generation algorithm. As a new question generation model is built, it becomes possible to provide even more accurate questions during future medical consultations.

[0679] As a concrete example, consider a case where a user diagnosed with hypertension inputs their diet and weekly exercise time, and based on that, the system automatically generates a question asking, "Should I seek specific dietary advice to manage my hypertension?" As part of this process, the prompt would be designed to read, "The user has been diagnosed with hypertension. Based on their daily salt intake, exercise time, and blood pressure readings, please generate questions that they should ask a professional."

[0680] This system allows users to communicate effectively with healthcare providers and improve the quality of medical consultations.

[0681] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0682] Step 1:

[0683] Users input health-related information using a mobile device or computer. This input includes data on diet, exercise levels, medical history, and current symptoms. This input data is saved to the device via the application interface. This allows users to record detailed information about their own health status.

[0684] Step 2:

[0685] The terminal encrypts health-related information entered by the user and sends it to the server using the HTTPS protocol. Encryption of the input data ensures information security during transmission. The server stores the received information in a database for processing. This step ensures secure data transfer and storage.

[0686] Step 3:

[0687] The server applies machine learning algorithms to the received health-related information and analyzes the data. Specific analysis includes assessing the user's health risks and identifying symptoms that require attention. The analysis is performed using libraries such as TensorFlow and scikit-learn, providing outputs to evaluate the user's health status and identify potential risks. The output obtained in this process is then used to generate questions in the next step.

[0688] Step 4:

[0689] The server uses a generative AI model based on the analysis results to generate questions for experts. The generative AI model employs natural language processing technology. Upon input of a prompt, specific and relevant questions tailored to the user's health condition are generated. The generated questions provide important information for the user to use in medical settings.

[0690] Step 5:

[0691] The terminal receives pre-generated questions sent from the server and presents them visually to the user. The user can review the questions on the screen and modify or add to them as needed. This process is important for the user to clearly communicate information in a healthcare setting. The completed questions are listed and output to the terminal.

[0692] Step 6:

[0693] After a user visits a medical institution and completes a consultation with a doctor, they input the resulting feedback into a terminal. The input feedback data is then sent back to the server by the terminal. The feedback obtained in this step will contribute to improving the accuracy of future analyses.

[0694] Step 7:

[0695] The server analyzes the feedback sent by the user and uses it to improve the question generation algorithm. This process involves learning from new data to increase generation accuracy. The questions generated for the user's next medical consultation will be based on the improved algorithm, resulting in more appropriate and helpful questions.

[0696] (Application Example 1)

[0697] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0698] In modern times, individuals often lack sufficient information when selecting products and services tailored to their specific health needs. This makes it difficult for them to find the optimal options based on their own health status and lifestyle. Furthermore, there is a lack of support in generating appropriate questions and facilitating meaningful information exchange during conversations with health professionals.

[0699] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0700] In this invention, the server includes means for acquiring information on lifestyle or health status via a user input unit, means for performing analysis based on the acquired information and generating questions for a third party, and means for recommending products or services based on the information. This makes it possible to generate appropriate questions and suggest products or services tailored to each individual's health condition.

[0701] The "user input section" is an interface for obtaining information about lifestyle habits or health status.

[0702] "Means of analysis" refers to a function that analyzes data based on acquired information and generates questions necessary for dialogue with a third party.

[0703] "Methods for generating questions" refers to the process of creating specific questions that a user should ask a third party, based on the analysis results.

[0704] "Means for presenting generated questions to the user" refers to a function that allows users to review and adjust automatically generated questions.

[0705] "Means of sending questions externally" refers to the process of sending a prepared question to an external service or third party.

[0706] "Means for obtaining feedback" refers to a function that receives responses and evaluations from external sources and uses them to inform future data analysis.

[0707] "Means of recommending products or services based on information" refers to a function that suggests the most suitable products or services according to the individual health data acquired.

[0708] "Encryption methods" are technologies used to encrypt information in order to communicate data securely and enable secure transmission and reception.

[0709] This system begins by utilizing a user input field to acquire data on the user's lifestyle and health status. Users input information such as their diet, exercise frequency, and current health status through a smartphone application. This information is sent from the device to a server for advanced data analysis. The analysis utilizes the Django framework using Python and Scikit-learn, a tool specifically designed for data analysis. This process lays the foundation for comprehensively evaluating the user's health status and recommending health-related services and products.

[0710] Next, the server generates specific questions that the user should ask a third party, such as store staff or a health professional, via a generative AI model. GPT-3 is used for question generation to support effective dialogue. The generated questions and suggested products and services are then displayed again on the smartphone, allowing the user to make product selections based on that information.

[0711] In this process, an example of a prompt might be, "Generate questions for a user who is aiming to improve their hypertension and has a history of high salt intake." Specifically, when a user enters data, the system might generate and present questions such as, "Are health foods rich in omega-3 fatty acids good?" or "Which supplements are best for reducing stress?" This allows users to make more informed decisions when choosing health-related products and services.

[0712] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0713] Step 1:

[0714] Users input information about their lifestyle and health status using a smartphone application. This includes diet, exercise frequency, medical history, and current symptoms. The entered data is immediately stored in the app and prepared for transmission to the server.

[0715] Step 2:

[0716] The terminal sends the information entered by the user to the server. Here, the data is encrypted, and protocols are configured to ensure secure transmission. The input data is stored on the server as foundational data for analysis.

[0717] Step 3:

[0718] The server uses Scikit-learn to analyze the received information. It comprehensively assesses the user's health status and identifies potential health risks and concerns. This assessment result is stored as internal data for use in the next step.

[0719] Step 4:

[0720] The server generates questions using GPT-3 based on the analysis results. This question generation process creates appropriate questions based on the user's health status and risks, and the content follows criteria defined as prompt statements. These generated questions are later presented to the user visually.

[0721] Step 5:

[0722] The generated questions and recommendations for products and services based on the analysis results are sent back to the device. Users receive these questions and recommendations through the app and can select the products and services that best suit their needs.

[0723] Step 6:

[0724] The user interacts with a third party based on the questions and recommendations received. After the interaction, the user inputs the acquired feedback information into the app and sends it back to the server. This feedback is used to improve the accuracy of the question generation algorithm.

[0725] Step 7:

[0726] The server receives feedback and uses it to improve the GPT-3 model. This allows for more appropriate output in subsequent question generation processes, accelerating the evolution of the user experience and improving the overall effectiveness of the system.

[0727] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0728] The system according to the present invention is an advanced system that supports dialogue with medical institutions by considering not only the user's lifestyle and health condition but also their emotional state. This system facilitates smooth communication with medical professionals through the acquisition, analysis, question generation, and adjustment of input and output of user input data.

[0729] First, the user inputs information about their health status, lifestyle, and emotions via the device. This emotional information is obtained through the user's facial expressions, tone of voice, and self-reporting. The device then organizes this data and prepares it for transmission to the server.

[0730] The server uses the received data to perform analysis to understand the user's overall profile. This analysis includes analyzing the user's emotions. The emotion engine recognizes the user's current emotional state and extracts factors such as stress, joy, and anxiety. By considering the user's emotions, the question generation process is personalized, enabling the creation of flexible questions that are tailored to the user's psychological state.

[0731] The generated questions are adjusted to suit the user's emotional state and sent to the device. The device displays the questions in a way that allows the user to easily understand and edit them as needed. By using these adjusted questions in a healthcare setting, users can have more effective consultations.

[0732] After a consultation with a medical institution, the user inputs the feedback received into their device and sends it to the server. The server analyzes this feedback to further improve the question generation algorithm and emotion engine. For example, if a user experiencing stress self-reports "what has been bothering them lately," the system can generate health consultation questions that take that emotional state into account. In this way, this system, which also responds to the user's emotions, can evolve conversations at medical institutions into something deeper and more personal.

[0733] The following describes the processing flow.

[0734] Step 1:

[0735] Users use the device to input lifestyle, health, and emotional information. Emotional information is automatically acquired through facial recognition or voice analysis, or it is entered by the user through self-reporting. The device appropriately formats this information and prepares it for transmission.

[0736] Step 2:

[0737] The device sends the collected data to the server. This transmission process uses encryption to ensure data security.

[0738] Step 3:

[0739] The server analyzes the received data to assess the user's health status and lifestyle, while simultaneously analyzing their emotional state using an emotion engine. The emotion engine detects stress, anxiety, joy, etc., and provides information to understand how these affect their health.

[0740] Step 4:

[0741] Based on the analysis results, the server generates specific and adaptive questions for medical professionals, taking into account the user's emotional state. This question generation process employs an approach that takes into account the user's psychological state.

[0742] Step 5:

[0743] The server sends a generated list of questions to the terminal. The terminal displays the questions so that the user can easily understand them and adjust them as needed.

[0744] Step 6:

[0745] Users can review the questions presented on their device and modify or add questions as needed. The user's edited list of questions is saved and ready for use during consultations at healthcare facilities.

[0746] Step 7:

[0747] After consulting with a medical institution, the user inputs the feedback received into their device and sends the feedback to the server.

[0748] Step 8:

[0749] The server analyzes the feedback and uses it to improve the question generation algorithm and sentiment engine. This allows the system to improve the accuracy and responsiveness of future user inquiries.

[0750] (Example 2)

[0751] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0752] Traditional medical consultation systems focused on information based on the user's lifestyle and health status, making it difficult to consider the user's emotional state. As a result, the user's stress, anxiety, and other emotions were not adequately reflected in the consultation, leading to insufficient communication with medical professionals.

[0753] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0754] In this invention, the server includes means for acquiring information on the user's lifestyle, health status, and emotions; means for analyzing the user's emotional state and generating questions for medical professionals; and means for adjusting the questions to suit the user's emotional state using a generative AI model. This enables deeper and more personal communication with medical professionals by considering the user's emotions and generating personalized questions.

[0755] A "user" refers to an individual who uses the system to input information about their lifestyle, health status, and emotions.

[0756] "Lifestyle habits" refers to information about a user's daily actions and habits. This includes things like diet, exercise, and sleep.

[0757] "Health status" refers to information about the user's physical health. This includes medical history, current illnesses, and overall physical condition.

[0758] "Emotional information" refers to data about the user's emotional state. This includes information obtained through facial expressions, tone of voice, and self-reporting.

[0759] "Analysis" refers to evaluating and analyzing a user's lifestyle, health status, and emotional state based on the information acquired.

[0760] "Question generation" refers to the process of formulating questions for medical professionals using analysis results.

[0761] A "generative AI model" refers to a computer program that uses artificial intelligence technology to extract useful information from data and generate appropriate questions.

[0762] "Adjustment" refers to modifying the generated questions so that they are in a form that is appropriate to the user's emotional state.

[0763] "Feedback" refers to the act of returning the results and answers obtained from the user's opinions and advice received at a medical institution to the system.

[0764] "Encryption" refers to a technology that transforms information in a specific way to securely transfer data and prevent unauthorized access.

[0765] "Medical institutions" refer to external organizations or professionals that provide medical services to users.

[0766] This invention is a system in which a user inputs information about their health status, lifestyle, and emotions via a terminal, and a server analyzes this data to generate questions useful for medical consultation. This system places particular emphasis on emotional information, enabling the generation of questions that take into account the user's emotional state.

[0767] Users input health-related information using devices such as smartphones and personal computers. This information includes not only lifestyle habits and health status, but also real-time emotional information of the user. Emotional information is obtained by analyzing facial expressions and voice tone using the camera and microphone built into the device. Users can also input their emotions through self-reporting.

[0768] The terminal organizes the acquired information and sends it to the server using encryption technology. Data security is ensured by using commonly used encryption protocols (e.g., TLS).

[0769] The server analyzes the user's health and emotional state based on the received data. An emotion analysis engine is used to identify emotional factors such as stress, joy, and anxiety that the user is experiencing. Based on the analysis results, a generative AI model generates questions for medical professionals that are tailored to the user's emotions. This enables flexible responses that are appropriate to the user's psychological state.

[0770] For example, the prompt message to the generating AI model could include instructions such as, "The user is feeling anxious. Please generate questions that address this state." This allows the system to provide consultation content optimized for the user's current situation. Based on this prompt message, the model generates appropriate questions and sends them from the server to the terminal.

[0771] The terminal presents the user with generated questions and allows the user to edit them as needed, creating a set of questions that the user can use when consulting with a medical institution. The feedback received after the consultation is then sent back to the server, and the system continues to improve its question generation algorithm and emotion engine based on this feedback. In this way, the system evolves with each use, enabling more personalized and effective medical consultations.

[0772] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0773] Step 1:

[0774] Users input health status, lifestyle habits, and emotional information using a device. The input data includes information directly entered by the user, as well as information from cameras and microphones that analyze facial expressions and voice tone. This allows for the acquisition of both the user's subjective state and objective emotional information.

[0775] Step 2:

[0776] The terminal organizes the acquired data and prepares it for transmission to the server using encryption technology. The input data is categorized as lifestyle, health status, and emotional information, and sent to the server after ensuring data reliability through encryption protocols such as TLS.

[0777] Step 3:

[0778] The server analyzes the data received from the terminal. Based on the input data, the emotion analysis engine identifies the user's emotional state (e.g., stress, anxiety, joy). As a result of the analysis, an overall health status and emotional profile of the user is generated.

[0779] Step 4:

[0780] The server uses a generative AI model based on the analysis results to generate questions that take into account the user's emotional state. The AI ​​model is given a prompt such as, "The user is feeling anxious. Generate questions that address this state," and an appropriate set of questions is generated.

[0781] Step 5:

[0782] The generated questions are sent from the server to the terminal. The terminal displays the questions to the user, helping them understand the questions and edit them as needed. The output here is a visualized list of questions provided to the user visually.

[0783] Step 6:

[0784] Users consult using questions generated by the medical institution. They then input the feedback they receive into their device. This feedback includes the medical professional's opinion based on the consultation and the user's response.

[0785] Step 7:

[0786] The terminal organizes the feedback data and sends it to the server. The server analyzes the feedback and uses it to improve the question generation algorithm and sentiment analysis engine. This process improves the accuracy and effectiveness of the system and prepares it for future use.

[0787] (Application Example 2)

[0788] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0789] Traditional service delivery systems based on users' lifestyles and health conditions suffer from insufficient personalization. They are unable to generate suggestions or questions that take into account the user's emotional state, thus failing to achieve adequate personalization. In particular, in the in-store shopping experience, product suggestions are not tailored to the user's current emotions or stress levels, making it difficult to improve customer satisfaction.

[0790] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0791] In this invention, the server includes means for acquiring information on lifestyle, health status, or emotional state via user input means, means for performing analysis based on the acquired information and generating questions or suggestions for experts, and means for presenting the generated questions or suggestions to the user and making them adjustable. This makes it possible to provide personalized product suggestions and questions in real time that are tailored to the user's emotions and health status.

[0792] "User input means" refers to a device or interface for receiving information from a user regarding their lifestyle, health status, or emotional state.

[0793] "Lifestyle habits" refer to the actions and habits that users perform on a daily basis, and include things like eating, exercise, and sleep patterns.

[0794] "Health status" is a concept that describes the overall physical and mental health of a user, and is often judged based on medical evaluations.

[0795] "Emotional state" refers to the emotions and moods a user feels at a particular time, and includes states such as stress, joy, and anxiety.

[0796] "Analysis means" refers to software or hardware devices used to analyze acquired information and extract meaning.

[0797] "Means for generating questions or suggestions" refers to a system for creating questions or product suggestions for users based on analyzed information.

[0798] "Adjustable means" refers to an interface or function that allows users to re-edit or modify generated questions or suggestions.

[0799] "Feedback" refers to information such as responses and opinions received from users, which can be used to improve the system.

[0800] The system implementing this invention includes a terminal used by the user, a server that analyzes the data, and an interface for displaying and adjusting the generated suggestions and questions. The user inputs information about their lifestyle, health status, and emotional state using a terminal such as a smartphone or smart glasses. The terminal transmits this information to the server.

[0801] The server uses an emotion analysis engine to analyze the received data. Specific examples include APIs from Microsoft Azure and IBM Watson. It analyzes the user's emotional state to understand conditions such as stress, joy, and anxiety. Based on the analyzed information, it utilizes an AI model to generate helpful questions or product suggestions for the user. The generating AI model provides personalized suggestions that take into account the user's health and emotional state. The generated questions and suggestions are sent to and displayed on the user's device.

[0802] Users can review the displayed questions or suggestions and make adjustments as needed. This adjusted information is then sent back to the server and stored as data. Based on the feedback, the server improves its generation algorithms and sentiment analysis engine, enabling more accurate suggestions.

[0803] As a concrete example, if stress is detected via smart glasses while a user is shopping in a store, relaxation-related product suggestions will be automatically displayed. An example of a prompt message to the generating AI model might be: "Consider the user's emotional state and generate a list of products best suited for relaxation. The user is experiencing stress, and recent health data includes: insufficient sleep and lack of exercise." This allows for specific and practical suggestions tailored to each user's individual needs.

[0804] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0805] Step 1:

[0806] Users input information about their lifestyle, health status, and emotional state using a device. The input information undergoes initial data formatting within the device and is then prepared for transmission to the server. Input can be in the form of text, audio, or images, and output is organized digital data.

[0807] Step 2:

[0808] The terminal sends the organized data to the server. The transmitted data is securely delivered to the server using a secure communication protocol. The input is formatted digital data, and the output is stored as a dataset on the server.

[0809] Step 3:

[0810] The server analyzes the received data and uses an emotion analysis engine to identify the user's emotional state. The input is a dataset stored on the server, and the output is the emotion analysis result. This process utilizes natural language processing and speech analysis techniques.

[0811] Step 4:

[0812] The server uses a generative AI model based on the analysis results to generate questions and product suggestions tailored to the user. The input is the sentiment analysis results, and the output is the generated questions or product suggestions. Prompt sentences are created in this step, serving as instructions for the AI.

[0813] Step 5:

[0814] The server sends the generated questions and suggestions to the user's terminal. The input is the generated questions and suggestions, and the output is what is displayed on the terminal. Communication is secure as it is performed using encryption technology.

[0815] Step 6:

[0816] The user reviews the questions and suggestions displayed on the terminal and makes adjustments as needed. The input is the user's judgment and adjustments, and the output is the adjusted data. The terminal then sends the adjusted data back to the server.

[0817] Step 7:

[0818] The server receives user feedback and uses it to improve its generation algorithms and sentiment analysis engine. The input is user feedback, and the output is data that contributes to improving the overall accuracy of the system. The improved algorithms are then reflected in subsequent analyses and generation processes.

[0819] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0820] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0821] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0822] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0823] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0824] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0825] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0826] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0827] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0828] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0829] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0830] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0831] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0832] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0833] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0834] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0835] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0836] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0837] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0838] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0839] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0840] The following is further disclosed regarding the embodiments described above.

[0841] (Claim 1)

[0842] A means of obtaining information about lifestyle habits or health status via a user input section,

[0843] A means of generating questions for medical professionals by performing analysis based on acquired information,

[0844] The generated questions are presented to the user, with adjustable means,

[0845] A means of sending the adjusted questions externally,

[0846] A means of obtaining external feedback and incorporating it into future analyses,

[0847] A system that includes this.

[0848] (Claim 2)

[0849] The system according to claim 1, further comprising means for improving the question generation algorithm using user feedback data.

[0850] (Claim 3)

[0851] The system according to claim 1, comprising encryption means for secure transmission and reception of data.

[0852] "Example 1"

[0853] (Claim 1)

[0854] A means for acquiring lifestyle information or health status information via a user input device,

[0855] Based on the acquired information, a means of generating questions for experts by performing analysis using machine learning algorithms,

[0856] The generated questions are presented to the user, and adjustable means are available.

[0857] A means of saving the adjusted questions and outputting them as information that can be brought to external organizations,

[0858] A means of obtaining feedback information and reflecting it in improving the analysis algorithm,

[0859] Information protection measures for securely sending and receiving data,

[0860] A system that includes this.

[0861] (Claim 2)

[0862] The system according to claim 1, further comprising means for improving the question generation method using feedback obtained from users.

[0863] (Claim 3)

[0864] The system according to claim 1, comprising an information protection method for encrypting and transmitting acquired data and for receiving data.

[0865] "Application Example 1"

[0866] (Claim 1)

[0867] A means of obtaining information about lifestyle habits or health status via a user input section,

[0868] A means of generating questions for a third party by performing analysis based on the acquired information,

[0869] The generated questions are presented to the user, with adjustable means,

[0870] A means of sending the adjusted questions externally,

[0871] A means of obtaining external feedback and incorporating it into future analyses,

[0872] Means of recommending products or services based on information,

[0873] A system that includes this.

[0874] (Claim 2)

[0875] The system according to claim 1, further comprising means for improving the question generation algorithm using user feedback data.

[0876] (Claim 3)

[0877] The system according to claim 1, comprising encryption means for secure transmission and reception of data.

[0878] "Example 2 of combining an emotion engine"

[0879] (Claim 1)

[0880] A means of acquiring emotional information in addition to the user's lifestyle or health status,

[0881] A means of analyzing the user's emotional state based on acquired information and generating questions for medical professionals,

[0882] A means of generating questions using a generative AI model and adjusting them to suit the user's emotional state,

[0883] Presenting users with adjusted questions and providing them with a means to edit them,

[0884] A means of sending the adjusted questions to an external medical institution,

[0885] A means of obtaining feedback from external medical experts and incorporating it into future improvements to the question generation algorithm,

[0886] Encryption methods for securely sending and receiving data,

[0887] A system that includes this.

[0888] (Claim 2)

[0889] The system according to claim 1, further comprising means for improving the question generation algorithm using user sentiment information.

[0890] (Claim 3)

[0891] The system according to claim 1, which is capable of acquiring emotional information via a user input unit.

[0892] "Application example 2 when combining with an emotional engine"

[0893] (Claim 1)

[0894] A means for acquiring information on lifestyle, health status, or emotional state via user input means,

[0895] A means of performing analysis based on acquired information and generating questions or suggestions for experts,

[0896] The generated questions or suggestions are presented to the user, with adjustable means,

[0897] A means of sending the adjusted questions or suggestions externally,

[0898] A means of obtaining external feedback and reflecting it in future analysis and generation,

[0899] A system that includes this.

[0900] (Claim 2)

[0901] The system according to claim 1, further comprising means for improving a generation algorithm or sentiment analysis engine using user feedback data.

[0902] (Claim 3)

[0903] The system according to claim 1, comprising encryption technology for secure transmission and reception of data. [Explanation of Symbols]

[0904] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of obtaining information about lifestyle habits or health status via a user input section, A means of generating questions for medical professionals by performing analysis based on acquired information, The generated questions are presented to the user, with adjustable means, A means of sending the adjusted questions externally, A means of obtaining external feedback and incorporating it into future analyses, A system that includes this.

2. The system according to claim 1, further comprising means for improving the question generation algorithm using user feedback data.

3. The system according to claim 1, comprising encryption means for secure transmission and reception of data.

Images