System

Abstract

To provide a system.SOLUTION: A system includes: means for analyzing numerical data included in the result of health screening provided by a user that indicates the health state of the user; means for inputting a specific prompt sentence to a generative AI model and thereby providing, on the basis of the analysis result, advice on the health habit and meal of the user and the exercise performed by the user, as advice on the health of the user; means for providing, on the basis of the analysis result, the user with a one-week recipe customized in conformity with the heath state of the user; means for analyzing the photographs of the meal registered by the user and proposing a plan for improving the meal of the user; and optimization means for optimizing the advice on health and the recipe on the basis of emotion by using an emotion engine that recognizes the emotion of the user and the generative AI model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, there are individual services that offer health management and dietary improvement suggestions based on the results of health checkups, but there is no system that can manage these services in a centralized manner and consistently provide recipe suggestions tailored to the user's health condition, as well as the purchasing and delivery of ingredients. [Means for solving the problem]

[0005] The system of the present disclosure includes: means for analyzing numerical data provided by a user and included in the results of a health check-up indicating the user's health condition; means for providing the user with health advice regarding the user's lifestyle habits, the user's diet, and the exercise the user performs based on the results of the analysis by inputting a specific prompt sentence into a generative AI model; means for suggesting to the user a week's worth of recipes customized to the user's health condition based on the results of the analysis; means for analyzing photos of meals registered by the user and suggesting suggestions for improving the user's diet; an emotion engine for recognizing the user's emotions; and optimization means for optimizing the health advice and the recipes based on the emotions using a generative AI model. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16] FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. [Figure 19] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system according to the embodiment 2 of the second embodiment in the case where an emotion engine is combined. [Figure 20] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when combined with an emotion engine. [Figure 21] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the embodiment 3 of the form example 3 in the case where an emotion engine is combined. [Figure 22] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The system of the present invention is provided with an interface for importing the results of a medical checkup. This interface can import the results of a medical checkup as digital data. For example, if the results of a medical checkup are provided in PDF format, the interface can read the PDF and

[0029] , and has the function of extracting it as text data.

[0030] "Example 2"

[0031] The system of the present invention has a function for analyzing the results of a medical checkup that has been imported. This analysis function analyzes various numerical data (blood pressure, blood glucose level, cholesterol level, etc.) contained in the results of the medical checkup to evaluate the user's health condition. For example, if the blood glucose level is high, it will determine that there is a risk of diabetes and provide advice on this.

[0032] "Example 3"

[0033] The system of the present invention has a function to suggest recipes for one week based on the analysis results. This recipe suggestion function selects and suggests recipes from a database that are suited to the user's health condition. For example, low-carbohydrate recipes are suggested to a user who is at risk of diabetes.

[0034] "Example 4"

[0035] The system of the present invention has a function to analyze photos of meals registered by users and propose improvements. This function identifies ingredients and dishes from the photos of meals and evaluates their nutritional balance. For example, for a user who eats a diet high in meat, a meal high in vegetables is suggested.

[0036] "Example 5"

[0037] The system of the present invention has a function to purchase and deliver ingredients based on a proposed recipe. This function links with existing ingredient purchasing and delivery services to purchase and deliver ingredients based on the ingredient list of the proposed recipe. For example, if the proposed recipe includes "chicken breast," the ingredient is sent to the purchasing and delivery service and delivered to the user's home.

[0038] The process flow of each embodiment will be described below.

[0039] "Example 1"

[0040] Step 1: The user uploads the results of a medical checkup in PDF format to the system. Step 2: The system reads the PDF and extracts it as text data.

[0041] "Example 2"

[0042] Step 1: The system analyzes various numerical data (blood pressure, blood sugar levels, cholesterol levels, etc.) from the text data extracted.

[0043] Step 2: Based on the analysis results, the user's health condition is evaluated and advice is provided.

[0044] Step 1: The system assesses the user's health status.

[0045] Step 2: Based on the evaluation results, select an appropriate recipe from the database and suggest it to the user.

[0046] "Example 4"

[0047] Step 1: A user uploads a photo of their meal into the system.

[0048] Step 2: The system identifies ingredients and dishes from the photo of the meal and evaluates its nutritional balance.

[0049] Step 3: Based on the evaluation results, suggest improvement ideas to the user.

[0050] "Example 5"

[0051] Step 1: Create an ingredients list for the recipe suggested by the system.

[0052] Step 2: Based on the ingredient list, connect with existing ingredient purchasing and delivery services to purchase and deliver the ingredients.

[0053] Example 1

[0054] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0055] Conventional health management systems required users to manually enter medical checkup results, which was time-consuming and labor-intensive. In addition, there was a lack of a way to efficiently analyze medical checkup results provided in PDF format and extract the necessary information. Furthermore, there was a need for a system that could provide health advice and recipe suggestions based on the extracted information.

[0056] The specification process by the specification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for reading the results of a medical checkup in PDF format and extracting it as text data, a means for formatting necessary information from the extracted text data and storing it in a database, a means for providing health advice based on the analysis results, a means for proposing recipes for one week based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, and a means for purchasing and delivering ingredients based on the proposed recipes. This enables the user to efficiently import and analyze the results of a medical checkup and manage his / her health.

[0057] "Results of a health checkup" refers to the results of detailed medical tests conducted as part of a health checkup, and are usually provided as digital data in PDF format or similar.

[0058] "Means of analysis" refers to software and algorithms used to read the results of health checkups provided as digital data, extract the necessary information, and format it.

[0059] "Means for providing health advice" refers to a system or function for evaluating a user's health condition based on the analyzed results of a medical checkup and providing appropriate lifestyle and medical advice.

[0060] "Means for suggesting a week's worth of recipes" refers to a system or function for creating a balanced meal plan and suggesting a week's worth of recipes based on the user's health condition.

[0061] "Means of analyzing food photos and suggesting improvement suggestions" refers to a system or function that uses image analysis technology to analyze food photos registered by users, evaluates nutritional balance, calories, etc., and suggests improvement suggestions.

[0062] "Means for purchasing and delivering ingredients" refers to a system or service for purchasing the necessary ingredients based on the proposed recipe and delivering them to a location specified by the user.

[0063] "Means of reading medical checkup results in PDF format and extracting them as text data" refers to software or algorithms for reading medical checkup results provided as a PDF file and extracting the contents as text data.

[0064] "Means for formatting the necessary information from the extracted text data and storing it in a database" refers to a system or function for analyzing the extracted text data, formatting the necessary information, and storing it in a database.

[0065] The present invention is a system for efficiently acquiring and analyzing the results of a medical checkup, and providing health advice and recipe suggestions to a user. A specific embodiment of this system will be described below.

[0066] Hardware and Software Used

[0067] The system consists of three main components: a server, a terminal, and a user. The server receives, analyzes, stores, and generates advice and recipe suggestions. The terminal is a device through which the user accesses the system and uploads data. The user is an individual who uses the system to manage their health.

[0068] The server uses the following software and libraries:

[0069] PDF analysis library: "PyMuPDF"

[0070] Database Management System: "SQLite"

[0071] Generative AI models: AI models for generating health advice and recipe suggestions

[0072] Program Processing

[0073] The server receives the results of medical checkups in PDF format uploaded by the user. The server uses the "PyMuPDF" library to open the PDF file and extract the text data. The extracted text data is formatted into the required information (e.g. blood test results, height, weight, etc.) using regular expressions. The formatted data is saved in the "SQLite" database.

[0074] Examples

[0075] As a concrete example, consider the case where a user uploads a file called "health checkup results 2023.pdf". The user accesses the system's web interface using a terminal browser, selects "health checkup results 2023.pdf" and uploads it. The server receives the uploaded PDF file and opens it using the "PyMuPDF" library. The server extracts the text data in the PDF file, extracts information such as blood test results and height and weight using regular expressions, and formats it. The formatted data is stored in the "SQLite" database.

[0076] Examples of prompt statements

[0077] Examples of prompt sentences to be input to a generative AI model include the following:

[0078] Prompt: I have uploaded "Health Check Results 2023.pdf". Please extract the blood test results and height and weight information.

[0079] Using this prompt, the generative AI model can extract the necessary information from the PDF file and provide it to the user.

[0080] The above is a specific embodiment for carrying out the present invention. This system enables a user to efficiently import and analyze the results of a medical checkup and manage his / her health.

[0081] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0082] Step 1:

[0083] A user uploads a PDF file.

[0084] A user accesses the system's web interface using a browser on the terminal. The user clicks the upload button and selects a PDF file, such as "health check result 2023.pdf", from the local storage. After selection, the user clicks the "upload" button. The input is the PDF file and the output is an upload request to the server.

[0085] Step 2:

[0086] The server receives the PDF file.

[0087] The server receives the PDF file uploaded by the user as an HTTP request. The server temporarily stores the received file. The input is the upload request, and the output is the temporarily stored PDF file.

[0088] Step 3:

[0089] The server parses the PDF file.

[0090] The server opens the temporarily saved PDF file using the PyMuPDF library. Specifically, the server reads the PDF file and prepares each page for parsing. The input is the temporarily saved PDF file, and the output is a parsable PDF object.

[0091] Step 4:

[0092] The server extracts the text data.

[0093] The server loops through each page in the PDF file and extracts the text data. The server takes the text of each page and combines it into a single text data. The input is a parsable PDF object and the output is the extracted text data.

[0094] Step 5:

[0095] The server formats the required information.

[0096] The server extracts necessary information from the extracted text data (for example, blood test results, height, weight, etc.) using regular expressions and formats it. Specifically, the server analyzes the text data and extracts information that matches a specific pattern. The input is the extracted text data, and the output is the formatted information.

[0097] Step 6:

[0098] The server stores the formatted data in a database.

[0099] The server stores the formatted data in a SQLite database. Specifically, the server connects to the database and inserts the formatted information into the appropriate tables. The input is the formatted information and the output is the data stored in the database.

[0100] The above are the specific processing steps of the program for this system, which enables users to efficiently import and analyze the results of medical checkups and manage their health.

[0101] (Application example 1)

[0102] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0103] Health management is an important issue in modern society, but it is not easy to accurately grasp an individual's health condition and provide appropriate advice and measures. In particular, there is a demand for a system that can efficiently import and analyze the results of medical checkups and quickly respond when abnormalities are detected. Other challenges include automating dietary suggestions based on the user's health condition and emergency responses in the event of an abnormality.

[0104] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0105] In this invention, the server includes a means for importing and analyzing the results of the medical checkup, a means for providing health advice based on the analysis results, a means for proposing recipes for one week based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, a means for purchasing and delivering ingredients based on the proposed recipes, a means for importing the results of the medical checkup in PDF format and extracting them as text data, a means for analyzing the extracted text data and notifying the user if an abnormality is detected, and a means for automatically contacting an emergency contact or a medical institution if an abnormality is detected. This makes it possible to efficiently manage the user's health condition and quickly respond to an abnormality if it is detected.

[0106] A "health checkup" is a type of medical examination, a medical service that aims to comprehensively examine each part and function of the body.

[0107] "Analysis" is a method for examining data or information in detail and understanding its structure and meaning.

[0108] "Health advice" refers to guidance and suggestions regarding lifestyle habits, diet, exercise, etc., provided based on the user's health condition.

[0109] A "recipe" is a list of steps and ingredients for making a particular dish.

[0110] A "photo of food" is image data of a meal taken by the user.

[0111] An "improvement proposal" is a specific suggestion or method for improving the current situation.

[0112] "Food purchase and delivery" is a service that purchases ingredients needed for cooking and delivers them to a specified location.

[0113] "PDF format" is an abbreviation for Portable Document Format, a file format for electronically storing and displaying documents.

[0114] "Text data" is data that represents character information in a digital format.

[0115] "Abnormal" refers to a state that deviates from the normal or norm, especially in the context of health, which may indicate a disease or disorder.

[0116] "User" refers to an individual who uses this system.

[0117] "Notification" is the act of notifying a user of specific information.

[0118] "Emergency contacts" refers to contact information for people or organizations to contact in the event of an emergency.

[0119] A "medical institution" is a facility that provides medical services, such as a hospital or clinic.

[0120] The system for implementing this invention is configured as follows. First, the server has a means for importing and analyzing the results of a medical checkup. This means has a function for reading the results of a medical checkup provided in PDF format and extracting them as text data. Specifically, a library such as PyMuPDF is used to extract text from the PDF.

[0121] The server then has the means to analyze the extracted text data and notify the user if an anomaly is detected. This analysis is done by checking whether it contains certain keywords (e.g. "anomaly"). If an anomaly is detected, a notification is sent to the user.

[0122] Furthermore, the server has a means for automatically contacting emergency contacts and medical institutions when an abnormality is detected. This means automatically contacts emergency contacts and medical institutions based on the information of the emergency contacts and medical institutions registered in advance by the user when an abnormality is detected.

[0123] The server also has a means for providing health advice based on the analysis results. This advice is generated based on the user's health condition and includes specific guidance and suggestions regarding lifestyle habits, diet, exercise, etc.

[0124] Furthermore, the server has a means for suggesting recipes for a week based on the analysis results. The recipe suggestions are customized according to the user's health condition. A means for analyzing photos of meals registered by the user and suggesting improvement suggestions is also included. The analysis is performed using image recognition technology to evaluate the user's diet and provide healthy improvement suggestions.

[0125] Finally, the server has a means for purchasing and delivering ingredients based on the suggested recipe, in cooperation with existing ingredient purchasing and delivery services, to automatically order ingredients required for the recipe selected by the user and deliver them to a specified location.

[0126] As a concrete example, a user uploads the results of a medical checkup in PDF format to the app. The server extracts text from the PDF and evaluates the health condition. If an abnormality is detected, a notification is sent to the user and emergency contacts and medical institutions are automatically contacted. In addition, recipes for a week are suggested based on the user's health condition, and the necessary ingredients are automatically ordered.

[0127] An example of a prompt for the generative AI model is, "Please upload the results of your health check in PDF format. We will evaluate your health condition and notify you if any abnormalities are detected."

[0128] In this way, it is possible to efficiently manage the user's health condition and quickly respond if an abnormality is detected.

[0129] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0130] Step 1:

[0131] The user uploads the results of his / her medical checkup in PDF format to the terminal.

[0132] Input: Medical checkup results (PDF file)

[0133] Output: Uploaded PDF file

[0134] Specific operations: The user opens the application, selects the PDF file containing the results of the health check, and presses the upload button.

[0135] Step 2:

[0136] The server receives the uploaded PDF file and extracts it as text data.

[0137] Input: Uploaded PDF file

[0138] Output: Extracted text data

[0139] What happens next: The server uses the PyMuPDF library to open the PDF file and extract the text from each page.

[0140] Step 3:

[0141] The server analyzes the extracted text data and evaluates the health status.

[0142] Input: Extracted text data

[0143] Output: Health status assessment results

[0144] Specific operation: The server searches for specific keywords (e.g., "abnormal") in the text data and evaluates the health status.

[0145] Step 4:

[0146] The server notifies the user if any abnormality is detected based on the health status evaluation results.

[0147] Input: Health status assessment results

[0148] Output:Notification to the user

[0149] Specific operation: The server checks the evaluation results, and if an abnormality is detected, it sends a notification to the user's device.

[0150] Step 5:

[0151] If the server detects an abnormality, it will automatically contact emergency contacts and medical institutions.

[0152] Input: Health assessment results, emergency contact information

[0153] Output: Notification to emergency contacts and medical institutions

[0154] Specific operation: The server automatically contacts emergency contacts and medical institutions that the user has registered in advance if an abnormality is detected.

[0155] Step 6:

[0156] The server provides health advice based on the analysis results.

[0157] Input: Health status assessment results

[0158] Output: Health advice

[0159] Specific operation: Based on the evaluation results, the server generates specific advice regarding lifestyle habits, diet, exercise, etc. and provides it to the user.

[0160] Step 7:

[0161] The server will suggest recipes for the week based on the analysis results.

[0162] Input: Health status assessment results

[0163] Output: Recipe suggestions for a week

[0164] Specific operation: The server generates recipes customized to the user's health condition and suggests them to the user.

[0165] Step 8:

[0166] The server analyzes the photos of meals registered by the user and suggests improvement suggestions.

[0167] Input: Food photo

[0168] Output: Dietary improvement suggestions

[0169] Specific operation: The server uses image recognition technology to analyze photos of food, evaluate the user's diet, and provide healthy improvement suggestions.

[0170] Step 9:

[0171] The server purchases and delivers ingredients based on the suggested recipe.

[0172] Input: Suggested recipe

[0173] Output: Purchase and delivery of ingredients

[0174] Specific operation: The server works with existing ingredient purchasing and delivery services to automatically order ingredients needed for the recipe selected by the user and deliver them to the specified location.

[0175] Example 2

[0176] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0177] Conventional health management systems are required to not only analyze the results of medical checkups, but also to provide specific health advice to users. However, these systems provide poor quality advice based on the analysis results, making it difficult to provide specific advice appropriate to the user's health condition. In addition, suggestions for improving the user's diet and purchasing and delivery of ingredients are insufficient, making comprehensive health management difficult.

[0178] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0179] In this invention, the server includes means for importing and analyzing the results of a medical checkup, means for providing health advice based on the analysis results, means for generating prompt sentences based on the analysis results and inputting them into the generative AI model, means for providing the user with advice returned from the generative AI model, means for proposing recipes for one week based on the analysis results, means for analyzing photos of meals registered by the user and proposing improvement plans, and means for purchasing and delivering ingredients based on the proposed recipes. This allows for the provision of specific, high-quality advice based on the user's health condition, enabling comprehensive health management.

[0180] The "results of a medical checkup" are the results of tests conducted as part of a health checkup, and include various numerical data such as blood pressure, blood sugar level, and cholesterol level.

[0181] The "analyzing means" refers to a device or program that performs processing to evaluate the user's health condition based on the captured data.

[0182] The "means for providing health advice" refers to a device or program that provides the user with appropriate health management guidance and suggestions based on the analysis results.

[0183] A "means for generating a prompt sentence" is a device or program that creates a sentence to be input into a generative AI model based on the analysis results.

[0184] A "generative AI model" is an artificial intelligence model that uses natural language generation technology to generate appropriate advice based on input prompt sentences.

[0185] "Means for inputting to a generative AI model" refers to a device or program that sends the generated prompt sentence to the generative AI model and causes it to generate advice.

[0186] A "means for providing generated advice to a user" is a device or program that communicates the advice returned from the generative AI model to a user.

[0187] The "means for suggesting recipes" refers to a device or program that suggests meal menus suitable for the user's health condition based on the analysis results.

[0188] The "means for analyzing food photos" refers to a device or program that analyzes food photos registered by users and evaluates the nutrients and calories.

[0189] The "means for suggesting improvements" is a device or program that makes suggestions for healthier meals based on the analyzed food photos.

[0190] The "means for purchasing and delivering ingredients" refers to a device or program that purchases the necessary ingredients based on the proposed recipe and delivers them to the user.

[0191] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0192] This invention is a system that analyzes the results of a medical checkup provided by a user and provides health advice. The system operates in cooperation with a server, a terminal, and a user.

[0193] 1. Generating the system program

[0194] The server generates a program to analyze the results of the medical checkup provided by the user. This program is developed using Python, and libraries such as Pandas and NumPy are used for data analysis. A model using natural language generation technology (e.g., GPT-4 (registered trademark)) is used as the generative AI model.

[0195] 2. Explanation of program processing

[0196] The server receives the results of medical checkups uploaded by users. These results are often provided as CSV or Excel files. The server reads these files and converts them into a data frame using Pandas.

[0197] The server then analyzes the various numerical data in the data frame (blood pressure, blood glucose level, cholesterol level, etc.). For example, if the blood glucose level is above a certain threshold, the server determines that the person is at risk for diabetes. This determination is based on pre-defined criteria.

[0198] Based on the analysis results, the server generates advice for the user. This advice is created using a generative AI model. Natural language generation models such as GPT-4 are used for the generative AI model. The server inputs the analysis results into the generative AI model as prompt sentences to generate appropriate advice.

[0199] 3. Specific Examples

[0200] For example, if the user’s blood glucose level is 150 mg / dL, the server will input the following prompt into the generative AI model:

[0201] User's blood sugar level is 150 mg / dL. He may be at risk for diabetes. Please provide appropriate advice.

[0202] Based on this prompt, the generative AI model would generate advice like this:

[0203] Your blood sugar level is 150 mg / dL, so you are at risk for diabetes. First, review your diet and limit your sugar intake. It is also important to incorporate regular exercise. We recommend that you see a doctor and start appropriate treatment.

[0204] The server provides this advice to the user, who can review it through the system's web interface.

[0205] In addition, the server will suggest recipes for the week based on the analysis results. It is also possible to analyze photos of meals registered by the user and suggest improvements. It also includes a means to purchase and deliver ingredients based on the suggested recipes, providing comprehensive support for the user's health management.

[0206] In this way, the system can provide specific, high-quality advice based on the user's health condition, enabling comprehensive health management.

[0207] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0208] Step 1:

[0209] A user uploads the results of a medical checkup.

[0210] The user logs into the system's web interface, clicks the "Choose File" button to select the medical examination result file from his / her computer, and then clicks the "Upload" button. The input is a file in CSV or Excel format, and the output is the file sent to the server.

[0211] Step 2:

[0212] The server receives the uploaded data.

[0213] The server receives the file uploaded by the user and checks the format of the file. For example, if it is in CSV format, it uses the Pandas read_csv function to read the data. The input is the file sent by the user, and the output is the data converted to a data frame.

[0214] Step 3:

[0215] The server parses the data.

[0216] The server analyzes various numerical data (blood pressure, blood glucose level, cholesterol level, etc.) in the data frame. For example, it checks whether the blood glucose level exceeds a certain threshold. Pre-defined reference values ​​are used for the analysis. The input is the data frame, and the output is the analysis result.

[0217] Step 4:

[0218] The server generates a prompt based on the parsing results.

[0219] The server creates a prompt sentence to be input to the generative AI model based on the analysis results. For example, if the blood glucose level is 150 mg / dL, the following prompt sentence is generated: "The user's blood glucose level is 150 mg / dL. There is a risk of diabetes. Please provide appropriate advice." The input is the analysis result, and the output is the prompt sentence.

[0220] Step 5:

[0221] The server inputs a prompt sentence into the generative AI model.

[0222] The server sends the generated prompt sentence to the generative AI model. The generative AI model uses a natural language generation model such as GPT-4. The input is the prompt sentence, and the output is the advice returned from the generative AI model.

[0223] Step 6:

[0224] The server provides the generated advice to the user.

[0225] The server receives the advice returned by the generative AI model and provides it to the user. The user can review the advice through the system's web interface. The input is the advice from the generative AI model, and the output is the advice provided to the user.

[0226] Step 7:

[0227] The server will suggest recipes for the week based on the analysis results.

[0228] The server then proposes a week's worth of recipes suited to the user's health condition based on the analysis results. The input is the analysis results, and the output is the proposed recipes.

[0229] Step 8:

[0230] The server analyzes the photos of meals registered by the user and suggests improvements.

[0231] Users upload photos of their meals to the system. The server uses image analysis technology to analyze the photos and evaluate their nutritional and calorie content. The input is the photo of the meal, and the output is the analysis results and improvement suggestions.

[0232] Step 9:

[0233] The server purchases and delivers ingredients based on the suggested recipe.

[0234] The server purchases the necessary ingredients based on the proposed recipe and delivers them to the user. The input is the proposed recipe and the output is the purchased ingredients and their delivery.

[0235] (Application example 2)

[0236] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0237] Conventional health management systems do not only analyze the results of medical checkups, but also do not adequately monitor the user's daily diet and health condition. In addition, users have to manually enter data, which is time-consuming and makes it difficult to accurately evaluate their health condition. Furthermore, they do not provide specific advice or suggestions for improvement, making it unclear how users should maintain their health.

[0238] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0239] In this invention, the server includes a means for importing and analyzing the results of a medical checkup, a means for providing health advice based on the analysis results, a means for proposing a week's worth of recipes based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, a means for purchasing and delivering ingredients based on the proposed recipes, a means for extracting text from images using optical character recognition technology, a means for analyzing numerical data from the extracted text, and a means for evaluating a health condition based on the numerical data and providing specific advice. This allows a user to easily import and analyze the results of a medical checkup and manage their daily health. The server includes: means for analyzing numerical data provided by a user and included in the results of a medical checkup indicating the user's health condition; means for providing the user with health advice regarding the user's lifestyle habits, the user's diet, and the exercise the user performs based on the results of the analysis by inputting a specific prompt sentence into a generative AI model; means for suggesting to the user a week's worth of recipes customized to the user's health condition based on the results of the analysis; means for analyzing meal photos registered by the user and suggesting suggestions for improving the user's diet; an emotion engine for recognizing the user's emotions; and optimization means for optimizing the health advice and the recipes based on the emotions using a generative AI model. The server includes: means for analyzing numerical data provided by a user and included in the results of a medical checkup indicating the user's health condition; means for providing the user with health advice regarding the user's lifestyle habits, the user's diet, and the exercise the user performs based on the results of the analysis by inputting a specific prompt sentence into a generative AI model; means for suggesting to the user a week's worth of recipes customized to the user's health condition based on the results of the analysis; means for analyzing meal photos registered by the user and suggesting suggestions for improving the user's diet; an emotion engine for recognizing the user's emotions; and optimization means for optimizing the health advice and the recipes based on the emotions using a generative AI model.

[0240] The "means for importing and analyzing medical checkup results" refers to technology for importing the results of a medical checkup taken by a user as digital data and analyzing that data.

[0241] The "means for providing health advice based on analysis results" refers to a technology that evaluates the user's health condition based on analyzed health checkup data and provides specific advice for maintaining or improving health.

[0242] The "means for proposing a week's worth of recipes based on the analysis results" is a technology that proposes a week's worth of meal menus suited to the user's health condition based on the analysis results of a health checkup.

[0243] The "means for analyzing photos of meals registered by users and proposing improvement suggestions" refers to a technology that analyzes photos of meals taken and registered by users, evaluates their content, and proposes improvement suggestions for healthier meals.

[0244] "Means for purchasing and delivering ingredients based on the proposed recipe" refers to technology for purchasing ingredients needed for the proposed meal menu and delivering them to the user.

[0245] "Means of extracting text from an image using optical character recognition technology" refers to technology that uses OCR (optical character recognition) technology to extract character information from image data.

[0246] "Means for analyzing numerical data from extracted text" refers to a technology for analyzing numerical data such as blood pressure, blood sugar levels, and cholesterol levels from character information extracted using OCR technology.

[0247] The "means for evaluating health condition based on numerical data and providing specific advice" refers to a technology for evaluating a user's health condition based on analyzed numerical data and providing specific advice for maintaining or improving health.

[0248] A system for implementing this invention is configured as follows. First, a user takes a picture of the results of a medical checkup using the camera of a smartphone. The taken image is processed by an application in the smartphone. The application extracts text from the image using optical character recognition technology (OCR).

[0249] The extracted text is analyzed by the analysis module, which uses Python's pytesseract library for OCR processing and regular expressions to extract numerical data such as blood pressure, blood glucose, cholesterol levels, etc. The extracted numerical data is then sent to the evaluation module for further evaluation of health status.

[0250] The evaluation module evaluates the user's health condition based on the extracted numerical data and generates specific advice that is customized according to the user's health condition. For example, if the blood sugar level is high, it indicates a risk of diabetes and provides specific advice on diet and exercise.

[0251] In addition, it also includes a module that suggests recipes for a week based on the analysis results. This module generates and suggests recipes that are tailored to the user's health condition. It also includes a module that purchases and delivers ingredients based on the suggested recipes, and delivers ingredients to the user in cooperation with existing food purchasing and delivery services.

[0252] As a concrete example, by inputting the following prompt sentences into a generative AI model, it is possible to design a detailed application and generate code:

[0253] Example prompt:

[0254] Develop an application that analyzes images taken by users on their smartphones during medical checkups and evaluates their health condition. The following requirements must be met:

[0255] 1. Extract text from images using OCR technology.

[0256] 2. Extract blood pressure, blood glucose, and cholesterol values ​​from the extracted text using regular expressions.

[0257] 3. Evaluate health status based on extracted numerical data and provide specific advice.

[0258] 4. The libraries used will be Python's pytesseract and re.

[0259] In this way, users can easily import and analyze the results of their medical checkups and manage their health on a daily basis.

[0260] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0261] Step 1:

[0262] The user takes a picture of the results of the medical checkup using the smartphone camera.

[0263] Input: An image of a paper showing the results of a health check.

[0264] Output: Image files stored on your smartphone.

[0265] Specific operation: The user starts the camera app on their smartphone and takes a picture of the paper containing the results of their medical checkup. The captured image is saved in the smartphone.

[0266] Step 2:

[0267] The device uses optical character recognition technology (OCR) to extract text from the image.

[0268] Input: An image file stored on your smartphone.

[0269] Output: The extracted text data.

[0270] How it works: An application in the smartphone uses the pytesseract library to extract text information from the saved image file. The extracted text data is stored in memory.

[0271] Step 3:

[0272] The terminal parses numerical data from the extracted text.

[0273] Input: The extracted text data.

[0274] Output: Numerical data such as blood pressure, blood glucose, cholesterol levels, etc.

[0275] Specific operation: The application uses regular expressions to parse numerical data such as blood pressure, blood glucose level, and cholesterol level from the extracted text data, and stores this numerical data in variables.

[0276] Step 4:

[0277] The device assesses your health condition based on numerical data and provides specific advice.

[0278] Input: Numerical data such as blood pressure, blood glucose, cholesterol levels, etc.

[0279] Output: Health status assessment results and specific recommendations.

[0280] Specific operation: The application evaluates the user's health condition based on the analyzed numerical data. For example, if the blood sugar level is high, it may determine that the user is at risk for diabetes, and generate specific advice regarding diet and exercise. The generated advice is displayed to the user.

[0281] Step 5:

[0282] The device will suggest recipes for the week based on the analysis results.

[0283] Input: Health status assessment results.

[0284] Output: A week's worth of recipes.

[0285] Specific behavior: The application generates and suggests a week's worth of meals based on the user's health status. The suggested recipes are displayed to the user.

[0286] Step 6:

[0287] The terminal purchases and delivers ingredients based on the suggested recipe.

[0288] Enter: a week's worth of recipes.

[0289] Output: Ingredients shopping list and delivery schedule.

[0290] What it does: The application generates a list of ingredients needed for the suggested recipe, then integrates with existing food shopping and delivery services to deliver the ingredients to the user, and notifies the user of the delivery schedule.

[0291] Example 3

[0292] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0293] Conventional health management systems do not adequately provide specific dietary improvement suggestions or recipe suggestions based on the user's health condition. In addition, they lack the functionality to purchase and deliver ingredients based on the suggested recipes, making it difficult for users to actually follow the suggested meals. Furthermore, they lack the functionality to analyze photos of meals to evaluate their nutritional balance and suggest improvement suggestions. To solve these issues, a system is needed that integrates specific recipe suggestions based on the user's health condition with the purchasing and delivery of ingredients.

[0294] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0295] In this invention, the server includes means for importing and analyzing the results of a medical checkup, means for providing health advice based on the analysis results, means for proposing recipes for one week based on the analysis results, means for analyzing images of meals registered by the user and proposing improvement plans, means for purchasing and delivering ingredients based on the proposed recipes, means for analyzing images of meals using image analysis software, means for evaluating nutritional balance by comparing with a nutrition database, and means for purchasing and delivering ingredients in cooperation with an ingredient purchasing and delivery service, thereby enabling specific recipe suggestions based on the user's health condition and the purchase and delivery of ingredients.

[0296] "Results of a health checkup" refers to data obtained as a result of a health examination or test.

[0297] An "analytical means" is a method or device for analyzing data and extracting meaningful information.

[0298] A "means for providing health advice" is a method or device that provides health advice or suggestions to a user based on the analysis results.

[0299] A "means for suggesting a week's worth of recipes" is a method or device that suggests a week's worth of meal menus based on the user's health condition and preferences.

[0300] "Meal images registered by a user" refers to photo data of meals that a user uploads to the system.

[0301] The "means for proposing improvement plans" refers to a method or device that makes suggestions for improving the user's diet based on the analysis results.

[0302] "Means for purchasing and delivering ingredients based on a proposed recipe" refers to a method or device for purchasing ingredients required for a proposed recipe and delivering them to the user.

[0303] "Image analysis software" is software for analyzing image data and extracting specific information.

[0304] A "nutrition database" is a database that collects information about the nutritional components of foods.

[0305] A "means for evaluating nutritional balance" is a method or device for evaluating the nutritional balance of meal contents.

[0306] The "foodstuff purchase and delivery service" is a service that allows a user to purchase the foodstuffs they need and have them delivered to a specified location.

[0307] The system of the present invention has the function of suggesting recipes for a week based on the user's health condition, analyzing photos of meals to provide improvement suggestions, and purchasing and delivering ingredients based on the suggested recipes. This system operates in cooperation with the server, the terminal, and the user.

[0308] Hardware and Software Used

[0309] Server: A server with high-performance computing power is used to analyze data, suggest recipes, purchase ingredients, and manage deliveries.

[0310] Device: A device, such as a smartphone or tablet, that a user uses to take photos of food, check recipes, and receive notifications.

[0311] Image analysis software: Analyze the food photos using image analysis software such as TENSORFLOW® or OpenCV.

[0312] Nutrition Databases: Use nutrition databases such as the USDA Food Database to get nutrition information for ingredients.

[0313] Grocery shopping and delivery services: Integrate with existing grocery shopping and delivery services such as Amazon Fresh and Instacart.

[0314] Specific operation of the system

[0315] 1. The user takes a photo of the meal on the device and uploads it to the system. The user can easily send the photo of the meal to the system using a dedicated application.

[0316] 2. The device sends the captured photo to the server. The device sends the photo data and metadata such as the user ID and timestamp to the server via the Internet.

[0317] 3. The server analyzes the received photos using image analysis software, using TensorFlow and OpenCV to identify the ingredients and dishes in the photos.

[0318] 4. The server compares the analysis results with the nutrition database to evaluate the nutritional balance. The server compares the identified ingredients and dish information with the USDA food database to evaluate the nutritional balance of the meal.

[0319] 5. The server generates dietary improvement suggestions based on the evaluation results. For example, for a user who eats a diet high in meat, the server suggests a diet high in vegetables.

[0320] 6. The server sends the generated improvement proposal to the user's device and notifies the user. The user receives the notification on the device and confirms the improvement proposal.

[0321] 7. The user selects a recipe suggested by the system. The user checks the list of recipes suggested by the system on their device and selects the recipe they like.

[0322] 8. The server generates an ingredient list for the selected recipe. The server obtains the details of the selected recipe and generates the required ingredient list.

[0323] 9. The server communicates with the food purchasing and delivery service and transmits the list of ingredients. The server communicates with the food purchasing and delivery service through the API and transmits the list of ingredients.

[0324] 10. The food purchasing and delivery service purchases the ingredients based on the list and delivers them to the user's home. The user receives the delivered ingredients and prepares the dish based on the suggested recipe.

[0325] Examples

[0326] If a photo contains steak and potatoes, the server will identify it, evaluate its nutritional balance, and suggest a recipe for a salad that includes lots of vegetables.

[0327] If the user selects the recipe "chicken breast and vegetable stir fry," the server generates an ingredients list (such as chicken breast, broccoli, and carrots) and sends it to an ingredient purchasing and delivery service, which purchases the ingredients and delivers them to the user's home.

[0328] Examples of prompt statements

[0329] "Analyze the meal in this photo, evaluate its nutritional balance, and suggest improvements."

[0330] "Purchase the ingredients needed for this recipe and have them delivered to the user's home."

[0331] The above is a specific embodiment of the system of the present invention. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0332] System program processing flow

[0333] Example 4: Food photo analysis and improvement proposal function

[0334] Step 1:

[0335] Users take photos of their meals with their devices and upload them to the system.

[0336] Input: A photo of a meal taken by the user.

[0337] Specific operations: Users use the camera on their smartphone or tablet to take a photo of their meal and upload it to the system via a dedicated application.

[0338] Output: The uploaded photo data.

[0339] Step 2:

[0340] The device sends the photograph to the server.

[0341] Input: The uploaded photo data.

[0342] Specific operation: The device sends the photo data and metadata such as user ID and timestamp to the server via the Internet.

[0343] Output: The photo data sent to the server.

[0344] Step 3:

[0345] The photos received by the server are analyzed using image analysis software.

[0346] Input: The photo data sent to the server.

[0347] How it works: The server uses image analysis software such as TensorFlow and OpenCV to identify the ingredients and dishes in the photo.

[0348] Output: Identified ingredients and dish information.

[0349] Step 4:

[0350] The server compares the analysis results with a nutritional database to evaluate nutritional balance.

[0351] Input: Identified ingredients and dish information.

[0352] What it does: The server checks the identified ingredients and dish information against the USDA food database to obtain the nutritional content of each ingredient, then evaluates the nutritional balance of the entire meal.

[0353] Output: Nutritional balance assessment results.

[0354] Step 5:

[0355] The server generates dietary improvement suggestions based on the evaluation results.

[0356] Input: Nutritional balance assessment results.

[0357] Specific operation: Based on the evaluation results, the server generates improvement proposals that take into account the user's health condition and dietary balance. For example, for a user who eats a diet high in meat, the server suggests a diet high in vegetables.

[0358] Output: Dietary improvement suggestions.

[0359] Step 6:

[0360] The improvement proposals generated by the server are sent to the user's device and notified.

[0361] Input: Dietary improvement suggestions.

[0362] Specific operation: The server sends the generated improvement proposal to the user's device and notifies the user. The user receives the notification on the device and confirms the improvement proposal.

[0363] Output: The improvement suggestions sent to the user's device.

[0364] Example 5: Recipe-based food purchasing and delivery function

[0365] Step 1:

[0366] The user selects a recipe suggested by the system.

[0367] Input: A list of suggested recipes.

[0368] Specific operation: The user checks the list of recipes suggested by the system on their device and selects the recipe they like.

[0369] Output: The selected recipe information.

[0370] Step 2:

[0371] The server generates an ingredient list for the selected recipe.

[0372] Input: Selected recipe information.

[0373] Specific operation: The server retrieves detailed information about the selected recipe and generates a list of ingredients, including the exact ingredients and the quantities required.

[0374] Output: Ingredients list.

[0375] Step 3:

[0376] The server cooperates with the food purchasing and delivery service to transmit the food list.

[0377] Input: Ingredient list.

[0378] Specific operation: The server communicates with the food purchasing and delivery service via API and sends the food list.

[0379] Output: A list of ingredients sent to the grocery shopping and delivery service.

[0380] Step 4:

[0381] A food purchasing and delivery service purchases food based on the list and delivers it to the user's home.

[0382] Input: Ingredient list.

[0383] Specific operation: The food purchasing and delivery service purchases ingredients based on the received ingredient list and delivers them to the user's home.

[0384] Output: Groceries delivered to the user's home.

[0385] Step 5:

[0386] The user receives the delivered ingredients and prepares the dish based on the suggested recipe.

[0387] Input: Delivered ingredients.

[0388] Specific behavior: The user receives the delivered ingredients and prepares a dish based on the suggested recipe. During the cooking process, the user refers to the recipe instructions provided by the system.

[0389] Output: The finished dish.

[0390] The above are the specific processing steps of the system of the present invention.

[0391] (Application example 3)

[0392] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0393] In modern society, health management is an important issue, but there is no system that can suggest meals tailored to individual health conditions or consistently purchase and deliver ingredients. There is also a lack of means to evaluate the nutritional balance of the meals that users consume on a daily basis and provide suggestions for improvement. This makes it difficult for users to obtain specific guidelines for maintaining a healthy diet.

[0394] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0395] In this invention, the server includes a means for importing and analyzing the results of a medical checkup, a means for providing health advice based on the analysis results, a means for proposing recipes for one week based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, a means for purchasing and delivering ingredients based on the proposed recipes, and a means including a smartphone application for proposing recipes based on the user's health condition, analyzing photos of meals to provide improvement plans, and purchasing and delivering ingredients based on the proposed recipes. This allows the user to receive meal suggestions tailored to their own health condition, and further enables the easy purchase and delivery of the ingredients.

[0396] "Results of a health checkup" refer to the results of detailed tests conducted as part of a health checkup, and are data used to evaluate an individual's health condition.

[0397] An "analyzing means" is a method or device for analyzing input data and extracting specific information.

[0398] A "means for providing health advice" is a method or device that presents specific guidelines for maintaining or improving health to a user based on the analysis results.

[0399] A "means for suggesting a week's worth of recipes" is a method or device that suggests meal menus for a seven-day period based on the user's health condition and preferences.

[0400] The "means for analyzing photos of meals and proposing improvement suggestions" refers to a method or device that analyzes images of meals registered by users, evaluates their nutritional balance, and makes suggestions for improvement.

[0401] "Means for purchasing and delivering ingredients" refers to a method or device for purchasing the necessary ingredients based on the proposed recipe and delivering them to a location specified by the user.

[0402] A "smartphone application" is software that runs on a smartphone and allows users to use functions such as health management, meal suggestions, food purchasing and delivery, etc.

[0403] "User's health condition" refers to the physical health condition, disease risk, allergy information, and so on, of each individual user.

[0404] "Existing food purchasing and delivery services" refers to service providers that purchase and deliver food ingredients that are already available in the market.

[0405] A system for implementing this invention includes a smartphone application that suggests recipes based on the user's health condition, analyzes photos of meals to provide improvement suggestions, and purchases and delivers ingredients based on the suggested recipes.

[0406] Hardware and Software Used

[0407] Hardware: Smartphone

[0408] software:

[0409] TensorFlow / Keras: A deep learning model used to analyze food photos

[0410] OpenCV: Image processing library

[0411] Requests: A library for sending HTTP requests.

[0412] Data processing and calculation

[0413] server

[0414] The server receives and analyzes the medical checkup results sent by the user. Based on the analysis results, the server evaluates the user's health condition and provides health advice. It also suggests recipes for the week based on the analysis results. It also analyzes photos of meals registered by the user, evaluates their nutritional balance, and suggests improvements.

[0415] Device (smartphone)

[0416] The smartphone application suggests recipes based on the user's health status and analyzes photos of meals to provide improvement suggestions. When a user uploads a photo of a meal, the application uses TensorFlow / Keras to identify ingredients and dishes, and OpenCV to process the image. The analysis results are fed back to the user, and improvement suggestions are provided if necessary.

[0417] User

[0418] Through the smartphone application, users can receive recipe suggestions tailored to their health condition. They can then purchase the ingredients needed based on the suggested recipe and have them delivered to their home using a delivery service. This gives users concrete guidance on how to maintain a healthy diet.

[0419] Examples

[0420] For example, if User A, age 30, is at risk of diabetes, the application will suggest low-carb recipes. When User A uploads a photo of a meal, the application will analyze the photo, evaluate the nutritional balance, and provide suggestions for improvement. Based on the suggested recipe, the user can purchase ingredients such as chicken breast and vegetables and have them delivered to their home using a delivery service.

[0421] Examples of prompt statements

[0422] Enter your health data:

[0423] Age: 30

[0424] Weight: 70

[0425] Height: 175

[0426] Diabetes risk: Yes

[0427] Allergies: Peanuts

[0428] Would you like to purchase and deliver ingredients based on the suggested recipe? (Yes / No)

[0429] In this way, a system will be created that allows users to receive meal suggestions tailored to their own health condition and handle the entire process from purchasing ingredients to having them delivered.

[0430] The flow of the specific process in the application example 3 will be described with reference to FIG.

[0431] Step 1:

[0432] The server takes in the medical checkup results sent by the user. The input is the medical checkup result data, and the output is the data ready for analysis. Specifically, the server receives the user's health data and stores it in a database.

[0433] Step 2:

[0434] The server analyzes the imported medical checkup results. The input is the health data stored in step 1, and the output is the analysis results. Specifically, the server runs the health data through an analysis algorithm to evaluate the user's health condition.

[0435] Step 3:

[0436] The server provides health advice based on the analysis results. The input is the analysis result from step 2, and the output is health advice. Specifically, the server generates appropriate health advice for the user based on the analysis results and sends it to the user's device.

[0437] Step 4:

[0438] The server proposes recipes for one week based on the analysis results. The input is the analysis results from step 2, and the output is recipes for one week. Specifically, the server selects and proposes recipes from the database that are suitable for the user's health condition.

[0439] Step 5:

[0440] A user uploads a photo of a meal through a smartphone application. The input is the photo of the meal taken by the user, and the output is the image data sent to the server. Specifically, a user opens the application, takes a photo of the meal, and uploads it.

[0441] Step 6:

[0442] The server analyzes the uploaded photos of the food. The input is the image data sent in step 5, and the output is the identification results of the ingredients and dishes. Specifically, the server uses a TensorFlow / Keras model to analyze the images and identify the ingredients and dishes.

[0443] Step 7:

[0444] The server proposes meal improvement suggestions based on the analysis results. The input is the classification result from step 6, and the output is improvement suggestions. Specifically, the server evaluates nutritional balance based on the classification results, and generates and sends improvement suggestions to the user.

[0445] Step 8:

[0446] The server purchases and delivers ingredients based on the proposed recipe. The input is the recipe from step 4, and the output is an ingredient purchase and delivery request. Specifically, the server lists the ingredients required for the proposed recipe and sends a request to an existing ingredient purchase and delivery service.

[0447] Step 9:

[0448] A user checks the purchase and delivery status of ingredients through a smartphone application. The input is the status of the purchase and delivery request sent from the server, and the output is the delivery status displayed to the user. Specifically, a user can open the application and check the purchase and delivery status of ingredients.

[0449] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0450] "Example 1"

[0451] In one embodiment of the present invention, an emotion engine is incorporated. This emotion engine recognizes the user's emotions and uses this information throughout the system. Specifically, when the user speaks to the system or inputs text, the emotion engine infers the user's emotional state from the content. This emotional state is taken into account when providing health advice and suggesting recipes. For example, if it is estimated that the user is feeling stressed, recipes that include ingredients with a relaxation effect are suggested. Health advice is also optimized according to the user's emotional state.

[0452] "Example 2"

[0453] In another embodiment of the present invention, the emotion engine suggests dietary improvement suggestions based on the user's emotional state. Specifically, when a user uploads a photo of a meal, the system evaluates the nutritional balance of the meal. The emotion engine then suggests dietary improvement suggestions taking into account the user's emotional state. For example, if the user is estimated to be feeling joy, dietary improvement suggestions to maintain the joy are suggested.

[0454] "Example 3"

[0455] In yet another embodiment of the present invention, the emotion engine adjusts the purchase and delivery of ingredients based on the user's emotional state. Specifically, the system creates an ingredient list for the recipe suggested by the system, and links with an ingredient purchase and delivery service based on the list. In this case, the emotion engine adjusts the selection of ingredients and the delivery time taking into account the user's emotional state. For example, if it is estimated that the user is feeling sad, ingredients that will ease the sadness are selected, and the delivery is made at a time when the user is likely to feel happy.

[0456] The process flow of each embodiment will be described below.

[0457] "Example 1"

[0458] Step 1: The user inputs speech or text into the system.

[0459] Step 2: The emotion engine infers the user's emotional state from the input.

[0460] Step 3: Based on the estimated emotional state, the system provides health advice and recipe suggestions.

[0461] "Example 2"

[0462] Step 1: A user uploads a photo of their meal into the system.

[0463] Step 2: The system evaluates the nutritional balance of the uploaded meal.

[0464] Step 3: The emotion engine takes the user's emotional state into account and suggests dietary improvements.

[0465] "Example 3"

[0466] Step 1: Create an ingredients list for the recipe suggested by the system.

[0467] Step 2: Based on the ingredients list, the system connects with the ingredients purchasing and delivery service.

[0468] Step 3: The emotion engine takes the user's emotional state into account and adjusts the ingredient selection and delivery time.

[0469] Example 1

[0470] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0471] Conventional health management systems require users to manually input data when importing medical checkup results, which is time-consuming and can lead to input errors. In addition, they provide uniform health advice and recipes without considering the user's emotional state, which means that they are unable to provide optimal suggestions for individual users. Furthermore, there is insufficient integration with food purchasing and delivery services, which means that it is time-consuming for users to purchase ingredients based on suggested recipes.

[0472] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0473] In this invention, the server includes a means for importing and analyzing the results of a medical checkup, a means for providing health advice based on the analysis results, a means for proposing recipes for one week based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, a means for purchasing and delivering ingredients based on the proposed recipes, a means for reading the results of the medical checkup in PDF format and extracting them as text data, and a means for recognizing the user's emotional state and optimizing health advice and recipes based on that information. This allows the user to import accurate medical checkup results without any hassle, and to receive optimal health advice and recipes according to the individual emotional state. In addition, by linking with an ingredient purchasing and delivery service, ingredients can be easily purchased based on the proposed recipes.

[0474] "Results of medical checkup" refers to data showing the results of detailed medical tests conducted as part of a health checkup.

[0475] "Means for analyzing" refers to a method or device for analyzing captured data and extracting meaningful information.

[0476] "Means for providing health advice" refers to a method or device that provides specific instructions or suggestions to a user for maintaining or improving their health based on the analysis results.

[0477] The "means for suggesting a week's worth of recipes" refers to a method or device for suggesting a week's worth of meal menus based on the user's health condition.

[0478] "Means for analyzing photos of meals registered by a user and suggesting improvement suggestions" refers to a method or device that analyzes images of meals taken and registered by a user and makes suggestions for improvement based on nutritional balance and health condition.

[0479] "Means for purchasing and delivering ingredients" refers to a method or device for purchasing the necessary ingredients based on the proposed recipe and delivering them to the user.

[0480] "Means for reading medical checkup results in PDF format and extracting them as text data" refers to a method or device for reading medical checkup results provided as a PDF file and extracting the contents as text data.

[0481] "Means for recognizing a user's emotional state and optimizing health advice and recipes based on that information" refers to a method or device for analyzing a user's emotions and adjusting the content of health advice and recipes based on the results.

[0482] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0483] The system of the present invention imports and analyzes the results of medical checkups and provides users with health advice and recipes. This system is composed of the following main components:

[0484] 1. Upload PDF file and extract text

[0485] The user uploads a PDF file containing the results of the medical checkup to the system. The server receives the uploaded PDF file and uses a PDF analysis library such as "Apache (registered trademark) PDFBox" to extract text data from the PDF. This text data is used for subsequent analysis processing.

[0486] 2. Recognizing Emotional States

[0487] The user inputs their recent emotional state into the system as text. For example, they input, "I've been feeling a lot of stress at work lately." The device sends this text data to the emotion engine. The emotion engine uses emotion analysis APIs such as "IBM Watson (registered trademark)" and "Microsoft (registered trademark) Azure (registered trademark) Cognitive Services" to estimate the user's emotional state. This emotional state is sent to the server and used throughout the system.

[0488] 3. Health advice and recipe generation

[0489] The server generates health advice and recipes based on the extracted text data and emotional state. For example, if the user is estimated to be stressed, the server will suggest recipes that contain ingredients that have a relaxation effect. The server uses a generative AI model to generate appropriate advice and recipes.

[0490] 4. Displaying the results

[0491] The terminal displays the health advice and recipes received from the server to the user, who can then review the suggested advice and recipes through the system's interface.

[0492] Examples

[0493] A user uploads the results of a medical checkup in PDF format to the system. The server extracts text data from the PDF and analyzes the data. Next, if the user enters text such as "I've been feeling a lot of stress at work lately," the emotion engine infers that the user is feeling stressed. Based on this information, the server suggests recipes that include ingredients that have a relaxing effect.

[0494] Examples of prompt statements

[0495] "Please upload the results of your medical checkup in PDF format. After that, please enter text about your recent emotional state. For example, 'I've been feeling a lot of stress at work lately.'"

[0496] In this way, the system can take into account the user's overall health and emotional state to provide optimal health advice and recipes.

[0497] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0498] Step 1:

[0499] A user uploads a PDF file containing the results of a medical checkup to the system. The user opens a file selection dialog through the system interface and selects the target PDF file. The input is the PDF file, and the output is the PDF file sent to the server.

[0500] Step 2:

[0501] The server receives the uploaded PDF file. Then, the server uses the "Apache PDFBox" library to extract text data from the PDF file. Specifically, it uses the PDFBox functions to parse the PDF and obtain the text data. The input is the PDF file, and the output is the extracted text data.

[0502] Step 3:

[0503] The user inputs a text of his / her recent emotional state to the system. For example, the user inputs "I've been feeling a lot of stress at work lately." The input is text data indicating the user's emotional state, and the output is the text data sent to the terminal.

[0504] Step 4:

[0505] The device sends text data entered by the user to the emotion engine. The emotion engine uses emotion analysis APIs such as "IBM Watson" and "Microsoft Azure Cognitive Services" to estimate the user's emotional state. Specifically, the device sends text data to the emotion analysis API and obtains the emotional state. The input is text data indicating the user's emotional state, and the output is the estimated emotional state.

[0506] Step 5:

[0507] The server generates health advice and recipes based on the extracted text data and the estimated emotional state. It uses a generative AI model to generate appropriate advice and recipes. In concrete terms, it inputs a prompt sentence into the generative AI model and obtains the generated advice and recipes. The input is the text data and the emotional state, and the output is the generated health advice and recipes.

[0508] Step 6:

[0509] The terminal displays the health advice and recipes received from the server to the user. The user can check the suggested advice and recipes through the system interface. The specific operation is to display the advice and recipes on the terminal screen. The input is the health advice and recipes, and the output is the information displayed to the user.

[0510] (Application example 1)

[0511] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0512] Conventional health management systems can analyze the results of medical checkups and provide health advice and recipe suggestions, but they cannot provide advice or suggestions that take into account the user's emotional state, making it difficult to provide optimal health management that matches the user's psychological state.In addition, users cannot receive real-time health advice or product suggestions while shopping in a physical store, making it difficult for them to select appropriate products on the spot.

[0513] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0514] In this invention, the server includes a means for importing and analyzing the results of a medical checkup, a means for providing health advice based on the analysis results, a means for proposing recipes for one week based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, a means for purchasing and delivering ingredients based on the proposed recipes, a means for recognizing the user's emotional state and optimizing health advice and recipe suggestions based on the information, and a means for providing health advice and product suggestions in real time through smart glasses. This enables optimal health management according to the user's emotional state, and allows the user to receive appropriate health advice and product suggestions in real time even while shopping at a physical store. The server includes a means for providing health advice and product suggestions in real time through smart glasses.

[0515] The "results of a medical checkup" are data on the test results obtained in a medical checkup, which is a type of health checkup.

[0516] An "analyzing means" is a device or software for analyzing captured data and extracting meaningful information.

[0517] The "means for providing health advice" refers to a device or software that provides the user with advice on health management based on the analysis results.

[0518] The "means for suggesting a week's worth of recipes" refers to a device or software that suggests a week's worth of meal menus to the user based on the analysis results.

[0519] The "means for analyzing food photos and proposing improvement suggestions" refers to a device or software that analyzes food photos registered by a user and provides improvement suggestions for making the diet healthier.

[0520] The "means for purchasing and delivering ingredients" is a device or software for purchasing the necessary ingredients based on the proposed recipe and delivering them to the user.

[0521] The "means for recognizing an emotional state and optimizing health advice or recipe suggestions based on that information" refers to a device or software that analyzes a user's emotions and adjusts health advice or recipe suggestions based on the results.

[0522] The "means for providing real-time health advice and product suggestions through smart glasses" refers to a device or software that displays real-time health advice and product suggestions to a user using the smart glasses. The server includes the means for providing real-time health advice and product suggestions through the smart glasses.

[0523] A system for carrying out the present invention has the following configuration: The server includes means for importing and analyzing the results of a medical checkup, means for providing health advice based on the analysis results, means for proposing recipes for a week based on the analysis results, means for analyzing photos of meals registered by the user and proposing improvement plans, means for purchasing and delivering ingredients based on the proposed recipes, means for recognizing the emotional state of the user and optimizing health advice and recipe suggestions based on that information, and means for providing health advice and product suggestions in real time through the smart glasses.

[0524] The server first imports the results of the medical checkup in PDF format and extracts them as text data using pytesseract. It then analyzes the extracted data and evaluates the user's health condition. Based on the analysis results, it provides the user with health advice. For example, if the user has high blood pressure, it advises them to limit their salt intake.

[0525] Furthermore, the system will suggest recipes for the week based on the analysis results. The system takes into account not only the user's health condition but also their emotional state when suggesting recipes. For example, if the user is feeling stressed, the system will suggest recipes that include ingredients that have a relaxing effect.

[0526] The system analyzes photos of meals uploaded by users and suggests improvements to make them healthier. To do this, it uses image analysis technology to recognize the contents of the meal and evaluate the nutritional balance. For example, if there are not enough vegetables, it suggests adding more vegetables.

[0527] Also included is a means for purchasing and delivering the necessary ingredients to the user based on the suggested recipe, in cooperation with existing ingredient purchasing and delivery services.

[0528] The server also includes a means for providing health advice and product suggestions in real time through the smart glasses. When a user wears the smart glasses and goes shopping in a brick-and-mortar store, the camera recognizes the user's emotional state and suggests appropriate products based on that information. For example, if the user is feeling stressed, products with a relaxation effect (such as herbal tea or aromatic oil) are suggested. The server includes a means for providing health advice and product suggestions in real time through the smart glasses.

[0529] Below are some example prompts that can be used to provide optimal health advice and product suggestions based on a user’s emotional state using a generative AI model:

[0530] "If the user is feeling stressed, suggest products that have a relaxation effect, such as herbal tea or aromatic oils."

[0531] In this way, optimal health management based on the user's emotional state is possible, and the user can receive appropriate health advice and product suggestions in real time while shopping in a physical store.

[0532] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0533] Step 1:

[0534] The server imports the results of the medical checkup in PDF format and extracts them as text data using pytesseract. The input is a PDF file and the output is text data. Specifically, the server reads the PDF file and extracts text information using image analysis technology.

[0535] Step 2:

[0536] The server analyzes the extracted text data and evaluates the user's health condition. The input is the text data, and the output is the health condition evaluation result. Specifically, the server analyzes the text data and evaluates health indicators such as blood pressure and blood sugar level.

[0537] Step 3:

[0538] The server provides health advice based on the analysis results. The input is the health status evaluation result, and the output is health advice. Specifically, the server generates advice such as "reduce salt intake" based on the evaluation result.

[0539] Step 4:

[0540] The server proposes recipes for one week based on the analysis results. The input is the health assessment result, and the output is recipes for one week. Specifically, the server generates recipes suitable for the user's health condition based on the assessment result.

[0541] Step 5:

[0542] The server analyzes photos of meals registered by users and proposes improvement suggestions. The input is a photo of the meal, and the output is improvement suggestions. Specifically, the server uses image analysis technology to recognize the contents of the meal, evaluates the nutritional balance, and generates improvement suggestions.

[0543] Step 6:

[0544] The server purchases the necessary ingredients based on the proposed recipe and delivers them to the user. The input is the recipe, and the output is a shopping list and delivery instructions. Specifically, the server makes a list of the necessary ingredients based on the recipe, and coordinates with existing ingredient purchasing and delivery services to arrange for delivery.

[0545] Step 7:

[0546] The server uses an emotion engine to recognize the user's emotional state and optimizes health advice and recipe suggestions based on that information. The input is the user's emotional data, and the output is optimized advice and recipes. Specifically, the server analyzes the user's speech and text input, estimates the user's emotional state, and adjusts the advice and recipes.

[0547] Step 8:

[0548] The terminal provides health advice and product suggestions in real time through the smart glasses. The input is the user's emotional state and video data in the store, and the output is advice and product suggestions displayed on the smart glasses. Specifically, the terminal recognizes the user's emotional state through a camera and suggests appropriate products based on that information. The server includes a means for providing health advice and product suggestions in real time through the smart glasses.

[0549] Example 2

[0550] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0551] Conventional health management systems had difficulty not only assessing the user's health condition but also providing dietary improvement suggestions that took into account the user's emotional state. There was also a need to utilize generative AI models to provide more accurate advice and suggestions. Furthermore, there was a lack of functionality to analyze photos of meals uploaded by users and provide specific improvement suggestions.

[0552] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0553] In this invention, the server includes a means for importing and analyzing the results of a medical checkup, a means for providing health advice based on the analysis results, a means for proposing recipes for one week based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, a means for purchasing and delivering ingredients based on the proposed recipes, a means for estimating the user's emotional state and proposing improvement plans for meals based on the emotional state, and a means for generating health advice and improvement plans for meals using a generative AI model. This makes it possible to comprehensively evaluate the user's health state and emotional state and provide appropriate advice and suggestions. The server includes a means for generating improvement plans for the meals based on the user's emotions using a generative AI model, and a means for adjusting the selection of ingredients and the delivery time of the ingredients based on the user's emotions.

[0554] The "means for receiving and analyzing the results of a medical checkup" refers to a combination of software and hardware for receiving medical checkup data provided by a user and analyzing the data.

[0555] "Means for providing health advice based on analysis results" refers to a combination of software and hardware for evaluating the user's health condition and generating appropriate health management advice based on analyzed health checkup data.

[0556] The "means for suggesting a week's worth of recipes based on the analysis results" is a combination of software and hardware for suggesting a balanced meal plan based on the user's health condition.

[0557] "Means for analyzing photos of meals registered by users and proposing improvement suggestions" refers to a combination of software and hardware for analyzing images of meals uploaded by users, evaluating their nutritional balance, and providing improvement suggestions.

[0558] "Means for purchasing and delivering ingredients based on a suggested recipe" is a combination of software and hardware for interfacing with the service to purchase ingredients needed for a suggested recipe and deliver them to the user.

[0559] The "means for estimating the user's emotional state and suggesting meal improvement suggestions based on that emotional state" is a combination of software and hardware for analyzing the user's emotional state and providing meal improvement suggestions based on that state.

[0560] "Means for generating health advice and dietary improvement suggestions using a generative AI model" refers to a combination of software and hardware for using artificial intelligence technology to generate advice and suggestions based on the health and emotional state of a user. The server includes means for generating the dietary improvement suggestions based on the user's emotions using a generative AI model, and means for adjusting the selection of ingredients and the delivery time of the ingredients based on the user's emotions.

[0561] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0562] The present invention is a system for comprehensively evaluating a user's health and emotional state and providing appropriate advice and suggestions. The system has the following main functions:

[0563] 1. Medical checkup results analysis function

[0564] The server receives the results of medical checkups uploaded by users. These results include numerical data such as blood pressure, blood glucose level, cholesterol level, etc. The server uses a Python-based data analysis library (e.g., Pandas, NumPy) to analyze this data.

[0565] The server evaluates the user's health condition based on the analysis results. For example, if the blood sugar level is high, the server determines that the user is at risk for diabetes and generates advice accordingly. This advice includes specific action guidelines for improving the user's health condition.

[0566] Examples:

[0567] A user uploads the results of a medical checkup.

[0568] The server detects that the blood glucose level is 140 mg / dL.

[0569] The server determines if a person is at high risk for diabetes and provides advice, including recommendations for improved diet and exercise.

[0570] Example prompt:

[0571] "If the user's blood sugar level is 140 mg / dL, determine that they are at high risk for diabetes and provide appropriate advice."

[0572] 2. Diet improvement suggestions function based on emotion engine

[0573] A user uploads a photo of their meal, and the server uses image analysis software (e.g., OpenCV, TensorFlow) to evaluate the nutritional balance of the meal.

[0574] The server estimates the user's emotional state using an emotion engine, which includes algorithms for estimating the user's current emotional state based on the user's past behavioral data and input data.

[0575] The server may provide dietary suggestions based on the user's emotional state, for example providing dietary suggestions to maintain the user's joy if the user is estimated to be feeling joy.

[0576] Examples:

[0577] A user uploads a photo of a meal.

[0578] The server evaluates the nutritional balance of meals from the photos and detects whether they are high in carbohydrates.

[0579] The server infers that the user is feeling happy and suggests a vegetable-rich diet to maintain that feeling.

[0580] Example prompt:

[0581] "If the user is feeling pleasure, suggest dietary changes to maintain that feeling. For example, if their diet is high in carbohydrates, suggest a diet high in vegetables."

[0582] 3. Recipe suggestions and ingredient purchasing and delivery features

[0583] The server then suggests recipes for the week based on the analysis results. These recipes are customized to the user's health condition. For example, low-carb recipes are suggested to a user who is at high risk of diabetes.

[0584] The server cooperates with a service to purchase ingredients based on the suggested recipe and deliver them to the user, so that the user can easily follow the suggested recipe.

[0585] This system supports users in managing their health by comprehensively evaluating their health and emotional state and providing appropriate advice and suggestions.The hardware and software used include servers (e.g., cloud services), user terminals (e.g., smartphones, PCs), Python, Pandas, NumPy, OpenCV, TensorFlow, etc.

[0586] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0587] Medical checkup result analysis function

[0588] Step 1: User uploads medical examination results

[0589] Input: The user uploads the results of the medical checkup in PDF or image format using a smartphone or PC.

[0590] Operation: The user selects the health check result file through the system interface and clicks the upload button.

[0591] Output: The uploaded file is sent to the server.

[0592] Step 2: The server receives the data and prepares it for analysis

[0593] Input: Uploaded PDF or image files.

[0594] How it works: The server checks the format of the file it receives and, if it's a PDF, converts it to text using OCR (Optical Character Recognition) software (e.g. Tesseract), and similarly for images.

[0595] Output: Medical checkup results converted into text data.

[0596] Step 3: The server parses the data

[0597] Input: Medical examination results converted into text data.

[0598] How it works: The server uses Python's Pandas library to convert the text data into a data frame, then uses NumPy to parse various numerical data (blood pressure, blood glucose levels, cholesterol levels, etc.).

[0599] Output: Parsed medical examination data.

[0600] Step 4: Server evaluates health and generates advice

[0601] Input: Parsed medical examination data.

[0602] How it works: The server evaluates the user's health status based on the analysis results. For example, if the blood glucose level is 140 mg / dL, it may determine that the user is at high risk for diabetes. It then uses a generative AI model to generate appropriate advice, which may include recommendations for dietary improvements and exercise.

[0603] Output: The generated health advice.

[0604] Step 5: The server provides advice to the user

[0605] Input: The generated health advice.

[0606] Operation: The server sends the generated advice to the user's device, where the user can check the advice on their smartphone or PC.

[0607] Output: Health advice displayed on the user's device.

[0608] Meal improvement suggestions function based on emotion engine

[0609] Step 1: User uploads a photo of their meal

[0610] Input: A photo of a meal taken by a user using their smartphone.

[0611] Operation: A user selects a photo of a meal through the system's interface and clicks the upload button.

[0612] Output: The uploaded photos are sent to the server.

[0613] Step 2: The server receives the images and prepares them for analysis

[0614] Input: Uploaded food photos.

[0615] How it works: The server uses image analysis software (e.g. OpenCV, TensorFlow) to analyze the images it receives.

[0616] Output: Image data ready for analysis.

[0617] Step 3: The server evaluates the nutritional balance of the meal

[0618] Input: Image data ready for analysis.

[0619] How it works: The server evaluates the nutritional balance of a meal based on the results of image analysis, for example calculating the percentages of carbohydrates, proteins, and fats.

[0620] Output: Evaluated nutritional balance data.

[0621] Step 4: The server estimates the user's emotional state

[0622] Input: User's past behavioral data and input data.

[0623] Operation: The server uses an emotion engine to estimate the user's emotional state. The emotion engine contains algorithms that estimate the user's current emotional state based on the user's past behavioral data and input data.

[0624] Output: Estimated emotional state data.

[0625] Step 5: The server generates meal improvement suggestions

[0626] Input: Evaluated nutritional balance data and estimated emotional state data.

[0627] Operation: The server generates meal improvement suggestions based on the user's emotional state. For example, if the user is estimated to be feeling joy, the server provides meal improvement suggestions to maintain the joy. The server includes means for generating the meal improvement suggestions based on the user's emotion using a generative AI model, and means for adjusting ingredient selection and delivery time of the ingredients based on the user's emotion.

[0628] Output: Generated dietary improvement suggestions.

[0629] Step 6: The server provides the user with suggestions for improvement

[0630] Input: Generated dietary improvement suggestions.

[0631] Operation: The server sends the generated improvement proposal to the user's device. The user can then check the improvement proposal on their smartphone or PC.

[0632] Output: Dietary improvement suggestions displayed on the user's device.

[0633] (Application example 2)

[0634] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0635] Conventional health management systems can provide advice and recipe suggestions based on the user's health condition, but they cannot provide dietary improvement suggestions that take into account the user's emotional state. As a result, it is not possible to provide optimal dietary suggestions based on the user's emotional state, making it difficult to increase user satisfaction. In addition, there is a lack of functionality that allows users to order the suggested meals as is, which reduces user convenience.

[0636] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0637] In this invention, the server includes means for importing and analyzing the results of a medical checkup, means for providing health advice based on the analysis results, means for proposing recipes for one week based on the analysis results, means for analyzing photos of meals registered by the user and proposing improvement plans, means for purchasing and delivering ingredients based on the proposed recipes, means for analyzing the emotional state of the user and proposing meal improvement plans based on the emotions, and means for proposing and ordering optimal meals based on the user's health and emotional states. This makes it possible to propose optimal meals that comprehensively take into account the user's health and emotional states, thereby increasing user satisfaction and convenience.

[0638] "Results of a medical checkup" refers to various health data (blood pressure, blood sugar level, cholesterol level, etc.) obtained during a medical checkup, which is a type of health checkup.

[0639] "Means for analyzing" refers to a method or device for analyzing the captured data and assessing the user's health or emotional state.

[0640] The "means for providing health advice" refers to a method or device that provides the user with specific guidance or suggestions for maintaining or improving their health based on the analysis results.

[0641] The "means for suggesting a week's worth of recipes" refers to a method or device that provides a week's worth of nutritionally balanced meal menus based on the user's health condition.

[0642] The "means for analyzing photos of meals and proposing improvement suggestions" refers to a method or device that analyzes images of meals registered by a user and suggests improvements to the meal content based on nutritional balance and health condition.

[0643] "Means for purchasing and delivering ingredients" refers to a method or device for purchasing the necessary ingredients based on the proposed recipe and delivering them to the user.

[0644] The "means for analyzing an emotional state and proposing dietary improvement suggestions based on the emotions" refers to a method or device for analyzing a user's emotional state and proposing improvements to dietary content according to the emotions.

[0645] The "means for proposing and ordering the most suitable meal" refers to a method or device that proposes the most suitable meal by taking into consideration the user's overall health and emotional state, and enables the user to order that meal.

[0646] A system for implementing the present invention comprises a server, a user terminal, and related software components. A specific embodiment of the system will be described below.

[0647] System Configuration

[0648] 1. Server:

[0649] The server has the function of importing and analyzing the medical checkup results sent by users, using software libraries such as Python, Scikit-learn, and Keras.

[0650] The server generates health advice based on the analysis results and provides it to the user.

[0651] The server has a function of suggesting recipes for a week based on the user's health condition.

[0652] The server analyzes photos of meals registered by the user and suggests improvements to the meal content based on nutritional balance and health condition.

[0653] The server analyzes the user's emotional state and proposes dietary improvement suggestions based on the emotion. For emotion analysis, a deep learning model using OpenCV and Keras is used.

[0654] The server provides a function that suggests optimal meals by taking into consideration the user's overall health and emotional state, and allows the user to order the meals directly.

[0655] 2. User device:

[0656] The user terminal is a device such as a smartphone or tablet, which uses a camera to take photos of the meal and transmits them to the server.

[0657] The user terminal displays health advice and recipe suggestions provided by the server.

[0658] The user terminal provides an interface for directly ordering the suggested meals.

[0659] Data processing and calculation

[0660] 1. Health analysis:

[0661] The server receives the health checkup results sent by the user and standardizes the data using Scikit-learn.

[0662] Based on standardized data, health status is assessed and health advice, such as diabetes risk, is generated.

[0663] 2. Emotional state analysis:

[0664] The server receives the food photos uploaded by users and pre-processes the images using OpenCV.

[0665] The preprocessed images are input into a Keras deep learning model to analyze the emotional state.

[0666] Based on the analysis results, dietary improvement suggestions are generated according to emotions.

[0667] 3. Meal Suggestions and Ordering:

[0668] The server takes into consideration the user's overall health and emotional state and suggests the most appropriate meal.

[0669] The user terminal displays the suggested meals and provides an interface through which the user can place an order.

[0670] Examples

[0671] If a user uploads the results of a health check and their blood sugar level is assessed as high, the server will suggest a low-carb meal.

[0672] If a user uploads a photo of a meal and the emotion engine detects "joy," the server will suggest balanced meals to maintain that emotion.

[0673] Examples of prompt statements

[0674] Evaluate the user's health status and suggest appropriate dietary recommendations using the following data:

[0675] Blood pressure: 120

[0676] Blood sugar level: 7.0

[0677] Cholesterol level: 200

[0678] Also, analyze the user's emotional state and suggest dietary improvements based on their emotions. Please use the image below.

[0679] Image Path: 'user_meal.jpg'

[0680] In this way, a system is realized that makes optimal meal suggestions that take into consideration the user's overall health and emotional state.

[0681] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0682] Step 1:

[0683] A user uploads the results of a medical checkup.

[0684] Input: The user uses a smartphone or tablet to input the results of a health check (blood pressure, blood sugar level, cholesterol level, etc.).

[0685] Output: The medical checkup result data sent to the server.

[0686] Specific operations: The user opens the dedicated application, enters the results of the health checkup according to the specified format, and presses the send button.

[0687] Step 2:

[0688] The server analyzes the results of the health check.

[0689] Input: Medical checkup result data submitted by the user.

[0690] Output: Health status assessment results and health advice.

[0691] How it works: The server uses Scikit-learn to standardize the received data and evaluates the health status. For example, if blood sugar level is high, it evaluates the risk of diabetes and suggests a low-sugar diet.

[0692] Step 3:

[0693] The server will suggest recipes for the week.

[0694] Input: Health status assessment results.

[0695] Output: A week's worth of recipes based on the user's health status.

[0696] Specific operation: The server generates nutritionally balanced recipes based on the health assessment results and provides them to the user.

[0697] Step 4:

[0698] A user uploads a photo of a meal.

[0699] Input: A photo of a meal taken by the user using a smartphone or tablet.

[0700] Output: Food photo data sent to the server.

[0701] Specific actions: The user opens the dedicated application, takes a photo of the meal, and presses the send button.

[0702] Step 5:

[0703] The server analyzes photos of meals and suggests improvements.

[0704] Input: Food photo data submitted by the user.

[0705] Output: Suggested dietary improvements.

[0706] Specific operation: The server preprocesses the images using OpenCV, inputs them into the Keras deep learning model to analyze nutritional balance, and generates recommendations for improving the diet based on the results.

[0707] Step 6:

[0708] The server analyzes the user's emotional state.

[0709] Input: Food photo data submitted by the user.

[0710] Output: Evaluation of the user's emotional state.

[0711] Specific operation: The server uses the photos of food to analyze the emotional state with an emotion engine (a Keras deep learning model). For example, if the user feels happy, it generates meal improvement suggestions to maintain that emotion.

[0712] Step 7:

[0713] The server will suggest the best meal and allow the user to order.

[0714] Input: Health status assessment results and emotional status assessment results.

[0715] Output: Optimal meal suggestions and ordering interface based on the user's health and emotional state.

[0716] Specific operation: The server considers the health and emotional state of the user and proposes the optimal meal. The user terminal displays the proposed meal and provides an interface that allows the user to order it.

[0717] In this way, a system is realized that makes optimal meal suggestions that take into consideration the user's overall health and emotional state.

[0718] Example 3

[0719] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0720] In modern society, improving dietary management and nutritional balance according to the individual's health condition is an important issue. However, it is time-consuming and requires specialized knowledge for users to select appropriate recipes based on their own health condition and purchase and deliver ingredients. In addition, it is difficult for conventional systems to suggest meals and adjust ingredient delivery while taking into account the user's emotional state. To solve these issues, a system is needed that automatically suggests appropriate recipes and purchases and delivers ingredients based on the user's health and emotional state.

[0721] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0722] In this invention, the server includes means for importing and analyzing the results of a medical checkup, means for providing health advice based on the analysis results, means for proposing recipes for one week based on the analysis results, means for analyzing images of meals registered by the user and proposing improvement plans, means for purchasing and delivering ingredients based on the proposed recipes, and means for analyzing the emotional state of the user and adjusting the selection of ingredients and delivery time based on the emotional state. This makes it possible to propose appropriate recipes and purchase and deliver ingredients based on the user's health and emotional state.

[0723] "Results of a health checkup" refers to data obtained as a result of a health examination or test.

[0724] An "analytical means" is a method or device for analyzing data and extracting meaningful information.

[0725] A "means for providing health advice" is a method or device that provides health advice or suggestions to a user based on the analysis results.

[0726] The "means for suggesting a week's worth of recipes" refers to a method or device that suggests a week's worth of meal menus based on the user's health condition.

[0727] "Meal images registered by a user" refers to photo data of meals that a user uploads to the system.

[0728] The "means for proposing improvement suggestions" refers to a method or device that makes suggestions for improving the user's diet based on the analysis results.

[0729] "Means for purchasing and delivering ingredients" refers to a method or device for purchasing the necessary ingredients based on the proposed recipe and delivering them to a location specified by the user.

[0730] The "means for analyzing an emotional state" refers to a method or device for analyzing the user's emotions and grasping that state.

[0731] A "means for adjusting ingredient selection and delivery time based on emotional state" is a method or device for selecting appropriate ingredients and adjusting delivery time by taking into account the emotional state of the user.

[0732] The system of the present invention automatically proposes appropriate recipes and purchases and delivers ingredients based on the user's health and emotional state. This system exchanges data between the server, the terminal, and the user, and performs various analyses and proposals.

[0733] Hardware and Software Used

[0734] Server: A server with high-performance computing power is used to analyze data, suggest recipes, purchase ingredients, and process deliveries.

[0735] Device: A device used by a user, such as a smartphone or tablet, that allows the user to take photos of food, select recipes, and input emotional states.

[0736] Image analysis software: Uses TensorFlow and OpenCV to identify ingredients and dishes from food photos.

[0737] Nutrient databases: Evaluate the nutritional value of ingredients using databases such as the USDA National Nutrient Database.

[0738] Emotion Analysis Software: Affectiva and IBM Watson Tone Analyzer are used to analyze the user’s emotional state.

[0739] Program Processing

[0740] 1. A user takes a photo of a meal on a device and uploads it to the system. The device then sends the uploaded photo to the server.

[0741] 2. The server analyzes the received photos using image analysis software (e.g., TensorFlow or OpenCV) to identify the ingredients and dishes.

[0742] 3. The server checks the identified ingredients and dishes against a nutritional database (e.g., USDA National Nutrient Database) to assess their nutritional balance.

[0743] 4. Based on the evaluation results, the server generates dietary improvement suggestions for the user and sends them to the user's device.

[0744] 5. The user selects a suggested recipe, and the device sends the selected recipe information to the server.

[0745] 6. The server generates an ingredients list for the selected recipe and connects with an ingredient purchasing and delivery service (e.g., Amazon Fresh or Instacart) to purchase and deliver the ingredients.

[0746] 7. The server analyzes the user’s emotional state using emotion analysis software (e.g., Affectiva or IBM Watson Tone Analyzer) and adjusts ingredient selection and delivery time based on the user’s emotional state.

[0747] Examples

[0748] Food photo analysis and improvement suggestions: If a user takes a photo that contains steak and potatoes, the system will identify this and suggest a salad that is high in vegetables.

[0749] Purchase and delivery of ingredients based on a recipe: If a suggested recipe includes “chicken breast,” the system will send the ingredients to a purchasing and delivery service to be delivered to the user’s home.

[0750] Emotion engine adjustment: If the system estimates that the user is feeling sad, it will choose food items like chocolate and fruit and deliver them in the evening when they are more relaxed.

[0751] Examples of prompt statements

[0752] "Analyze this photo of a meal, evaluate its nutritional balance, and suggest improvements."

[0753] "Generate an ingredients list based on this recipe and connect with an ingredient shopping and delivery service to deliver the ingredients."

[0754] "Analyze the user's emotional state, adjust the ingredients list based on that state, and deliver at the right time."

[0755] As described above, the system of the present invention provides a specific means for realizing appropriate recipe suggestions, purchasing and delivery of ingredients based on the health and emotional state of the user. The flow of the specific process in the third embodiment will be described with reference to FIG.

[0756] Step 1:

[0757] Users take photos of their meals with their devices and upload them to the system.

[0758] Input: A photo of a meal taken by the user.

[0759] Specific actions: The user takes a photo of their meal using their smartphone's camera app, opens the dedicated app, selects the photo, and presses the upload button.

[0760] Output: The uploaded photo data is saved on the device.

[0761] Step 2:

[0762] The device sends the uploaded photos to the server.

[0763] Input: The uploaded photo data.

[0764] How it works: The device sends the uploaded photo data to the server via an Internet connection, along with metadata such as the user ID and timestamp.

[0765] Output: Photo data and metadata are sent to the server.

[0766] Step 3:

[0767] The photos received by the server are analyzed using image analysis software (e.g., TensorFlow or OpenCV).

[0768] Input: The received photo data.

[0769] How it works: The server feeds the received photo data into image analysis software and uses pre-trained models to identify ingredients and dishes.

[0770] Output: Information on ingredients and dishes identified as analysis results.

[0771] Step 4:

[0772] The server identifies ingredients and dishes from the analysis results.

[0773] Input: Output of image analysis software.

[0774] How it works: The server uses the output of image analysis software to identify ingredients and dishes in a photo, for example identifying that a photo contains "steak" and "potatoes."

[0775] Output: A list of identified ingredients and dishes.

[0776] Step 5:

[0777] The server may compare identified ingredients and dishes against a nutritional database (e.g., the USDA National Nutrient Database) to assess their nutritional balance.

[0778] Input: A list of identified ingredients and / or dishes.

[0779] Specific operation: The server compares the identified ingredients and dish information with the nutrition database to obtain the nutritional value of each ingredient, for example, the calories, protein, and fat of "steak."

[0780] Output: Nutritional value information for each ingredient.

[0781] Step 6:

[0782] The server generates dietary improvement suggestions for the user based on the evaluation results.

[0783] Input: Nutritional value information for each ingredient.

[0784] Specific operation: The server evaluates the user's dietary balance based on the acquired nutritional information and generates improvement suggestions, such as "add a salad that contains a lot of vegetables."

[0785] Output: Dietary improvement suggestions.

[0786] Step 7:

[0787] The server sends the improvement proposal generated to the user's terminal and notifies the user.

[0788] Input: Dietary improvement suggestions.

[0789] Specific operation: The server sends the generated improvement proposal to the user's device, and the device notifies the user of the received improvement proposal and displays it within the app.

[0790] Output: The suggested improvements displayed on the user's device.

[0791] Step 8:

[0792] The user selects a suggested recipe.

[0793] Input: A list of suggested recipes.

[0794] Specific behavior: The user reviews the list of suggested recipes within the app, selects the recipe they like, and presses the purchase button.

[0795] Output: The selected recipe information.

[0796] Step 9:

[0797] The terminal transmits the selected recipe information to the server.

[0798] Input: Selected recipe information.

[0799] Specific operation: The device sends information about the selected recipe to the server, along with metadata such as the user ID and recipe ID.

[0800] Output: The recipe information sent to the server.

[0801] Step 10:

[0802] The server generates an ingredient list for the selected recipe.

[0803] Input: Selected recipe information.

[0804] Specific operation: The server generates a list of ingredients based on the selected recipe information. For example, it creates an ingredients list such as "chicken breast," "carrot," and "onion."

[0805] Output: Ingredients list.

[0806] Step 11:

[0807] The server cooperates with a food purchasing and delivery service (e.g., food purchasing and delivery service) to purchase and deliver food.

[0808] Input: Ingredient list.

[0809] Specific operation: The server uses the API of the food purchasing and delivery service to send the list of ingredients, and the service checks the inventory and completes the purchase process.

[0810] Output: The ingredient list sent to the purchasing and delivery service.

[0811] Step 12:

[0812] The server sends the list of ingredients to a purchasing and delivery service and arranges for the ingredients to be delivered to the user's home.

[0813] Input: A confirmation message from the purchasing and delivery service.

[0814] Specific operation: The server receives a confirmation message from the purchase and delivery service and sends the user's delivery information. The delivery service delivers the groceries to the user's home at the specified date and time.

[0815] Output: Groceries delivered to the user's home.

[0816] Step 13:

[0817] The server analyzes the user's emotional state using emotion analysis software, e.g.

[0818] Input: User emotional state data.

[0819] How it works: The user inputs their emotional state within the app or analyzes their facial expressions through the camera. The emotion analysis software analyzes the user's emotional state and generates a result.

[0820] Output: Emotional state analysis result.

[0821] Step 14:

[0822] The server adjusts the ingredients list of a suggested recipe based on the user's emotional state.

[0823] Input: Emotional state analysis results.

[0824] What it does: The server adjusts the ingredient list based on the emotional state data, for example adding ingredients like chocolate and fruit if the user is feeling sad.

[0825] Output: The adjusted ingredient list.

[0826] Step 15:

[0827] The server purchases ingredients and links with delivery services based on the adjusted ingredient list.

[0828] Input: The adjusted ingredient list.

[0829] Specific operation: The server sends the adjusted ingredient list to the ingredient purchasing and delivery service, which checks the ingredient inventory and completes the purchase process.

[0830] Output: The adjusted ingredient list sent to the purchasing and delivery service.

[0831] Step 16:

[0832] The server takes into account the user's emotional state and adjusts the selection of ingredients and delivery time.

[0833] Input: tuned ingredient list and parsed emotional state.

[0834] Specific behavior: The server takes into account the user's emotional state and adjusts the delivery time, for example arranging for the delivery to occur at a time when the user is likely to feel happy.

[0835] Output: The delivered ingredients based on the adjusted delivery times.

[0836] (Application example 3)

[0837] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0838] Conventional health management systems suggested recipes and purchased and delivered ingredients based on the user's health condition, but did not select ingredients or adjust delivery times based on the user's emotional state. As a result, it was not possible to provide appropriate meal suggestions or delivery services according to the user's emotional state, making it difficult to improve user satisfaction.

[0839] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0840] In this invention, the server includes a means for importing and analyzing the results of a medical checkup, a means for providing health advice based on the analysis results, a means for proposing recipes for one week based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, a means for purchasing and delivering ingredients based on the proposed recipes, and a means for analyzing the emotional state of the user and adjusting the selection of ingredients and delivery time based on the emotional state. This makes it possible to make appropriate meal suggestions and purchase and deliver ingredients taking into account both the health and emotional states of the user.

[0841] A "health checkup" is a medical service that provides a comprehensive health check.

[0842] An "analyzing means" is a method or device for analyzing data and extracting specific information.

[0843] A "means for providing health advice" is a method or device for providing appropriate lifestyle and dietary advice based on the user's health condition.

[0844] A "means for suggesting a week's worth of recipes" is a method or device for providing a week's worth of meal menus based on the user's health condition and preferences.

[0845] The "means for analyzing food photos and proposing improvement suggestions" refers to a method or device for analyzing food photos registered by a user and providing improvement suggestions regarding nutritional balance and ingredient selection.

[0846] "Means for purchasing and delivering ingredients" refers to a method or device for purchasing the necessary ingredients based on the proposed recipe and delivering them to a location specified by the user.

[0847] "Means for analyzing an emotional state and adjusting ingredient selection or delivery time based on the emotional state" refers to a method or device for analyzing a user's emotional state and optimizing ingredient selection or delivery time based on the results.

[0848] A system for carrying out this invention has the following configuration: The server includes means for inputting and analyzing the results of a medical checkup, means for providing health advice based on the analysis results, means for proposing recipes for the week based on the analysis results, means for analyzing photos of meals registered by the user and proposing improvement plans, means for purchasing and delivering ingredients based on the proposed recipes, and means for analyzing the user's emotional state and adjusting the selection of ingredients and delivery time based on the emotional state.

[0849] Hardware and Software Used

[0850] Hardware:

[0851] Smartphone

[0852] server

[0853] software:

[0854] Python

[0855] PIL (Python Imaging Library)

[0856] requests (HTTP library)

[0857] Sentiment Analysis Engine

[0858] Data processing and calculation

[0859] 1. Means of importing and analyzing medical checkup results:

[0860] The server receives the user's health checkup data and analyzes the data using an analysis engine, thereby evaluating the user's health condition.

[0861] 2. Means of Providing Health Advice:

[0862] Based on the analysis results, the server generates health advice appropriate for the user and notifies the smartphone.

[0863] 3. How to suggest recipes for the week:

[0864] The server uses a generative AI model to suggest recipes for a week based on the user's health condition. For example, it suggests low-carb recipes to a user who is at risk of diabetes.

[0865] 4. Methods for analyzing food photos and suggesting improvements:

[0866] The user takes a photo of the meal with their smartphone and sends it to the server, which analyzes the image using PIL. Based on the analysis results, the system provides suggestions for improving nutritional balance.

[0867] 5. Means of purchasing and delivering food:

[0868] The server generates a list of ingredients needed based on the proposed recipe, and cooperates with existing ingredient purchasing and delivery services to purchase the ingredients and deliver them to the user's specified location.

[0869] 6. A means to analyze emotional states and adjust food selections and delivery times based on emotional states:

[0870] The server evaluates the user's emotional state using an analysis engine and optimizes the selection of ingredients and delivery time based on the results. For example, if the user is feeling sad, the server selects ingredients that will ease that emotion and delivers them at a time when the user is likely to feel happy.

[0871] Examples

[0872] User ID: user123

[0873] Meal photo URL: https: / / example.com / meal_photo.jpg

[0874] Example prompt for a generative AI model:

[0875] User ID: user123

[0876] Meal photo URL: https: / / example.com / meal_photo.jpg

[0877] Suggest recipes for the week based on the user's health and emotional state, analyze food photos to evaluate nutritional balance and provide suggestions for improvement, purchase and deliver ingredients based on the suggested recipes, and use an emotion engine to adjust ingredient selection and delivery times.

[0878] In this way, it becomes possible to provide appropriate meal suggestions, as well as food purchasing and delivery, that take into account both the user's health and emotional state.

[0879] The flow of the specific process in the application example 3 will be described with reference to FIG.

[0880] Step 1:

[0881] The server receives the results of the user's medical checkup and analyzes the data using an analysis engine.

[0882] Input: Medical checkup result data

[0883] Data processing: Analysis of medical examination data

[0884] Output: User's health status rating

[0885] Specific operation: The server receives the user's health checkup data and uses an analysis engine to analyze health indicators such as blood sugar level, blood pressure, and cholesterol level, thereby evaluating the user's health condition.

[0886] Step 2:

[0887] Based on the analysis results, the server generates health advice appropriate for the user and notifies the smartphone.

[0888] Input: Health status assessment results

[0889] Data processing: generating health advice

[0890] Output: Health advice notice

[0891] Specific operation: Based on the health status evaluation results, the server generates lifestyle and dietary advice and notifies the smartphone, including recommendations for exercise and restrictions on the intake of certain foods.

[0892] Step 3:

[0893] The server uses a generative AI model to suggest a week's worth of recipes based on the user's health condition.

[0894] Input: Health status assessment results

[0895] Data processing: Recipe generation

[0896] Output: Recipe suggestions for a week

[0897] How it works: The server uses a generative AI model to generate a week's worth of recipes suited to the user's health condition. For example, it suggests low-carb recipes to a user who is at risk of diabetes.

[0898] Step 4:

[0899] A user takes a photo of their meal with their smartphone and sends it to the server, where the image is analyzed using PIL.

[0900] Input: Food photo

[0901] Data processing: Image analysis

[0902] Output: Nutritional balance evaluation of meals

[0903] How it works: A user takes a photo of a meal with their smartphone and sends it to the server. The server uses PIL to analyze the image, identify the types and amounts of ingredients, and evaluate the nutritional balance.

[0904] Step 5:

[0905] The server provides suggestions for improving nutritional balance based on the analysis results.

[0906] Input: Nutritional balance assessment of meals

[0907] Data processing: generating improvement proposals

[0908] Output: Nutritional balance improvement proposal

[0909] Specific operation: The server provides the user with specific improvement suggestions based on the nutritional balance evaluation results of the meal, such as a suggestion to increase the intake of vegetables.

[0910] Step 6:

[0911] The server generates a list of ingredients needed based on the proposed recipe, and cooperates with existing ingredient purchasing and delivery services to purchase the ingredients and deliver them to the user's specified location.

[0912] Input: Suggested recipe

[0913] Data processing: Generating ingredient lists

[0914] Output: Food purchase and delivery instructions

[0915] Specific operation: The server generates a list of ingredients needed based on the proposed recipe, and cooperates with existing ingredient purchasing and delivery services to purchase the ingredients and deliver them to the user's specified location.

[0916] Step 7:

[0917] The server uses an analytical engine to evaluate the user's emotional state and optimizes ingredient selection and delivery times based on the results.

[0918] Input: The user's emotional state.

[0919] Data processing: Optimization based on emotional state

[0920] Output: Optimized ingredient selection and delivery times

[0921] Specific operation: The server evaluates the user's emotional state using an analysis engine, and optimizes the selection of ingredients and delivery time based on the results. For example, if the user is feeling sad, the server selects ingredients that will ease the emotion and delivers them at a time when the user is likely to feel happy.

[0922] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0923] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0924] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0925] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0926] [Second embodiment]

[0927] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0928] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0929] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0930] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0931] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0932] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0933] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0934] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0935] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0936] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0937] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0938] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0939] "Example 1"

[0940] The system of the present invention is equipped with an interface for importing the results of a medical checkup. This interface can import the results of a medical checkup as digital data. For example, if the results of a medical checkup are provided in PDF format, the interface has a function for reading the PDF and extracting it as text data.

[0941] "Example 2"

[0942] The system of the present invention has a function for analyzing the results of a medical checkup that has been imported. This analysis function analyzes various numerical data (blood pressure, blood glucose level, cholesterol level, etc.) contained in the results of the medical checkup to evaluate the user's health condition. For example, if the blood glucose level is high, it will determine that there is a risk of diabetes and provide advice on this.

[0943] "Example 3"

[0944] The system of the present invention has a function to suggest recipes for one week based on the analysis results. This recipe suggestion function selects and suggests recipes from a database that are suited to the user's health condition. For example, low-carbohydrate recipes are suggested to a user who is at risk of diabetes.

[0945] "Example 4"

[0946] The system of the present invention has a function to analyze photos of meals registered by users and propose improvements. This function identifies ingredients and dishes from the photos of meals and evaluates their nutritional balance. For example, for a user who eats a diet high in meat, a meal high in vegetables is suggested.

[0947] "Example 5"

[0948] The system of the present invention has a function to purchase and deliver ingredients based on a proposed recipe. This function links with existing ingredient purchasing and delivery services to purchase and deliver ingredients based on the ingredient list of the proposed recipe. For example, if the proposed recipe includes "chicken breast," the ingredient is sent to the purchasing and delivery service and delivered to the user's home.

[0949] The process flow of each embodiment will be described below.

[0950] "Example 1"

[0951] Step 1: The user uploads the results of a medical checkup in PDF format to the system.

[0952] Step 2: The system reads the PDF and extracts it as text data.

[0953] "Example 2"

[0954] Step 1: The system analyzes various numerical data (blood pressure, blood sugar levels, cholesterol levels, etc.) from the text data extracted.

[0955] Step 2: Based on the analysis results, the user's health condition is evaluated and advice is provided.

[0956] Step 1: The system assesses the user's health status.

[0957] Step 2: Based on the evaluation results, select an appropriate recipe from the database and suggest it to the user.

[0958] "Example 4"

[0959] Step 1: A user uploads a photo of their meal into the system.

[0960] Step 2: The system identifies ingredients and dishes from the photo of the meal and evaluates its nutritional balance.

[0961] Step 3: Based on the evaluation results, suggest improvement ideas to the user.

[0962] "Example 5"

[0963] Step 1: Create an ingredients list for the recipe suggested by the system.

[0964] Step 2: Based on the ingredient list, connect with existing ingredient purchasing and delivery services to purchase and deliver the ingredients.

[0965] Example 1

[0966] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0967] Conventional health management systems required users to manually enter medical checkup results, which was time-consuming and labor-intensive. In addition, there was a lack of a way to efficiently analyze medical checkup results provided in PDF format and extract the necessary information. Furthermore, there was a need for a system that could provide health advice and recipe suggestions based on the extracted information.

[0968] The specification process by the specification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for reading the results of a medical checkup in PDF format and extracting it as text data, a means for formatting necessary information from the extracted text data and storing it in a database, a means for providing health advice based on the analysis results, a means for proposing recipes for one week based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, and a means for purchasing and delivering ingredients based on the proposed recipes. This enables the user to efficiently import and analyze the results of a medical checkup and manage his / her health.

[0969] "Results of a health checkup" refers to the results of detailed medical tests conducted as part of a health checkup, and are usually provided as digital data in PDF format or similar.

[0970] "Means of analysis" refers to software and algorithms used to read the results of health checkups provided as digital data, extract the necessary information, and format it.

[0971] "Means for providing health advice" refers to a system or function for evaluating a user's health condition based on the analyzed results of a medical checkup and providing appropriate lifestyle and medical advice.

[0972] "Means for suggesting a week's worth of recipes" refers to a system or function for creating a balanced meal plan and suggesting a week's worth of recipes based on the user's health condition.

[0973] "Means of analyzing food photos and suggesting improvement suggestions" refers to a system or function that uses image analysis technology to analyze food photos registered by users, evaluates nutritional balance, calories, etc., and suggests improvement suggestions.

[0974] "Means for purchasing and delivering ingredients" refers to a system or service for purchasing the necessary ingredients based on the proposed recipe and delivering them to a location specified by the user.

[0975] "Means of reading medical checkup results in PDF format and extracting them as text data" refers to software or algorithms for reading medical checkup results provided as a PDF file and extracting the contents as text data.

[0976] "Means for formatting the necessary information from the extracted text data and storing it in a database" refers to a system or function for analyzing the extracted text data, formatting the necessary information, and storing it in a database.

[0977] The present invention is a system for efficiently acquiring and analyzing the results of a medical checkup, and providing health advice and recipe suggestions to a user. A specific embodiment of this system will be described below.

[0978] Hardware and Software Used

[0979] The system consists of three main components: a server, a terminal, and a user. The server receives, analyzes, stores, and generates advice and recipe suggestions. The terminal is a device through which the user accesses the system and uploads data. The user is an individual who uses the system to manage their health.

[0980] The server uses the following software and libraries:

[0981] PDF analysis library: "PyMuPDF"

[0982] Database Management System: "SQLite"

[0983] Generative AI models: AI models for generating health advice and recipe suggestions

[0984] Program Processing

[0985] The server receives the results of medical checkups in PDF format uploaded by the user. The server uses the "PyMuPDF" library to open the PDF file and extract the text data. The extracted text data is formatted into the required information (e.g. blood test results, height, weight, etc.) using regular expressions. The formatted data is saved in the "SQLite" database.

[0986] Examples

[0987] As a concrete example, consider the case where a user uploads a file called "health checkup results 2023.pdf". The user accesses the system's web interface using a terminal browser, selects "health checkup results 2023.pdf" and uploads it. The server receives the uploaded PDF file and opens it using the "PyMuPDF" library. The server extracts the text data in the PDF file, extracts information such as blood test results and height and weight using regular expressions, and formats it. The formatted data is stored in the "SQLite" database.

[0988] Examples of prompt statements

[0989] Examples of prompt sentences to be input to a generative AI model include the following:

[0990] Prompt: I have uploaded "Health Check Results 2023.pdf". Please extract the blood test results and height and weight information.

[0991] Using this prompt, the generative AI model can extract the necessary information from the PDF file and provide it to the user.

[0992] The above is a specific embodiment for carrying out the present invention. This system enables a user to efficiently import and analyze the results of a medical checkup and manage his / her health.

[0993] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0994] Step 1:

[0995] A user uploads a PDF file.

[0996] A user accesses the system's web interface using a browser on the terminal. The user clicks the upload button and selects a PDF file, such as "health check result 2023.pdf", from the local storage. After selection, the user clicks the "upload" button. The input is the PDF file and the output is an upload request to the server.

[0997] Step 2:

[0998] The server receives the PDF file.

[0999] The server receives the PDF file uploaded by the user as an HTTP request. The server temporarily stores the received file. The input is the upload request, and the output is the temporarily stored PDF file.

[1000] Step 3:

[1001] The server parses the PDF file.

[1002] The server opens the temporarily saved PDF file using the PyMuPDF library. Specifically, the server reads the PDF file and prepares each page for parsing. The input is the temporarily saved PDF file, and the output is a parsable PDF object.

[1003] Step 4:

[1004] The server extracts the text data.

[1005] The server loops through each page in the PDF file and extracts the text data. The server takes the text of each page and combines it into a single text data. The input is a parsable PDF object and the output is the extracted text data.

[1006] Step 5:

[1007] The server formats the required information.

[1008] The server extracts necessary information from the extracted text data (for example, blood test results, height, weight, etc.) using regular expressions and formats it. Specifically, the server analyzes the text data and extracts information that matches a specific pattern. The input is the extracted text data, and the output is the formatted information.

[1009] Step 6:

[1010] The server stores the formatted data in a database.

[1011] The server stores the formatted data in a SQLite database. Specifically, the server connects to the database and inserts the formatted information into the appropriate tables. The input is the formatted information and the output is the data stored in the database.

[1012] The above are the specific processing steps of the program for this system, which enables users to efficiently import and analyze the results of medical checkups and manage their health.

[1013] (Application example 1)

[1014] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1015] Health management is an important issue in modern society, but it is not easy to accurately grasp an individual's health condition and provide appropriate advice and measures. In particular, there is a demand for a system that can efficiently import and analyze the results of medical checkups and quickly respond when abnormalities are detected. Other challenges include automating dietary suggestions based on the user's health condition and emergency responses in the event of an abnormality.

[1016] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1017] In this invention, the server includes a means for importing and analyzing the results of the medical checkup, a means for providing health advice based on the analysis results, a means for proposing recipes for one week based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, a means for purchasing and delivering ingredients based on the proposed recipes, a means for importing the results of the medical checkup in PDF format and extracting them as text data, a means for analyzing the extracted text data and notifying the user if an abnormality is detected, and a means for automatically contacting an emergency contact or a medical institution if an abnormality is detected. This makes it possible to efficiently manage the user's health condition and quickly respond to an abnormality if it is detected.

[1018] A "health checkup" is a type of medical examination, a medical service that aims to comprehensively examine each part and function of the body.

[1019] "Analysis" is a method for examining data or information in detail and understanding its structure and meaning.

[1020] "Health advice" refers to guidance and suggestions regarding lifestyle habits, diet, exercise, etc., provided based on the user's health condition.

[1021] A "recipe" is a list of steps and ingredients for making a particular dish.

[1022] A "photo of food" is image data of a meal taken by the user.

[1023] An "improvement proposal" is a specific suggestion or method for improving the current situation.

[1024] "Food purchase and delivery" is a service that purchases ingredients needed for cooking and delivers them to a specified location.

[1025] "PDF format" is an abbreviation for Portable Document Format, a file format for electronically storing and displaying documents.

[1026] "Text data" is data that represents character information in a digital format.

[1027] "Abnormal" refers to a state that deviates from the normal or norm, especially in the context of health, which may indicate a disease or disorder.

[1028] "User" refers to an individual who uses this system.

[1029] "Notification" is the act of notifying a user of specific information.

[1030] "Emergency contacts" refers to contact information for people or organizations to contact in the event of an emergency.

[1031] A "medical institution" is a facility that provides medical services, such as a hospital or clinic.

[1032] The system for implementing this invention is configured as follows. First, the server has a means for importing and analyzing the results of a medical checkup. This means has a function for reading the results of a medical checkup provided in PDF format and extracting them as text data. Specifically, a library such as PyMuPDF is used to extract text from the PDF.

[1033] The server then has the means to analyze the extracted text data and notify the user if an anomaly is detected. This analysis is done by checking whether it contains certain keywords (e.g. "anomaly"). If an anomaly is detected, a notification is sent to the user.

[1034] Furthermore, the server has a means for automatically contacting emergency contacts and medical institutions when an abnormality is detected. This means automatically contacts emergency contacts and medical institutions based on the information of the emergency contacts and medical institutions registered in advance by the user when an abnormality is detected.

[1035] The server also has a means for providing health advice based on the analysis results. This advice is generated based on the user's health condition and includes specific guidance and suggestions regarding lifestyle habits, diet, exercise, etc.

[1036] Furthermore, the server has a means for suggesting recipes for a week based on the analysis results. The recipe suggestions are customized according to the user's health condition. A means for analyzing photos of meals registered by the user and suggesting improvement suggestions is also included. The analysis is performed using image recognition technology to evaluate the user's diet and provide healthy improvement suggestions.

[1037] Finally, the server has a means for purchasing and delivering ingredients based on the suggested recipe, in cooperation with existing ingredient purchasing and delivery services, to automatically order ingredients required for the recipe selected by the user and deliver them to a specified location.

[1038] As a concrete example, a user uploads the results of a medical checkup in PDF format to the app. The server extracts text from the PDF and evaluates the health condition. If an abnormality is detected, a notification is sent to the user and emergency contacts and medical institutions are automatically contacted. In addition, recipes for a week are suggested based on the user's health condition, and the necessary ingredients are automatically ordered.

[1039] An example of a prompt for the generative AI model is, "Please upload the results of your health check in PDF format. We will evaluate your health condition and notify you if any abnormalities are detected."

[1040] In this way, it is possible to efficiently manage the user's health condition and quickly respond if an abnormality is detected.

[1041] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1042] Step 1:

[1043] The user uploads the results of his / her medical checkup in PDF format to the terminal.

[1044] Input: Medical checkup results (PDF file)

[1045] Output: Uploaded PDF file

[1046] Specific operations: The user opens the application, selects the PDF file containing the results of the health check, and presses the upload button.

[1047] Step 2:

[1048] The server receives the uploaded PDF file and extracts it as text data.

[1049] Input: Uploaded PDF file

[1050] Output: Extracted text data

[1051] What happens next: The server uses the PyMuPDF library to open the PDF file and extract the text from each page.

[1052] Step 3:

[1053] The server analyzes the extracted text data and evaluates the health status.

[1054] Input: Extracted text data

[1055] Output: Health status assessment results

[1056] Specific operation: The server searches for specific keywords (e.g., "abnormal") in the text data and evaluates the health status.

[1057] Step 4:

[1058] The server notifies the user if any abnormality is detected based on the health status evaluation results.

[1059] Input: Health status assessment results

[1060] Output:Notification to the user

[1061] Specific operation: The server checks the evaluation results and, if an abnormality is detected, sends a notification to the user's device.

[1062] Step 5:

[1063] If the server detects an abnormality, it will automatically contact emergency contacts and medical institutions.

[1064] Input: Health assessment results, emergency contact information

[1065] Output: Notification to emergency contacts and medical institutions

[1066] Specific operation: The server automatically contacts emergency contacts and medical institutions that the user has registered in advance if an abnormality is detected.

[1067] Step 6:

[1068] The server provides health advice based on the analysis results.

[1069] Input: Health status assessment results

[1070] Output: Health advice

[1071] Specific operation: Based on the evaluation results, the server generates specific advice regarding lifestyle habits, diet, exercise, etc. and provides it to the user.

[1072] Step 7:

[1073] The server will suggest recipes for the week based on the analysis results.

[1074] Input: Health status assessment results

[1075] Output: Recipe suggestions for a week

[1076] Specific operation: The server generates recipes customized to the user's health condition and suggests them to the user.

[1077] Step 8:

[1078] The server analyzes the photos of meals registered by the user and suggests improvement suggestions.

[1079] Input: Food photo

[1080] Output: Dietary improvement suggestions

[1081] Specific operation: The server uses image recognition technology to analyze photos of food, evaluate the user's diet, and provide healthy improvement suggestions.

[1082] Step 9:

[1083] The server purchases and delivers ingredients based on the suggested recipe.

[1084] Input: Suggested recipe

[1085] Output: Purchase and delivery of ingredients

[1086] Specific operation: The server works with existing ingredient purchasing and delivery services to automatically order ingredients needed for the recipe selected by the user and deliver them to the specified location.

[1087] Example 2

[1088] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[1089] Conventional health management systems are required to not only analyze the results of medical checkups, but also to provide specific health advice to users. However, these systems provide poor quality advice based on the analysis results, making it difficult to provide specific advice appropriate to the user's health condition. In addition, suggestions for improving the user's diet and purchasing and delivery of ingredients are insufficient, making comprehensive health management difficult.

[1090] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1091] In this invention, the server includes means for importing and analyzing the results of a medical checkup, means for providing health advice based on the analysis results, means for generating prompt sentences based on the analysis results and inputting them into the generative AI model, means for providing the user with advice returned from the generative AI model, means for proposing recipes for one week based on the analysis results, means for analyzing photos of meals registered by the user and proposing improvement plans, and means for purchasing and delivering ingredients based on the proposed recipes. This allows for the provision of specific, high-quality advice based on the user's health condition, enabling comprehensive health management.

[1092] The "results of a medical checkup" are the results of tests conducted as part of a health checkup, and include various numerical data such as blood pressure, blood sugar level, and cholesterol level.

[1093] The "analyzing means" refers to a device or program that performs processing to evaluate the user's health condition based on the captured data.

[1094] The "means for providing health advice" refers to a device or program that provides the user with appropriate health management guidance and suggestions based on the analysis results.

[1095] A "means for generating a prompt sentence" is a device or program that creates a sentence to be input into a generative AI model based on the analysis results.

[1096] A "generative AI model" is an artificial intelligence model that uses natural language generation technology to generate appropriate advice based on input prompt sentences.

[1097] "Means for inputting to a generative AI model" refers to a device or program that sends the generated prompt sentence to the generative AI model and causes it to generate advice.

[1098] A "means for providing generated advice to a user" is a device or program that communicates the advice returned from the generative AI model to a user.

[1099] The "means for suggesting recipes" refers to a device or program that suggests meal menus suitable for the user's health condition based on the analysis results.

[1100] The "means for analyzing food photos" refers to a device or program that analyzes food photos registered by users and evaluates the nutrients and calories.

[1101] The "means for suggesting improvements" is a device or program that makes suggestions for healthier meals based on the analyzed food photos.

[1102] The "means for purchasing and delivering ingredients" refers to a device or program that purchases the necessary ingredients based on the proposed recipe and delivers them to the user.

[1103] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1104] This invention is a system that analyzes the results of a medical checkup provided by a user and provides health advice. The system operates in cooperation with a server, a terminal, and a user.

[1105] 1. Generating the system program

[1106] The server generates a program to analyze the results of the medical checkup provided by the user. This program is developed using Python, and libraries such as Pandas and NumPy are used for data analysis. A model using natural language generation technology (e.g., GPT-4) is used as the generative AI model.

[1107] 2. Explanation of program processing

[1108] The server receives the results of medical checkups uploaded by users. These results are often provided as CSV or Excel files. The server reads these files and converts them into a data frame using Pandas.

[1109] The server then analyzes the various numerical data in the data frame (blood pressure, blood glucose level, cholesterol level, etc.). For example, if the blood glucose level is above a certain threshold, the server determines that the person is at risk for diabetes. This determination is based on pre-defined criteria.

[1110] Based on the analysis results, the server generates advice for the user. This advice is created using a generative AI model. Natural language generation models such as GPT-4 are used for the generative AI model. The server inputs the analysis results into the generative AI model as prompt sentences to generate appropriate advice.

[1111] 3. Specific Examples

[1112] For example, if the user’s blood glucose level is 150 mg / dL, the server will input the following prompt into the generative AI model:

[1113] User's blood sugar level is 150 mg / dL. He may be at risk for diabetes. Please provide appropriate advice.

[1114] Based on this prompt, the generative AI model would generate advice like this:

[1115] Your blood sugar level is 150 mg / dL, so you are at risk for diabetes. First, review your diet and limit your sugar intake. It is also important to incorporate regular exercise. We recommend that you see a doctor and start appropriate treatment.

[1116] The server provides this advice to the user, who can review it through the system's web interface.

[1117] In addition, the server will suggest recipes for the week based on the analysis results. It is also possible to analyze photos of meals registered by the user and suggest improvements. It also includes a means to purchase and deliver ingredients based on the suggested recipes, providing comprehensive support for the user's health management.

[1118] In this way, the system can provide specific, high-quality advice based on the user's health condition, enabling comprehensive health management.

[1119] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1120] Step 1:

[1121] A user uploads the results of a medical checkup.

[1122] The user logs into the system's web interface, clicks the "Choose File" button to select the medical examination result file from his / her computer, and then clicks the "Upload" button. The input is a file in CSV or Excel format, and the output is the file sent to the server.

[1123] Step 2:

[1124] The server receives the uploaded data.

[1125] The server receives the file uploaded by the user and checks the format of the file. For example, if it is in CSV format, it uses the Pandas read_csv function to read the data. The input is the file sent by the user, and the output is the data converted to a data frame.

[1126] Step 3:

[1127] The server parses the data.

[1128] The server analyzes various numerical data (blood pressure, blood glucose level, cholesterol level, etc.) in the data frame. For example, it checks whether the blood glucose level exceeds a certain threshold. Pre-defined reference values ​​are used for the analysis. The input is the data frame, and the output is the analysis result.

[1129] Step 4:

[1130] The server generates a prompt based on the parsing results.

[1131] The server creates a prompt sentence to be input to the generative AI model based on the analysis results. For example, if the blood glucose level is 150 mg / dL, the following prompt sentence is generated: "The user's blood glucose level is 150 mg / dL. There is a risk of diabetes. Please provide appropriate advice." The input is the analysis result, and the output is the prompt sentence.

[1132] Step 5:

[1133] The server inputs a prompt sentence into the generative AI model.

[1134] The server sends the generated prompt sentence to the generative AI model. The generative AI model uses a natural language generation model such as GPT-4. The input is the prompt sentence, and the output is the advice returned from the generative AI model.

[1135] Step 6:

[1136] The server provides the generated advice to the user.

[1137] The server receives the advice returned by the generative AI model and provides it to the user. The user can review the advice through the system's web interface. The input is the advice from the generative AI model, and the output is the advice provided to the user.

[1138] Step 7:

[1139] The server will suggest recipes for the week based on the analysis results.

[1140] The server then proposes a week's worth of recipes suited to the user's health condition based on the analysis results. The input is the analysis results, and the output is the proposed recipes.

[1141] Step 8:

[1142] The server analyzes the photos of meals registered by the user and suggests improvements.

[1143] Users upload photos of their meals to the system. The server uses image analysis technology to analyze the photos and evaluate their nutritional and calorie content. The input is the photo of the meal, and the output is the analysis results and improvement suggestions.

[1144] Step 9:

[1145] The server purchases and delivers ingredients based on the suggested recipe.

[1146] The server purchases the necessary ingredients based on the proposed recipe and delivers them to the user. The input is the proposed recipe and the output is the purchased ingredients and their delivery.

[1147] (Application example 2)

[1148] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1149] Conventional health management systems do not only analyze the results of medical checkups, but also do not adequately monitor the user's daily diet and health condition. In addition, users have to manually enter data, which is time-consuming and makes it difficult to accurately evaluate their health condition. Furthermore, they do not provide specific advice or suggestions for improvement, making it unclear how users should maintain their health.

[1150] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1151] In this invention, the server includes a means for importing and analyzing the results of a medical checkup, a means for providing health advice based on the analysis results, a means for proposing a week's worth of recipes based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, a means for purchasing and delivering ingredients based on the proposed recipes, a means for extracting text from images using optical character recognition technology, a means for analyzing numerical data from the extracted text, and a means for evaluating a health condition based on the numerical data and providing specific advice. This allows a user to easily import and analyze the results of a medical checkup and manage their daily health. The server includes: means for analyzing numerical data provided by a user and included in the results of a medical checkup indicating the user's health condition; means for providing the user with health advice regarding the user's lifestyle habits, the user's diet, and the exercise the user performs based on the results of the analysis by inputting a specific prompt sentence into a generative AI model; means for suggesting to the user a week's worth of recipes customized to the user's health condition based on the results of the analysis; means for analyzing meal photos registered by the user and suggesting suggestions for improving the user's diet; an emotion engine for recognizing the user's emotions; and optimization means for optimizing the health advice and the recipes based on the emotions using a generative AI model.

[1152] The "means for importing and analyzing medical checkup results" refers to technology for importing the results of a medical checkup taken by a user as digital data and analyzing that data.

[1153] The "means for providing health advice based on analysis results" refers to a technology that evaluates the user's health condition based on analyzed health checkup data and provides specific advice for maintaining or improving health.

[1154] The "means for proposing a week's worth of recipes based on the analysis results" is a technology that proposes a week's worth of meal menus suited to the user's health condition based on the analysis results of a health checkup.

[1155] The "means for analyzing photos of meals registered by users and proposing improvement suggestions" refers to a technology that analyzes photos of meals taken and registered by users, evaluates their content, and proposes improvement suggestions for healthier meals.

[1156] "Means for purchasing and delivering ingredients based on the proposed recipe" refers to technology for purchasing ingredients needed for the proposed meal menu and delivering them to the user.

[1157] "Means of extracting text from an image using optical character recognition technology" refers to technology that uses OCR (optical character recognition) technology to extract character information from image data.

[1158] "Means for analyzing numerical data from extracted text" refers to a technology for analyzing numerical data such as blood pressure, blood sugar levels, and cholesterol levels from character information extracted using OCR technology.

[1159] The "means for evaluating health condition based on numerical data and providing specific advice" refers to a technology for evaluating a user's health condition based on analyzed numerical data and providing specific advice for maintaining or improving health.

[1160] A system for implementing this invention is configured as follows. First, a user takes a picture of the results of a medical checkup using the camera of a smartphone. The taken image is processed by an application in the smartphone. The application extracts text from the image using optical character recognition technology (OCR).

[1161] The extracted text is analyzed by the analysis module, which uses Python's pytesseract library for OCR processing and regular expressions to extract numerical data such as blood pressure, blood glucose, cholesterol levels, etc. The extracted numerical data is then sent to the evaluation module for further evaluation of health status.

[1162] The evaluation module evaluates the user's health condition based on the extracted numerical data and generates specific advice that is customized according to the user's health condition. For example, if the blood sugar level is high, it indicates a risk of diabetes and provides specific advice on diet and exercise.

[1163] In addition, it also includes a module that suggests recipes for a week based on the analysis results. This module generates and suggests recipes that are tailored to the user's health condition. It also includes a module that purchases and delivers ingredients based on the suggested recipes, and delivers ingredients to the user in cooperation with existing food purchasing and delivery services.

[1164] As a concrete example, by inputting the following prompt sentences into a generative AI model, it is possible to design a detailed application and generate code:

[1165] Example prompt:

[1166] Develop an application that analyzes images taken by users on their smartphones during medical checkups and evaluates their health condition. The following requirements must be met:

[1167] 1. Extract text from images using OCR technology.

[1168] 2. Extract blood pressure, blood glucose, and cholesterol values ​​from the extracted text using regular expressions.

[1169] 3. Evaluate health status based on extracted numerical data and provide specific advice.

[1170] 4. The libraries used will be Python's pytesseract and re.

[1171] In this way, users can easily import and analyze the results of their medical checkups and manage their health on a daily basis.

[1172] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1173] Step 1:

[1174] The user takes a picture of the results of the medical checkup using the smartphone camera.

[1175] Input: An image of a paper showing the results of a health check.

[1176] Output: Image files stored on your smartphone.

[1177] Specific operation: The user starts the camera app on their smartphone and takes a picture of the paper containing the results of their medical checkup. The captured image is saved in the smartphone.

[1178] Step 2:

[1179] The device uses optical character recognition technology (OCR) to extract text from the image.

[1180] Input: An image file stored on your smartphone.

[1181] Output: The extracted text data.

[1182] How it works: An application in the smartphone uses the pytesseract library to extract text information from the saved image file. The extracted text data is stored in memory.

[1183] Step 3:

[1184] The terminal parses numerical data from the extracted text.

[1185] Input: The extracted text data.

[1186] Output: Numerical data such as blood pressure, blood glucose, cholesterol levels, etc.

[1187] Specific operation: The application uses regular expressions to parse numerical data such as blood pressure, blood glucose level, and cholesterol level from the extracted text data, and stores this numerical data in variables.

[1188] Step 4:

[1189] The device assesses your health condition based on numerical data and provides specific advice.

[1190] Input: Numerical data such as blood pressure, blood glucose, cholesterol levels, etc.

[1191] Output: Health status assessment results and specific recommendations.

[1192] Specific operation: The application evaluates the user's health condition based on the analyzed numerical data. For example, if the blood sugar level is high, it may determine that the user is at risk for diabetes, and generate specific advice regarding diet and exercise. The generated advice is displayed to the user.

[1193] Step 5:

[1194] The device will suggest recipes for the week based on the analysis results.

[1195] Input: Health status assessment results.

[1196] Output: A week's worth of recipes.

[1197] Specific behavior: The application generates and suggests a week's worth of meals based on the user's health status. The suggested recipes are displayed to the user.

[1198] Step 6:

[1199] The terminal purchases and delivers ingredients based on the suggested recipe.

[1200] Enter: a week's worth of recipes.

[1201] Output: Ingredients shopping list and delivery schedule.

[1202] What it does: The application generates a list of ingredients needed for the suggested recipe, then integrates with existing food shopping and delivery services to deliver the ingredients to the user, and notifies the user of the delivery schedule.

[1203] Example 3

[1204] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1205] Conventional health management systems do not adequately provide specific dietary improvement suggestions or recipe suggestions based on the user's health condition. In addition, they lack the functionality to purchase and deliver ingredients based on the suggested recipes, making it difficult for users to actually follow the suggested meals. Furthermore, they lack the functionality to analyze photos of meals to evaluate their nutritional balance and suggest improvement suggestions. To solve these issues, a system is needed that integrates specific recipe suggestions based on the user's health condition with the purchasing and delivery of ingredients.

[1206] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1207] In this invention, the server includes means for importing and analyzing the results of a medical checkup, means for providing health advice based on the analysis results, means for proposing recipes for one week based on the analysis results, means for analyzing images of meals registered by the user and proposing improvement plans, means for purchasing and delivering ingredients based on the proposed recipes, means for analyzing images of meals using image analysis software, means for evaluating nutritional balance by comparing with a nutrition database, and means for purchasing and delivering ingredients in cooperation with an ingredient purchasing and delivery service, thereby enabling specific recipe suggestions based on the user's health condition and the purchase and delivery of ingredients.

[1208] "Results of a health checkup" refers to data obtained as a result of a health examination or test.

[1209] An "analytical means" is a method or device for analyzing data and extracting meaningful information.

[1210] A "means for providing health advice" is a method or device that provides health advice or suggestions to a user based on the analysis results.

[1211] A "means for suggesting a week's worth of recipes" is a method or device that suggests a week's worth of meal menus based on the user's health condition and preferences.

[1212] "Meal images registered by a user" refers to photo data of meals that a user uploads to the system.

[1213] The "means for proposing improvement plans" refers to a method or device that makes suggestions for improving the user's diet based on the analysis results.

[1214] "Means for purchasing and delivering ingredients based on a proposed recipe" refers to a method or device for purchasing ingredients required for a proposed recipe and delivering them to the user.

[1215] "Image analysis software" is software for analyzing image data and extracting specific information.

[1216] A "nutrition database" is a database that collects information about the nutritional components of foods.

[1217] A "means for evaluating nutritional balance" is a method or device for evaluating the nutritional balance of meal contents.

[1218] The "foodstuff purchase and delivery service" is a service that allows a user to purchase the foodstuffs they need and have them delivered to a specified location.

[1219] The system of the present invention has the function of suggesting recipes for a week based on the user's health condition, analyzing photos of meals to provide improvement suggestions, and purchasing and delivering ingredients based on the suggested recipes. This system operates in cooperation with the server, the terminal, and the user.

[1220] Hardware and Software Used

[1221] Server: A server with high-performance computing power is used to analyze data, suggest recipes, purchase ingredients, and manage deliveries.

[1222] Device: A device, such as a smartphone or tablet, that a user uses to take photos of food, check recipes, and receive notifications.

[1223] Image analysis software: Photos of food are analyzed using image analysis software such as TensorFlow and OpenCV.

[1224] Nutrition Databases: Use nutrition databases such as the USDA Food Database to get nutrition information for ingredients.

[1225] Grocery shopping and delivery services: Integrate with existing grocery shopping and delivery services such as Amazon Fresh and Instacart.

[1226] Specific operation of the system

[1227] 1. The user takes a photo of the meal on the device and uploads it to the system. The user can easily send the photo of the meal to the system using a dedicated application.

[1228] 2. The device sends the captured photo to the server. The device sends the photo data and metadata such as the user ID and timestamp to the server via the Internet.

[1229] 3. The server analyzes the received photos using image analysis software, using TensorFlow and OpenCV to identify the ingredients and dishes in the photos.

[1230] 4. The server compares the analysis results with the nutrition database to evaluate the nutritional balance. The server compares the identified ingredients and dish information with the USDA food database to evaluate the nutritional balance of the meal.

[1231] 5. The server generates dietary improvement suggestions based on the evaluation results. For example, for a user who eats a diet high in meat, the server suggests a diet high in vegetables.

[1232] 6. The server sends the generated improvement proposal to the user's device and notifies the user. The user receives the notification on the device and confirms the improvement proposal.

[1233] 7. The user selects a recipe suggested by the system. The user checks the list of recipes suggested by the system on their device and selects the recipe they like.

[1234] 8. The server generates an ingredient list for the selected recipe. The server obtains the details of the selected recipe and generates the required ingredient list.

[1235] 9. The server communicates with the food purchasing and delivery service and transmits the list of ingredients. The server communicates with the food purchasing and delivery service through the API and transmits the list of ingredients.

[1236] 10. The food purchasing and delivery service purchases the ingredients based on the list and delivers them to the user's home. The user receives the delivered ingredients and prepares the dish based on the suggested recipe.

[1237] Examples

[1238] If a photo contains steak and potatoes, the server will identify it, evaluate its nutritional balance, and suggest a recipe for a salad that includes lots of vegetables.

[1239] If the user selects the recipe "chicken breast and vegetable stir fry," the server generates an ingredients list (such as chicken breast, broccoli, and carrots) and sends it to an ingredient purchasing and delivery service, which purchases the ingredients and delivers them to the user's home.

[1240] Examples of prompt statements

[1241] "Analyze the meal in this photo, evaluate its nutritional balance, and suggest improvements."

[1242] "Purchase the ingredients needed for this recipe and have them delivered to the user's home."

[1243] The above is a specific embodiment of the system of the present invention. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1244] System program processing flow

[1245] Example 4: Food photo analysis and improvement proposal function

[1246] Step 1:

[1247] Users take photos of their meals with their devices and upload them to the system.

[1248] Input: A photo of a meal taken by the user.

[1249] Specific operations: Users use the camera on their smartphone or tablet to take a photo of their meal and upload it to the system via a dedicated application.

[1250] Output: The uploaded photo data.

[1251] Step 2:

[1252] The device sends the photograph to the server.

[1253] Input: The uploaded photo data.

[1254] Specific operation: The device sends the photo data and metadata such as user ID and timestamp to the server via the Internet.

[1255] Output: The photo data sent to the server.

[1256] Step 3:

[1257] The photos received by the server are analyzed using image analysis software.

[1258] Input: The photo data sent to the server.

[1259] How it works: The server uses image analysis software such as TensorFlow and OpenCV to identify the ingredients and dishes in the photo.

[1260] Output: Identified ingredients and dish information.

[1261] Step 4:

[1262] The server compares the analysis results with a nutritional database to evaluate nutritional balance.

[1263] Input: Identified ingredients and dish information.

[1264] What it does: The server checks the identified ingredients and dish information against the USDA food database to obtain the nutritional content of each ingredient, then evaluates the nutritional balance of the entire meal.

[1265] Output: Nutritional balance assessment results.

[1266] Step 5:

[1267] The server generates dietary improvement suggestions based on the evaluation results.

[1268] Input: Nutritional balance assessment results.

[1269] Specific operation: Based on the evaluation results, the server generates improvement proposals that take into account the user's health condition and dietary balance. For example, for a user who eats a diet high in meat, the server suggests a diet high in vegetables.

[1270] Output: Dietary improvement suggestions.

[1271] Step 6:

[1272] The improvement proposals generated by the server are sent to the user's device and notified.

[1273] Input: Dietary improvement suggestions.

[1274] Specific operation: The server sends the generated improvement proposal to the user's device and notifies the user. The user receives the notification on the device and confirms the improvement proposal.

[1275] Output: The improvement suggestions sent to the user's device.

[1276] Example 5: Recipe-based food purchasing and delivery function

[1277] Step 1:

[1278] The user selects a recipe suggested by the system.

[1279] Input: A list of suggested recipes.

[1280] Specific operation: The user checks the list of recipes suggested by the system on their device and selects the recipe they like.

[1281] Output: The selected recipe information.

[1282] Step 2:

[1283] The server generates an ingredient list for the selected recipe.

[1284] Input: Selected recipe information.

[1285] Specific operation: The server retrieves detailed information about the selected recipe and generates a list of ingredients, including the exact ingredients and the quantities required.

[1286] Output: Ingredients list.

[1287] Step 3:

[1288] The server cooperates with the food purchasing and delivery service to transmit the food list.

[1289] Input: Ingredient list.

[1290] Specific operation: The server communicates with the food purchasing and delivery service via API and sends the food list.

[1291] Output: A list of ingredients sent to the grocery shopping and delivery service.

[1292] Step 4:

[1293] A food purchasing and delivery service purchases food based on the list and delivers it to the user's home.

[1294] Input: Ingredient list.

[1295] Specific operation: The food purchasing and delivery service purchases ingredients based on the received ingredient list and delivers them to the user's home.

[1296] Output: Groceries delivered to the user's home.

[1297] Step 5:

[1298] The user receives the delivered ingredients and prepares the dish based on the suggested recipe.

[1299] Input: Delivered ingredients.

[1300] Specific behavior: The user receives the delivered ingredients and prepares a dish based on the suggested recipe. During the cooking process, the user refers to the recipe instructions provided by the system.

[1301] Output: The finished dish.

[1302] The above are the specific processing steps of the system of the present invention.

[1303] (Application example 3)

[1304] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1305] In modern society, health management is an important issue, but there is no system that can suggest meals tailored to individual health conditions or consistently purchase and deliver ingredients. There is also a lack of means to evaluate the nutritional balance of the meals that users consume on a daily basis and provide suggestions for improvement. This makes it difficult for users to obtain specific guidelines for maintaining a healthy diet.

[1306] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1307] In this invention, the server includes a means for importing and analyzing the results of a medical checkup, a means for providing health advice based on the analysis results, a means for proposing recipes for one week based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, a means for purchasing and delivering ingredients based on the proposed recipes, and a means including a smartphone application for proposing recipes based on the user's health condition, analyzing photos of meals to provide improvement plans, and purchasing and delivering ingredients based on the proposed recipes. This allows the user to receive meal suggestions tailored to their own health condition, and further enables the easy purchase and delivery of the ingredients.

[1308] "Results of a health checkup" refer to the results of detailed tests conducted as part of a health checkup, and are data used to evaluate an individual's health condition.

[1309] An "analyzing means" is a method or device for analyzing input data and extracting specific information.

[1310] A "means for providing health advice" is a method or device that presents specific guidelines for maintaining or improving health to a user based on the analysis results.

[1311] A "means for suggesting a week's worth of recipes" is a method or device that suggests meal menus for a seven-day period based on the user's health condition and preferences.

[1312] The "means for analyzing photos of meals and proposing improvement suggestions" refers to a method or device that analyzes images of meals registered by users, evaluates their nutritional balance, and makes suggestions for improvement.

[1313] "Means for purchasing and delivering ingredients" refers to a method or device for purchasing the necessary ingredients based on the proposed recipe and delivering them to a location specified by the user.

[1314] A "smartphone application" is software that runs on a smartphone and allows users to use functions such as health management, meal suggestions, food purchasing and delivery, etc.

[1315] "User's health condition" refers to the physical health condition, disease risk, allergy information, and so on, of each individual user.

[1316] "Existing food purchasing and delivery services" refers to service providers that purchase and deliver food ingredients that are already available in the market.

[1317] A system for implementing this invention includes a smartphone application that suggests recipes based on the user's health condition, analyzes photos of meals to provide improvement suggestions, and purchases and delivers ingredients based on the suggested recipes.

[1318] Hardware and Software Used

[1319] Hardware: Smartphone

[1320] software:

[1321] TensorFlow / Keras: A deep learning model used to analyze food photos

[1322] OpenCV: Image processing library

[1323] Requests: A library for sending HTTP requests.

[1324] Data processing and calculation

[1325] server

[1326] The server receives and analyzes the medical checkup results sent by the user. Based on the analysis results, the server evaluates the user's health condition and provides health advice. It also suggests recipes for the week based on the analysis results. It also analyzes photos of meals registered by the user, evaluates their nutritional balance, and suggests improvements.

[1327] Device (smartphone)

[1328] The smartphone application suggests recipes based on the user's health status and analyzes photos of meals to provide improvement suggestions. When a user uploads a photo of a meal, the application uses TensorFlow / Keras to identify ingredients and dishes, and OpenCV to process the image. The analysis results are fed back to the user, and improvement suggestions are provided if necessary.

[1329] User

[1330] Through the smartphone application, users can receive recipe suggestions tailored to their health condition. They can then purchase the ingredients needed based on the suggested recipe and have them delivered to their home using a delivery service. This gives users concrete guidance on how to maintain a healthy diet.

[1331] Examples

[1332] For example, if User A, age 30, is at risk of diabetes, the application will suggest low-carb recipes. When User A uploads a photo of a meal, the application will analyze the photo, evaluate the nutritional balance, and provide suggestions for improvement. Based on the suggested recipe, the user can purchase ingredients such as chicken breast and vegetables and have them delivered to their home using a delivery service.

[1333] Examples of prompt statements

[1334] Enter your health data:

[1335] Age: 30

[1336] Weight: 70

[1337] Height: 175

[1338] Diabetes risk: Yes

[1339] Allergens: Peanuts

[1340] Would you like to purchase and deliver ingredients based on the suggested recipe? (Yes / No)

[1341] In this way, a system will be created that allows users to receive meal suggestions tailored to their own health condition and handle the entire process from purchasing ingredients to having them delivered.

[1342] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1343] Step 1:

[1344] The server takes in the medical checkup results sent by the user. The input is the medical checkup result data, and the output is the data ready for analysis. Specifically, the server receives the user's health data and stores it in a database.

[1345] Step 2:

[1346] The server analyzes the imported medical checkup results. The input is the health data stored in step 1, and the output is the analysis results. Specifically, the server runs the health data through an analysis algorithm to evaluate the user's health condition.

[1347] Step 3:

[1348] The server provides health advice based on the analysis results. The input is the analysis result from step 2, and the output is health advice. Specifically, the server generates appropriate health advice for the user based on the analysis results and sends it to the user's device.

[1349] Step 4:

[1350] The server proposes recipes for one week based on the analysis results. The input is the analysis results from step 2, and the output is recipes for one week. Specifically, the server selects and proposes recipes from the database that are suitable for the user's health condition.

[1351] Step 5:

[1352] A user uploads a photo of a meal through a smartphone application. The input is the photo of the meal taken by the user, and the output is the image data sent to the server. Specifically, a user opens the application, takes a photo of the meal, and uploads it.

[1353] Step 6:

[1354] The server analyzes the uploaded photos of the food. The input is the image data sent in step 5, and the output is the identification results of the ingredients and dishes. Specifically, the server uses a TensorFlow / Keras model to analyze the images and identify the ingredients and dishes.

[1355] Step 7:

[1356] The server proposes meal improvement suggestions based on the analysis results. The input is the classification result from step 6, and the output is improvement suggestions. Specifically, the server evaluates nutritional balance based on the classification results, and generates and sends improvement suggestions to the user.

[1357] Step 8:

[1358] The server purchases and delivers ingredients based on the proposed recipe. The input is the recipe from step 4, and the output is an ingredient purchase and delivery request. Specifically, the server lists the ingredients required for the proposed recipe and sends a request to an existing ingredient purchase and delivery service.

[1359] Step 9:

[1360] A user checks the purchase and delivery status of ingredients through a smartphone application. The input is the status of the purchase and delivery request sent from the server, and the output is the delivery status displayed to the user. Specifically, a user can open the application and check the purchase and delivery status of ingredients.

[1361] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1362] "Example 1"

[1363] In one embodiment of the present invention, an emotion engine is incorporated. This emotion engine recognizes the user's emotions and uses this information throughout the system. Specifically, when the user speaks to the system or inputs text, the emotion engine infers the user's emotional state from the content. This emotional state is taken into account when providing health advice and suggesting recipes. For example, if it is estimated that the user is feeling stressed, recipes that include ingredients with a relaxation effect are suggested. Health advice is also optimized according to the user's emotional state.

[1364] "Example 2"

[1365] In another embodiment of the present invention, the emotion engine suggests dietary improvement suggestions based on the user's emotional state. Specifically, when a user uploads a photo of a meal, the system evaluates the nutritional balance of the meal. The emotion engine then suggests dietary improvement suggestions taking into account the user's emotional state. For example, if the user is estimated to be feeling joy, dietary improvement suggestions to maintain the joy are suggested.

[1366] "Example 3"

[1367] In yet another embodiment of the present invention, the emotion engine adjusts the purchase and delivery of ingredients based on the user's emotional state. Specifically, the system creates an ingredient list for the recipe suggested by the system, and links with an ingredient purchase and delivery service based on the list. In this case, the emotion engine adjusts the selection of ingredients and the delivery time taking into account the user's emotional state. For example, if it is estimated that the user is feeling sad, ingredients that will ease the sadness are selected, and the delivery is made at a time when the user is likely to feel happy.

[1368] The process flow of each embodiment will be described below.

[1369] "Example 1"

[1370] Step 1: The user inputs speech or text into the system.

[1371] Step 2: The emotion engine infers the user's emotional state from the input.

[1372] Step 3: Based on the estimated emotional state, the system provides health advice and recipe suggestions.

[1373] "Example 2"

[1374] Step 1: A user uploads a photo of their meal into the system.

[1375] Step 2: The system evaluates the nutritional balance of the uploaded meal.

[1376] Step 3: The emotion engine takes the user's emotional state into account and suggests dietary improvements.

[1377] "Example 3"

[1378] Step 1: Create an ingredients list for the recipe suggested by the system.

[1379] Step 2: Based on the ingredients list, the system connects with the ingredients purchasing and delivery service.

[1380] Step 3: The emotion engine takes the user's emotional state into account and adjusts the ingredient selection and delivery time.

[1381] Example 1

[1382] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1383] Conventional health management systems require users to manually input data when importing medical checkup results, which is time-consuming and can lead to input errors. In addition, they provide uniform health advice and recipes without considering the user's emotional state, which means that they are unable to provide optimal suggestions for individual users. Furthermore, there is insufficient integration with food purchasing and delivery services, which means that it is time-consuming for users to purchase ingredients based on suggested recipes.

[1384] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1385] In this invention, the server includes a means for importing and analyzing the results of a medical checkup, a means for providing health advice based on the analysis results, a means for proposing recipes for one week based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, a means for purchasing and delivering ingredients based on the proposed recipes, a means for reading the results of the medical checkup in PDF format and extracting them as text data, and a means for recognizing the user's emotional state and optimizing health advice and recipes based on that information. This allows the user to import accurate medical checkup results without any hassle, and to receive optimal health advice and recipes according to the individual emotional state. In addition, by linking with an ingredient purchasing and delivery service, ingredients can be easily purchased based on the proposed recipes.

[1386] "Results of medical checkup" refers to data showing the results of detailed medical tests conducted as part of a health checkup.

[1387] "Means for analyzing" refers to a method or device for analyzing captured data and extracting meaningful information.

[1388] "Means for providing health advice" refers to a method or device that provides specific instructions or suggestions to a user for maintaining or improving their health based on the analysis results.

[1389] The "means for suggesting a week's worth of recipes" refers to a method or device for suggesting a week's worth of meal menus based on the user's health condition.

[1390] "Means for analyzing photos of meals registered by a user and suggesting improvement suggestions" refers to a method or device that analyzes images of meals taken and registered by a user and makes suggestions for improvement based on nutritional balance and health condition.

[1391] "Means for purchasing and delivering ingredients" refers to a method or device for purchasing the necessary ingredients based on the proposed recipe and delivering them to the user.

[1392] "Means for reading medical checkup results in PDF format and extracting them as text data" refers to a method or device for reading medical checkup results provided as a PDF file and extracting the contents as text data.

[1393] "Means for recognizing a user's emotional state and optimizing health advice and recipes based on that information" refers to a method or device for analyzing a user's emotions and adjusting the content of health advice and recipes based on the results.

[1394] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1395] The system of the present invention imports and analyzes the results of medical checkups and provides users with health advice and recipes. This system is composed of the following main components:

[1396] 1. Upload PDF file and extract text

[1397] The user uploads a PDF file containing the results of the medical checkup to the system. The server receives the uploaded PDF file and uses a PDF analysis library such as "Apache PDFBox" to extract text data from the PDF. This text data is used for subsequent analysis processing.

[1398] 2. Recognizing Emotional States

[1399] The user inputs their recent emotional state into the system as text. For example, they might input, "I've been feeling stressed at work lately." The device then sends this text data to the emotion engine. The emotion engine uses emotion analysis APIs such as IBM Watson and Microsoft Azure Cognitive Services to estimate the user's emotional state. This emotional state is then sent to the server and used throughout the system.

[1400] 3. Health advice and recipe generation

[1401] The server generates health advice and recipes based on the extracted text data and emotional state. For example, if the user is estimated to be stressed, the server will suggest recipes that contain ingredients that have a relaxation effect. The server uses a generative AI model to generate appropriate advice and recipes.

[1402] 4. Displaying the results

[1403] The terminal displays the health advice and recipes received from the server to the user, who can then review the suggested advice and recipes through the system's interface.

[1404] Examples

[1405] A user uploads the results of a medical checkup in PDF format to the system. The server extracts text data from the PDF and analyzes the data. Next, if the user enters text such as "I've been feeling a lot of stress at work lately," the emotion engine infers that the user is feeling stressed. Based on this information, the server suggests recipes that include ingredients that have a relaxing effect.

[1406] Examples of prompt statements

[1407] "Please upload the results of your medical checkup in PDF format. After that, please enter text about your recent emotional state. For example, 'I've been feeling a lot of stress at work lately.'"

[1408] In this way, the system can take into account the user's overall health and emotional state to provide optimal health advice and recipes.

[1409] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1410] Step 1:

[1411] A user uploads a PDF file containing the results of a medical checkup to the system. The user opens a file selection dialog through the system interface and selects the target PDF file. The input is the PDF file, and the output is the PDF file sent to the server.

[1412] Step 2:

[1413] The server receives the uploaded PDF file. Then, the server uses the "Apache PDFBox" library to extract text data from the PDF file. Specifically, it uses the PDFBox functions to parse the PDF and obtain the text data. The input is the PDF file, and the output is the extracted text data.

[1414] Step 3:

[1415] The user inputs a text of his / her recent emotional state to the system. For example, the user inputs "I've been feeling a lot of stress at work lately." The input is text data indicating the user's emotional state, and the output is the text data sent to the terminal.

[1416] Step 4:

[1417] The device sends text data entered by the user to the emotion engine. The emotion engine uses emotion analysis APIs such as "IBM Watson" and "Microsoft Azure Cognitive Services" to estimate the user's emotional state. Specifically, the device sends text data to the emotion analysis API and obtains the emotional state. The input is text data indicating the user's emotional state, and the output is the estimated emotional state.

[1418] Step 5:

[1419] The server generates health advice and recipes based on the extracted text data and the estimated emotional state. It uses a generative AI model to generate appropriate advice and recipes. In concrete terms, it inputs a prompt sentence into the generative AI model and obtains the generated advice and recipes. The input is the text data and the emotional state, and the output is the generated health advice and recipes.

[1420] Step 6:

[1421] The terminal displays the health advice and recipes received from the server to the user. The user can check the suggested advice and recipes through the system interface. The specific operation is to display the advice and recipes on the terminal screen. The input is the health advice and recipes, and the output is the information displayed to the user.

[1422] (Application example 1)

[1423] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1424] Conventional health management systems can analyze the results of medical checkups and provide health advice and recipe suggestions, but they cannot provide advice or suggestions that take into account the user's emotional state, making it difficult to provide optimal health management that matches the user's psychological state.In addition, users cannot receive real-time health advice or product suggestions while shopping in a physical store, making it difficult for them to select appropriate products on the spot.

[1425] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1426] In this invention, the server includes a means for importing and analyzing the results of a medical checkup, a means for providing health advice based on the analysis results, a means for proposing recipes for one week based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, a means for purchasing and delivering ingredients based on the proposed recipes, a means for recognizing the user's emotional state and optimizing health advice and recipe suggestions based on the information, and a means for providing health advice and product suggestions in real time through smart glasses. This enables optimal health management according to the user's emotional state, and allows the user to receive appropriate health advice and product suggestions in real time even while shopping at a physical store. The server includes a means for providing health advice and product suggestions in real time through smart glasses.

[1427] The "results of a medical checkup" are data on the test results obtained in a medical checkup, which is a type of health checkup.

[1428] An "analyzing means" is a device or software for analyzing captured data and extracting meaningful information.

[1429] The "means for providing health advice" refers to a device or software that provides the user with advice on health management based on the analysis results.

[1430] The "means for suggesting a week's worth of recipes" refers to a device or software that suggests a week's worth of meal menus to the user based on the analysis results.

[1431] The "means for analyzing food photos and proposing improvement suggestions" refers to a device or software that analyzes food photos registered by a user and provides improvement suggestions for making the diet healthier.

[1432] The "means for purchasing and delivering ingredients" is a device or software for purchasing the necessary ingredients based on the proposed recipe and delivering them to the user.

[1433] The "means for recognizing an emotional state and optimizing health advice or recipe suggestions based on that information" refers to a device or software that analyzes a user's emotions and adjusts health advice or recipe suggestions based on the results.

[1434] The "means for providing health advice and product suggestions in real time through smart glasses" refers to a device or software that displays health advice and product suggestions to a user in real time using smart glasses.

[1435] A system for carrying out the present invention has the following configuration: The server includes means for importing and analyzing the results of a medical checkup, means for providing health advice based on the analysis results, means for proposing recipes for a week based on the analysis results, means for analyzing photos of meals registered by the user and proposing improvement plans, means for purchasing and delivering ingredients based on the proposed recipes, means for recognizing the emotional state of the user and optimizing health advice and recipe suggestions based on that information, and means for providing health advice and product suggestions in real time through the smart glasses.

[1436] The server first imports the results of the medical checkup in PDF format and extracts them as text data using pytesseract. It then analyzes the extracted data and evaluates the user's health condition. Based on the analysis results, it provides the user with health advice. For example, if the user has high blood pressure, it advises them to limit their salt intake.

[1437] Furthermore, the system will suggest recipes for the week based on the analysis results. The system takes into account not only the user's health condition but also their emotional state when suggesting recipes. For example, if the user is feeling stressed, the system will suggest recipes that include ingredients that have a relaxing effect.

[1438] The system analyzes photos of meals uploaded by users and suggests improvements to make them healthier. To do this, it uses image analysis technology to recognize the contents of the meal and evaluate the nutritional balance. For example, if there are not enough vegetables, it suggests adding more vegetables.

[1439] Also included is a means for purchasing and delivering the necessary ingredients to the user based on the suggested recipe, in cooperation with existing ingredient purchasing and delivery services.

[1440] This also includes a means of providing real-time health advice and product suggestions through smart glasses. When a user wears the smart glasses and goes shopping in a brick-and-mortar store, the camera will recognize the user's emotional state and suggest appropriate products based on that information. For example, if the user is feeling stressed, products with a relaxation effect (such as herbal tea or aromatic oil) will be suggested.

[1441] Below are some example prompts that can be used to provide optimal health advice and product suggestions based on a user’s emotional state using a generative AI model:

[1442] "If the user is feeling stressed, suggest products that have a relaxation effect, such as herbal tea or aromatic oils."

[1443] In this way, optimal health management based on the user's emotional state is possible, and the user can receive appropriate health advice and product suggestions in real time while shopping in a physical store.

[1444] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1445] Step 1:

[1446] The server imports the results of the medical checkup in PDF format and extracts them as text data using pytesseract. The input is a PDF file and the output is text data. Specifically, the server reads the PDF file and extracts text information using image analysis technology.

[1447] Step 2:

[1448] The server analyzes the extracted text data and evaluates the user's health condition. The input is the text data, and the output is the health condition evaluation result. Specifically, the server analyzes the text data and evaluates health indicators such as blood pressure and blood sugar level.

[1449] Step 3:

[1450] The server provides health advice based on the analysis results. The input is the health status evaluation result, and the output is health advice. Specifically, the server generates advice such as "reduce salt intake" based on the evaluation result.

[1451] Step 4:

[1452] The server proposes recipes for one week based on the analysis results. The input is the health assessment result, and the output is recipes for one week. Specifically, the server generates recipes suitable for the user's health condition based on the assessment result.

[1453] Step 5:

[1454] The server analyzes photos of meals registered by users and proposes improvement suggestions. The input is a photo of the meal, and the output is improvement suggestions. Specifically, the server uses image analysis technology to recognize the contents of the meal, evaluates the nutritional balance, and generates improvement suggestions.

[1455] Step 6:

[1456] The server purchases the necessary ingredients based on the proposed recipe and delivers them to the user. The input is the recipe, and the output is a shopping list and delivery instructions. Specifically, the server makes a list of the necessary ingredients based on the recipe, and coordinates with existing ingredient purchasing and delivery services to arrange for delivery.

[1457] Step 7:

[1458] The server uses an emotion engine to recognize the user's emotional state and optimizes health advice and recipe suggestions based on that information. The input is the user's emotional data, and the output is optimized advice and recipes. Specifically, the server analyzes the user's speech and text input, estimates the user's emotional state, and adjusts the advice and recipes.

[1459] Step 8:

[1460] The terminal provides real-time health advice and product suggestions through the smart glasses. The input is the user's emotional state and video data from within the store, and the output is advice and product suggestions displayed on the smart glasses. Specifically, the terminal recognizes the user's emotional state through a camera and suggests appropriate products based on that information.

[1461] Example 2

[1462] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[1463] Conventional health management systems had difficulty not only assessing the user's health condition but also providing dietary improvement suggestions that took into account the user's emotional state. There was also a need to utilize generative AI models to provide more accurate advice and suggestions. Furthermore, there was a lack of functionality to analyze photos of meals uploaded by users and provide specific improvement suggestions.

[1464] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1465] In this invention, the server includes a means for importing and analyzing the results of a medical checkup, a means for providing health advice based on the analysis results, a means for proposing recipes for one week based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, a means for purchasing and delivering ingredients based on the proposed recipes, a means for estimating the user's emotional state and proposing improvement plans for meals based on the emotional state, and a means for generating health advice and improvement plans for meals using a generative AI model. This makes it possible to comprehensively evaluate the user's health state and emotional state and provide appropriate advice and suggestions. The server includes a means for generating improvement plans for the meals based on the user's emotions using a generative AI model, and a means for adjusting the selection of ingredients and the delivery time of the ingredients based on the user's emotions.

[1466] The "means for receiving and analyzing the results of a medical checkup" refers to a combination of software and hardware for receiving medical checkup data provided by a user and analyzing the data.

[1467] "Means for providing health advice based on analysis results" refers to a combination of software and hardware for evaluating the user's health condition and generating appropriate health management advice based on analyzed health checkup data.

[1468] The "means for suggesting a week's worth of recipes based on the analysis results" is a combination of software and hardware for suggesting a balanced meal plan based on the user's health condition.

[1469] "Means for analyzing photos of meals registered by users and proposing improvement suggestions" refers to a combination of software and hardware for analyzing images of meals uploaded by users, evaluating their nutritional balance, and providing improvement suggestions.

[1470] "Means for purchasing and delivering ingredients based on a suggested recipe" is a combination of software and hardware for interfacing with the service to purchase ingredients needed for a suggested recipe and deliver them to the user.

[1471] The "means for estimating the user's emotional state and suggesting meal improvement suggestions based on that emotional state" is a combination of software and hardware for analyzing the user's emotional state and providing meal improvement suggestions based on that state.

[1472] "Means for generating health advice and dietary improvement suggestions using a generative AI model" refers to a combination of software and hardware for using artificial intelligence technology to generate advice and suggestions based on the health and emotional state of a user. The server includes means for generating the dietary improvement suggestions based on the user's emotions using a generative AI model, and means for adjusting the selection of ingredients and the delivery time of the ingredients based on the user's emotions.

[1473] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1474] The present invention is a system for comprehensively evaluating a user's health and emotional state and providing appropriate advice and suggestions. The system has the following main functions:

[1475] 1. Medical checkup results analysis function

[1476] The server receives the results of medical checkups uploaded by users. These results include numerical data such as blood pressure, blood glucose level, cholesterol level, etc. The server uses a Python-based data analysis library (e.g., Pandas, NumPy) to analyze this data.

[1477] The server evaluates the user's health condition based on the analysis results. For example, if the blood sugar level is high, the server determines that the user is at risk for diabetes and generates advice accordingly. This advice includes specific action guidelines for improving the user's health condition.

[1478] Examples:

[1479] A user uploads the results of a medical checkup.

[1480] The server detects that the blood glucose level is 140 mg / dL.

[1481] The server determines if a person is at high risk for diabetes and provides advice, including recommendations for improved diet and exercise.

[1482] Example prompt:

[1483] "If the user's blood sugar level is 140 mg / dL, determine that they are at high risk for diabetes and provide appropriate advice."

[1484] 2. Diet improvement suggestions function based on emotion engine

[1485] A user uploads a photo of their meal, and the server uses image analysis software (e.g., OpenCV, TensorFlow) to evaluate the nutritional balance of the meal.

[1486] The server estimates the user's emotional state using an emotion engine, which includes algorithms for estimating the user's current emotional state based on the user's past behavioral data and input data.

[1487] The server provides dietary suggestions based on the user's emotional state, for example providing dietary suggestions to maintain the user's joy if the user is estimated to be feeling joy.

[1488] Examples:

[1489] A user uploads a photo of a meal.

[1490] The server evaluates the nutritional balance of meals from the photos and detects whether they are high in carbohydrates.

[1491] The server infers that the user is feeling happy and suggests a vegetable-rich diet to maintain that feeling.

[1492] Example prompt:

[1493] "If the user is feeling pleasure, suggest dietary changes to maintain that feeling. For example, if their diet is high in carbohydrates, suggest a diet high in vegetables."

[1494] 3. Recipe suggestions and ingredient purchasing and delivery features

[1495] The server then suggests recipes for the week based on the analysis results. These recipes are customized to the user's health condition. For example, low-carb recipes are suggested to a user who is at high risk for diabetes.

[1496] The server cooperates with a service to purchase ingredients based on the suggested recipe and deliver them to the user, so that the user can easily follow the suggested recipe.

[1497] This system supports users in managing their health by comprehensively evaluating their health and emotional state and providing appropriate advice and suggestions.The hardware and software used include servers (e.g., cloud services), user terminals (e.g., smartphones, PCs), Python, Pandas, NumPy, OpenCV, TensorFlow, etc.

[1498] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1499] Medical checkup result analysis function

[1500] Step 1: User uploads medical examination results

[1501] Input: The user uploads the results of the medical checkup in PDF or image format using a smartphone or PC.

[1502] Operation: The user selects the health check result file through the system interface and clicks the upload button.

[1503] Output: The uploaded file is sent to the server.

[1504] Step 2: The server receives the data and prepares it for analysis

[1505] Input: Uploaded PDF or image files.

[1506] How it works: The server checks the format of the file it receives and, if it's a PDF, converts it to text using OCR (Optical Character Recognition) software (e.g. Tesseract), and similarly for images.

[1507] Output: Medical checkup results converted into text data.

[1508] Step 3: The server parses the data

[1509] Input: Medical examination results converted into text data.

[1510] How it works: The server uses Python's Pandas library to convert the text data into a data frame, then uses NumPy to parse various numerical data (blood pressure, blood glucose levels, cholesterol levels, etc.).

[1511] Output: Parsed medical examination data.

[1512] Step 4: Server evaluates health and generates advice

[1513] Input: Parsed medical examination data.

[1514] How it works: The server evaluates the user's health status based on the analysis results. For example, if the blood glucose level is 140 mg / dL, it may determine that the user is at high risk for diabetes. It then uses a generative AI model to generate appropriate advice, which may include recommendations for dietary improvements and exercise.

[1515] Output: The generated health advice.

[1516] Step 5: The server provides advice to the user

[1517] Input: The generated health advice.

[1518] Operation: The server sends the generated advice to the user's device, where the user can check the advice on their smartphone or PC.

[1519] Output: Health advice displayed on the user's device.

[1520] Meal improvement suggestions function based on emotion engine

[1521] Step 1: User uploads a photo of their meal

[1522] Input: A photo of a meal taken by a user using their smartphone.

[1523] Operation: A user selects a photo of a meal through the system's interface and clicks the upload button.

[1524] Output: The uploaded photos are sent to the server.

[1525] Step 2: The server receives the images and prepares them for analysis

[1526] Input: Uploaded food photos.

[1527] How it works: The server uses image analysis software (e.g. OpenCV, TensorFlow) to analyze the images it receives.

[1528] Output: Image data ready for analysis.

[1529] Step 3: The server evaluates the nutritional balance of the meal

[1530] Input: Image data ready for analysis.

[1531] How it works: The server evaluates the nutritional balance of a meal based on the results of image analysis, for example calculating the percentages of carbohydrates, proteins, and fats.

[1532] Output: Evaluated nutritional balance data.

[1533] Step 4: The server estimates the user's emotional state

[1534] Input: User's past behavioral data and input data.

[1535] Operation: The server uses an emotion engine to estimate the user's emotional state. The emotion engine contains algorithms that estimate the user's current emotional state based on the user's past behavioral data and input data.

[1536] Output: Estimated emotional state data.

[1537] Step 5: The server generates meal improvement suggestions

[1538] Input: Evaluated nutritional balance data and estimated emotional state data.

[1539] Operation: The server generates dietary suggestions based on the user's emotional state, e.g., if the user is estimated to be feeling joyful, the server provides dietary suggestions to maintain that joy.

[1540] Output: Generated dietary improvement suggestions.

[1541] Step 6: The server provides the user with suggestions for improvement

[1542] Input: Generated dietary improvement suggestions.

[1543] Operation: The server sends the generated improvement proposal to the user's device. The user can then check the improvement proposal on their smartphone or PC.

[1544] Output: Dietary improvement suggestions displayed on the user's device.

[1545] (Application example 2)

[1546] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1547] Conventional health management systems can provide advice and recipe suggestions based on the user's health condition, but they cannot provide dietary improvement suggestions that take into account the user's emotional state. As a result, it is not possible to provide optimal dietary suggestions based on the user's emotional state, making it difficult to increase user satisfaction. In addition, there is a lack of functionality that allows users to order the suggested meals as is, which reduces user convenience.

[1548] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1549] In this invention, the server includes means for importing and analyzing the results of a medical checkup, means for providing health advice based on the analysis results, means for proposing recipes for one week based on the analysis results, means for analyzing photos of meals registered by the user and proposing improvement plans, means for purchasing and delivering ingredients based on the proposed recipes, means for analyzing the emotional state of the user and proposing meal improvement plans based on the emotions, and means for proposing and ordering optimal meals based on the user's health and emotional states. This makes it possible to propose optimal meals that comprehensively take into account the user's health and emotional states, thereby increasing user satisfaction and convenience.

[1550] "Results of a medical checkup" refers to various health data (blood pressure, blood sugar level, cholesterol level, etc.) obtained during a medical checkup, which is a type of health checkup.

[1551] "Means for analyzing" refers to a method or device for analyzing the captured data and assessing the user's health or emotional state.

[1552] The "means for providing health advice" refers to a method or device that provides the user with specific guidance or suggestions for maintaining or improving their health based on the analysis results.

[1553] The "means for suggesting a week's worth of recipes" refers to a method or device that provides a week's worth of nutritionally balanced meal menus based on the user's health condition.

[1554] The "means for analyzing photos of meals and proposing improvement suggestions" refers to a method or device that analyzes images of meals registered by a user and suggests improvements to the meal content based on nutritional balance and health condition.

[1555] "Means for purchasing and delivering ingredients" refers to a method or device for purchasing the necessary ingredients based on the proposed recipe and delivering them to the user.

[1556] The "means for analyzing an emotional state and proposing dietary improvement suggestions based on the emotions" refers to a method or device for analyzing a user's emotional state and proposing improvements to dietary content according to the emotions.

[1557] The "means for proposing and ordering the most suitable meal" refers to a method or device that proposes the most suitable meal by taking into consideration the user's overall health and emotional state, and enables the user to order that meal.

[1558] A system for implementing the present invention comprises a server, a user terminal, and related software components. A specific embodiment of the system will be described below.

[1559] System Configuration

[1560] 1. Server:

[1561] The server has the function of importing and analyzing the medical checkup results sent by users, using software libraries such as Python, Scikit-learn, and Keras.

[1562] The server generates health advice based on the analysis results and provides it to the user.

[1563] The server has a function of suggesting recipes for a week based on the user's health condition.

[1564] The server analyzes photos of meals registered by the user and suggests improvements to the meal content based on nutritional balance and health condition.

[1565] The server analyzes the user's emotional state and proposes dietary improvement suggestions based on the emotion. For emotion analysis, a deep learning model using OpenCV and Keras is used.

[1566] The server provides a function that suggests optimal meals by taking into consideration the user's overall health and emotional state, and allows the user to order the meals directly.

[1567] 2. User device:

[1568] The user terminal is a device such as a smartphone or tablet, which uses a camera to take photos of the meal and transmits them to the server.

[1569] The user terminal displays health advice and recipe suggestions provided by the server.

[1570] The user terminal provides an interface for directly ordering the suggested meals.

[1571] Data processing and calculation

[1572] 1. Health analysis:

[1573] The server receives the health checkup results sent by the user and standardizes the data using Scikit-learn.

[1574] Based on standardized data, health status is assessed and health advice, such as diabetes risk, is generated.

[1575] 2. Emotional state analysis:

[1576] The server receives the food photos uploaded by users and pre-processes the images using OpenCV.

[1577] The preprocessed images are input into a Keras deep learning model to analyze the emotional state.

[1578] Based on the analysis results, dietary improvement suggestions are generated according to emotions.

[1579] 3. Meal Suggestions and Ordering:

[1580] The server takes into consideration the user's overall health and emotional state and suggests the most appropriate meal.

[1581] The user terminal displays the suggested meals and provides an interface through which the user can place an order.

[1582] Examples

[1583] If a user uploads the results of a health check and their blood sugar level is assessed as high, the server will suggest a low-carb meal.

[1584] If a user uploads a photo of a meal and the emotion engine detects "joy," the server will suggest balanced meals to maintain that emotion.

[1585] Examples of prompt statements

[1586] Evaluate the user's health status and suggest appropriate dietary recommendations using the following data:

[1587] Blood pressure: 120

[1588] Blood sugar level: 7.0

[1589] Cholesterol level: 200

[1590] Also, analyze the user's emotional state and suggest dietary improvements based on their emotions. Please use the image below.

[1591] Image Path: 'user_meal.jpg'

[1592] In this way, a system is realized that makes optimal meal suggestions that take into consideration the user's overall health and emotional state.

[1593] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1594] Step 1:

[1595] A user uploads the results of a medical checkup.

[1596] Input: The user uses a smartphone or tablet to input the results of a health check (blood pressure, blood sugar level, cholesterol level, etc.).

[1597] Output: The medical checkup result data sent to the server.

[1598] Specific operations: The user opens the dedicated application, enters the results of the health checkup according to the specified format, and presses the send button.

[1599] Step 2:

[1600] The server analyzes the results of the health check.

[1601] Input: Medical checkup result data submitted by the user.

[1602] Output: Health status assessment results and health advice.

[1603] How it works: The server uses Scikit-learn to standardize the received data and evaluate the health status. For example, if blood sugar level is high, it evaluates the risk of diabetes and suggests a low-sugar diet.

[1604] Step 3:

[1605] The server will suggest recipes for the week.

[1606] Input: Health status assessment results.

[1607] Output: A week's worth of recipes based on the user's health status.

[1608] Specific operation: The server generates nutritionally balanced recipes based on the health assessment results and provides them to the user.

[1609] Step 4:

[1610] A user uploads a photo of a meal.

[1611] Input: A photo of a meal taken by the user using a smartphone or tablet.

[1612] Output: Food photo data sent to the server.

[1613] Specific actions: The user opens the dedicated application, takes a photo of the meal, and presses the send button.

[1614] Step 5:

[1615] The server analyzes photos of meals and suggests improvements.

[1616] Input: Food photo data submitted by the user.

[1617] Output: Suggested dietary improvements.

[1618] Specific operation: The server preprocesses the images using OpenCV, inputs them into the Keras deep learning model to analyze nutritional balance, and generates recommendations for improving the diet based on the results.

[1619] Step 6:

[1620] The server analyzes the user's emotional state.

[1621] Input: Food photo data submitted by the user.

[1622] Output: Evaluation of the user's emotional state.

[1623] Specific operation: The server uses the photos of food to analyze the emotional state with an emotion engine (a Keras deep learning model). For example, if the user feels happy, it generates meal improvement suggestions to maintain that emotion.

[1624] Step 7:

[1625] The server will suggest the best meal and allow the user to order.

[1626] Input: Health status assessment results and emotional status assessment results.

[1627] Output: Optimal meal suggestions and ordering interface based on the user's health and emotional state.

[1628] Specific operation: The server considers the health and emotional state of the user and proposes the optimal meal. The user terminal displays the proposed meal and provides an interface that allows the user to order it.

[1629] In this way, a system is realized that makes optimal meal suggestions that take into consideration the user's overall health and emotional state.

[1630] Example 3

[1631] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1632] In modern society, improving dietary management and nutritional balance according to the individual's health condition is an important issue. However, it is time-consuming and requires specialized knowledge for users to select appropriate recipes based on their own health condition and purchase and deliver ingredients. In addition, it is difficult for conventional systems to suggest meals and adjust ingredient delivery while taking into account the user's emotional state. To solve these issues, a system is needed that automatically suggests appropriate recipes and purchases and delivers ingredients based on the user's health and emotional state.

[1633] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1634] In this invention, the server includes means for importing and analyzing the results of a medical checkup, means for providing health advice based on the analysis results, means for proposing recipes for one week based on the analysis results, means for analyzing images of meals registered by the user and proposing improvement plans, means for purchasing and delivering ingredients based on the proposed recipes, and means for analyzing the emotional state of the user and adjusting the selection of ingredients and delivery time based on the emotional state. This makes it possible to propose appropriate recipes and purchase and deliver ingredients based on the user's health and emotional state.

[1635] "Results of a health checkup" refers to data obtained as a result of a health examination or test.

[1636] An "analytical means" is a method or device for analyzing data and extracting meaningful information.

[1637] A "means for providing health advice" is a method or device that provides health advice or suggestions to a user based on the analysis results.

[1638] The "means for suggesting a week's worth of recipes" refers to a method or device that suggests a week's worth of meal menus based on the user's health condition.

[1639] "Meal images registered by a user" refers to photo data of meals that a user uploads to the system.

[1640] The "means for proposing improvement suggestions" refers to a method or device that makes suggestions for improving the user's diet based on the analysis results.

[1641] "Means for purchasing and delivering ingredients" refers to a method or device for purchasing the necessary ingredients based on the proposed recipe and delivering them to a location specified by the user.

[1642] The "means for analyzing an emotional state" refers to a method or device for analyzing the user's emotions and understanding that state.

[1643] A "means for adjusting ingredient selection and delivery time based on emotional state" is a method or device for selecting appropriate ingredients and adjusting delivery time by taking into account the user's emotional state.

[1644] The system of the present invention automatically proposes appropriate recipes and purchases and delivers ingredients based on the user's health and emotional state. This system exchanges data between the server, the terminal, and the user, and performs various analyses and proposals.

[1645] Hardware and Software Used

[1646] Server: A server with high-performance computing power is used to analyze data, suggest recipes, purchase ingredients, and process deliveries.

[1647] Device: A device used by a user, such as a smartphone or tablet, that allows the user to take photos of food, select recipes, and input emotional states.

[1648] Image analysis software: Uses TensorFlow and OpenCV to identify ingredients and dishes from food photos.

[1649] Nutrient databases: Evaluate the nutritional value of ingredients using databases such as the USDA National Nutrient Database.

[1650] Emotion Analysis Software: Affectiva and IBM Watson Tone Analyzer are used to analyze the user’s emotional state.

[1651] Program Processing

[1652] 1. A user takes a photo of a meal on a device and uploads it to the system. The device sends the uploaded photo to the server.

[1653] 2. The server analyzes the received photos using image analysis software (e.g., TensorFlow or OpenCV) to identify the ingredients and dishes.

[1654] 3. The server checks the identified ingredients and dishes against a nutritional database (e.g., USDA National Nutrient Database) to assess their nutritional balance.

[1655] 4. Based on the evaluation results, the server generates dietary improvement suggestions for the user and sends them to the user's device.

[1656] 5. The user selects a suggested recipe, and the device sends the selected recipe information to the server.

[1657] 6. The server generates an ingredients list for the selected recipe and connects with an ingredient purchasing and delivery service (e.g., Amazon Fresh or Instacart) to purchase and deliver the ingredients.

[1658] 7. The server analyzes the user’s emotional state using emotion analysis software (e.g., Affectiva or IBM Watson Tone Analyzer) and adjusts ingredient selection and delivery time based on the user’s emotional state.

[1659] Examples

[1660] Food photo analysis and improvement suggestions: If a user takes a photo that contains steak and potatoes, the system will identify this and suggest a salad that is high in vegetables.

[1661] Purchase and delivery of ingredients based on a recipe: If a suggested recipe includes “chicken breast,” the system will send the ingredients to a purchasing and delivery service to be delivered to the user’s home.

[1662] Emotion engine adjustment: If the system estimates that the user is feeling sad, it will choose food items like chocolate and fruit and deliver them in the evening when they are more relaxed.

[1663] Examples of prompt statements

[1664] "Analyze this photo of a meal, evaluate its nutritional balance, and suggest improvements."

[1665] "Generate an ingredients list based on this recipe and connect with an ingredient shopping and delivery service to deliver the ingredients."

[1666] "Analyze the user's emotional state, adjust the ingredients list based on that state, and deliver at the right time."

[1667] As described above, the system of the present invention provides a specific means for realizing appropriate recipe suggestions, purchasing and delivery of ingredients based on the health and emotional state of the user. The flow of the specific process in the third embodiment will be described with reference to FIG.

[1668] Step 1:

[1669] Users take photos of their meals with their devices and upload them to the system.

[1670] Input: A photo of a meal taken by the user.

[1671] Specific actions: The user takes a photo of their meal using their smartphone's camera app, opens the dedicated app, selects the photo, and presses the upload button.

[1672] Output: The uploaded photo data is saved on the device.

[1673] Step 2:

[1674] The device sends the uploaded photos to the server.

[1675] Input: The uploaded photo data.

[1676] How it works: The device sends the uploaded photo data to the server via an Internet connection, along with metadata such as the user ID and timestamp.

[1677] Output: Photo data and metadata are sent to the server.

[1678] Step 3:

[1679] The photos received by the server are analyzed using image analysis software (e.g., TensorFlow or OpenCV).

[1680] Input: The received photo data.

[1681] How it works: The server feeds the received photo data into image analysis software and uses pre-trained models to identify ingredients and dishes.

[1682] Output: Information on ingredients and dishes identified as a result of the analysis.

[1683] Step 4:

[1684] The server identifies ingredients and dishes from the analysis results.

[1685] Input: Output of image analysis software.

[1686] How it works: The server uses the output of image analysis software to identify ingredients and dishes in a photo, for example identifying that a photo contains "steak" and "potatoes."

[1687] Output: A list of identified ingredients and dishes.

[1688] Step 5:

[1689] The server may compare identified ingredients and dishes against a nutritional database (e.g., the USDA National Nutrient Database) to assess their nutritional balance.

[1690] Input: A list of identified ingredients and / or dishes.

[1691] Specific operation: The server compares the identified ingredients and dish information with the nutrition database to obtain the nutritional value of each ingredient, for example, the calories, protein, and fat of "steak."

[1692] Output: Nutritional value information for each ingredient.

[1693] Step 6:

[1694] The server generates dietary improvement suggestions for the user based on the evaluation results.

[1695] Input: Nutritional value information for each ingredient.

[1696] Specific operation: The server evaluates the user's dietary balance based on the acquired nutritional information and generates improvement suggestions, such as "add a salad that contains a lot of vegetables."

[1697] Output: Dietary improvement suggestions.

[1698] Step 7:

[1699] The server sends the improvement proposal generated to the user's terminal and notifies the user.

[1700] Input: Dietary improvement suggestions.

[1701] Specific operation: The server sends the generated improvement proposal to the user's device, and the device notifies the user of the received improvement proposal and displays it within the app.

[1702] Output: The suggested improvements displayed on the user's device.

[1703] Step 8:

[1704] The user selects a suggested recipe.

[1705] Input: A list of suggested recipes.

[1706] Specific behavior: The user reviews the list of suggested recipes within the app, selects the recipe they like, and presses the purchase button.

[1707] Output: The selected recipe information.

[1708] Step 9:

[1709] The terminal transmits the selected recipe information to the server.

[1710] Input: Selected recipe information.

[1711] Specific operation: The device sends information about the selected recipe to the server, along with metadata such as the user ID and recipe ID.

[1712] Output: The recipe information sent to the server.

[1713] Step 10:

[1714] The server generates an ingredient list for the selected recipe.

[1715] Input: Selected recipe information.

[1716] Specific operation: The server generates a list of ingredients based on the selected recipe information. For example, it creates an ingredients list such as "chicken breast," "carrot," and "onion."

[1717] Output: Ingredients list.

[1718] Step 11:

[1719] The server cooperates with a food purchasing and delivery service (e.g., food purchasing and delivery service) to purchase and deliver food.

[1720] Input: Ingredient list.

[1721] Specific operation: The server uses the API of the food purchasing and delivery service to send the list of ingredients, and the service checks the inventory and completes the purchase process.

[1722] Output: The ingredient list sent to the purchasing and delivery service.

[1723] Step 12:

[1724] The server sends the list of ingredients to a purchasing and delivery service and arranges for the ingredients to be delivered to the user's home.

[1725] Input: A confirmation message from the purchasing and delivery service.

[1726] Specific operation: The server receives a confirmation message from the purchase and delivery service and sends the user's delivery information. The delivery service delivers the groceries to the user's home at the specified date and time.

[1727] Output: Groceries delivered to the user's home.

[1728] Step 13:

[1729] The server analyzes the user's emotional state using emotion analysis software, e.g.

[1730] Input: User emotional state data.

[1731] How it works: The user inputs their emotional state within the app or analyzes their facial expressions through the camera. The emotion analysis software analyzes the user's emotional state and generates a result.

[1732] Output: Emotional state analysis result.

[1733] Step 14:

[1734] The server adjusts the ingredients list of a suggested recipe based on the user's emotional state.

[1735] Input: Emotional state analysis results.

[1736] What it does: The server adjusts the ingredient list based on the emotional state data, for example adding ingredients like chocolate and fruit if the user is feeling sad.

[1737] Output: The adjusted ingredient list.

[1738] Step 15:

[1739] The server purchases ingredients and links with delivery services based on the adjusted ingredient list.

[1740] Input: The adjusted ingredient list.

[1741] Specific operation: The server sends the adjusted ingredient list to the ingredient purchasing and delivery service, which checks the ingredient inventory and completes the purchase process.

[1742] Output: The adjusted ingredient list sent to the purchasing and delivery service.

[1743] Step 16:

[1744] The server takes into account the user's emotional state and adjusts the selection of ingredients and delivery time.

[1745] Input: tuned ingredient list and parsed emotional state.

[1746] Specific behavior: The server takes into account the user's emotional state and adjusts the delivery time, for example arranging for the delivery to occur at a time when the user is likely to feel happy.

[1747] Output: The delivered ingredients based on the adjusted delivery times.

[1748] (Application example 3)

[1749] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1750] Conventional health management systems suggested recipes and purchased and delivered ingredients based on the user's health condition, but did not select ingredients or adjust delivery times based on the user's emotional state. As a result, it was not possible to provide appropriate meal suggestions or delivery services according to the user's emotional state, making it difficult to improve user satisfaction.

[1751] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1752] In this invention, the server includes a means for importing and analyzing the results of a medical checkup, a means for providing health advice based on the analysis results, a means for proposing recipes for one week based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, a means for purchasing and delivering ingredients based on the proposed recipes, and a means for analyzing the emotional state of the user and adjusting the selection of ingredients and delivery time based on the emotional state. This makes it possible to make appropriate meal suggestions and purchase and deliver ingredients taking into account both the health and emotional states of the user.

[1753] A "health checkup" is a medical service that provides a comprehensive health check.

[1754] An "analyzing means" is a method or device for analyzing data and extracting specific information.

[1755] A "means for providing health advice" is a method or device for providing appropriate lifestyle and dietary advice based on the user's health condition.

[1756] A "means for suggesting a week's worth of recipes" is a method or device for providing a week's worth of meal menus based on the user's health condition and preferences.

[1757] The "means for analyzing food photos and proposing improvement suggestions" refers to a method or device for analyzing food photos registered by a user and providing improvement suggestions regarding nutritional balance and ingredient selection.

[1758] "Means for purchasing and delivering ingredients" refers to a method or device for purchasing the necessary ingredients based on the proposed recipe and delivering them to a location specified by the user.

[1759] "Means for analyzing an emotional state and adjusting ingredient selection or delivery time based on the emotional state" refers to a method or device for analyzing a user's emotional state and optimizing ingredient selection or delivery time based on the results.

[1760] A system for carrying out this invention has the following configuration: The server includes means for inputting and analyzing the results of a medical checkup, means for providing health advice based on the analysis results, means for proposing recipes for the week based on the analysis results, means for analyzing photos of meals registered by the user and proposing improvement plans, means for purchasing and delivering ingredients based on the proposed recipes, and means for analyzing the user's emotional state and adjusting the selection of ingredients and delivery time based on the emotional state.

[1761] Hardware and Software Used

[1762] Hardware:

[1763] Smartphone

[1764] server

[1765] software:

[1766] Python

[1767] PIL (Python Imaging Library)

[1768] requests (HTTP library)

[1769] Sentiment Analysis Engine

[1770] Data processing and calculation

[1771] 1. Means of importing and analyzing medical checkup results:

[1772] The server receives the user's health checkup data and analyzes the data using an analysis engine, thereby evaluating the user's health condition.

[1773] 2. Means of Providing Health Advice:

[1774] Based on the analysis results, the server generates health advice appropriate for the user and notifies the smartphone.

[1775] 3. How to suggest recipes for the week:

[1776] The server uses a generative AI model to suggest recipes for a week based on the user's health condition. For example, it suggests low-carb recipes to a user who is at risk of diabetes.

[1777] 4. Methods for analyzing food photos and suggesting improvements:

[1778] The user takes a photo of the meal with their smartphone and sends it to the server, which analyzes the image using PIL. Based on the analysis results, the system provides suggestions for improving nutritional balance.

[1779] 5. Means of purchasing and delivering food:

[1780] The server generates a list of ingredients needed based on the proposed recipe, and cooperates with existing ingredient purchasing and delivery services to purchase the ingredients and deliver them to the user's specified location.

[1781] 6. A means to analyze emotional states and adjust food selections and delivery times based on emotional states:

[1782] The server evaluates the user's emotional state using an analysis engine and optimizes the selection of ingredients and delivery time based on the results. For example, if the user is feeling sad, the server selects ingredients that will ease that emotion and delivers them at a time when the user is likely to feel happy.

[1783] Examples

[1784] User ID: user123

[1785] Meal photo URL: https: / / example.com / meal_photo.jpg

[1786] Example prompt for a generative AI model:

[1787] User ID: user123

[1788] Meal photo URL: https: / / example.com / meal_photo.jpg

[1789] Suggest recipes for the week based on the user's health and emotional state, analyze food photos to evaluate nutritional balance and provide improvement suggestions, purchase and deliver ingredients based on the suggested recipes, and use an emotion engine to adjust ingredient selection and delivery times.

[1790] In this way, it becomes possible to provide appropriate meal suggestions, as well as food purchasing and delivery, that take into account both the user's health and emotional state.

[1791] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1792] Step 1:

[1793] The server receives the results of the user's medical checkup and analyzes the data using an analysis engine.

[1794] Input: Medical checkup result data

[1795] Data processing: Analysis of medical examination data

[1796] Output: User's health status rating

[1797] Specific operation: The server receives the user's health checkup data and uses an analysis engine to analyze health indicators such as blood sugar level, blood pressure, and cholesterol level, thereby evaluating the user's health condition.

[1798] Step 2:

[1799] Based on the analysis results, the server generates health advice appropriate for the user and notifies the smartphone.

[1800] Input: Health status assessment results

[1801] Data processing: generating health advice

[1802] Output: Health advice notice

[1803] Specific operation: Based on the health status evaluation results, the server generates lifestyle and dietary advice and notifies the smartphone, including recommendations for exercise and restrictions on the intake of certain foods.

[1804] Step 3:

[1805] The server uses a generative AI model to suggest a week's worth of recipes based on the user's health condition.

[1806] Input: Health status assessment results

[1807] Data processing: Recipe generation

[1808] Output: Recipe suggestions for a week

[1809] How it works: The server uses a generative AI model to generate a week's worth of recipes suited to the user's health condition. For example, it suggests low-carb recipes to a user who is at risk of diabetes.

[1810] Step 4:

[1811] A user takes a photo of their meal with their smartphone and sends it to the server, where the image is analyzed using PIL.

[1812] Input: Food photo

[1813] Data processing: Image analysis

[1814] Output: Nutritional balance evaluation of meals

[1815] How it works: A user takes a photo of a meal with their smartphone and sends it to the server. The server uses PIL to analyze the image, identify the types and amounts of ingredients, and evaluate the nutritional balance.

[1816] Step 5:

[1817] The server provides suggestions for improving nutritional balance based on the analysis results.

[1818] Input: Nutritional balance assessment of meals

[1819] Data processing: generating improvement proposals

[1820] Output: Nutritional balance improvement proposal

[1821] Specific operation: The server provides the user with specific improvement suggestions based on the nutritional balance evaluation results of the meal, such as a suggestion to increase the intake of vegetables.

[1822] Step 6:

[1823] The server generates a list of ingredients needed based on the proposed recipe, and cooperates with existing ingredient purchasing and delivery services to purchase the ingredients and deliver them to the user's specified location.

[1824] Input: Suggested recipe

[1825] Data processing: Generating ingredient lists

[1826] Output: Food purchase and delivery instructions

[1827] Specific operation: The server generates a list of ingredients needed based on the proposed recipe, and cooperates with existing ingredient purchasing and delivery services to purchase the ingredients and deliver them to the user's specified location.

[1828] Step 7:

[1829] The server uses an analytical engine to evaluate the user's emotional state and optimizes ingredient selection and delivery times based on the results.

[1830] Input: The user's emotional state.

[1831] Data processing: Optimization based on emotional state

[1832] Output: Optimized ingredient selection and delivery times

[1833] Specific operation: The server evaluates the user's emotional state using an analysis engine, and optimizes the selection of ingredients and delivery time based on the results. For example, if the user is feeling sad, the server selects ingredients that will ease the emotion and delivers them at a time when the user is likely to feel happy.

[1834] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1835] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1836] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1837] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1838] [Third embodiment]

[1839] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1840] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1841] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1842] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1843] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1844] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1845] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1846] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1847] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1848] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1849] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1850] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1851] "Example 1"

[1852] The system of the present invention is equipped with an interface for importing the results of a medical checkup. This interface can import the results of a medical checkup as digital data. For example, if the results of a medical checkup are provided in PDF format, the interface has a function for reading the PDF and extracting it as text data.

[1853] "Example 2"

[1854] The system of the present invention has a function for analyzing the results of a medical checkup that has been imported. This analysis function analyzes various numerical data (blood pressure, blood glucose level, cholesterol level, etc.) contained in the results of the medical checkup to evaluate the user's health condition. For example, if the blood glucose level is high, it will determine that there is a risk of diabetes and provide advice on this.

[1855] "Example 3"

[1856] The system of the present invention has a function to suggest recipes for one week based on the analysis results. This recipe suggestion function selects and suggests recipes from a database that are suited to the user's health condition. For example, low-carbohydrate recipes are suggested to a user who is at risk of diabetes.

[1857] "Example 4"

[1858] The system of the present invention has a function to analyze photos of meals registered by users and propose improvements. This function identifies ingredients and dishes from the photos of meals and evaluates their nutritional balance. For example, for a user who eats a diet high in meat, a meal high in vegetables is suggested.

[1859] "Example 5"

[1860] The system of the present invention has a function to purchase and deliver ingredients based on a proposed recipe. This function links with existing ingredient purchasing and delivery services to purchase and deliver ingredients based on the ingredient list of the proposed recipe. For example, if the proposed recipe includes "chicken breast," the ingredient is sent to the purchasing and delivery service and delivered to the user's home.

[1861] The process flow of each embodiment will be described below.

[1862] "Example 1"

[1863] Step 1: The user uploads the results of a medical checkup in PDF format to the system. Step 2: The system reads the PDF and extracts it as text data.

[1864] "Example 2"

[1865] Step 1: The system analyzes various numerical data (blood pressure, blood sugar levels, cholesterol levels, etc.) from the text data extracted.

[1866] Step 2: Based on the analysis results, the user's health condition is evaluated and advice is provided.

[1867] Step 1: The system assesses the user's health status.

[1868] Step 2: Based on the evaluation results, select an appropriate recipe from the database and suggest it to the user.

[1869] "Example 4"

[1870] Step 1: A user uploads a photo of their meal into the system.

[1871] Step 2: The system identifies ingredients and dishes from the photo of the meal and evaluates its nutritional balance.

[1872] Step 3: Based on the evaluation results, suggest improvement ideas to the user.

[1873] "Example 5"

[1874] Step 1: Create an ingredients list for the recipe suggested by the system.

[1875] Step 2: Based on the ingredient list, connect with existing ingredient purchasing and delivery services to purchase and deliver the ingredients.

[1876] Example 1

[1877] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1878] Conventional health management systems required users to manually enter medical checkup results, which was time-consuming and labor-intensive. In addition, there was a lack of a way to efficiently analyze medical checkup results provided in PDF format and extract the necessary information. Furthermore, there was a need for a system that could provide health advice and recipe suggestions based on the extracted information.

[1879] The specification process by the specification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for reading the results of a medical checkup in PDF format and extracting it as text data, a means for formatting necessary information from the extracted text data and storing it in a database, a means for providing health advice based on the analysis results, a means for proposing recipes for one week based on the analysis results, a means for analyzing photos of meals registered by the user and proposing improvement plans, and a means for purchasing and delivering ingredients based on the proposed recipes. This enables the user to efficiently import and analyze the results of a medical checkup and manage his / her health.

[1880] "Results of a health checkup" refers to the results of detailed medical tests conducted as part of a health checkup, and are usually provided as digital data in PDF format or similar.

[1881] "Means of analysis" refers to software and algorithms used to read the results of health checkups provided as digital data, extract the necessary information, and format it.

[1882] "Means for providing health advice" refers to a system or function for evaluating a user's health condition based on the analyzed results of a medical checkup and providing appropriate lifestyle and medical advice.

[1883] "Means for suggesting a week's worth of recipes" refers to a system or function for creating a balanced meal plan and suggesting a week's worth of recipes based on the user's health condition.

[1884] "Means of analyzing food photos and suggesting improvement suggestions" refers to a system or function that uses image analysis technology to analyze food photos registered by users, evaluates nutritional balance, calories, etc., and suggests improvement suggestions.

[1885] "Means for purchasing and delivering ingredients" refers to a system or service for purchasing the necessary ingredients based on the proposed recipe and delivering them to a location specified by the user.

[1886] "Means of reading medical checkup results in PDF format and extracting them as text data" refers to software or algorithms for reading medical checkup results provided as a PDF file and extracting the contents as text data.

[1887] "Means for formatting the necessary information from the extracted text data and storing it in a database" refers to a system or function for analyzing the extracted text data, formatting the necessary information, and storing it in a database.

[1888] The present invention is a system for efficiently acquiring and analyzing the results of a medical checkup, and providing health advice and recipe suggestions to a user. A specific embodiment of this system will be described below.

[1889] Hardware and Software Used

[1890] The system consists of three main components: a server, a terminal, and a user. The server receives, analyzes, stores, and generates advice and recipe suggestions. The terminal is a device through which the user accesses the system and uploads data. The user is an individual who uses the system to manage their health.

[1891] The server uses the following software and libraries:

[1892] PDF analysis library: "PyMuPDF"

[1893] Database Management System: "SQLite"

[1894] Generative AI models: AI models for generating health advice and recipe suggestions

[1895] Program Processing

[1896] The server receives the results of medical checkups in PDF format uploaded by the user. The server uses the "PyMuPDF" library to open the PDF file and extract the text data. The extracted text data is formatted into the required information (e.g. blood test results, height, weight, etc.) using regular expressions. The formatted data is saved in the "SQLite" database.

[1897] Examples

[1898] As a concrete example, consider the case where a user uploads a file called "health checkup results 2023.pdf". The user accesses the system's web interface using a terminal browser, selects "health checkup results 2023.pdf" and uploads it. The server receives the uploaded PDF file and opens it using the "PyMuPDF" library. The server extracts the text data in the PDF file, extracts information such as blood test results and height and weight using regular expressions, and formats it. The formatted data is stored in the "SQLite" database.

[1899] Examples of prompt statements

[1900] Examples of prompt sentences to be input to a generative AI model include the following:

[1901] Prompt: I have uploaded "Health Check Res...

Claims

1. A means for analyzing numerical data included in a result of a medical checkup that is provided by a user and indicates the health condition of the user; A means for providing the user with health advice based on the results of the analysis, the advice being on the user's lifestyle habits, the user's diet, and the user's exercise, by inputting a specific prompt sentence into a generative AI model; A means for suggesting to the user a week's worth of recipes that are customized according to the health condition of the user based on the results of the analysis; A means for analyzing a photo of a meal registered by the user and proposing a meal improvement plan to the user; an emotion engine that recognizes the emotion of the user; an optimization means for optimizing the health advice and the recipe based on the emotion using a generative AI model; A system including:

2. The system of claim 1 , further comprising means for providing real-time health advice and product suggestions through the smart glasses.

3. means for generating, using a generative AI model, suggestions for improving the diet based on the emotions; means for adjusting a selection of ingredients and a delivery time of the ingredients based on the emotion; The system of claim 1 , comprising:

Images