System

Abstract

A system is provided.SOLUTION: A system, comprising: means for inputting medical examination data of a user; means for transmitting the input medical examination data to a server; means for storing the medical examination data received by the server; analysis means including deep learning technology and multi-modal AI technology for analyzing the stored medical examination data; and means for providing an individual health risk and a solution thereof to the user based on an analysis result generated by the analysis means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, the increase in health checkups in medical settings has required physicians to visually review large volumes of test results and images in a short period of time and make comprehensive judgments. As a result, the risk of oversights and misdiagnosis has increased. This issue can have a significant impact, especially for diseases where early detection is important. Conventional systems have limited ability to integrate and analyze multiple data sources, making it difficult to provide users with comprehensive and accurate health risk assessments and countermeasures. [Means for solving the problem]

[0005] To solve this problem, the present invention provides the following system. It includes a means for inputting a user's health checkup data and a means for transmitting the input data to a server. The server has a means for storing the received data and an analysis means for analyzing the stored data, including deep learning and multimodal AI technologies. The analysis means detects abnormal patterns that the user may easily overlook, particularly by analyzing the correlation between blood test values ​​and image data. It also has a means for providing the user with individual health risks and solutions based on the generated analysis results. The analysis results are shared with a specialist, and a final report including the specialist's opinion is generated, reducing the risk of oversights and misdiagnosis and enabling the provision of a more detailed and objective health risk assessment.

[0006] "User" refers to an individual or multiple people who use the system, and is the entity that inputs health checkup data and checks analysis results.

[0007] "Health checkup data" refers to various data that indicate the user's health status, such as blood test results from a health checkup the user has undergone, and image data (e.g., X-rays and CT scan images).

[0008] "Input means" refers to the devices or software that users use to input health checkup data into the system, including dedicated applications and interfaces.

[0009] "Transmission means" refers to a communication means for transferring the input health checkup data to the server, including the Internet and dedicated data communication protocols.

[0010] "Server" refers to the core computer system used to store and analyze received data.

[0011] "Storage means" refers to a function or database for safely and effectively storing received health checkup data.

[0012] "Analysis means" refers to programs and algorithms for analyzing stored health checkup data using deep learning technology and multimodal AI technology.

[0013] "Deep learning technology" is a technology that uses large-scale neural networks to automatically learn the characteristics and patterns of data and generate highly accurate analysis results.

[0014] "Multimodal AI" is an artificial intelligence technology that performs more comprehensive understanding and judgment through the integrated analysis of multiple different data sources (e.g., text data, image data).

[0015] "Analysis results" refers to the results of health risk assessment and abnormality detection based on health checkup data generated by the analysis means.

[0016] "Health risks" refer to the possibility of potential problems or abnormalities in the user's health, including diseases or symptoms for which early detection is important.

[0017] "Solutions" refer to recommended countermeasures, treatments, and lifestyle improvements for detected health risks.

[0018] A "specialist" is a doctor with expertise and experience in a particular medical field, and is responsible for reviewing and providing additional opinions based on the analysis results.

[0019] "Final Report" refers to a detailed report that integrates the analysis results and expert medical opinions, including a health risk assessment and countermeasures provided to users. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] The present invention is a system that allows users to input health checkup data and receive analysis results. Specific processing details of each of the users, terminals, and servers will be explained below.

[0042] User interaction

[0043] 1. Enter the data:

[0044] After the diagnosis, the user launches a dedicated application on their device (smartphone or PC) and logs in.

[0045] Follow the app's instructions to take a photo of your health check results and enter data such as blood test results.

[0046] 2. Data transmission:

[0047] The entered health checkup data is sent from the terminal to the server using a secure communication method (e.g. HTTPS) for use over the Internet.

[0048] Server processing

[0049] 1. Data Receipt and Storage:

[0050] The server receives the health checkup data sent from the terminal and first stores the data in a database.

[0051] The format of the saved data is checked for consistency, and the data format is converted if necessary.

[0052] 2. Data Analysis:

[0053] The data stored in the database is analyzed using deep learning algorithms and multimodal AI. Deep learning technology uses large-scale neural networks to learn features and patterns that appear in blood test values ​​and image data, and detect abnormalities.

[0054] Multimodal AI technology analyzes correlations between blood test values ​​and image data to draw comprehensive conclusions from multiple data sources.

[0055] 3. Results generation and notification:

[0056] Based on the analysis results, a detailed diagnostic report is generated for the user, including details of any abnormalities detected, a health risk assessment, and recommended actions (e.g., retesting or consultation with a specialist).

[0057] The server notifies the terminal that the analysis results are ready.

[0058] 4. Collaboration with specialists:

[0059] If necessary, the analysis results are sent to a specialist for further comment, and a final report incorporating the specialist's feedback is generated.

[0060] The final report will be notified to the user again and will be available to view through the app.

[0061] Specific examples

[0062] As a specific example, consider the case where User A, a 45-year-old male, undergoes a health checkup. User A uses a dedicated smartphone app to take blood test results and chest X-ray images obtained after the checkup and enter them into the app. The app then sends this data to the server.

[0063] The server stores the data and begins analyzing it using a deep learning system. For example, if a blood test shows high cholesterol and an X-ray shows mild abnormalities, the AI ​​can connect these features to detect a risk of arteriosclerosis.

[0064] A detailed report containing this risk assessment is generated and sent from the server to User A's smartphone. User A can then review the report through the app and take necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist).

[0065] Furthermore, if a serious abnormality is detected, the server sends the analysis results to a specialist for further opinion. A final report including the specialist's opinion is generated and notified to User A again. In this way, the present invention significantly reduces the risk of oversight or misdiagnosis and provides useful health management information to users.

[0066] The processing flow will be explained below.

[0067] Step 1:

[0068] After the user has undergone a health checkup, they launch the dedicated app on their device (smartphone or PC) and log in. They then follow the instructions in the app to begin entering their health checkup results (e.g., blood test values, image data). If necessary, they can take photos of the results or enter the values ​​manually.

[0069] Step 2:

[0070] The device sends the entered health checkup data to the server. A secure communication protocol (e.g., HTTPS) is used over the Internet to ensure data confidentiality.

[0071] Step 3:

[0072] The server receives the health checkup data sent from the device. The received data is stored in a database and a data consistency check is performed. If there is a problem with the data format, it is converted into an appropriate format.

[0073] Step 4:

[0074] The server prepares the stored data to be sent to the analysis engine, where preprocessing such as resizing and noise removal is performed on the image data to improve the accuracy of the analysis.

[0075] Step 5:

[0076] The server launches an analysis engine (deep learning model and multimodal AI), which analyzes correlations between blood test values ​​and image data to detect abnormal patterns and health risks.

[0077] Step 6:

[0078] The server generates a detailed diagnostic report based on the analysis results, including any abnormalities detected, a health risk assessment, and recommended actions (e.g., retesting or consultation with a specialist).

[0079] Step 7:

[0080] The server notifies the user that the analysis results are ready. A push notification is sent to the device with the message "Analysis results are ready. Please check the app."

[0081] Step 8:

[0082] The user checks the notification on their device and launches the dedicated app to view the analysis results. The app displays a detailed diagnostic report, allowing the user to learn specific measures to address health risks.

[0083] Step 9:

[0084] If necessary, the server sends the analysis results to a specialist, who reviews them and provides additional opinions and advice.

[0085] Step 10:

[0086] The server generates a final report including the expert's opinion. The server notifies the user again that the final report is ready, and the user can view the final report through the app.

[0087] Step 11:

[0088] The server provides healthcare support functions, allowing users to receive in-app reminders and advice on daily health management (e.g., drinking more water, improving exercise habits).

[0089] Step 12:

[0090] Users manage their health based on the advice provided, and use the app's support features to implement healthy lifestyle habits and regularly monitor their health.

[0091] Example 1

[0092] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0093] Conventional health checkup data management and analysis systems often make it difficult for users to efficiently input and transmit health checkup data and quickly receive analysis results. Furthermore, due to insufficient accuracy in data analysis and insufficient collaboration with specialists, it is difficult to provide users with reliable health management information. For these reasons, there is a growing need for a system that can detect health risks early and suggest appropriate countermeasures.

[0094] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0095] In this invention, the server includes a means for a user to input health checkup data, a means for transmitting the input health checkup data from a terminal to the server, and a means for storing the health checkup data received by the server and checking the format consistency. This enables efficient data input, transmission, and consistency check. The server also includes a means for analyzing the stored data using deep learning and multimodal AI technologies, a means for generating a detailed diagnostic report for the user based on the analysis results generated by the analysis means and notifying the user via push notification or email, a means for sharing the analysis results with a specialist as needed and incorporating the specialist's opinion into the final report, and a means for notifying the user of the final report and making it viewable via an application. This allows the user to receive analysis results quickly and comprehensively, enabling them to obtain reliable diagnostic information that reflects the specialist's opinion.

[0096] "User" means an individual who uses the system to input health checkup data and receive analysis results.

[0097] "Health checkup data" refers to health-related information such as blood test results and image data entered by the user.

[0098] "Input means" refers to the method or device by which users input their health checkup data into the system. Specifically, this applies to dedicated applications on smartphones or PCs.

[0099] A "terminal" is a device that a user uses to access the system, such as a smartphone or PC.

[0100] A "server" is the central computer device of the entire system that receives, stores, and analyzes data sent from terminals.

[0101] "Transmission means" refers to a method or device for transmitting the entered medical examination data from the terminal to the server. Specifically, this refers to a secure communication method via the Internet (e.g., HTTPS).

[0102] The "receiving means" refers to a method or device by which the server receives the medical examination data transmitted from the terminal.

[0103] The "storage means" refers to a method or device for storing the received medical examination data in the server. Specifically, this corresponds to a database.

[0104] "Consistency check means" refers to a method or device for verifying whether the received data conforms to a specified format, whether required fields have been entered, and so on.

[0105] "Deep learning technology" is a technique that uses large-scale neural networks to analyze data.

[0106] "Multimodal AI technology" refers to technology that combines and analyzes different types of data (e.g., text data and image data) to draw comprehensive conclusions.

[0107] "Analysis means" refers to a method or device for analyzing health checkup data using deep learning technology and multimodal AI technology.

[0108] A "diagnostic report" is a report generated based on the analysis results, which includes detected abnormalities, an assessment of health risks, and recommended actions to take.

[0109] "Notification means" refers to a method or device for notifying the user of the completion of the analysis results preparation or other information from the server. Specifically, this applies to push notifications and emails.

[0110] "Expert" means a medical professional with medical expertise who provides additional input on the results of the analysis.

[0111] "Sharing means" refers to a method or device for sharing analysis results with specialists.

[0112] "Final report" refers to the final diagnostic report that reflects the opinion of the specialist.

[0113] An "application" is software that allows users to input data and view analysis results.

[0114] This invention is a system that allows users to input health checkup data and receive analysis results. This system works in cooperation with three entities: the user, the terminal, and the server, to detect health risks early and propose appropriate countermeasures.

[0115] User interaction

[0116] Users launch the dedicated application on their smartphone or PC and log in. They follow the app's instructions to take pictures of their health checkup results and enter data such as blood test values. The dedicated application then imports images from the camera roll and provides an interface for manually entering numerical data.

[0117] Sending data by the device

[0118] The device formats the input data and sends it to the server using a secure communication method (e.g., HTTPS). Specifically, the captured image and input numerical data are combined into JSON format and sent as an HTTP POST request. This ensures reliable data transmission.

[0119] Receiving and storing data by the server

[0120] The server receives the health checkup data sent from the terminal. The received data is first checked for format consistency before being saved in the database. Specifically, it verifies whether the data conforms to the specified format and whether required fields have been entered. Data that passes this verification is saved in the database, and if there is any inconsistency, an error message is returned to the user.

[0121] Data analysis by the server

[0122] The stored data is analyzed using deep learning algorithms and multimodal AI technology. Deep learning technology uses large-scale neural networks to learn features and patterns that appear in blood test values ​​and image data and detect abnormalities. Multimodal AI technology combines and analyzes different data sources (blood test values ​​and image data) to provide a comprehensive health assessment.

[0123] Generation and notification of analysis results

[0124] Based on the analysis results, a detailed diagnostic report is generated for the user, including any detected abnormalities, an assessment of health risks, and recommended actions (e.g., retesting or consulting a specialist). Once the analysis results are ready, the server notifies the user via push notification or email.

[0125] Collaboration with specialists

[0126] If any significant abnormalities are detected, the server sends the analysis results to a specialist for further opinion. A final report incorporating the specialist's opinion is generated and notified to the user. The user can view this final report through the application.

[0127] Specific examples

[0128] Consider the case where User A, a 45-year-old male, undergoes a health checkup. User A uses a dedicated smartphone app to take blood test results and chest X-ray images obtained after the checkup and enter them into the app. The app then sends this data to the server.

[0129] The server stores the data and begins analyzing it using a deep learning system. For example, if a blood test shows high cholesterol and an X-ray shows mild abnormalities, the AI ​​can connect these features to detect a risk of arteriosclerosis.

[0130] A detailed report containing this risk assessment is generated and sent from the server to User A's smartphone. User A can then review the report through the app and take necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist).

[0131] Furthermore, if a serious abnormality is detected, the server sends the analysis results to a specialist for further opinion. A final report including the specialist's opinion is generated and notified to User A again. In this way, the present invention significantly reduces the risk of oversight or misdiagnosis and provides useful health management information to users.

[0132] Prompt Sentence Examples

[0133] "Please create a specific scenario in which a 45-year-old male user enters the results of a health check into a smartphone app and receives the results of an analysis by AI. The analysis will use deep learning technology to generate a detailed report including a risk assessment based on high cholesterol and abnormal chest X-ray findings."

[0134] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0135] Step 1:

[0136] The user launches the dedicated application on their smartphone or PC and logs in.

[0137] Input: User account information (e.g., user ID, password)

[0138] Output: User's dashboard on successful login, error message on failure

[0139] The user launches the dedicated application and enters their account information to log in. If the login is successful, the user's dashboard will be displayed and they can proceed to the next step.

[0140] Step 2:

[0141] Users take pictures of their health checkup results and enter data such as blood test results into a dedicated app.

[0142] Input: Images of health checkup results, blood test results, and other data

[0143] Output: Formatted health check data (e.g., JSON format)

[0144] The user follows the app's instructions to take a photo of the health checkup results and enter data such as blood test values, which are then formatted into the appropriate format (e.g., JSON) within the app.

[0145] Step 3:

[0146] The terminal transmits the input health checkup data to the server.

[0147] Input: Formatted health check data (e.g., JSON format)

[0148] Output: Data transmission result to the server (success or failure message)

[0149] The device sends the formatted health check data to the server using HTTPS, sending the data as an HTTP POST request and receiving a success or failure result.

[0150] Step 4:

[0151] The server receives the health checkup data sent from the terminal and stores it in a database.

[0152] Input: Health checkup data sent from the device

[0153] Output: Save result to database (success or failure message)

[0154] The server receives the data sent from the terminal and first checks the format for consistency. If this check is successful, it stores the data in the database. If there is a mismatch, it returns an error message to the terminal.

[0155] Step 5:

[0156] The server analyzes the stored health checkup data using deep learning algorithms and multimodal AI technology.

[0157] Input: Medical examination data stored in a database

[0158] Output: Analysis results (abnormal locations, risk assessment, etc.)

[0159] The server applies deep learning technology to the stored data to detect abnormalities in blood test values ​​and image data, and uses multimodal AI technology to analyze correlations between different data sources to perform a comprehensive health risk assessment.

[0160] Step 6:

[0161] Based on the analysis results, a detailed diagnostic report is generated and notified to the user.

[0162] Input: Analysis results

[0163] Output: Diagnostic report, notification (push notification or email)

[0164] The server generates a diagnostic report based on the analysis results, which includes detected abnormalities, a health risk assessment, and recommended actions to take. Based on the report, the server sends a notification to the user's device.

[0165] Step 7:

[0166] If necessary, the analysis results will be sent to a specialist for additional opinion.

[0167] Input: Analysis results

[0168] Output: Expert feedback, final report

[0169] If any significant abnormalities are detected, the server sends the analysis results to a specialist for further opinion, generates a final report incorporating the specialist's opinion, and notifies the user again.

[0170] Step 8:

[0171] The user reviews the final report and takes any recommended actions.

[0172] Input: Final Report

[0173] Output: User behavior change (e.g., diet change, increased exercise, specialist consultation)

[0174] Users receive a notification and can launch the app to view the final report, which can then be used to take specific measures such as improving their diet, increasing exercise, or consulting a specialist.

[0175] (Application example 1)

[0176] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0177] Health management has become an extremely important issue in modern life. However, due to the lack of individual health risk assessments based on health checkup results and the lack of solutions, it is difficult for users to respond appropriately. Furthermore, it is difficult to recommend and deliver dietary recommendations based on health risks with current technology.

[0178] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0179] In this invention, the server includes a means for inputting a user's health checkup data, a means for transmitting the input health checkup data to the server, and a means for storing the health checkup data received by the server. This enables an analysis means including deep learning technology and multimodal AI technology for analyzing the stored health checkup data. The system also includes a means for providing the user with individual health risks and solutions based on the analysis results generated by the analysis means, and a means for generating dietary recommendations based on the health data and transmitting them to a delivery device. This enables health risk assessment and customized dietary suggestions and delivery for the user, providing more comprehensive health management.

[0180] "User" refers to a person who inputs health checkup data and receives analysis results.

[0181] "Health checkup data" refers to information such as blood test results and image data obtained by a user as a result of a health checkup.

[0182] "Input means" refers to a method or device that allows a user to input health checkup data into the system.

[0183] "Server" refers to the computer system that receives, stores, and analyzes health checkup data sent by users.

[0184] The "storing means" refers to a method or device for storing the received medical examination data in a storage device such as a database.

[0185] "Analysis means" refers to a method or device for analyzing health checkup data and conducting health risk assessments using deep learning technology and multimodal AI technology.

[0186] "Deep learning technology" is an advanced machine learning technology that uses artificial neural networks to learn the characteristics and patterns of data.

[0187] "Multimodal AI technology" refers to artificial intelligence technology that integrates and analyzes multiple types of data (e.g., blood test values ​​and image data).

[0188] "Means based on the analysis results" refers to a method or device that utilizes the analysis results obtained by the analysis means to provide information or suggestions to the user.

[0189] "Dietary recommendations based on health data" means proposing the optimal meal menu for each individual based on the user's health checkup data.

[0190] A "delivery device" is a robot or machine that delivers meals to users.

[0191] The present invention is a system for assessing individual health risks based on health checkup data, proposing healthy meals to users based on the results, and providing the meals via a delivery device. Specific embodiments of the system are described below.

[0192] Hardware and Software Configuration

[0193] 1. User Device

[0194] Hardware: Smartphone, tablet, or personal computer

[0195] Software: Dedicated application, camera, internet connection

[0196] 2. Server

[0197] Hardware: High-performance server computer

[0198] Software: Database management systems, deep learning algorithms (e.g., Keras), multimodal AI technology

[0199] 3. Delivery Device

[0200] Hardware: Food delivery robot

[0201] Software: Route optimization algorithm, communication module

[0202] System processing flow

[0203] User operations

[0204] 1. Enter the data:

[0205] After undergoing a health checkup, the user launches a dedicated application on their device and logs in. Following the instructions, they take an image of the health checkup results and enter blood test results and other diagnostic data.

[0206] 2. Data transmission:

[0207] The entered health checkup data is sent from the terminal to a server via the Internet using a secure protocol (e.g., HTTPS).

[0208] Server Processing

[0209] 1. Data Receipt and Storage:

[0210] The server receives the health checkup data sent from the terminal, first checks whether the data is in the correct format, and then stores it in a database.

[0211] 2. Data Analysis:

[0212] The stored data is analyzed using deep learning and multimodal AI technologies. Deep learning models (e.g., using Keras) are used to learn features that appear in blood test values ​​and image data and detect abnormalities. Multimodal AI technology integrates information from multiple data sources (e.g., blood test values ​​and image data) to perform more accurate analysis.

[0213] 3. Results generation and notification:

[0214] Based on the analysis results, a detailed report is automatically generated, including a health risk assessment, any abnormalities detected, and recommended actions (e.g., dietary changes, exercise recommendations). Based on this report, optimal meal menu recommendations are also generated.

[0215] 4. Send to delivery device:

[0216] A meal menu personalized for the user is sent to the delivery device, which then delivers the meal to the user's location via the optimal route.

[0217] Specific examples

[0218] example:

[0219] User A, a 45-year-old man, undergoes a health check and enters the results into a smartphone app. When his blood test results and chest X-ray image are sent to the server, deep learning technology begins analysis. If the blood test results indicate high cholesterol and the X-ray image shows abnormalities, the AI ​​will link these features and detect a risk of arteriosclerosis.

[0220] A detailed report including this risk assessment is generated, and recommended meals such as "low-fat meals" and "high-fiber meals" are suggested. The server sends this information to a food delivery robot, which then delivers healthy meals to User A via the optimal route. The process can be initiated with a prompt that reads, "Your blood test results indicate high cholesterol, and your X-rays have revealed abnormalities."

[0221] This system allows users to easily manage their health at home, offering customized meal suggestions and prompt delivery, which helps users maintain their health and reduce medical costs.

[0222] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0223] Step 1:

[0224] After the health check, the user launches a dedicated application and logs in. The user takes an image of the health check results and inputs data such as blood test values. The application receives this input data. The input data is received in the form of image data and numerical data. The user's device formats this data appropriately and generates a prompt to proceed to the next step.

[0225] Step 2:

[0226] The terminal sends the entered health checkup data to the server. This communication uses HTTPS to protect the confidentiality of the data. The data sent includes numerical data of blood test values ​​and health checkup images. The terminal notifies the user that the data has been sent successfully.

[0227] Step 3:

[0228] The server receives the health checkup data sent from the terminal. The received data is first stored in a database. A consistency check is performed on the stored data, and the data format is converted as necessary. After the data format conversion on the server side, the data is in a format that can be processed by the analysis system.

[0229] Step 4:

[0230] The server analyzes the stored health checkup data. Deep learning technology (e.g., Keras) is used to learn the characteristics and patterns of blood test values ​​and image data and detect abnormalities. Multimodal AI technology is used to analyze the correlation between blood test values ​​and image data and perform a comprehensive health risk assessment. The input data is passed to an analysis model, which generates analysis results.

[0231] Step 5:

[0232] Based on the analysis results, the server generates a detailed health risk assessment report for the user, which includes details of the detected abnormalities, a health risk assessment, and recommended measures (e.g., dietary changes and exercise recommendations).The analysis results also simultaneously generate recommendations for optimal meal menus based on the health data.

[0233] Step 6:

[0234] The server sends the generated report and meal recommendations to the delivery device using an appropriate communication protocol (e.g., MQTT, HTTP). The delivery device uses the received information to prepare meals optimal for the user's health condition and schedules the delivery. After the data is sent, the server confirms that the delivery was successfully initiated and notifies the user.

[0235] Step 7:

[0236] The delivery device delivers the meal to the user via the optimal route based on the received meal menu. The delivery device uses GPS and route optimization algorithms to reach the user efficiently. After delivery, the user can confirm that the delivery has been completed through the application and provide any necessary feedback to help improve the accuracy of the entire system.

[0237] The above is the specific processing flow of the system that realizes this application example, and clarifies what operations are performed at each step and what output data is generated from what input data.

[0238] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0239] This invention is a system that allows users to input and submit health checkup data, receive analysis results based on that data, and also combines it with an emotion engine that recognizes the user's emotional state. This system makes it possible to provide more personalized health risk assessments and solutions. The specific processing details for each of the users, terminals, and servers are explained below.

[0240] User interaction

[0241] 1. Enter the data:

[0242] After the diagnosis, the user launches a dedicated application on their device (smartphone or PC) and logs in.

[0243] Follow the app's instructions to enter your health checkup results (e.g., blood test results, image data). If necessary, take a photo of the results or enter the values ​​manually.

[0244] It also enables voice input and facial expression recognition functions to capture data on the user's voice and facial expressions.

[0245] 2. Data transmission:

[0246] The entered health checkup data and emotion data are sent from the device to a server using a secure communication protocol (e.g., SSL / TLS) over the Internet to ensure data confidentiality.

[0247] Server processing

[0248] 1. Data Receipt and Storage:

[0249] The server receives the health check data and emotion data sent from the device. The received data is stored in a database and a data integrity check is performed. If there is a problem with the data format, it is converted into an appropriate format.

[0250] 2. Data Analysis:

[0251] The data stored on the server is sent to the analysis engine, where preprocessing such as resizing and noise removal is performed on the image data to improve the accuracy of the analysis.

[0252] Using deep learning models and multimodal AI technology, the system analyzes correlations between blood test values ​​and image data to detect abnormal patterns and health risks.

[0253] 3. Emotional Data Analysis:

[0254] The server uses an emotion engine to analyze the user's voice and facial expression data, taking into account the user's recognized emotional state (e.g., stress, anxiety, relief) and adjusts the health risk assessment accordingly.

[0255] 4. Results generation and notification:

[0256] Based on the analysis results, a detailed diagnostic report is generated for the user, including detected abnormalities, a health risk assessment, and recommended actions (e.g., retesting or seeing a specialist) based on the user's emotional state.

[0257] The server notifies the user when the analysis results are ready.

[0258] 5. Collaboration with specialists:

[0259] If necessary, the analysis results are sent to a specialist for further comment, and a final report incorporating the specialist's feedback is generated.

[0260] The final report will be notified to the user again and made available for viewing through the app.

[0261] Specific examples

[0262] As a specific example, consider the case of User A, a 45-year-old male, after undergoing a health checkup. User A uses a dedicated smartphone app to take blood test results and chest X-ray images obtained after the checkup and enter them into the app. User A also has the app recognize voice comments and facial expressions to record his emotions regarding the diagnosis results (e.g., anxiety, stress).

[0263] The server receives this data and begins analyzing it using a deep learning system. For example, if a blood test shows high cholesterol and an X-ray shows mild abnormalities, the AI ​​will connect these features and detect a risk of arteriosclerosis. At the same time, the emotion engine will recognize that User A is experiencing high levels of stress.

[0264] A detailed report including this risk assessment and emotional state is generated and sent from the server to User A's smartphone. User A can review this report through the app and determine necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist). In addition, if any serious abnormalities are detected, the analysis results are sent to a specialist, and a final report incorporating the specialist's opinion is generated and notified to User A again.

[0265] The present invention provides a more accurate and personalized health risk assessment by taking into account the user's emotional state, significantly reducing the risk of oversight or misdiagnosis, and providing comprehensive health management information that allows users to take measures with greater peace of mind.

[0266] The processing flow will be explained below.

[0267] Step 1:

[0268] After the user has undergone a health checkup, they launch a dedicated app on their device (smartphone or PC) and log in. They follow the instructions within the app and begin inputting their health checkup results (e.g., blood test values, image data). If necessary, they can take a photo of the results or manually enter the values. They can also turn on voice input and facial expression recognition functions to capture data on the user's voice and facial expressions.

[0269] Step 2:

[0270] The device sends the entered health check data and emotion data to the server. The data is transmitted via the Internet using a secure communication protocol (e.g., HTTPS) to ensure confidentiality.

[0271] Step 3:

[0272] The server receives the health check data and emotion data sent from the device. The received data is stored in a database and a data integrity check is performed. If there is a problem with the data format, it is converted into an appropriate format.

[0273] Step 4:

[0274] The server prepares the stored data to be sent to the analysis engine, where preprocessing such as resizing and noise removal is performed on the image data to improve the accuracy of the analysis.

[0275] Step 5:

[0276] The server launches an analysis engine (deep learning model and multimodal AI), which analyzes correlations between blood test values ​​and image data to detect abnormal patterns and health risks.

[0277] Step 6:

[0278] The server analyzes the user's voice data and facial expression data using an emotion engine. The emotion engine recognizes the user's emotional state (e.g., stress, anxiety, relief) and reflects it in the analysis results.

[0279] Step 7:

[0280] The server generates a detailed diagnostic report based on the analysis results and emotional state, including any abnormalities detected, a health risk assessment, and recommended actions (e.g., retesting or consultation with a specialist).

[0281] Step 8:

[0282] The server notifies the user that the analysis results are ready. A push notification is sent to the device with the message "Analysis results are ready. Please check the app."

[0283] Step 9:

[0284] The user checks the notification on their device and launches the dedicated app to view the analysis results. The app displays a detailed diagnostic report, allowing the user to learn specific measures to address health risks.

[0285] Step 10:

[0286] If necessary, the server sends the analysis results to a specialist, who reviews them and provides additional opinions and advice.

[0287] Step 11:

[0288] The server generates a final report including the expert's opinion. The server notifies the user again that the final report is ready, and the user can view the final report through the app.

[0289] Step 12:

[0290] The server provides healthcare support functions, allowing users to receive in-app reminders and advice on daily health management (e.g., drinking more water, improving exercise habits).

[0291] Step 13:

[0292] Users manage their health based on the advice provided, and use the app's support features to implement healthy lifestyle habits and regularly monitor their health.

[0293] Example 2

[0294] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0295] Conventional health checkup data analysis systems provide analysis results without considering the user's emotional state, which means they are unable to reflect the user's psychological factors.In addition, it is difficult to provide users with more reliable information because it is difficult to perform detailed analysis of diagnosis results or create reports that incorporate the opinions of specialists.

[0296] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0297] In this invention, the server includes means for inputting a user's health checkup data and emotional data, means for transmitting the input health checkup data and emotional data to the server, means for storing and preprocessing the health checkup data and emotional data received by the server and analyzing them using deep learning and multimodal AI technologies, means for providing individual health risks and solutions that take the user's emotional state into consideration based on the analysis results generated by the analysis means and means for notifying the user of the analysis results, and means for transmitting the analysis results to a specialist, if necessary, and generating and notifying the specialist of a final report including the specialist's opinion. This makes it possible to provide more accurate and individualized health risk assessments and solutions that take the user's emotional state into consideration.

[0298] A "user" is an individual who uses the system to input medical and emotional data.

[0299] "Health checkup data" refers to medical data used to evaluate a user's health condition, such as blood test results and image data.

[0300] "Emotional data" is data obtained through a user's voice and facial expressions that indicates the user's emotional state (e.g., stress, anxiety, relief).

[0301] A "server" is a computer system that receives, stores, pre-processes, analyzes, and notifies users of data submitted by the server.

[0302] "Preprocessing" refers to processing for improving the quality of received data, and includes, for example, noise removal and resizing of image data.

[0303] "Deep learning technology" is an advanced machine learning technology that uses neural networks to analyze data and recognize patterns.

[0304] "Multimodal AI technology" is an artificial intelligence technology that integrates and analyzes multiple data formats (e.g., numerical data, image data, audio data).

[0305] "Analysis means" refers to a means for analyzing stored health checkup data and emotion data using deep learning technology and multimodal AI technology.

[0306] "Analysis results" are results generated by the analysis means that indicate the user's health risks and abnormality patterns.

[0307] "Individual Health Risk" is a risk assessment based on a user's particular health condition and indicates a health problem specific to the user.

[0308] "Solutions" are methods or measures for reducing or improving health risks that are provided to users based on the analysis results.

[0309] A "specialist" is a medical professional with advanced knowledge and experience in the medical field, and is responsible for providing expert opinions on the analysis results.

[0310] "Final Report" means a detailed medical examination report provided to the user, including analysis results and expert opinions.

[0311] "Notification" is the act of informing the user that the analysis results or final report are ready.

[0312] This invention is a system that provides comprehensive health risk assessment and solutions by allowing users to input health checkup data and emotional data and send them to a server.The system uses deep learning and multimodal AI technologies to analyze the health checkup data and emotional data and provide individual countermeasures based on the analysis results.

[0313] User interaction

[0314] Users launch a dedicated application on their smartphone or computer and log in. Next, they follow the app's instructions to enter their health checkup results. This input method includes manually entering numerical data for blood test results and uploading image data of the checkup results. Voice input and facial expression recognition functions are also enabled to capture the user's emotional state.

[0315] Sending data

[0316] When the user presses the "Send" button, the entered health checkup data and emotion data are sent from the device to the server. This transmission is securely transmitted using the SSL / TLS protocol.

[0317] Server processing

[0318] The server stores the received data in a database and checks its integrity. If the data has defects, it is converted to a standard format. Next, the image data is preprocessed, such as noise removal and resizing. Libraries such as OpenCV can be used for this.

[0319] Once the data preprocessing is complete, the server analyzes the data using deep learning techniques (e.g., TensorFlow, PyTorch) and multimodal AI techniques. Specifically, it analyzes correlations between blood test values ​​and image data to detect abnormal patterns. It also evaluates the user's emotional state (e.g., stress, anxiety, relief) through analysis of voice and facial expression data.

[0320] Analysis results and notifications

[0321] Once the analysis results are generated, a detailed diagnostic report is created for the user, including any detected health risks, details of any abnormalities, and recommended actions (e.g., retesting or seeing a specialist) based on the user's emotional state. The server then notifies the user of the diagnostic report, typically via push notification or email.

[0322] Collaboration with specialists

[0323] If necessary, the server sends the analysis results to a specialist for further comment. A final report incorporating the specialist's feedback is generated and sent back to the user, allowing them to receive a more detailed and reliable diagnosis and treatment plan.

[0324] Specific examples

[0325] As a concrete example, consider the case where a 45-year-old male user undergoes a health checkup. Using a dedicated app, the user takes a blood test result and a chest X-ray image and inputs them into the app. The user also comments via voice on his / her feelings about the diagnosis (e.g., anxiety, stress), and facial expression data is also captured. The server receives this data and analyzes it using deep learning and multimodal AI technologies.

[0326] Through analysis, for example, high cholesterol blood test results and minor abnormalities in a chest X-ray can be detected as indicators of atherosclerosis risk. At the same time, high stress levels can be identified from emotional data. A detailed report that takes into account the analysis results and emotional state is provided to the user, suggesting necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist). If serious abnormalities are found, the analysis results are sent to a specialist, and the user is notified of a final report including the specialist's opinion.

[0327] Prompt Sentence Examples

[0328] "Analyze the blood test results and chest X-ray images of a 45-year-old man after he undergoes a health checkup to assess his risk of arteriosclerosis. Also, take into account the user's emotional state (e.g., anxiety, stress) in response to the diagnosis, and suggest appropriate countermeasures."

[0329] The system will enable accurate and personalized health risk assessment and solutions that also take into account the user's emotional state.

[0330] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0331] Step 1: Entering User Data

[0332] How it works: The user launches a dedicated application on their smartphone or computer and logs in.

[0333] Input: Health checkup results (e.g., blood test numerical data, image data), voice data, facial expression data

[0334] Specific operation: The app displays a health check result input form. The user enters numerical data into the form and takes and uploads image data using the camera. The voice input and facial expression recognition functions are enabled, and the app captures voice comments and facial expression data in real time.

[0335] Output: Health check result data and emotion data are input into the app.

[0336] Step 2: Sending user data

[0337] Action: The user presses the "Submit" button, which sends the entered data to the server.

[0338] Input: Health check result data and emotion data

[0339] How it works: The app uses the SSL / TLS protocol to send health checkup data and emotion data to the server via secure communication.

[0340] Output: Health and emotion data are transmitted securely.

[0341] Step 3: Receiving and storing data on the server

[0342] How it works: The server receives the data and stores it in a database.

[0343] Input: Health check data and emotion data

[0344] What happens: The server checks the integrity of the data and converts any abnormal formatting into a standard format, for example, standardizing date and number formats.

[0345] Output: The integrity-checked health and emotion data are stored in a database.

[0346] Step 4: Data Preprocessing on the Server

[0347] Action: The server preprocesses the received data.

[0348] Input: Medical examination data (especially image data)

[0349] Specific operation: Uses the OpenCV library to remove noise and resize image data. Specifically, it removes noise using a Gaussian filter and unifies the image size.

[0350] Output: Preprocessed image data is generated.

[0351] Step 5: Server Data Analysis

[0352] How it works: The server uses deep learning and multimodal AI techniques to analyze health checkup data.

[0353] Input: Preprocessed health checkup data (blood test results, processed image data), emotion data

[0354] How it works: Using TensorFlow and PyTorch, the system analyzes the correlation between blood test values ​​and image data. For example, if blood test values ​​indicate high cholesterol and the image shows an abnormality, the system can link these to detect the risk of arteriosclerosis.

[0355] Output: Health risk assessment results and abnormal patterns are generated.

[0356] Step 6: Emotional Data Analysis on the Server

[0357] How it works: The server uses the emotion engine to analyze the emotion data.

[0358] Input: User's voice and facial expression data

[0359] How it works: Voice data is subjected to phonemic analysis, and facial expression data is analyzed using facial expression recognition algorithms (e.g., FaceNet). For example, stress or anxiety can be detected from the user's tone of voice and facial expressions.

[0360] Output: The user's emotional state assessment result is generated.

[0361] Step 7: Server result generation and notification

[0362] How it works: The server generates a detailed diagnostic report based on the analysis results and notifies the user.

[0363] Input: Health risk assessment results, abnormal patterns, emotional state assessment results

[0364] What it does: Creates a diagnostic report for the user based on the analysis results, including recommended actions based on detected anomalies, health risks, and emotional state. Sends a push notification to the user's device and allows them to view the report through the app.

[0365] Output: Detailed diagnostic reports and notifications

[0366] Step 8: Collaborate with a server specialist

[0367] Actions: If necessary, the server sends the analysis results to a specialist for further opinion.

[0368] Input: Analysis results

[0369] What it does: Receives feedback from the specialist and incorporates it into the final report, which is then communicated back to the user and made available for review through the app.

[0370] Output: Final report and notification including expert opinion

[0371] At each step, specific data processing and calculations are carried out, and the results are passed on to the next step, providing users with comprehensive and accurate health risk assessments and countermeasures.

[0372] (Application example 2)

[0373] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0374] In modern society, there are many systems that assess individual health risks and provide solutions, but few take the user's emotional state into account. Furthermore, most systems do not provide personalized meal plans that combine health and emotional data. Therefore, there is a need for a system that enables more effective health management by providing appropriate meal plans based on the user's health status.

[0375] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting a user's health checkup data and emotional data; means for transmitting the input health checkup data and emotional data to the server; means for storing the health checkup data and emotional data received by the server; analysis means including deep learning technology and multimodal AI technology for analyzing the stored health checkup data and emotional data; and means for providing the user with individual health risks, solutions, and a personalized meal plan based on the analysis results generated by the analysis means. This enables more accurate and personalized health risk assessments and solutions, as well as the provision of appropriate meal plans, based on the user's health and emotional state.

[0376] "User's health checkup data" refers to numerical data and image data obtained as a result of a user's health checkup, such as blood test results and X-ray images.

[0377] "Emotion data" is data that represents the user's emotional state, obtained from the user's voice input and facial expression data.

[0378] "Server" refers to the central system for receiving, storing and analyzing data sent from the User's device.

[0379] "Deep learning technology" is a technology that uses artificial neural networks to learn and analyze patterns and features from large amounts of data.

[0380] "Multimodal AI technology" is an artificial intelligence technology that integrates multiple different types of data (e.g., text data, audio data, image data), correlates them, and analyzes them.

[0381] "Analysis means" refers to a means for analyzing health checkup data and emotion data using deep learning technology and multimodal AI technology to identify abnormal patterns and health risks.

[0382] "Individual health risk" refers to the health risk that differs for each user and is assessed based on each user's health checkup data and emotional data.

[0383] "Solution" refers to a recommended measure or treatment for a health risk identified by the analytical means.

[0384] A "personalized meal plan" is a meal suggestion created based on a user's health and emotional data, taking into account their individual health and emotional state.

[0385] A "specialist" refers to a medical professional with advanced expertise in medicine and health.

[0386] The "final report" is a report that includes detailed diagnostic results and recommended measures, based on the analytical results obtained by the analytical means and the opinions of specialists.

[0387] The present invention is a system that analyzes a user's health checkup data and emotional data to provide an individualized health risk assessment and personalized meal plan. This system aims to improve the efficiency of health management and provide more accurate countermeasures. The overall configuration of the system and specific processing are described below.

[0388] System configuration

[0389] 1. User's Device

[0390] Hardware: Smartphones, tablets, computers

[0391] Software: Dedicated application, camera, microphone

[0392] Users can launch a dedicated application and input health checkup data (e.g., blood test results, image data), and can also use the application to collect emotional data from facial expressions and voice.

[0393] 2. Server System

[0394] Hardware: Server, GPU (Graphics Processing Unit)

[0395] Software: Database management systems (e.g., MySQL, PostgreSQL), deep learning analysis engines (e.g., TensorFlow), sentiment analysis engines (e.g., OpenAI)

[0396] It receives, stores, and analyzes data sent by users, including checking the integrity of the received data, preprocessing the data, and analyzing the data using deep learning technology and a sentiment analysis engine.

[0397] Processing flow

[0398] 1. Enter and submit data

[0399] Users input their health checkup data and emotion data through a dedicated application and send it to the server using a secure communication protocol (e.g., SSL / TLS).

[0400] 2. Receipt and storage of data

[0401] The server receives the data sent by the user and stores it in a database, where it undergoes consistency checks and format conversion.

[0402] 3. Data Analysis

[0403] The server sends the stored data to a deep learning analysis engine, which preprocesses the health check data and image data. The emotion analysis engine then analyzes the emotion data. Deep learning and multimodal AI technologies are integrated to assess health risks.

[0404] 4. Generating and notifying results

[0405] Based on the analysis results, the system provides users with an individual health risk assessment and solutions, and generates a personalized diet plan. The analysis results and diet plan are sent to the user's device, and a final report incorporating the opinion of a specialist may also be provided.

[0406] Specific examples

[0407] For example, consider the case of a 45-year-old male user undergoing a health checkup. The user obtains the results of a blood test and takes a photo of a chest X-ray with their smartphone, which they then enter into the app. At the same time, they record their emotions about the results (e.g., anxiety or stress) using audio and facial recording. This data is sent to a server and analyzed by a deep learning system. For example, if high cholesterol is detected and there are abnormalities in the X-ray image, the risk of arteriosclerosis is identified. At the same time, it is also analyzed that the user is in a state of high stress. Based on this, a meal plan with a relaxing effect, mainly consisting of low-fat foods, is suggested.

[0408] Prompt Sentence Examples

[0409] "45-year-old male, recent medical checkup revealed high cholesterol. High stress levels. Can you suggest a low-fat, relaxing diet plan?"

[0410] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0411] Step 1:

[0412] Users enter their health checkup data (blood test results and image data) through a dedicated application, and then record their emotional data. This data is collected using a specific input screen within the app. For example, they can use the camera function to take a photo of their checkup results, and record voice comments and facial expressions as emotional data. The entered data is sent to the server using a secure communication protocol (e.g., SSL / TLS).

[0413] Input: health check data, voice data, facial expression data

[0414] Output: User data sent to the server

[0415] Step 2:

[0416] The server receives the health checkup data and emotion data sent by the user. The received data is stored in a database, where it is checked for consistency and formatted for conversion. This allows inaccurate data or data with different formats to be stored in a unified format.

[0417] Input: Submitted user data

[0418] Output: Consistency-checked data stored in the database

[0419] Step 3:

[0420] The server uses a deep learning analysis engine to preprocess the health checkup data (blood test results and image data) based on the stored data. Specifically, the image is resized and noise is removed. Deep learning technology then begins analysis to assess health risks.

[0421] Input: Integrity checked data

[0422] Output: Preprocessed data

[0423] Step 4:

[0424] The server sends the stored emotional data to an emotion analysis engine for analysis. The user's emotional state (e.g., stress, anxiety, relief) is extracted from the voice and facial expression data. This allows the user's current psychological state to be understood.

[0425] Input: Emotion data that has been integrity checked

[0426] Output: Parsed emotional state data

[0427] Step 5:

[0428] The server combines the results from the deep learning analysis engine and the sentiment analysis engine to assess the user's health risks and generate a personalized diet plan based on this. For example, if a high cholesterol risk is identified, a low-fat diet plan will be suggested.

[0429] Input: Health risk analysis results, emotional state data

[0430] Output: Integrated health risk assessment and meal plan

[0431] Step 6:

[0432] The server notifies the user of the generated analysis results and meal plan. Notifications are made using the notification function within the application or push notifications, and the user can check them through the application.

[0433] Input: Integrated health risk assessment and meal plan

[0434] Output: User notification

[0435] Step 7:

[0436] If necessary, the server will share the generated analysis results with a specialist and generate a final report based on the specialist's opinion. This final report will also be notified to the user and can be viewed through the application.

[0437] Input: Integrated health risk assessment and meal plan

[0438] Output: Final report including expert opinion and notification to user

[0439] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0440] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0441] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0442] [Second embodiment]

[0443] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0444] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0445] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0446] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0447] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0448] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0449] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0450] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0451] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0452] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0453] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0454] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0455] The present invention is a system that allows users to input health checkup data and receive analysis results. Specific processing details of each of the users, terminals, and servers will be explained below.

[0456] User interaction

[0457] 1. Enter the data:

[0458] After the diagnosis, the user launches a dedicated application on their device (smartphone or PC) and logs in.

[0459] Follow the app's instructions to take a photo of your health check results and enter data such as blood test results.

[0460] 2. Data transmission:

[0461] The entered health checkup data is sent from the terminal to the server using a secure communication method (e.g. HTTPS) for use over the Internet.

[0462] Server processing

[0463] 1. Data Receipt and Storage:

[0464] The server receives the health checkup data sent from the terminal and first stores the data in a database.

[0465] The format of the saved data is checked for consistency, and the data format is converted if necessary.

[0466] 2. Data Analysis:

[0467] The data stored in the database is analyzed using deep learning algorithms and multimodal AI. Deep learning technology uses large-scale neural networks to learn features and patterns that appear in blood test values ​​and image data, and detect abnormalities.

[0468] Multimodal AI technology analyzes correlations between blood test values ​​and image data to draw comprehensive conclusions from multiple data sources.

[0469] 3. Results generation and notification:

[0470] Based on the analysis results, a detailed diagnostic report is generated for the user, including details of any abnormalities detected, a health risk assessment, and recommended actions (e.g., retesting or consultation with a specialist).

[0471] The server notifies the terminal that the analysis results are ready.

[0472] 4. Collaboration with specialists:

[0473] If necessary, the analysis results are sent to a specialist for further comment, and a final report incorporating the specialist's feedback is generated.

[0474] The final report will be notified to the user again and will be available to view through the app.

[0475] Specific examples

[0476] As a specific example, consider the case where User A, a 45-year-old male, undergoes a health checkup. User A uses a dedicated smartphone app to take blood test results and chest X-ray images obtained after the checkup and enter them into the app. The app then sends this data to the server.

[0477] The server stores the data and begins analyzing it using a deep learning system. For example, if a blood test shows high cholesterol and an X-ray shows mild abnormalities, the AI ​​can connect these features to detect a risk of arteriosclerosis.

[0478] A detailed report containing this risk assessment is generated and sent from the server to User A's smartphone. User A can then review the report through the app and take necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist).

[0479] Furthermore, if a serious abnormality is detected, the server sends the analysis results to a specialist for further opinion. A final report including the specialist's opinion is generated and notified to User A again. In this way, the present invention significantly reduces the risk of oversight or misdiagnosis and provides useful health management information to users.

[0480] The processing flow will be explained below.

[0481] Step 1:

[0482] After the user has undergone a health checkup, they launch the dedicated app on their device (smartphone or PC) and log in. They then follow the instructions in the app to begin entering their health checkup results (e.g., blood test values, image data). If necessary, they can take photos of the results or enter the values ​​manually.

[0483] Step 2:

[0484] The device sends the entered health checkup data to the server. A secure communication protocol (e.g., HTTPS) is used over the Internet to ensure data confidentiality.

[0485] Step 3:

[0486] The server receives the health checkup data sent from the device. The received data is stored in a database and a data consistency check is performed. If there is a problem with the data format, it is converted into an appropriate format.

[0487] Step 4:

[0488] The server prepares the stored data to be sent to the analysis engine, where preprocessing such as resizing and noise removal is performed on the image data to improve the accuracy of the analysis.

[0489] Step 5:

[0490] The server launches an analysis engine (deep learning model and multimodal AI), which analyzes correlations between blood test values ​​and image data to detect abnormal patterns and health risks.

[0491] Step 6:

[0492] The server generates a detailed diagnostic report based on the analysis results, including any abnormalities detected, a health risk assessment, and recommended actions (e.g., retesting or consultation with a specialist).

[0493] Step 7:

[0494] The server notifies the user that the analysis results are ready. A push notification is sent to the device with the message "Analysis results are ready. Please check the app."

[0495] Step 8:

[0496] The user checks the notification on their device and launches the dedicated app to view the analysis results. The app displays a detailed diagnostic report, allowing the user to learn specific measures to address health risks.

[0497] Step 9:

[0498] If necessary, the server sends the analysis results to a specialist, who reviews them and provides additional opinions and advice.

[0499] Step 10:

[0500] The server generates a final report including the expert's opinion. The server notifies the user again that the final report is ready, and the user can view the final report through the app.

[0501] Step 11:

[0502] The server provides healthcare support functions, allowing users to receive in-app reminders and advice on daily health management (e.g., drinking more water, improving exercise habits).

[0503] Step 12:

[0504] Users manage their health based on the advice provided, and use the app's support features to implement healthy lifestyle habits and regularly monitor their health.

[0505] Example 1

[0506] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0507] Conventional health checkup data management and analysis systems often make it difficult for users to efficiently input and transmit health checkup data and quickly receive analysis results. Furthermore, due to insufficient accuracy in data analysis and insufficient collaboration with specialists, it is difficult to provide users with reliable health management information. For these reasons, there is a growing need for a system that can detect health risks early and suggest appropriate countermeasures.

[0508] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0509] In this invention, the server includes a means for a user to input health checkup data, a means for transmitting the input health checkup data from a terminal to the server, and a means for storing the health checkup data received by the server and checking the format consistency. This enables efficient data input, transmission, and consistency check. The server also includes a means for analyzing the stored data using deep learning and multimodal AI technologies, a means for generating a detailed diagnostic report for the user based on the analysis results generated by the analysis means and notifying the user via push notification or email, a means for sharing the analysis results with a specialist as needed and incorporating the specialist's opinion into the final report, and a means for notifying the user of the final report and making it viewable via an application. This allows the user to receive analysis results quickly and comprehensively, enabling them to obtain reliable diagnostic information that reflects the specialist's opinion.

[0510] "User" means an individual who uses the system to input health checkup data and receive analysis results.

[0511] "Health checkup data" refers to health-related information such as blood test results and image data entered by the user.

[0512] "Input means" refers to the method or device by which users input their health checkup data into the system. Specifically, this applies to dedicated applications on smartphones or PCs.

[0513] A "terminal" is a device that a user uses to access the system, such as a smartphone or PC.

[0514] A "server" is the central computer device of the entire system that receives, stores, and analyzes data sent from terminals.

[0515] "Transmission means" refers to a method or device for transmitting the entered medical examination data from the terminal to the server. Specifically, this refers to a secure communication method via the Internet (e.g., HTTPS).

[0516] The "receiving means" refers to a method or device by which the server receives the medical examination data transmitted from the terminal.

[0517] The "storage means" refers to a method or device for storing the received medical examination data in the server. Specifically, this corresponds to a database.

[0518] "Consistency check means" refers to a method or device for verifying whether the received data conforms to a specified format, whether required fields have been entered, and so on.

[0519] "Deep learning technology" is a technique that uses large-scale neural networks to analyze data.

[0520] "Multimodal AI technology" refers to technology that combines and analyzes different types of data (e.g., text data and image data) to draw comprehensive conclusions.

[0521] "Analysis means" refers to a method or device for analyzing health checkup data using deep learning technology and multimodal AI technology.

[0522] A "diagnostic report" is a report generated based on the analysis results, which includes detected abnormalities, an assessment of health risks, and recommended actions to take.

[0523] "Notification means" refers to a method or device for notifying the user of the completion of the analysis results preparation or other information from the server. Specifically, this applies to push notifications and emails.

[0524] "Expert" means a medical professional with medical expertise who provides additional input on the results of the analysis.

[0525] "Sharing means" refers to a method or device for sharing analysis results with specialists.

[0526] "Final report" refers to the final diagnostic report that reflects the opinion of the specialist.

[0527] An "application" is software that allows users to input data and view analysis results.

[0528] This invention is a system that allows users to input health checkup data and receive analysis results. This system works in cooperation with three entities: the user, the terminal, and the server, to detect health risks early and propose appropriate countermeasures.

[0529] User interaction

[0530] Users launch the dedicated application on their smartphone or PC and log in. They follow the app's instructions to take pictures of their health checkup results and enter data such as blood test values. The dedicated application then imports images from the camera roll and provides an interface for manually entering numerical data.

[0531] Sending data by the device

[0532] The device formats the input data and sends it to the server using a secure communication method (e.g., HTTPS). Specifically, the captured image and input numerical data are combined into JSON format and sent as an HTTP POST request. This ensures reliable data transmission.

[0533] Receiving and storing data by the server

[0534] The server receives the health checkup data sent from the terminal. The received data is first checked for format consistency before being saved in the database. Specifically, it verifies whether the data conforms to the specified format and whether required fields have been entered. Data that passes this verification is saved in the database, and if there is any inconsistency, an error message is returned to the user.

[0535] Data analysis by the server

[0536] The stored data is analyzed using deep learning algorithms and multimodal AI technology. Deep learning technology uses large-scale neural networks to learn features and patterns that appear in blood test values ​​and image data and detect abnormalities. Multimodal AI technology combines and analyzes different data sources (blood test values ​​and image data) to provide a comprehensive health assessment.

[0537] Generation and notification of analysis results

[0538] Based on the analysis results, a detailed diagnostic report is generated for the user, including any detected abnormalities, an assessment of health risks, and recommended actions (e.g., retesting or consulting a specialist). Once the analysis results are ready, the server notifies the user via push notification or email.

[0539] Collaboration with specialists

[0540] If any significant abnormalities are detected, the server sends the analysis results to a specialist for further opinion. A final report incorporating the specialist's opinion is generated and notified to the user. The user can view this final report through the application.

[0541] Specific examples

[0542] Consider the case where User A, a 45-year-old male, undergoes a health checkup. User A uses a dedicated smartphone app to take blood test results and chest X-ray images obtained after the checkup and enter them into the app. The app then sends this data to the server.

[0543] The server stores the data and begins analyzing it using a deep learning system. For example, if a blood test shows high cholesterol and an X-ray shows mild abnormalities, the AI ​​can connect these features to detect a risk of arteriosclerosis.

[0544] A detailed report containing this risk assessment is generated and sent from the server to User A's smartphone. User A can then review the report through the app and take necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist).

[0545] Furthermore, if a serious abnormality is detected, the server sends the analysis results to a specialist for further opinion. A final report including the specialist's opinion is generated and notified to User A again. In this way, the present invention significantly reduces the risk of oversight or misdiagnosis and provides useful health management information to users.

[0546] Prompt Sentence Examples

[0547] "Please create a specific scenario in which a 45-year-old male user enters the results of a health check into a smartphone app and receives the results of an analysis by AI. The analysis will use deep learning technology to generate a detailed report including a risk assessment based on high cholesterol and abnormal chest X-ray findings."

[0548] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0549] Step 1:

[0550] The user launches the dedicated application on their smartphone or PC and logs in.

[0551] Input: User account information (e.g., user ID, password)

[0552] Output: User's dashboard on successful login, error message on failure

[0553] The user launches the dedicated application and enters their account information to log in. If the login is successful, the user's dashboard will be displayed and they can proceed to the next step.

[0554] Step 2:

[0555] Users take pictures of their health checkup results and enter data such as blood test results into a dedicated app.

[0556] Input: Images of health checkup results, blood test results, and other data

[0557] Output: Formatted health check data (e.g., JSON format)

[0558] The user follows the app's instructions to take a photo of the health checkup results and enter data such as blood test values, which are then formatted into the appropriate format (e.g., JSON) within the app.

[0559] Step 3:

[0560] The terminal transmits the input health checkup data to the server.

[0561] Input: Formatted health check data (e.g., JSON format)

[0562] Output: Data transmission result to the server (success or failure message)

[0563] The device sends the formatted health check data to the server using HTTPS, sending the data as an HTTP POST request and receiving a success or failure result.

[0564] Step 4:

[0565] The server receives the health checkup data sent from the terminal and stores it in a database.

[0566] Input: Health checkup data sent from the device

[0567] Output: Save result to database (success or failure message)

[0568] The server receives the data sent from the terminal and first checks the format for consistency. If this check is successful, it stores the data in the database. If there is a mismatch, it returns an error message to the terminal.

[0569] Step 5:

[0570] The server analyzes the stored health checkup data using deep learning algorithms and multimodal AI technology.

[0571] Input: Medical examination data stored in a database

[0572] Output: Analysis results (abnormal locations, risk assessment, etc.)

[0573] The server applies deep learning technology to the stored data to detect abnormalities in blood test values ​​and image data, and uses multimodal AI technology to analyze correlations between different data sources to perform a comprehensive health risk assessment.

[0574] Step 6:

[0575] Based on the analysis results, a detailed diagnostic report is generated and notified to the user.

[0576] Input: Analysis results

[0577] Output: Diagnostic report, notification (push notification or email)

[0578] The server generates a diagnostic report based on the analysis results, which includes detected abnormalities, a health risk assessment, and recommended actions to take. Based on the report, the server sends a notification to the user's device.

[0579] Step 7:

[0580] If necessary, the analysis results will be sent to a specialist for additional opinion.

[0581] Input: Analysis results

[0582] Output: Expert feedback, final report

[0583] If any significant abnormalities are detected, the server sends the analysis results to a specialist for further opinion, generates a final report incorporating the specialist's opinion, and notifies the user again.

[0584] Step 8:

[0585] The user reviews the final report and takes any recommended actions.

[0586] Input: Final Report

[0587] Output: User behavior change (e.g., diet change, increased exercise, specialist consultation)

[0588] Users receive a notification and can launch the app to view the final report, which can then be used to take specific measures such as improving their diet, increasing exercise, or consulting a specialist.

[0589] (Application example 1)

[0590] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0591] Health management has become an extremely important issue in modern life. However, due to the lack of individual health risk assessments based on health checkup results and the lack of solutions, it is difficult for users to respond appropriately. Furthermore, it is difficult to recommend and deliver dietary recommendations based on health risks with current technology.

[0592] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0593] In this invention, the server includes a means for inputting a user's health checkup data, a means for transmitting the input health checkup data to the server, and a means for storing the health checkup data received by the server. This enables an analysis means including deep learning technology and multimodal AI technology for analyzing the stored health checkup data. The system also includes a means for providing the user with individual health risks and solutions based on the analysis results generated by the analysis means, and a means for generating dietary recommendations based on the health data and transmitting them to a delivery device. This enables health risk assessment and customized dietary suggestions and delivery for the user, providing more comprehensive health management.

[0594] "User" refers to a person who inputs health checkup data and receives analysis results.

[0595] "Health checkup data" refers to information such as blood test results and image data obtained by a user as a result of a health checkup.

[0596] "Input means" refers to a method or device that allows a user to input health checkup data into the system.

[0597] "Server" refers to the computer system that receives, stores, and analyzes health checkup data sent by users.

[0598] The "storing means" refers to a method or device for storing the received medical examination data in a storage device such as a database.

[0599] "Analysis means" refers to a method or device for analyzing health checkup data and conducting health risk assessments using deep learning technology and multimodal AI technology.

[0600] "Deep learning technology" is an advanced machine learning technology that uses artificial neural networks to learn the characteristics and patterns of data.

[0601] "Multimodal AI technology" refers to artificial intelligence technology that integrates and analyzes multiple types of data (e.g., blood test values ​​and image data).

[0602] "Means based on the analysis results" refers to a method or device that utilizes the analysis results obtained by the analysis means to provide information or suggestions to the user.

[0603] "Dietary recommendations based on health data" means proposing the optimal meal menu for each individual based on the user's health checkup data.

[0604] A "delivery device" is a robot or machine that delivers meals to users.

[0605] The present invention is a system for assessing individual health risks based on health checkup data, proposing healthy meals to users based on the results, and providing the meals via a delivery device. Specific embodiments of the system are described below.

[0606] Hardware and Software Configuration

[0607] 1. User Device

[0608] Hardware: Smartphone, tablet, or personal computer

[0609] Software: Dedicated application, camera, internet connection

[0610] 2. Server

[0611] Hardware: High-performance server computer

[0612] Software: Database management systems, deep learning algorithms (e.g., Keras), multimodal AI technology

[0613] 3. Delivery Device

[0614] Hardware: Food delivery robot

[0615] Software: Route optimization algorithm, communication module

[0616] System processing flow

[0617] User operations

[0618] 1. Enter the data:

[0619] After undergoing a health checkup, the user launches a dedicated application on their device and logs in. Following the instructions, they take an image of the health checkup results and enter blood test results and other diagnostic data.

[0620] 2. Data transmission:

[0621] The entered health checkup data is sent from the terminal to a server via the Internet using a secure protocol (e.g., HTTPS).

[0622] Server Processing

[0623] 1. Data Receipt and Storage:

[0624] The server receives the health checkup data sent from the terminal, first checks whether the data is in the correct format, and then stores it in a database.

[0625] 2. Data Analysis:

[0626] The stored data is analyzed using deep learning and multimodal AI technologies. Deep learning models (e.g., using Keras) are used to learn features that appear in blood test values ​​and image data and detect abnormalities. Multimodal AI technology integrates information from multiple data sources (e.g., blood test values ​​and image data) to perform more accurate analysis.

[0627] 3. Results generation and notification:

[0628] Based on the analysis results, a detailed report is automatically generated, including a health risk assessment, any abnormalities detected, and recommended actions (e.g., dietary changes, exercise recommendations). Based on this report, optimal meal menu recommendations are also generated.

[0629] 4. Send to delivery device:

[0630] A meal menu personalized for the user is sent to the delivery device, which then delivers the meal to the user's location via the optimal route.

[0631] Specific examples

[0632] example:

[0633] User A, a 45-year-old man, undergoes a health check and enters the results into a smartphone app. When his blood test results and chest X-ray image are sent to the server, deep learning technology begins analysis. If the blood test results indicate high cholesterol and the X-ray image shows abnormalities, the AI ​​will link these features and detect a risk of arteriosclerosis.

[0634] A detailed report including this risk assessment is generated, and recommended meals such as "low-fat meals" and "high-fiber meals" are suggested. The server sends this information to a food delivery robot, which then delivers healthy meals to User A via the optimal route. The process can be initiated with a prompt that reads, "Your blood test results indicate high cholesterol, and your X-rays have revealed abnormalities."

[0635] This system allows users to easily manage their health at home, offering customized meal suggestions and prompt delivery, which helps users maintain their health and reduce medical costs.

[0636] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0637] Step 1:

[0638] After the health check, the user launches a dedicated application and logs in. The user takes an image of the health check results and inputs data such as blood test values. The application receives this input data. The input data is received in the form of image data and numerical data. The user's device formats this data appropriately and generates a prompt to proceed to the next step.

[0639] Step 2:

[0640] The terminal sends the entered health checkup data to the server. This communication uses HTTPS to protect the confidentiality of the data. The data sent includes numerical data of blood test values ​​and health checkup images. The terminal notifies the user that the data has been sent successfully.

[0641] Step 3:

[0642] The server receives the health checkup data sent from the terminal. The received data is first stored in a database. A consistency check is performed on the stored data, and the data format is converted as necessary. After the data format conversion on the server side, the data is in a format that can be processed by the analysis system.

[0643] Step 4:

[0644] The server analyzes the stored health checkup data. Deep learning technology (e.g., Keras) is used to learn the characteristics and patterns of blood test values ​​and image data and detect abnormalities. Multimodal AI technology is used to analyze the correlation between blood test values ​​and image data and perform a comprehensive health risk assessment. The input data is passed to an analysis model, which generates analysis results.

[0645] Step 5:

[0646] Based on the analysis results, the server generates a detailed health risk assessment report for the user, which includes details of the detected abnormalities, a health risk assessment, and recommended measures (e.g., dietary changes and exercise recommendations).The analysis results also simultaneously generate recommendations for optimal meal menus based on the health data.

[0647] Step 6:

[0648] The server sends the generated report and meal recommendations to the delivery device using an appropriate communication protocol (e.g., MQTT, HTTP). The delivery device uses the received information to prepare meals optimal for the user's health condition and schedules the delivery. After the data is sent, the server confirms that the delivery was successfully initiated and notifies the user.

[0649] Step 7:

[0650] The delivery device delivers the meal to the user via the optimal route based on the received meal menu. The delivery device uses GPS and route optimization algorithms to reach the user efficiently. After delivery, the user can confirm that the delivery has been completed through the application and provide any necessary feedback to help improve the accuracy of the entire system.

[0651] The above is the specific processing flow of the system that realizes this application example, and clarifies what operations are performed at each step and what output data is generated from what input data.

[0652] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0653] This invention is a system that allows users to input and submit health checkup data, receive analysis results based on that data, and also combines it with an emotion engine that recognizes the user's emotional state. This system makes it possible to provide more personalized health risk assessments and solutions. The specific processing details for each of the users, terminals, and servers are explained below.

[0654] User interaction

[0655] 1. Enter the data:

[0656] After the diagnosis, the user launches a dedicated application on their device (smartphone or PC) and logs in.

[0657] Follow the app's instructions to enter your health checkup results (e.g., blood test results, image data). If necessary, take a photo of the results or enter the values ​​manually.

[0658] It also enables voice input and facial expression recognition functions to capture data on the user's voice and facial expressions.

[0659] 2. Data transmission:

[0660] The entered health checkup data and emotion data are sent from the device to a server using a secure communication protocol (e.g., SSL / TLS) over the Internet to ensure data confidentiality.

[0661] Server processing

[0662] 1. Data Receipt and Storage:

[0663] The server receives the health check data and emotion data sent from the device. The received data is stored in a database and a data integrity check is performed. If there is a problem with the data format, it is converted into an appropriate format.

[0664] 2. Data Analysis:

[0665] The data stored on the server is sent to the analysis engine, where preprocessing such as resizing and noise removal is performed on the image data to improve the accuracy of the analysis.

[0666] Using deep learning models and multimodal AI technology, the system analyzes correlations between blood test values ​​and image data to detect abnormal patterns and health risks.

[0667] 3. Emotional Data Analysis:

[0668] The server uses an emotion engine to analyze the user's voice and facial expression data, taking into account the user's recognized emotional state (e.g., stress, anxiety, relief) and adjusts the health risk assessment accordingly.

[0669] 4. Result generation and notification:

[0670] Based on the analysis results, a detailed diagnostic report is generated for the user, including detected abnormalities, a health risk assessment, and recommended actions (e.g., retesting or seeing a specialist) based on the user's emotional state.

[0671] The server notifies the user when the analysis results are ready.

[0672] 5. Collaboration with specialists:

[0673] If necessary, the analysis results are sent to a specialist for further comment, and a final report incorporating the specialist's feedback is generated.

[0674] The final report will be notified to the user again and made available for viewing through the app.

[0675] Specific examples

[0676] As a specific example, consider the case of User A, a 45-year-old male, after undergoing a health checkup. User A uses a dedicated smartphone app to take blood test results and chest X-ray images obtained after the checkup and enter them into the app. User A also has the app recognize voice comments and facial expressions to record his emotions regarding the diagnosis results (e.g., anxiety, stress).

[0677] The server receives this data and begins analyzing it using a deep learning system. For example, if a blood test shows high cholesterol and an X-ray shows mild abnormalities, the AI ​​will connect these features and detect a risk of arteriosclerosis. At the same time, the emotion engine will recognize that User A is experiencing high levels of stress.

[0678] A detailed report including this risk assessment and emotional state is generated and sent from the server to User A's smartphone. User A can review this report through the app and determine necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist). In addition, if any serious abnormalities are detected, the analysis results are sent to a specialist, and a final report incorporating the specialist's opinion is generated and notified to User A again.

[0679] The present invention provides a more accurate and personalized health risk assessment by taking into account the user's emotional state, significantly reducing the risk of oversight or misdiagnosis, and providing comprehensive health management information that allows users to take measures with greater peace of mind.

[0680] The processing flow will be explained below.

[0681] Step 1:

[0682] After the user has undergone a health checkup, they launch a dedicated app on their device (smartphone or PC) and log in. They follow the instructions within the app and begin inputting their health checkup results (e.g., blood test values, image data). If necessary, they can take a photo of the results or manually enter the values. They can also turn on voice input and facial expression recognition functions to capture data on the user's voice and facial expressions.

[0683] Step 2:

[0684] The device sends the entered health check data and emotion data to the server. The data is transmitted via the Internet using a secure communication protocol (e.g., HTTPS) to ensure confidentiality.

[0685] Step 3:

[0686] The server receives the health check data and emotion data sent from the device. The received data is stored in a database and a data integrity check is performed. If there is a problem with the data format, it is converted into an appropriate format.

[0687] Step 4:

[0688] The server prepares the stored data to be sent to the analysis engine, where preprocessing such as resizing and noise removal is performed on the image data to improve the accuracy of the analysis.

[0689] Step 5:

[0690] The server launches an analysis engine (deep learning model and multimodal AI), which analyzes correlations between blood test values ​​and image data to detect abnormal patterns and health risks.

[0691] Step 6:

[0692] The server analyzes the user's voice data and facial expression data using an emotion engine. The emotion engine recognizes the user's emotional state (e.g., stress, anxiety, relief) and reflects it in the analysis results.

[0693] Step 7:

[0694] The server generates a detailed diagnostic report based on the analysis results and emotional state, including any abnormalities detected, a health risk assessment, and recommended actions (e.g., retesting or consultation with a specialist).

[0695] Step 8:

[0696] The server notifies the user that the analysis results are ready. A push notification is sent to the device with the message "Analysis results are ready. Please check the app."

[0697] Step 9:

[0698] The user checks the notification on their device and launches the dedicated app to view the analysis results. The app displays a detailed diagnostic report, allowing the user to learn specific measures to address health risks.

[0699] Step 10:

[0700] If necessary, the server sends the analysis results to a specialist, who reviews them and provides additional opinions and advice.

[0701] Step 11:

[0702] The server generates a final report including the expert's opinion. The server notifies the user again that the final report is ready, and the user can view the final report through the app.

[0703] Step 12:

[0704] The server provides healthcare support functions, allowing users to receive in-app reminders and advice on daily health management (e.g., drinking more water, improving exercise habits).

[0705] Step 13:

[0706] Users manage their health based on the advice provided, and use the app's support features to implement healthy lifestyle habits and regularly monitor their health.

[0707] Example 2

[0708] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0709] Conventional health checkup data analysis systems provide analysis results without considering the user's emotional state, which means they are unable to reflect the user's psychological factors.In addition, it is difficult to provide users with more reliable information because it is difficult to perform detailed analysis of diagnosis results or create reports that incorporate the opinions of specialists.

[0710] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0711] In this invention, the server includes means for inputting a user's health checkup data and emotional data, means for transmitting the input health checkup data and emotional data to the server, means for storing and preprocessing the health checkup data and emotional data received by the server and analyzing them using deep learning and multimodal AI technologies, means for providing individual health risks and solutions that take the user's emotional state into consideration based on the analysis results generated by the analysis means and means for notifying the user of the analysis results, and means for transmitting the analysis results to a specialist, if necessary, and generating and notifying the specialist of a final report including the specialist's opinion. This makes it possible to provide more accurate and individualized health risk assessments and solutions that take the user's emotional state into consideration.

[0712] A "user" is an individual who uses the system to input medical and emotional data.

[0713] "Health checkup data" refers to medical data used to evaluate a user's health condition, such as blood test results and image data.

[0714] "Emotional data" is data obtained through a user's voice and facial expressions that indicates the user's emotional state (e.g., stress, anxiety, relief).

[0715] A "server" is a computer system that receives, stores, pre-processes, analyzes, and notifies users of data submitted by the server.

[0716] "Preprocessing" refers to processing for improving the quality of received data, and includes, for example, noise removal and resizing of image data.

[0717] "Deep learning technology" is an advanced machine learning technology that uses neural networks to analyze data and recognize patterns.

[0718] "Multimodal AI technology" is an artificial intelligence technology that integrates and analyzes multiple data formats (e.g., numerical data, image data, audio data).

[0719] "Analysis means" refers to a means for analyzing stored health checkup data and emotion data using deep learning technology and multimodal AI technology.

[0720] "Analysis results" are results generated by the analysis means that indicate the user's health risks and abnormality patterns.

[0721] "Individual Health Risk" is a risk assessment based on a user's particular health condition and indicates a health problem specific to the user.

[0722] "Solutions" are methods or measures for reducing or improving health risks that are provided to users based on the analysis results.

[0723] A "specialist" is a medical professional with advanced knowledge and experience in the medical field, and is responsible for providing expert opinions on the analysis results.

[0724] "Final Report" means a detailed medical examination report provided to the user, including analysis results and expert opinions.

[0725] "Notification" is the act of informing the user that the analysis results or final report are ready.

[0726] This invention is a system that provides comprehensive health risk assessment and solutions by allowing users to input health checkup data and emotional data and send them to a server.The system uses deep learning and multimodal AI technologies to analyze the health checkup data and emotional data and provide individual countermeasures based on the analysis results.

[0727] User interaction

[0728] Users launch a dedicated application on their smartphone or computer and log in. Next, they follow the app's instructions to enter their health checkup results. This input method includes manually entering numerical data for blood test results and uploading image data of the checkup results. Voice input and facial expression recognition functions are also enabled to capture the user's emotional state.

[0729] Sending data

[0730] When the user presses the "Send" button, the entered health checkup data and emotion data are sent from the device to the server. This transmission is securely transmitted using the SSL / TLS protocol.

[0731] Server processing

[0732] The server stores the received data in a database and checks its integrity. If the data has defects, it is converted to a standard format. Next, the image data is preprocessed, such as noise removal and resizing. Libraries such as OpenCV can be used for this.

[0733] Once the data preprocessing is complete, the server analyzes the data using deep learning techniques (e.g., TensorFlow, PyTorch) and multimodal AI techniques. Specifically, it analyzes correlations between blood test values ​​and image data to detect abnormal patterns. It also evaluates the user's emotional state (e.g., stress, anxiety, relief) through analysis of voice and facial expression data.

[0734] Analysis results and notifications

[0735] Once the analysis results are generated, a detailed diagnostic report is created for the user, including any detected health risks, details of any abnormalities, and recommended actions (e.g., retesting or seeing a specialist) based on the user's emotional state. The server then notifies the user of the diagnostic report, typically via push notification or email.

[0736] Collaboration with specialists

[0737] If necessary, the server sends the analysis results to a specialist for further comment. A final report incorporating the specialist's feedback is generated and sent back to the user, allowing them to receive a more detailed and reliable diagnosis and treatment plan.

[0738] Specific examples

[0739] As a concrete example, consider the case where a 45-year-old male user undergoes a health checkup. Using a dedicated app, the user takes a blood test result and a chest X-ray image and inputs them into the app. The user also comments via voice on his / her feelings about the diagnosis (e.g., anxiety, stress), and facial expression data is also captured. The server receives this data and analyzes it using deep learning and multimodal AI technologies.

[0740] Through analysis, for example, high cholesterol blood test results and minor abnormalities in a chest X-ray can be detected as indicators of atherosclerosis risk. At the same time, high stress levels can be identified from emotional data. A detailed report that takes into account the analysis results and emotional state is provided to the user, suggesting necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist). If serious abnormalities are found, the analysis results are sent to a specialist, and the user is notified of a final report including the specialist's opinion.

[0741] Prompt Sentence Examples

[0742] "Analyze the blood test results and chest X-ray images of a 45-year-old man after he undergoes a health checkup to assess his risk of arteriosclerosis. Also, take into account the user's emotional state (e.g., anxiety, stress) in response to the diagnosis, and suggest appropriate countermeasures."

[0743] The system will enable accurate and personalized health risk assessment and solutions that also take into account the user's emotional state.

[0744] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0745] Step 1: Entering User Data

[0746] How it works: The user launches a dedicated application on their smartphone or computer and logs in.

[0747] Input: Health checkup results (e.g., blood test numerical data, image data), voice data, facial expression data

[0748] Specific operation: The app displays a health check result input form. The user enters numerical data into the form and takes and uploads image data using the camera. The voice input and facial expression recognition functions are enabled, and the app captures voice comments and facial expression data in real time.

[0749] Output: Health check result data and emotion data are input into the app.

[0750] Step 2: Sending user data

[0751] Action: The user presses the "Submit" button, which sends the entered data to the server.

[0752] Input: Health check result data and emotion data

[0753] How it works: The app uses the SSL / TLS protocol to send health checkup data and emotion data to the server via secure communication.

[0754] Output: Health and emotion data are transmitted securely.

[0755] Step 3: Receiving and storing data on the server

[0756] How it works: The server receives the data and stores it in a database.

[0757] Input: Health check data and emotion data

[0758] What happens: The server checks the integrity of the data and converts any abnormal formatting into a standard format, for example, standardizing date and number formats.

[0759] Output: The integrity-checked health and emotion data are stored in a database.

[0760] Step 4: Data Preprocessing on the Server

[0761] Action: The server preprocesses the received data.

[0762] Input: Medical examination data (especially image data)

[0763] Specific operation: Uses the OpenCV library to remove noise and resize image data. Specifically, it removes noise using a Gaussian filter and unifies the image size.

[0764] Output: Preprocessed image data is generated.

[0765] Step 5: Server Data Analysis

[0766] How it works: The server uses deep learning and multimodal AI techniques to analyze health checkup data.

[0767] Input: Preprocessed health checkup data (blood test results, processed image data), emotion data

[0768] How it works: Using TensorFlow and PyTorch, the system analyzes the correlation between blood test values ​​and image data. For example, if blood test values ​​indicate high cholesterol and the image shows an abnormality, the system can link these to detect the risk of arteriosclerosis.

[0769] Output: Health risk assessment results and abnormal patterns are generated.

[0770] Step 6: Emotional Data Analysis on the Server

[0771] How it works: The server uses the emotion engine to analyze the emotion data.

[0772] Input: User's voice and facial expression data

[0773] How it works: Voice data is subjected to phonemic analysis, and facial expression data is analyzed using facial expression recognition algorithms (e.g., FaceNet). For example, stress or anxiety can be detected from the user's tone of voice and facial expressions.

[0774] Output: The user's emotional state assessment result is generated.

[0775] Step 7: Server result generation and notification

[0776] How it works: The server generates a detailed diagnostic report based on the analysis results and notifies the user.

[0777] Input: Health risk assessment results, abnormal patterns, emotional state assessment results

[0778] What it does: Creates a diagnostic report for the user based on the analysis results, including recommended actions based on detected anomalies, health risks, and emotional state. Sends a push notification to the user's device and allows them to view the report through the app.

[0779] Output: Detailed diagnostic reports and notifications

[0780] Step 8: Collaborate with a server specialist

[0781] Actions: If necessary, the server sends the analysis results to a specialist for further opinion.

[0782] Input: Analysis results

[0783] What it does: Receives feedback from the specialist and incorporates it into the final report, which is then communicated back to the user and made available for review through the app.

[0784] Output: Final report and notification including expert opinion

[0785] At each step, specific data processing and calculations are carried out, and the results are passed on to the next step, providing users with comprehensive and accurate health risk assessments and countermeasures.

[0786] (Application example 2)

[0787] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0788] In modern society, there are many systems that assess individual health risks and provide solutions, but few take the user's emotional state into account. Furthermore, most systems do not provide personalized meal plans that combine health and emotional data. Therefore, there is a need for a system that enables more effective health management by providing appropriate meal plans based on the user's health status.

[0789] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting a user's health checkup data and emotional data; means for transmitting the input health checkup data and emotional data to the server; means for storing the health checkup data and emotional data received by the server; analysis means including deep learning technology and multimodal AI technology for analyzing the stored health checkup data and emotional data; and means for providing the user with individual health risks, solutions, and a personalized meal plan based on the analysis results generated by the analysis means. This enables more accurate and personalized health risk assessments and solutions, as well as the provision of appropriate meal plans, based on the user's health and emotional state.

[0790] "User's health checkup data" refers to numerical data and image data obtained as a result of a user's health checkup, such as blood test results and X-ray images.

[0791] "Emotion data" is data that represents the user's emotional state, obtained from the user's voice input and facial expression data.

[0792] "Server" refers to the central system for receiving, storing and analyzing data sent from the User's device.

[0793] "Deep learning technology" is a technology that uses artificial neural networks to learn and analyze patterns and features from large amounts of data.

[0794] "Multimodal AI technology" is an artificial intelligence technology that integrates multiple different types of data (e.g., text data, audio data, image data), correlates them, and analyzes them.

[0795] "Analysis means" refers to a means for analyzing health checkup data and emotion data using deep learning technology and multimodal AI technology to identify abnormal patterns and health risks.

[0796] "Individual health risk" refers to the health risk that differs for each user and is assessed based on each user's health checkup data and emotional data.

[0797] "Solution" refers to a recommended measure or treatment for a health risk identified by the analytical means.

[0798] A "personalized meal plan" is a meal suggestion created based on a user's health and emotional data, taking into account their individual health and emotional state.

[0799] A "specialist" refers to a medical professional with advanced expertise in medicine and health.

[0800] The "final report" is a report that includes detailed diagnostic results and recommended measures, based on the analytical results obtained by the analytical means and the opinions of specialists.

[0801] The present invention is a system that analyzes a user's health checkup data and emotional data to provide an individualized health risk assessment and personalized meal plan. This system aims to improve the efficiency of health management and provide more accurate countermeasures. The overall configuration of the system and specific processing are described below.

[0802] System configuration

[0803] 1. User's Device

[0804] Hardware: Smartphones, tablets, computers

[0805] Software: Dedicated application, camera, microphone

[0806] Users can launch a dedicated application and input health checkup data (e.g., blood test results, image data), and can also use the application to collect emotional data from facial expressions and voice.

[0807] 2. Server System

[0808] Hardware: Server, GPU (Graphics Processing Unit)

[0809] Software: Database management systems (e.g., MySQL, PostgreSQL), deep learning analysis engines (e.g., TensorFlow), sentiment analysis engines (e.g., OpenAI)

[0810] It receives, stores, and analyzes data sent by users, including checking the integrity of the received data, preprocessing the data, and analyzing the data using deep learning technology and a sentiment analysis engine.

[0811] Processing flow

[0812] 1. Enter and submit data

[0813] Users input their health checkup data and emotion data through a dedicated application and send it to the server using a secure communication protocol (e.g., SSL / TLS).

[0814] 2. Receipt and storage of data

[0815] The server receives the data sent by the user and stores it in a database, where it undergoes consistency checks and format conversion.

[0816] 3. Data Analysis

[0817] The server sends the stored data to a deep learning analysis engine, which preprocesses the health check data and image data. The emotion analysis engine then analyzes the emotion data. Deep learning and multimodal AI technologies are integrated to assess health risks.

[0818] 4. Generating and notifying results

[0819] Based on the analysis results, the system provides users with an individual health risk assessment and solutions, and generates a personalized diet plan. The analysis results and diet plan are sent to the user's device, and a final report incorporating the opinion of a specialist may also be provided.

[0820] Specific examples

[0821] For example, consider the case of a 45-year-old male user undergoing a health checkup. The user obtains the results of a blood test and takes a photo of a chest X-ray with their smartphone, which they then enter into the app. At the same time, they record their emotions about the results (e.g., anxiety or stress) using audio and facial recording. This data is sent to a server and analyzed by a deep learning system. For example, if high cholesterol is detected and there are abnormalities in the X-ray image, the risk of arteriosclerosis is identified. At the same time, it is also analyzed that the user is in a state of high stress. Based on this, a meal plan with a relaxing effect, mainly consisting of low-fat foods, is suggested.

[0822] Prompt Sentence Examples

[0823] "45-year-old male, recent medical checkup revealed high cholesterol. High stress levels. Can you suggest a low-fat, relaxing diet plan?"

[0824] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0825] Step 1:

[0826] Users enter their health checkup data (blood test results and image data) through a dedicated application, and then record their emotional data. This data is collected using a specific input screen within the app. For example, they can use the camera function to take a photo of their checkup results, and record voice comments and facial expressions as emotional data. The entered data is sent to the server using a secure communication protocol (e.g., SSL / TLS).

[0827] Input: health check data, voice data, facial expression data

[0828] Output: User data sent to the server

[0829] Step 2:

[0830] The server receives the health checkup data and emotion data sent by the user. The received data is stored in a database, where it is checked for consistency and formatted for conversion. This allows inaccurate data or data with different formats to be stored in a unified format.

[0831] Input: Submitted user data

[0832] Output: Consistency-checked data stored in the database

[0833] Step 3:

[0834] The server uses a deep learning analysis engine to preprocess the health checkup data (blood test results and image data) based on the stored data. Specifically, the image is resized and noise is removed. Deep learning technology then begins analysis to assess health risks.

[0835] Input: Integrity checked data

[0836] Output: Preprocessed data

[0837] Step 4:

[0838] The server sends the stored emotional data to an emotion analysis engine for analysis. The user's emotional state (e.g., stress, anxiety, relief) is extracted from the voice and facial expression data. This allows the user's current psychological state to be understood.

[0839] Input: Emotion data that has been integrity checked

[0840] Output: Parsed emotional state data

[0841] Step 5:

[0842] The server combines the results from the deep learning analysis engine and the sentiment analysis engine to assess the user's health risks and generate a personalized diet plan based on this. For example, if a high cholesterol risk is identified, a low-fat diet plan will be suggested.

[0843] Input: Health risk analysis results, emotional state data

[0844] Output: Integrated health risk assessment and meal plan

[0845] Step 6:

[0846] The server notifies the user of the generated analysis results and meal plan. Notifications are made using the notification function within the application or push notifications, and the user can check them through the application.

[0847] Input: Integrated health risk assessment and meal plan

[0848] Output: User notification

[0849] Step 7:

[0850] If necessary, the server will share the generated analysis results with a specialist and generate a final report based on the specialist's opinion. This final report will also be notified to the user and can be viewed through the application.

[0851] Input: Integrated health risk assessment and meal plan

[0852] Output: Final report including expert opinion and notification to user

[0853] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0854] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0855] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0856] [Third embodiment]

[0857] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0858] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0859] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0860] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0861] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0862] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0863] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0864] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0865] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0866] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0867] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0868] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0869] The present invention is a system that allows users to input health checkup data and receive analysis results. Specific processing details of each of the users, terminals, and servers will be explained below.

[0870] User interaction

[0871] 1. Enter the data:

[0872] After the diagnosis, the user launches a dedicated application on their device (smartphone or PC) and logs in.

[0873] Follow the app's instructions to take a photo of your health check results and enter data such as blood test results.

[0874] 2. Data transmission:

[0875] The entered health checkup data is sent from the terminal to the server using a secure communication method (e.g. HTTPS) for use over the Internet.

[0876] Server processing

[0877] 1. Data Receipt and Storage:

[0878] The server receives the health checkup data sent from the terminal and first stores the data in a database.

[0879] The format of the saved data is checked for consistency, and the data format is converted if necessary.

[0880] 2. Data Analysis:

[0881] The data stored in the database is analyzed using deep learning algorithms and multimodal AI. Deep learning technology uses large-scale neural networks to learn features and patterns that appear in blood test values ​​and image data, and detect abnormalities.

[0882] Multimodal AI technology analyzes correlations between blood test values ​​and image data to draw comprehensive conclusions from multiple data sources.

[0883] 3. Results generation and notification:

[0884] Based on the analysis results, a detailed diagnostic report is generated for the user, including details of any abnormalities detected, a health risk assessment, and recommended actions (e.g., retesting or consultation with a specialist).

[0885] The server notifies the terminal that the analysis results are ready.

[0886] 4. Collaboration with specialists:

[0887] If necessary, the analysis results are sent to a specialist for further comment, and a final report incorporating the specialist's feedback is generated.

[0888] The final report will be notified to the user again and will be available to view through the app.

[0889] Specific examples

[0890] As a specific example, consider the case where User A, a 45-year-old male, undergoes a health checkup. User A uses a dedicated smartphone app to take blood test results and chest X-ray images obtained after the checkup and enter them into the app. The app then sends this data to the server.

[0891] The server stores the data and begins analyzing it using a deep learning system. For example, if a blood test shows high cholesterol and an X-ray shows mild abnormalities, the AI ​​can connect these features to detect a risk of arteriosclerosis.

[0892] A detailed report containing this risk assessment is generated and sent from the server to User A's smartphone. User A can then review the report through the app and take necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist).

[0893] Furthermore, if a serious abnormality is detected, the server sends the analysis results to a specialist for further opinion. A final report including the specialist's opinion is generated and notified to User A again. In this way, the present invention significantly reduces the risk of oversight or misdiagnosis and provides useful health management information to users.

[0894] The processing flow will be explained below.

[0895] Step 1:

[0896] After the user has undergone a health checkup, they launch the dedicated app on their device (smartphone or PC) and log in. They then follow the instructions in the app to begin entering their health checkup results (e.g., blood test values, image data). If necessary, they can take photos of the results or enter the values ​​manually.

[0897] Step 2:

[0898] The device sends the entered health checkup data to the server. A secure communication protocol (e.g., HTTPS) is used over the Internet to ensure data confidentiality.

[0899] Step 3:

[0900] The server receives the health checkup data sent from the device. The received data is stored in a database and a data consistency check is performed. If there is a problem with the data format, it is converted into an appropriate format.

[0901] Step 4:

[0902] The server prepares the stored data to be sent to the analysis engine, where preprocessing such as resizing and noise removal is performed on the image data to improve the accuracy of the analysis.

[0903] Step 5:

[0904] The server launches an analysis engine (deep learning model and multimodal AI), which analyzes correlations between blood test values ​​and image data to detect abnormal patterns and health risks.

[0905] Step 6:

[0906] The server generates a detailed diagnostic report based on the analysis results, including any abnormalities detected, a health risk assessment, and recommended actions (e.g., retesting or consultation with a specialist).

[0907] Step 7:

[0908] The server notifies the user that the analysis results are ready. A push notification is sent to the device with the message "Analysis results are ready. Please check the app."

[0909] Step 8:

[0910] The user checks the notification on their device and launches the dedicated app to view the analysis results. The app displays a detailed diagnostic report, allowing the user to learn specific measures to address health risks.

[0911] Step 9:

[0912] If necessary, the server sends the analysis results to a specialist, who reviews them and provides additional opinions and advice.

[0913] Step 10:

[0914] The server generates a final report including the expert's opinion. The server notifies the user again that the final report is ready, and the user can view the final report through the app.

[0915] Step 11:

[0916] The server provides healthcare support functions, allowing users to receive in-app reminders and advice (e.g., water intake, improving exercise habits) that are useful for daily health management.

[0917] Step 12:

[0918] Users manage their health based on the advice provided, and use the app's support features to implement healthy lifestyle habits and regularly monitor their health.

[0919] Example 1

[0920] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0921] Conventional health checkup data management and analysis systems often make it difficult for users to efficiently input and transmit health checkup data and quickly receive analysis results. Furthermore, due to insufficient accuracy in data analysis and insufficient collaboration with specialists, it is difficult to provide users with reliable health management information. For these reasons, there is a growing need for a system that can detect health risks early and suggest appropriate countermeasures.

[0922] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0923] In this invention, the server includes a means for a user to input health checkup data, a means for transmitting the input health checkup data from a terminal to the server, and a means for storing the health checkup data received by the server and checking the format consistency. This enables efficient data input, transmission, and consistency check. The server also includes a means for analyzing the stored data using deep learning and multimodal AI technologies, a means for generating a detailed diagnostic report for the user based on the analysis results generated by the analysis means and notifying the user via push notification or email, a means for sharing the analysis results with a specialist as needed and incorporating the specialist's opinion into the final report, and a means for notifying the user of the final report and making it viewable via an application. This allows the user to receive analysis results quickly and comprehensively, enabling them to obtain reliable diagnostic information that reflects the specialist's opinion.

[0924] "User" means an individual who uses the system to input health checkup data and receive analysis results.

[0925] "Health checkup data" refers to health-related information such as blood test results and image data entered by the user.

[0926] "Input means" refers to the method or device by which users input their health checkup data into the system. Specifically, this applies to dedicated applications on smartphones or PCs.

[0927] A "terminal" is a device that a user uses to access the system, such as a smartphone or PC.

[0928] A "server" is the central computer device of the entire system that receives, stores, and analyzes data sent from terminals.

[0929] "Transmission means" refers to a method or device for transmitting the entered medical examination data from the terminal to the server. Specifically, this refers to a secure communication method via the Internet (e.g., HTTPS).

[0930] The "receiving means" refers to a method or device by which the server receives the medical examination data transmitted from the terminal.

[0931] The "storage means" refers to a method or device for storing the received medical examination data in the server. Specifically, this corresponds to a database.

[0932] "Consistency check means" refers to a method or device for verifying whether the received data conforms to a specified format, whether required fields have been entered, and so on.

[0933] "Deep learning technology" is a technique that uses large-scale neural networks to analyze data.

[0934] "Multimodal AI technology" refers to technology that combines and analyzes different types of data (e.g., text data and image data) to draw comprehensive conclusions.

[0935] "Analysis means" refers to a method or device for analyzing health checkup data using deep learning technology and multimodal AI technology.

[0936] A "diagnostic report" is a report generated based on the analysis results, which includes detected abnormalities, an assessment of health risks, and recommended actions to take.

[0937] "Notification means" refers to a method or device for notifying the user of the completion of the analysis results preparation or other information from the server. Specifically, this applies to push notifications and emails.

[0938] "Expert" means a medical professional with medical expertise who provides additional input on the results of the analysis.

[0939] "Sharing means" refers to a method or device for sharing analysis results with specialists.

[0940] "Final report" refers to the final diagnostic report that reflects the opinion of the specialist.

[0941] An "application" is software that allows users to input data and view analysis results.

[0942] This invention is a system that allows users to input health checkup data and receive analysis results. This system works in cooperation with three entities: the user, the terminal, and the server, to detect health risks early and propose appropriate countermeasures.

[0943] User interaction

[0944] Users launch the dedicated application on their smartphone or PC and log in. They follow the app's instructions to take pictures of their health checkup results and enter data such as blood test values. The dedicated application then imports images from the camera roll and provides an interface for manually entering numerical data.

[0945] Sending data by the device

[0946] The device formats the input data and sends it to the server using a secure communication method (e.g., HTTPS). Specifically, the captured image and input numerical data are combined into JSON format and sent as an HTTP POST request. This ensures reliable data transmission.

[0947] Receiving and storing data by the server

[0948] The server receives the health checkup data sent from the terminal. The received data is first checked for format consistency before being saved in the database. Specifically, it verifies whether the data conforms to the specified format and whether required fields have been entered. Data that passes this verification is saved in the database, and if there is any inconsistency, an error message is returned to the user.

[0949] Data analysis by the server

[0950] The stored data is analyzed using deep learning algorithms and multimodal AI technology. Deep learning technology uses large-scale neural networks to learn features and patterns that appear in blood test values ​​and image data and detect abnormalities. Multimodal AI technology combines and analyzes different data sources (blood test values ​​and image data) to provide a comprehensive health assessment.

[0951] Generation and notification of analysis results

[0952] Based on the analysis results, a detailed diagnostic report is generated for the user, including any detected abnormalities, an assessment of health risks, and recommended actions (e.g., retesting or consulting a specialist). Once the analysis results are ready, the server notifies the user via push notification or email.

[0953] Collaboration with specialists

[0954] If any significant abnormalities are detected, the server sends the analysis results to a specialist for further opinion. A final report incorporating the specialist's opinion is generated and notified to the user. The user can view this final report through the application.

[0955] Specific examples

[0956] Consider the case where User A, a 45-year-old male, undergoes a health checkup. User A uses a dedicated smartphone app to take blood test results and chest X-ray images obtained after the checkup and enter them into the app. The app then sends this data to the server.

[0957] The server stores the data and begins analyzing it using a deep learning system. For example, if a blood test shows high cholesterol and an X-ray shows mild abnormalities, the AI ​​can connect these features to detect a risk of arteriosclerosis.

[0958] A detailed report containing this risk assessment is generated and sent from the server to User A's smartphone. User A can then review the report through the app and take necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist).

[0959] Furthermore, if a serious abnormality is detected, the server sends the analysis results to a specialist for further opinion. A final report including the specialist's opinion is generated and notified to User A again. In this way, the present invention significantly reduces the risk of oversight or misdiagnosis and provides useful health management information to users.

[0960] Prompt Sentence Examples

[0961] "Please create a specific scenario in which a 45-year-old male user enters the results of a health check into a smartphone app and receives the results of an analysis by AI. The analysis will use deep learning technology to generate a detailed report including a risk assessment based on high cholesterol and abnormal chest X-ray findings."

[0962] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0963] Step 1:

[0964] The user launches the dedicated application on their smartphone or PC and logs in.

[0965] Input: User account information (e.g., user ID, password)

[0966] Output: User's dashboard on successful login, error message on failure

[0967] The user launches the dedicated application and enters their account information to log in. If the login is successful, the user's dashboard will be displayed and they can proceed to the next step.

[0968] Step 2:

[0969] Users take pictures of their health checkup results and enter data such as blood test results into a dedicated app.

[0970] Input: Images of health checkup results, blood test results, and other data

[0971] Output: Formatted health check data (e.g., JSON format)

[0972] The user follows the app's instructions to take a photo of the health checkup results and enter data such as blood test values, which are then formatted into the appropriate format (e.g., JSON) within the app.

[0973] Step 3:

[0974] The terminal transmits the input health checkup data to the server.

[0975] Input: Formatted health check data (e.g., JSON format)

[0976] Output: Data transmission result to the server (success or failure message)

[0977] The device sends the formatted health check data to the server using HTTPS, sending the data as an HTTP POST request and receiving a success or failure result.

[0978] Step 4:

[0979] The server receives the health checkup data sent from the terminal and stores it in a database.

[0980] Input: Health checkup data sent from the device

[0981] Output: Save result to database (success or failure message)

[0982] The server receives the data sent from the terminal and first checks the format for consistency. If this check is successful, it stores the data in the database. If there is a mismatch, it returns an error message to the terminal.

[0983] Step 5:

[0984] The server analyzes the stored health checkup data using deep learning algorithms and multimodal AI technology.

[0985] Input: Medical examination data stored in a database

[0986] Output: Analysis results (abnormal locations, risk assessment, etc.)

[0987] The server applies deep learning technology to the stored data to detect abnormalities in blood test values ​​and image data, and uses multimodal AI technology to analyze correlations between different data sources to perform a comprehensive health risk assessment.

[0988] Step 6:

[0989] Based on the analysis results, a detailed diagnostic report is generated and notified to the user.

[0990] Input: Analysis results

[0991] Output: Diagnostic report, notification (push notification or email)

[0992] The server generates a diagnostic report based on the analysis results, which includes detected abnormalities, a health risk assessment, and recommended actions to take. Based on the report, the server sends a notification to the user's device.

[0993] Step 7:

[0994] If necessary, the analysis results will be sent to a specialist for additional opinion.

[0995] Input: Analysis results

[0996] Output: Expert feedback, final report

[0997] If any significant abnormalities are detected, the server sends the analysis results to a specialist for further opinion, generates a final report incorporating the specialist's opinion, and notifies the user again.

[0998] Step 8:

[0999] The user reviews the final report and takes any recommended actions.

[1000] Input: Final Report

[1001] Output: User behavior change (e.g., diet change, increased exercise, specialist consultation)

[1002] Users receive a notification and can launch the app to view the final report, which can then be used to take specific measures such as improving their diet, increasing exercise, or consulting a specialist.

[1003] (Application example 1)

[1004] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1005] Health management has become an extremely important issue in modern life. However, due to the lack of individual health risk assessments based on health checkup results and the lack of solutions, it is difficult for users to respond appropriately. Furthermore, it is difficult to recommend and deliver dietary recommendations based on health risks with current technology.

[1006] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1007] In this invention, the server includes a means for inputting a user's health checkup data, a means for transmitting the input health checkup data to the server, and a means for storing the health checkup data received by the server. This enables an analysis means including deep learning technology and multimodal AI technology for analyzing the stored health checkup data. The system also includes a means for providing the user with individual health risks and solutions based on the analysis results generated by the analysis means, and a means for generating dietary recommendations based on the health data and transmitting them to a delivery device. This enables health risk assessment and customized dietary suggestions and delivery for the user, providing more comprehensive health management.

[1008] "User" refers to a person who inputs health checkup data and receives analysis results.

[1009] "Health checkup data" refers to information such as blood test results and image data obtained by a user as a result of a health checkup.

[1010] "Input means" refers to a method or device that allows a user to input health checkup data into the system.

[1011] "Server" refers to the computer system that receives, stores, and analyzes health checkup data sent by users.

[1012] The "storing means" refers to a method or device for storing the received medical examination data in a storage device such as a database.

[1013] "Analysis means" refers to a method or device for analyzing health checkup data and conducting health risk assessments using deep learning technology and multimodal AI technology.

[1014] "Deep learning technology" is an advanced machine learning technology that uses artificial neural networks to learn the characteristics and patterns of data.

[1015] "Multimodal AI technology" refers to artificial intelligence technology that integrates and analyzes multiple types of data (e.g., blood test values ​​and image data).

[1016] "Means based on the analysis results" refers to a method or device that utilizes the analysis results obtained by the analysis means to provide information or suggestions to the user.

[1017] "Dietary recommendations based on health data" means proposing the optimal meal menu for each individual based on the user's health checkup data.

[1018] A "delivery device" is a robot or machine that delivers meals to users.

[1019] The present invention is a system for assessing individual health risks based on health checkup data, proposing healthy meals to users based on the results, and providing the meals via a delivery device. Specific embodiments of the system are described below.

[1020] Hardware and Software Configuration

[1021] 1. User Device

[1022] Hardware: Smartphone, tablet, or personal computer

[1023] Software: Dedicated application, camera, internet connection

[1024] 2. Server

[1025] Hardware: High-performance server computer

[1026] Software: Database management systems, deep learning algorithms (e.g., Keras), multimodal AI technology

[1027] 3. Delivery Device

[1028] Hardware: Food delivery robot

[1029] Software: Route optimization algorithm, communication module

[1030] System processing flow

[1031] User operations

[1032] 1. Enter the data:

[1033] After undergoing a health checkup, the user launches a dedicated application on their device and logs in. Following the instructions, they take an image of the health checkup results and enter blood test results and other diagnostic data.

[1034] 2. Data transmission:

[1035] The entered health checkup data is sent from the terminal to a server via the Internet using a secure protocol (e.g., HTTPS).

[1036] Server Processing

[1037] 1. Data Receipt and Storage:

[1038] The server receives the health checkup data sent from the terminal, first checks whether the data is in the correct format, and then stores it in a database.

[1039] 2. Data Analysis:

[1040] The stored data is analyzed using deep learning and multimodal AI technologies. Deep learning models (e.g., using Keras) are used to learn features that appear in blood test values ​​and image data and detect abnormalities. Multimodal AI technology integrates information from multiple data sources (e.g., blood test values ​​and image data) to perform more accurate analysis.

[1041] 3. Results generation and notification:

[1042] Based on the analysis results, a detailed report is automatically generated, including a health risk assessment, any abnormalities detected, and recommended actions (e.g., dietary changes, exercise recommendations). Based on this report, optimal meal menu recommendations are also generated.

[1043] 4. Send to delivery device:

[1044] A meal menu personalized for the user is sent to the delivery device, which then delivers the meal to the user's location via the optimal route.

[1045] Specific examples

[1046] example:

[1047] User A, a 45-year-old man, undergoes a health check and enters the results into a smartphone app. When his blood test results and chest X-ray image are sent to the server, deep learning technology begins analysis. If the blood test results indicate high cholesterol and the X-ray image shows abnormalities, the AI ​​will link these features and detect a risk of arteriosclerosis.

[1048] A detailed report including this risk assessment is generated, and recommended meals such as "low-fat meals" and "high-fiber meals" are suggested. The server sends this information to a food delivery robot, which then delivers healthy meals to User A via the optimal route. The process can be initiated with a prompt that reads, "Your blood test results indicate high cholesterol, and your X-rays have revealed abnormalities."

[1049] This system allows users to easily manage their health at home, offering customized meal suggestions and prompt delivery, which helps users maintain their health and reduce medical costs.

[1050] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1051] Step 1:

[1052] After the health check, the user launches a dedicated application and logs in. The user takes an image of the health check results and inputs data such as blood test values. The application receives this input data. The input data is received in the form of image data and numerical data. The user's device formats this data appropriately and generates a prompt to proceed to the next step.

[1053] Step 2:

[1054] The terminal sends the entered health checkup data to the server. This communication uses HTTPS to protect the confidentiality of the data. The data sent includes numerical data of blood test values ​​and health checkup images. The terminal notifies the user that the data has been sent successfully.

[1055] Step 3:

[1056] The server receives the health checkup data sent from the terminal. The received data is first stored in a database. A consistency check is performed on the stored data, and the data format is converted as necessary. After the data format conversion on the server side, the data is in a format that can be processed by the analysis system.

[1057] Step 4:

[1058] The server analyzes the stored health checkup data. Deep learning technology (e.g., Keras) is used to learn the characteristics and patterns of blood test values ​​and image data and detect abnormalities. Multimodal AI technology is used to analyze the correlation between blood test values ​​and image data and perform a comprehensive health risk assessment. The input data is passed to an analysis model, which generates analysis results.

[1059] Step 5:

[1060] Based on the analysis results, the server generates a detailed health risk assessment report for the user, which includes details of the detected abnormalities, a health risk assessment, and recommended measures (e.g., dietary changes and exercise recommendations).The analysis results also simultaneously generate recommendations for optimal meal menus based on the health data.

[1061] Step 6:

[1062] The server sends the generated report and meal recommendations to the delivery device using an appropriate communication protocol (e.g., MQTT, HTTP). The delivery device uses the received information to prepare meals optimal for the user's health condition and schedules the delivery. After the data is sent, the server confirms that the delivery was successfully initiated and notifies the user.

[1063] Step 7:

[1064] The delivery device delivers the meal to the user via the optimal route based on the received meal menu. The delivery device uses GPS and route optimization algorithms to reach the user efficiently. After delivery, the user can confirm that the delivery has been completed through the application and provide any necessary feedback to help improve the accuracy of the entire system.

[1065] The above is the specific processing flow of the system that realizes this application example, and clarifies what operations are performed at each step and what output data is generated from what input data.

[1066] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1067] This invention is a system that allows users to input and submit health checkup data, receive analysis results based on that data, and also combines it with an emotion engine that recognizes the user's emotional state. This system makes it possible to provide more personalized health risk assessments and solutions. The specific processing details for each of the users, terminals, and servers are explained below.

[1068] User interaction

[1069] 1. Enter the data:

[1070] After the diagnosis, the user launches a dedicated application on their device (smartphone or PC) and logs in.

[1071] Follow the app's instructions to enter your health checkup results (e.g., blood test results, image data). If necessary, take a photo of the results or enter the values ​​manually.

[1072] It also enables voice input and facial expression recognition functions to capture data on the user's voice and facial expressions.

[1073] 2. Data transmission:

[1074] The entered health checkup data and emotion data are sent from the device to a server using a secure communication protocol (e.g., SSL / TLS) over the Internet to ensure data confidentiality.

[1075] Server processing

[1076] 1. Data Receipt and Storage:

[1077] The server receives the health check data and emotion data sent from the device. The received data is stored in a database and a data integrity check is performed. If there is a problem with the data format, it is converted into an appropriate format.

[1078] 2. Data Analysis:

[1079] The data stored on the server is sent to the analysis engine, where preprocessing such as resizing and noise removal is performed on the image data to improve the accuracy of the analysis.

[1080] Using deep learning models and multimodal AI technology, the system analyzes correlations between blood test values ​​and image data to detect abnormal patterns and health risks.

[1081] 3. Emotional Data Analysis:

[1082] The server uses an emotion engine to analyze the user's voice and facial expression data, taking into account the user's recognized emotional state (e.g., stress, anxiety, relief) and adjusts the health risk assessment accordingly.

[1083] 4. Result generation and notification:

[1084] Based on the analysis results, a detailed diagnostic report is generated for the user, including detected abnormalities, a health risk assessment, and recommended actions (e.g., retesting or seeing a specialist) based on the user's emotional state.

[1085] The server notifies the user when the analysis results are ready.

[1086] 5. Collaboration with specialists:

[1087] If necessary, the analysis results are sent to a specialist for further comment, and a final report incorporating the specialist's feedback is generated.

[1088] The final report will be notified to the user again and made available for viewing through the app.

[1089] Specific examples

[1090] As a specific example, consider the case of User A, a 45-year-old male, after undergoing a health checkup. User A uses a dedicated smartphone app to take blood test results and chest X-ray images obtained after the checkup and enter them into the app. User A also has the app recognize voice comments and facial expressions to record his emotions regarding the diagnosis results (e.g., anxiety, stress).

[1091] The server receives this data and begins analyzing it using a deep learning system. For example, if a blood test shows high cholesterol and an X-ray shows mild abnormalities, the AI ​​will connect these features and detect a risk of arteriosclerosis. At the same time, the emotion engine will recognize that User A is experiencing high levels of stress.

[1092] A detailed report including this risk assessment and emotional state is generated and sent from the server to User A's smartphone. User A can review this report through the app and determine necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist). In addition, if any serious abnormalities are detected, the analysis results are sent to a specialist, and a final report incorporating the specialist's opinion is generated and notified to User A again.

[1093] The present invention provides a more accurate and personalized health risk assessment by taking into account the user's emotional state, significantly reducing the risk of oversight or misdiagnosis, and providing comprehensive health management information that allows users to take measures with greater peace of mind.

[1094] The processing flow will be explained below.

[1095] Step 1:

[1096] After the user has undergone a health checkup, they launch a dedicated app on their device (smartphone or PC) and log in. They follow the instructions within the app and begin inputting their health checkup results (e.g., blood test values, image data). If necessary, they can take a photo of the results or manually enter the values. They can also turn on voice input and facial expression recognition functions to capture data on the user's voice and facial expressions.

[1097] Step 2:

[1098] The device sends the entered health check data and emotion data to the server. The data is transmitted via the Internet using a secure communication protocol (e.g., HTTPS) to ensure confidentiality.

[1099] Step 3:

[1100] The server receives the health check data and emotion data sent from the device. The received data is stored in a database and a data integrity check is performed. If there is a problem with the data format, it is converted into an appropriate format.

[1101] Step 4:

[1102] The server prepares the stored data to be sent to the analysis engine, where preprocessing such as resizing and noise removal is performed on the image data to improve the accuracy of the analysis.

[1103] Step 5:

[1104] The server launches an analysis engine (deep learning model and multimodal AI), which analyzes correlations between blood test values ​​and image data to detect abnormal patterns and health risks.

[1105] Step 6:

[1106] The server analyzes the user's voice data and facial expression data using an emotion engine. The emotion engine recognizes the user's emotional state (e.g., stress, anxiety, relief) and reflects it in the analysis results.

[1107] Step 7:

[1108] The server generates a detailed diagnostic report based on the analysis results and emotional state, including any abnormalities detected, a health risk assessment, and recommended actions (e.g., retesting or consultation with a specialist).

[1109] Step 8:

[1110] The server notifies the user that the analysis results are ready. A push notification is sent to the device with the message "Analysis results are ready. Please check the app."

[1111] Step 9:

[1112] The user checks the notification on their device and launches the dedicated app to view the analysis results. The app displays a detailed diagnostic report, allowing the user to learn specific measures to address health risks.

[1113] Step 10:

[1114] If necessary, the server sends the analysis results to a specialist, who reviews them and provides additional opinions and advice.

[1115] Step 11:

[1116] The server generates a final report including the expert's opinion. The server notifies the user again that the final report is ready, and the user can view the final report through the app.

[1117] Step 12:

[1118] The server provides healthcare support functions, allowing users to receive in-app reminders and advice on daily health management (e.g., drinking more water, improving exercise habits).

[1119] Step 13:

[1120] Users manage their health based on the advice provided, and use the app's support features to implement healthy lifestyle habits and regularly monitor their health.

[1121] Example 2

[1122] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1123] Conventional health checkup data analysis systems provide analysis results without considering the user's emotional state, which means they are unable to reflect the user's psychological factors.In addition, it is difficult to provide users with more reliable information because it is difficult to perform detailed analysis of diagnosis results or create reports that incorporate the opinions of specialists.

[1124] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1125] In this invention, the server includes means for inputting a user's health checkup data and emotional data, means for transmitting the input health checkup data and emotional data to the server, means for storing and preprocessing the health checkup data and emotional data received by the server and analyzing them using deep learning and multimodal AI technologies, means for providing individual health risks and solutions that take the user's emotional state into consideration based on the analysis results generated by the analysis means and means for notifying the user of the analysis results, and means for transmitting the analysis results to a specialist, if necessary, and generating and notifying the specialist of a final report including the specialist's opinion. This makes it possible to provide more accurate and individualized health risk assessments and solutions that take the user's emotional state into consideration.

[1126] A "user" is an individual who uses the system to input medical and emotional data.

[1127] "Health checkup data" refers to medical data used to evaluate a user's health condition, such as blood test results and image data.

[1128] "Emotional data" is data obtained through a user's voice and facial expressions that indicates the user's emotional state (e.g., stress, anxiety, relief).

[1129] A "server" is a computer system that receives, stores, pre-processes, analyzes, and notifies users of data submitted by the server.

[1130] "Preprocessing" refers to processing for improving the quality of received data, and includes, for example, noise removal and resizing of image data.

[1131] "Deep learning technology" is an advanced machine learning technology that uses neural networks to analyze data and recognize patterns.

[1132] "Multimodal AI technology" is an artificial intelligence technology that integrates and analyzes multiple data formats (e.g., numerical data, image data, audio data).

[1133] "Analysis means" refers to a means for analyzing stored health checkup data and emotion data using deep learning technology and multimodal AI technology.

[1134] "Analysis results" are results generated by the analysis means that indicate the user's health risks and abnormality patterns.

[1135] "Individual Health Risk" is a risk assessment based on a user's particular health condition and indicates a health problem specific to the user.

[1136] "Solutions" are methods or measures for reducing or improving health risks that are provided to users based on the analysis results.

[1137] A "specialist" is a medical professional with advanced knowledge and experience in the medical field, and is responsible for providing expert opinions on the analysis results.

[1138] "Final Report" means a detailed medical examination report provided to the user, including analysis results and expert opinions.

[1139] "Notification" is the act of informing the user that the analysis results or final report are ready.

[1140] This invention is a system that provides comprehensive health risk assessment and solutions by allowing users to input health checkup data and emotional data and send them to a server.The system uses deep learning and multimodal AI technologies to analyze the health checkup data and emotional data and provide individual countermeasures based on the analysis results.

[1141] User interaction

[1142] Users launch a dedicated application on their smartphone or computer and log in. Next, they follow the app's instructions to enter their health checkup results. This input method includes manually entering numerical data for blood test results and uploading image data of the checkup results. Voice input and facial expression recognition functions are also enabled to capture the user's emotional state.

[1143] Sending data

[1144] When the user presses the "Send" button, the entered health checkup data and emotion data are sent from the device to the server. This transmission is securely transmitted using the SSL / TLS protocol.

[1145] Server processing

[1146] The server stores the received data in a database and checks its integrity. If the data has defects, it is converted to a standard format. Next, the image data is preprocessed, such as noise removal and resizing. Libraries such as OpenCV can be used for this.

[1147] Once the data preprocessing is complete, the server analyzes the data using deep learning techniques (e.g., TensorFlow, PyTorch) and multimodal AI techniques. Specifically, it analyzes correlations between blood test values ​​and image data to detect abnormal patterns. It also evaluates the user's emotional state (e.g., stress, anxiety, relief) through analysis of voice and facial expression data.

[1148] Analysis results and notifications

[1149] Once the analysis results are generated, a detailed diagnostic report is created for the user, including any detected health risks, details of any abnormalities, and recommended actions (e.g., retesting or seeing a specialist) based on the user's emotional state. The server then notifies the user of the diagnostic report, typically via push notification or email.

[1150] Collaboration with specialists

[1151] If necessary, the server sends the analysis results to a specialist for further comment. A final report incorporating the specialist's feedback is generated and sent back to the user, allowing them to receive a more detailed and reliable diagnosis and treatment plan.

[1152] Specific examples

[1153] As a concrete example, consider the case where a 45-year-old male user undergoes a health checkup. Using a dedicated app, the user takes a blood test result and a chest X-ray image and inputs them into the app. The user also comments via voice on his / her feelings about the diagnosis (e.g., anxiety, stress), and facial expression data is also captured. The server receives this data and analyzes it using deep learning and multimodal AI technologies.

[1154] Through analysis, for example, high cholesterol blood test results and minor abnormalities in a chest X-ray can be detected as indicators of atherosclerosis risk. At the same time, high stress levels can be identified from emotional data. A detailed report that takes into account the analysis results and emotional state is provided to the user, suggesting necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist). If serious abnormalities are found, the analysis results are sent to a specialist, and the user is notified of a final report including the specialist's opinion.

[1155] Prompt Sentence Examples

[1156] "Analyze the blood test results and chest X-ray images of a 45-year-old man after he undergoes a health checkup to assess his risk of arteriosclerosis. Also, take into account the user's emotional state (e.g., anxiety, stress) in response to the diagnosis, and suggest appropriate countermeasures."

[1157] The system will enable accurate and personalized health risk assessment and solutions that also take into account the user's emotional state.

[1158] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1159] Step 1: Entering User Data

[1160] How it works: The user launches a dedicated application on their smartphone or computer and logs in.

[1161] Input: Health checkup results (e.g., blood test numerical data, image data), voice data, facial expression data

[1162] Specific operation: The app displays a health check result input form. The user enters numerical data into the form and takes and uploads image data using the camera. The voice input and facial expression recognition functions are enabled, and the app captures voice comments and facial expression data in real time.

[1163] Output: Health check result data and emotion data are input into the app.

[1164] Step 2: Sending user data

[1165] Action: The user presses the "Submit" button, which sends the entered data to the server.

[1166] Input: Health check result data and emotion data

[1167] How it works: The app uses the SSL / TLS protocol to send health checkup data and emotion data to the server via secure communication.

[1168] Output: Health and emotion data are transmitted securely.

[1169] Step 3: Receiving and storing data on the server

[1170] How it works: The server receives the data and stores it in a database.

[1171] Input: Health check data and emotion data

[1172] What happens: The server checks the integrity of the data and converts any abnormal formatting into a standard format, for example, standardizing date and number formats.

[1173] Output: The integrity-checked health and emotion data are stored in a database.

[1174] Step 4: Data Preprocessing on the Server

[1175] Action: The server preprocesses the received data.

[1176] Input: Medical examination data (especially image data)

[1177] Specific operation: Uses the OpenCV library to remove noise and resize image data. Specifically, it removes noise using a Gaussian filter and unifies the image size.

[1178] Output: Preprocessed image data is generated.

[1179] Step 5: Server Data Analysis

[1180] How it works: The server uses deep learning and multimodal AI techniques to analyze health checkup data.

[1181] Input: Preprocessed health checkup data (blood test results, processed image data), emotion data

[1182] How it works: Using TensorFlow and PyTorch, the system analyzes the correlation between blood test values ​​and image data. For example, if blood test values ​​indicate high cholesterol and the image shows an abnormality, the system can link these to detect the risk of arteriosclerosis.

[1183] Output: Health risk assessment results and abnormal patterns are generated.

[1184] Step 6: Emotional Data Analysis on the Server

[1185] How it works: The server uses the emotion engine to analyze the emotion data.

[1186] Input: User's voice and facial expression data

[1187] How it works: Voice data is subjected to phonemic analysis, and facial expression data is analyzed using facial expression recognition algorithms (e.g., FaceNet). For example, stress or anxiety can be detected from the user's tone of voice and facial expressions.

[1188] Output: The user's emotional state assessment result is generated.

[1189] Step 7: Server result generation and notification

[1190] How it works: The server generates a detailed diagnostic report based on the analysis results and notifies the user.

[1191] Input: Health risk assessment results, abnormal patterns, emotional state assessment results

[1192] What it does: Creates a diagnostic report for the user based on the analysis results, including recommended actions based on detected anomalies, health risks, and emotional state. Sends a push notification to the user's device and allows them to view the report through the app.

[1193] Output: Detailed diagnostic reports and notifications

[1194] Step 8: Collaborate with a server specialist

[1195] Actions: If necessary, the server sends the analysis results to a specialist for further opinion.

[1196] Input: Analysis results

[1197] What it does: Receives feedback from the specialist and incorporates it into the final report, which is then communicated back to the user and made available for review through the app.

[1198] Output: Final report and notification including expert opinion

[1199] At each step, specific data processing and calculations are carried out, and the results are passed on to the next step, providing users with comprehensive and accurate health risk assessments and countermeasures.

[1200] (Application example 2)

[1201] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1202] In modern society, there are many systems that assess individual health risks and provide solutions, but few take the user's emotional state into account. Furthermore, most systems do not provide personalized meal plans that combine health and emotional data. Therefore, there is a need for a system that enables more effective health management by providing appropriate meal plans based on the user's health status.

[1203] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting a user's health checkup data and emotional data; means for transmitting the input health checkup data and emotional data to the server; means for storing the health checkup data and emotional data received by the server; analysis means including deep learning technology and multimodal AI technology for analyzing the stored health checkup data and emotional data; and means for providing the user with individual health risks, solutions, and a personalized meal plan based on the analysis results generated by the analysis means. This enables more accurate and personalized health risk assessments and solutions, as well as the provision of appropriate meal plans, based on the user's health and emotional state.

[1204] "User's health checkup data" refers to numerical data and image data obtained as a result of a user's health checkup, such as blood test results and X-ray images.

[1205] "Emotion data" is data that represents the user's emotional state, obtained from the user's voice input and facial expression data.

[1206] "Server" refers to the central system for receiving, storing and analyzing data sent from the User's device.

[1207] "Deep learning technology" is a technology that uses artificial neural networks to learn and analyze patterns and features from large amounts of data.

[1208] "Multimodal AI technology" is an artificial intelligence technology that integrates multiple different types of data (e.g., text data, audio data, image data), correlates them, and analyzes them.

[1209] "Analysis means" refers to a means for analyzing health checkup data and emotion data using deep learning technology and multimodal AI technology to identify abnormal patterns and health risks.

[1210] "Individual health risk" refers to the health risk that differs for each user and is assessed based on each user's health checkup data and emotional data.

[1211] "Solution" refers to a recommended measure or treatment for a health risk identified by the analytical means.

[1212] A "personalized meal plan" is a meal suggestion created based on a user's health and emotional data, taking into account their individual health and emotional state.

[1213] A "specialist" refers to a medical professional with advanced expertise in medicine and health.

[1214] The "final report" is a report that includes detailed diagnostic results and recommended measures, based on the analytical results obtained by the analytical means and the opinions of specialists.

[1215] The present invention is a system that analyzes a user's health checkup data and emotional data to provide an individualized health risk assessment and personalized meal plan. This system aims to improve the efficiency of health management and provide more accurate countermeasures. The overall configuration of the system and specific processing are described below.

[1216] System configuration

[1217] 1. User's Device

[1218] Hardware: Smartphones, tablets, computers

[1219] Software: Dedicated application, camera, microphone

[1220] Users can launch a dedicated application and input health checkup data (e.g., blood test results, image data), and can also use the application to collect emotional data from facial expressions and voice.

[1221] 2. Server System

[1222] Hardware: Server, GPU (Graphics Processing Unit)

[1223] Software: Database management systems (e.g., MySQL, PostgreSQL), deep learning analysis engines (e.g., TensorFlow), sentiment analysis engines (e.g., OpenAI)

[1224] It receives, stores, and analyzes data sent by users, including checking the integrity of the received data, preprocessing the data, and analyzing the data using deep learning technology and a sentiment analysis engine.

[1225] Processing flow

[1226] 1. Enter and submit data

[1227] Users input their health checkup data and emotion data through a dedicated application and send it to the server using a secure communication protocol (e.g., SSL / TLS).

[1228] 2. Receipt and storage of data

[1229] The server receives the data sent by the user and stores it in a database, where it undergoes consistency checks and format conversion.

[1230] 3. Data Analysis

[1231] The server sends the stored data to a deep learning analysis engine, which preprocesses the health check data and image data. The emotion analysis engine then analyzes the emotion data. Deep learning and multimodal AI technologies are integrated to assess health risks.

[1232] 4. Generating and notifying results

[1233] Based on the analysis results, the system provides users with an individual health risk assessment and solutions, and generates a personalized diet plan. The analysis results and diet plan are sent to the user's device, and a final report incorporating the opinion of a specialist may also be provided.

[1234] Specific examples

[1235] For example, consider the case of a 45-year-old male user undergoing a health checkup. The user obtains the results of a blood test and takes a photo of a chest X-ray with their smartphone, which they then enter into the app. At the same time, they record their emotions about the results (e.g., anxiety or stress) using audio and facial recordings. This data is sent to a server and analyzed by a deep learning system. For example, if high cholesterol is detected and there are abnormalities in the X-ray image, the risk of arteriosclerosis is identified. At the same time, it is also analyzed that the user is in a state of high stress. Based on this, a meal plan with a relaxing effect, mainly consisting of low-fat foods, is suggested.

[1236] Prompt Sentence Examples

[1237] "45-year-old male, recent medical checkup revealed high cholesterol. High stress levels. Can you suggest a low-fat, relaxing diet plan?"

[1238] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1239] Step 1:

[1240] Users enter their health checkup data (blood test results and image data) through a dedicated application, and then record their emotional data. This data is collected using a specific input screen within the app. For example, they can use the camera function to take a photo of their checkup results, and record voice comments and facial expressions as emotional data. The entered data is sent to the server using a secure communication protocol (e.g., SSL / TLS).

[1241] Input: health check data, voice data, facial expression data

[1242] Output: User data sent to the server

[1243] Step 2:

[1244] The server receives the health checkup data and emotion data sent by the user. The received data is stored in a database, where it is checked for consistency and formatted for conversion. This allows inaccurate data or data with different formats to be stored in a unified format.

[1245] Input: Submitted user data

[1246] Output: Consistency-checked data stored in the database

[1247] Step 3:

[1248] The server uses a deep learning analysis engine to preprocess the health checkup data (blood test results and image data) based on the stored data. Specifically, the image is resized and noise is removed. Deep learning technology then begins analysis to assess health risks.

[1249] Input: Integrity checked data

[1250] Output: Preprocessed data

[1251] Step 4:

[1252] The server sends the stored emotional data to an emotion analysis engine for analysis. The user's emotional state (e.g., stress, anxiety, relief) is extracted from the voice and facial expression data. This allows the user's current psychological state to be understood.

[1253] Input: Emotion data that has been integrity checked

[1254] Output: Parsed emotional state data

[1255] Step 5:

[1256] The server combines the results from the deep learning analysis engine and the sentiment analysis engine to assess the user's health risks and generate a personalized diet plan based on this. For example, if a high cholesterol risk is identified, a low-fat diet plan will be suggested.

[1257] Input: Health risk analysis results, emotional state data

[1258] Output: Integrated health risk assessment and meal plan

[1259] Step 6:

[1260] The server notifies the user of the generated analysis results and meal plan. Notifications are made using the notification function within the application or push notifications, and the user can check them through the application.

[1261] Input: Integrated health risk assessment and meal plan

[1262] Output: User notification

[1263] Step 7:

[1264] If necessary, the server will share the generated analysis results with a specialist and generate a final report based on the specialist's opinion. This final report will also be notified to the user and can be viewed through the application.

[1265] Input: Integrated health risk assessment and meal plan

[1266] Output: Final report including expert opinion and notification to user

[1267] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1268] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1269] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1270] [Fourth embodiment]

[1271] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1272] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1273] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1274] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1275] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1276] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1277] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1278] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1279] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1280] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1281] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1282] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1283] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1284] The present invention is a system that allows users to input health checkup data and receive analysis results. Specific processing details of each of the users, terminals, and servers will be explained below.

[1285] User interaction

[1286] 1. Enter the data:

[1287] After the diagnosis, the user launches a dedicated application on their device (smartphone or PC) and logs in.

[1288] Follow the app's instructions to take a photo of your health check results and enter data such as blood test results.

[1289] 2. Data transmission:

[1290] The entered health checkup data is sent from the terminal to the server using a secure communication method (e.g. HTTPS) for use over the Internet.

[1291] Server processing

[1292] 1. Data Receipt and Storage:

[1293] The server receives the health checkup data sent from the terminal and first stores the data in a database.

[1294] The format of the saved data is checked for consistency, and the data format is converted if necessary.

[1295] 2. Data Analysis:

[1296] The data stored in the database is analyzed using deep learning algorithms and multimodal AI. Deep learning technology uses large-scale neural networks to learn features and patterns that appear in blood test values ​​and image data, and detect abnormalities.

[1297] Multimodal AI technology analyzes correlations between blood test values ​​and image data to draw comprehensive conclusions from multiple data sources.

[1298] 3. Results generation and notification:

[1299] Based on the analysis results, a detailed diagnostic report is generated for the user, including details of any abnormalities detected, a health risk assessment, and recommended actions (e.g., retesting or consultation with a specialist).

[1300] The server notifies the terminal that the analysis results are ready.

[1301] 4. Collaboration with specialists:

[1302] If necessary, the analysis results are sent to a specialist for further comment, and a final report incorporating the specialist's feedback is generated.

[1303] The final report will be notified to the user again and will be available to view through the app.

[1304] Specific examples

[1305] As a specific example, consider the case where User A, a 45-year-old male, undergoes a health checkup. User A uses a dedicated smartphone app to take blood test results and chest X-ray images obtained after the checkup and enter them into the app. The app then sends this data to the server.

[1306] The server stores the data and begins analyzing it using a deep learning system. For example, if a blood test shows high cholesterol and an X-ray shows mild abnormalities, the AI ​​can connect these features to detect a risk of arteriosclerosis.

[1307] A detailed report containing this risk assessment is generated and sent from the server to User A's smartphone. User A can then review the report through the app and take necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist).

[1308] Furthermore, if a serious abnormality is detected, the server sends the analysis results to a specialist for further opinion. A final report including the specialist's opinion is generated and notified to User A again. In this way, the present invention significantly reduces the risk of oversight or misdiagnosis and provides useful health management information to users.

[1309] The processing flow will be explained below.

[1310] Step 1:

[1311] After the user has undergone a health checkup, they launch the dedicated app on their device (smartphone or PC) and log in. They then follow the instructions in the app to begin entering their health checkup results (e.g., blood test values, image data). If necessary, they can take photos of the results or enter the values ​​manually.

[1312] Step 2:

[1313] The device sends the entered health checkup data to the server. A secure communication protocol (e.g., HTTPS) is used over the Internet to ensure data confidentiality.

[1314] Step 3:

[1315] The server receives the health checkup data sent from the device. The received data is stored in a database and a data consistency check is performed. If there is a problem with the data format, it is converted into an appropriate format.

[1316] Step 4:

[1317] The server prepares the stored data to be sent to the analysis engine, where preprocessing such as resizing and noise removal is performed on the image data to improve the accuracy of the analysis.

[1318] Step 5:

[1319] The server launches an analysis engine (deep learning model and multimodal AI), which analyzes correlations between blood test values ​​and image data to detect abnormal patterns and health risks.

[1320] Step 6:

[1321] The server generates a detailed diagnostic report based on the analysis results, including any abnormalities detected, a health risk assessment, and recommended actions (e.g., retesting or consultation with a specialist).

[1322] Step 7:

[1323] The server notifies the user that the analysis results are ready. A push notification is sent to the device with the message "Analysis results are ready. Please check the app."

[1324] Step 8:

[1325] The user checks the notification on their device and launches the dedicated app to view the analysis results. The app displays a detailed diagnostic report, allowing the user to learn specific measures to address health risks.

[1326] Step 9:

[1327] If necessary, the server sends the analysis results to a specialist, who reviews them and provides additional opinions and advice.

[1328] Step 10:

[1329] The server generates a final report including the expert's opinion. The server notifies the user again that the final report is ready, and the user can view the final report through the app.

[1330] Step 11:

[1331] The server provides healthcare support functions, allowing users to receive in-app reminders and advice on daily health management (e.g., drinking more water, improving exercise habits).

[1332] Step 12:

[1333] Users manage their health based on the advice provided, and use the app's support features to implement healthy lifestyle habits and regularly monitor their health.

[1334] Example 1

[1335] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1336] Conventional health checkup data management and analysis systems often make it difficult for users to efficiently input and transmit health checkup data and quickly receive analysis results. Furthermore, due to insufficient accuracy in data analysis and insufficient collaboration with specialists, it is difficult to provide users with reliable health management information. For these reasons, there is a growing need for a system that can detect health risks early and suggest appropriate countermeasures.

[1337] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1338] In this invention, the server includes a means for a user to input health checkup data, a means for transmitting the input health checkup data from a terminal to the server, and a means for storing the health checkup data received by the server and checking the format consistency. This enables efficient data input, transmission, and consistency check. The server also includes a means for analyzing the stored data using deep learning and multimodal AI technologies, a means for generating a detailed diagnostic report for the user based on the analysis results generated by the analysis means and notifying the user via push notification or email, a means for sharing the analysis results with a specialist as needed and incorporating the specialist's opinion into the final report, and a means for notifying the user of the final report and making it viewable via an application. This allows the user to receive analysis results quickly and comprehensively, enabling them to obtain reliable diagnostic information that reflects the specialist's opinion.

[1339] "User" means an individual who uses the system to input health checkup data and receive analysis results.

[1340] "Health checkup data" refers to health-related information such as blood test results and image data entered by the user.

[1341] "Input means" refers to the method or device by which users input their health checkup data into the system. Specifically, this applies to dedicated applications on smartphones or PCs.

[1342] A "terminal" is a device that a user uses to access the system, such as a smartphone or PC.

[1343] A "server" is the central computer device of the entire system that receives, stores, and analyzes data sent from terminals.

[1344] "Transmission means" refers to a method or device for transmitting the entered medical examination data from the terminal to the server. Specifically, this refers to a secure communication method via the Internet (e.g., HTTPS).

[1345] The "receiving means" refers to a method or device by which the server receives the medical examination data transmitted from the terminal.

[1346] The "storage means" refers to a method or device for storing the received medical examination data in the server. Specifically, this corresponds to a database.

[1347] "Consistency check means" refers to a method or device for verifying whether the received data conforms to a specified format, whether required fields have been entered, and so on.

[1348] "Deep learning technology" is a technique that uses large-scale neural networks to analyze data.

[1349] "Multimodal AI technology" refers to technology that combines and analyzes different types of data (e.g., text data and image data) to draw comprehensive conclusions.

[1350] "Analysis means" refers to a method or device for analyzing health checkup data using deep learning technology and multimodal AI technology.

[1351] A "diagnostic report" is a report generated based on the analysis results, which includes detected abnormalities, an assessment of health risks, and recommended actions to take.

[1352] "Notification means" refers to a method or device for notifying the user of the completion of the analysis results preparation or other information from the server. Specifically, this applies to push notifications and emails.

[1353] "Expert" means a medical professional with medical expertise who provides additional input on the results of the analysis.

[1354] "Sharing means" refers to a method or device for sharing analysis results with specialists.

[1355] "Final report" refers to the final diagnostic report that reflects the opinion of the specialist.

[1356] An "application" is software that allows users to input data and view analysis results.

[1357] This invention is a system that allows users to input health checkup data and receive analysis results. This system works in cooperation with three entities: the user, the terminal, and the server, to detect health risks early and propose appropriate countermeasures.

[1358] User interaction

[1359] Users launch the dedicated application on their smartphone or PC and log in. They follow the app's instructions to take pictures of their health checkup results and enter data such as blood test values. The dedicated application then imports images from the camera roll and provides an interface for manually entering numerical data.

[1360] Sending data by the device

[1361] The device formats the input data and sends it to the server using a secure communication method (e.g., HTTPS). Specifically, the captured image and input numerical data are combined into JSON format and sent as an HTTP POST request. This ensures reliable data transmission.

[1362] Receiving and storing data by the server

[1363] The server receives the health checkup data sent from the terminal. The received data is first checked for format consistency before being saved in the database. Specifically, it verifies whether the data conforms to the specified format and whether required fields have been entered. Data that passes this verification is saved in the database, and if there is any inconsistency, an error message is returned to the user.

[1364] Data analysis by the server

[1365] The stored data is analyzed using deep learning algorithms and multimodal AI technology. Deep learning technology uses large-scale neural networks to learn features and patterns that appear in blood test values ​​and image data and detect abnormalities. Multimodal AI technology combines and analyzes different data sources (blood test values ​​and image data) to provide a comprehensive health assessment.

[1366] Generation and notification of analysis results

[1367] Based on the analysis results, a detailed diagnostic report is generated for the user, including any detected abnormalities, an assessment of health risks, and recommended actions (e.g., retesting or consulting a specialist). Once the analysis results are ready, the server notifies the user via push notification or email.

[1368] Collaboration with specialists

[1369] If any significant abnormalities are detected, the server sends the analysis results to a specialist for further opinion. A final report incorporating the specialist's opinion is generated and notified to the user. The user can view this final report through the application.

[1370] Specific examples

[1371] Consider the case where User A, a 45-year-old male, undergoes a health checkup. User A uses a dedicated smartphone app to take blood test results and chest X-ray images obtained after the checkup and enter them into the app. The app then sends this data to the server.

[1372] The server stores the data and begins analyzing it using a deep learning system. For example, if a blood test shows high cholesterol and an X-ray shows mild abnormalities, the AI ​​can connect these features to detect a risk of arteriosclerosis.

[1373] A detailed report containing this risk assessment is generated and sent from the server to User A's smartphone. User A can then review the report through the app and take necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist).

[1374] Furthermore, if a serious abnormality is detected, the server sends the analysis results to a specialist for further opinion. A final report including the specialist's opinion is generated and notified to User A again. In this way, the present invention significantly reduces the risk of oversight or misdiagnosis and provides useful health management information to users.

[1375] Prompt Sentence Examples

[1376] "Please create a specific scenario in which a 45-year-old male user enters the results of a health check into a smartphone app and receives the results of an analysis by AI. The analysis will use deep learning technology to generate a detailed report including a risk assessment based on high cholesterol and abnormal chest X-ray findings."

[1377] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1378] Step 1:

[1379] The user launches the dedicated application on their smartphone or PC and logs in.

[1380] Input: User account information (e.g., user ID, password)

[1381] Output: User's dashboard on successful login, error message on failure

[1382] The user launches the dedicated application and enters their account information to log in. If the login is successful, the user's dashboard will be displayed and they can proceed to the next step.

[1383] Step 2:

[1384] Users take pictures of their health checkup results and enter data such as blood test results into a dedicated app.

[1385] Input: Images of health checkup results, blood test results, and other data

[1386] Output: Formatted health check data (e.g., JSON format)

[1387] The user follows the app's instructions to take a photo of the health checkup results and enter data such as blood test values, which are then formatted into the appropriate format (e.g., JSON) within the app.

[1388] Step 3:

[1389] The terminal transmits the input health checkup data to the server.

[1390] Input: Formatted health check data (e.g., JSON format)

[1391] Output: Data transmission result to the server (success or failure message)

[1392] The device sends the formatted health check data to the server using HTTPS, sending the data as an HTTP POST request and receiving a success or failure result.

[1393] Step 4:

[1394] The server receives the health checkup data sent from the terminal and stores it in a database.

[1395] Input: Health checkup data sent from the device

[1396] Output: Save result to database (success or failure message)

[1397] The server receives the data sent from the terminal and first checks the format for consistency. If this check is successful, it stores the data in the database. If there is a mismatch, it returns an error message to the terminal.

[1398] Step 5:

[1399] The server analyzes the stored health checkup data using deep learning algorithms and multimodal AI technology.

[1400] Input: Medical examination data stored in a database

[1401] Output: Analysis results (abnormal locations, risk assessment, etc.)

[1402] The server applies deep learning technology to the stored data to detect abnormalities in blood test values ​​and image data, and uses multimodal AI technology to analyze correlations between different data sources to perform a comprehensive health risk assessment.

[1403] Step 6:

[1404] Based on the analysis results, a detailed diagnostic report is generated and notified to the user.

[1405] Input: Analysis results

[1406] Output: Diagnostic report, notification (push notification or email)

[1407] The server generates a diagnostic report based on the analysis results, which includes detected abnormalities, a health risk assessment, and recommended actions to take. Based on the report, the server sends a notification to the user's device.

[1408] Step 7:

[1409] If necessary, the analysis results will be sent to a specialist for additional opinion.

[1410] Input: Analysis results

[1411] Output: Expert feedback, final report

[1412] If any significant abnormalities are detected, the server sends the analysis results to a specialist for further opinion, generates a final report incorporating the specialist's opinion, and notifies the user again.

[1413] Step 8:

[1414] The user reviews the final report and takes any recommended actions.

[1415] Input: Final Report

[1416] Output: User behavior change (e.g., diet change, increased exercise, specialist consultation)

[1417] Users receive a notification and can launch the app to view the final report, which can then be used to take specific measures such as improving their diet, increasing exercise, or consulting a specialist.

[1418] (Application example 1)

[1419] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1420] Health management has become an extremely important issue in modern life. However, due to the lack of individual health risk assessments based on health checkup results and the lack of solutions, it is difficult for users to respond appropriately. Furthermore, it is difficult to recommend and deliver dietary recommendations based on health risks with current technology.

[1421] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1422] In this invention, the server includes a means for inputting a user's health checkup data, a means for transmitting the input health checkup data to the server, and a means for storing the health checkup data received by the server. This enables an analysis means including deep learning technology and multimodal AI technology for analyzing the stored health checkup data. The system also includes a means for providing the user with individual health risks and solutions based on the analysis results generated by the analysis means, and a means for generating dietary recommendations based on the health data and transmitting them to a delivery device. This enables health risk assessment and customized dietary suggestions and delivery for the user, providing more comprehensive health management.

[1423] "User" refers to a person who inputs health checkup data and receives analysis results.

[1424] "Health checkup data" refers to information such as blood test results and image data obtained by a user as a result of a health checkup.

[1425] "Input means" refers to a method or device that allows a user to input health checkup data into the system.

[1426] "Server" refers to the computer system that receives, stores, and analyzes health checkup data sent by users.

[1427] The "storing means" refers to a method or device for storing the received medical examination data in a storage device such as a database.

[1428] "Analysis means" refers to a method or device for analyzing health checkup data and conducting health risk assessments using deep learning technology and multimodal AI technology.

[1429] "Deep learning technology" is an advanced machine learning technology that uses artificial neural networks to learn the characteristics and patterns of data.

[1430] "Multimodal AI technology" refers to artificial intelligence technology that integrates and analyzes multiple types of data (e.g., blood test values ​​and image data).

[1431] "Means based on the analysis results" refers to a method or device that utilizes the analysis results obtained by the analysis means to provide information or suggestions to the user.

[1432] "Dietary recommendations based on health data" means proposing the optimal meal menu for each individual based on the user's health checkup data.

[1433] A "delivery device" is a robot or machine that delivers meals to users.

[1434] The present invention is a system for assessing individual health risks based on health checkup data, proposing healthy meals to users based on the results, and providing the meals via a delivery device. Specific embodiments of the system are described below.

[1435] Hardware and Software Configuration

[1436] 1. User Device

[1437] Hardware: Smartphone, tablet, or personal computer

[1438] Software: Dedicated application, camera, internet connection

[1439] 2. Server

[1440] Hardware: High-performance server computer

[1441] Software: Database management systems, deep learning algorithms (e.g., Keras), multimodal AI technology

[1442] 3. Delivery Device

[1443] Hardware: Food delivery robot

[1444] Software: Route optimization algorithm, communication module

[1445] System processing flow

[1446] User operations

[1447] 1. Enter the data:

[1448] After undergoing a health checkup, the user launches a dedicated application on their device and logs in. Following the instructions, they take an image of the health checkup results and enter blood test results and other diagnostic data.

[1449] 2. Data transmission:

[1450] The entered health checkup data is sent from the terminal to a server via the Internet using a secure protocol (e.g., HTTPS).

[1451] Server Processing

[1452] 1. Data Receipt and Storage:

[1453] The server receives the health checkup data sent from the terminal, first checks whether the data is in the correct format, and then stores it in a database.

[1454] 2. Data Analysis:

[1455] The stored data is analyzed using deep learning and multimodal AI technologies. Deep learning models (e.g., using Keras) are used to learn features that appear in blood test values ​​and image data and detect abnormalities. Multimodal AI technology integrates information from multiple data sources (e.g., blood test values ​​and image data) to perform more accurate analysis.

[1456] 3. Results generation and notification:

[1457] Based on the analysis results, a detailed report is automatically generated, including a health risk assessment, any abnormalities detected, and recommended actions (e.g., dietary changes, exercise recommendations). Based on this report, optimal meal menu recommendations are also generated.

[1458] 4. Send to delivery device:

[1459] A meal menu personalized for the user is sent to the delivery device, which then delivers the meal to the user's location via the optimal route.

[1460] Specific examples

[1461] example:

[1462] User A, a 45-year-old man, undergoes a health check and enters the results into a smartphone app. When his blood test results and chest X-ray image are sent to the server, deep learning technology begins analysis. If the blood test results indicate high cholesterol and the X-ray image shows abnormalities, the AI ​​will link these features and detect a risk of arteriosclerosis.

[1463] A detailed report including this risk assessment is generated, and recommended meals such as "low-fat meals" and "high-fiber meals" are suggested. The server sends this information to a food delivery robot, which then delivers healthy meals to User A via the optimal route. The process can be initiated with a prompt that reads, "Your blood test results indicate high cholesterol, and your X-rays have revealed abnormalities."

[1464] This system allows users to easily manage their health at home, offering customized meal suggestions and prompt delivery, which helps users maintain their health and reduce medical costs.

[1465] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1466] Step 1:

[1467] After the health check, the user launches a dedicated application and logs in. The user takes an image of the health check results and inputs data such as blood test values. The application receives this input data. The input data is received in the form of image data and numerical data. The user's device formats this data appropriately and generates a prompt to proceed to the next step.

[1468] Step 2:

[1469] The terminal sends the entered health checkup data to the server. This communication uses HTTPS to protect the confidentiality of the data. The data sent includes numerical data of blood test values ​​and health checkup images. The terminal notifies the user that the data has been sent successfully.

[1470] Step 3:

[1471] The server receives the health checkup data sent from the terminal. The received data is first stored in a database. A consistency check is performed on the stored data, and the data format is converted as necessary. After the data format conversion on the server side, the data is in a format that can be processed by the analysis system.

[1472] Step 4:

[1473] The server analyzes the stored health checkup data. Deep learning technology (e.g., Keras) is used to learn the characteristics and patterns of blood test values ​​and image data and detect abnormalities. Multimodal AI technology is used to analyze the correlation between blood test values ​​and image data and perform a comprehensive health risk assessment. The input data is passed to an analysis model, which generates analysis results.

[1474] Step 5:

[1475] Based on the analysis results, the server generates a detailed health risk assessment report for the user, which includes details of the detected abnormalities, a health risk assessment, and recommended measures (e.g., dietary changes and exercise recommendations).The analysis results also simultaneously generate recommendations for optimal meal menus based on the health data.

[1476] Step 6:

[1477] The server sends the generated report and meal recommendations to the delivery device using an appropriate communication protocol (e.g., MQTT, HTTP). The delivery device uses the received information to prepare meals optimal for the user's health condition and schedules the delivery. After the data is sent, the server confirms that the delivery was successfully initiated and notifies the user.

[1478] Step 7:

[1479] The delivery device delivers the meal to the user via the optimal route based on the received meal menu. The delivery device uses GPS and route optimization algorithms to reach the user efficiently. After delivery, the user can confirm that the delivery has been completed through the application and provide any necessary feedback to help improve the accuracy of the entire system.

[1480] The above is the specific processing flow of the system that realizes this application example, and clarifies what operations are performed at each step and what output data is generated from what input data.

[1481] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1482] This invention is a system that allows users to input and submit health checkup data, receive analysis results based on that data, and also combines it with an emotion engine that recognizes the user's emotional state. This system makes it possible to provide more personalized health risk assessments and solutions. The specific processing details for each of the users, terminals, and servers are explained below.

[1483] User interaction

[1484] 1. Enter the data:

[1485] After the diagnosis, the user launches a dedicated application on their device (smartphone or PC) and logs in.

[1486] Follow the app's instructions to enter your health checkup results (e.g., blood test results, image data). If necessary, take a photo of the results or enter the values ​​manually.

[1487] It also enables voice input and facial expression recognition functions to capture data on the user's voice and facial expressions.

[1488] 2. Data transmission:

[1489] The entered health checkup data and emotion data are sent from the device to a server using a secure communication protocol (e.g., SSL / TLS) over the Internet to ensure data confidentiality.

[1490] Server processing

[1491] 1. Data Receipt and Storage:

[1492] The server receives the health check data and emotion data sent from the device. The received data is stored in a database and a data integrity check is performed. If there is a problem with the data format, it is converted into an appropriate format.

[1493] 2. Data Analysis:

[1494] The data stored on the server is sent to the analysis engine, where preprocessing such as resizing and noise removal is performed on the image data to improve the accuracy of the analysis.

[1495] Using deep learning models and multimodal AI technology, the system analyzes correlations between blood test values ​​and image data to detect abnormal patterns and health risks.

[1496] 3. Emotional Data Analysis:

[1497] The server uses an emotion engine to analyze the user's voice and facial expression data, taking into account the user's recognized emotional state (e.g., stress, anxiety, relief) and adjusts the health risk assessment accordingly.

[1498] 4. Result generation and notification:

[1499] Based on the analysis results, a detailed diagnostic report is generated for the user, including detected abnormalities, a health risk assessment, and recommended actions (e.g., retesting or seeing a specialist) based on the user's emotional state.

[1500] The server notifies the user when the analysis results are ready.

[1501] 5. Collaboration with specialists:

[1502] If necessary, the analysis results are sent to a specialist for further comment, and a final report incorporating the specialist's feedback is generated.

[1503] The final report will be notified to the user again and made available for viewing through the app.

[1504] Specific examples

[1505] As a specific example, consider the case of User A, a 45-year-old male, after undergoing a health checkup. User A uses a dedicated smartphone app to take blood test results and chest X-ray images obtained after the checkup and enter them into the app. User A also has the app recognize voice comments and facial expressions to record his emotions regarding the diagnosis results (e.g., anxiety, stress).

[1506] The server receives this data and begins analyzing it using a deep learning system. For example, if a blood test shows high cholesterol and an X-ray shows mild abnormalities, the AI ​​will connect these features and detect a risk of arteriosclerosis. At the same time, the emotion engine will recognize that User A is experiencing high levels of stress.

[1507] A detailed report including this risk assessment and emotional state is generated and sent from the server to User A's smartphone. User A can review this report through the app and determine necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist). In addition, if any serious abnormalities are detected, the analysis results are sent to a specialist, and a final report incorporating the specialist's opinion is generated and notified to User A again.

[1508] The present invention provides a more accurate and personalized health risk assessment by taking into account the user's emotional state, significantly reducing the risk of oversight or misdiagnosis, and providing comprehensive health management information that allows users to take measures with greater peace of mind.

[1509] The processing flow will be explained below.

[1510] Step 1:

[1511] After the user has undergone a health checkup, they launch a dedicated app on their device (smartphone or PC) and log in. They follow the instructions within the app and begin inputting their health checkup results (e.g., blood test values, image data). If necessary, they can take a photo of the results or manually enter the values. They can also turn on voice input and facial expression recognition functions to capture data on the user's voice and facial expressions.

[1512] Step 2:

[1513] The device sends the entered health check data and emotion data to the server. The data is transmitted via the Internet using a secure communication protocol (e.g., HTTPS) to ensure confidentiality.

[1514] Step 3:

[1515] The server receives the health check data and emotion data sent from the device. The received data is stored in a database and a data integrity check is performed. If there is a problem with the data format, it is converted into an appropriate format.

[1516] Step 4:

[1517] The server prepares the stored data to be sent to the analysis engine, where preprocessing such as resizing and noise removal is performed on the image data to improve the accuracy of the analysis.

[1518] Step 5:

[1519] The server launches an analysis engine (deep learning model and multimodal AI), which analyzes correlations between blood test values ​​and image data to detect abnormal patterns and health risks.

[1520] Step 6:

[1521] The server analyzes the user's voice data and facial expression data using an emotion engine. The emotion engine recognizes the user's emotional state (e.g., stress, anxiety, relief) and reflects it in the analysis results.

[1522] Step 7:

[1523] The server generates a detailed diagnostic report based on the analysis results and emotional state, including any abnormalities detected, a health risk assessment, and recommended actions (e.g., retesting or consultation with a specialist).

[1524] Step 8:

[1525] The server notifies the user that the analysis results are ready. A push notification is sent to the device with the message "Analysis results are ready. Please check the app."

[1526] Step 9:

[1527] The user checks the notification on their device and launches the dedicated app to view the analysis results. The app displays a detailed diagnostic report, allowing the user to learn specific measures to address health risks.

[1528] Step 10:

[1529] If necessary, the server sends the analysis results to a specialist, who reviews them and provides additional opinions and advice.

[1530] Step 11:

[1531] The server generates a final report including the expert's opinion. The server notifies the user again that the final report is ready, and the user can view the final report through the app.

[1532] Step 12:

[1533] The server provides healthcare support functions, allowing users to receive in-app reminders and advice on daily health management (e.g., drinking more water, improving exercise habits).

[1534] Step 13:

[1535] Users manage their health based on the advice provided, and use the app's support features to implement healthy lifestyle habits and regularly monitor their health.

[1536] Example 2

[1537] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1538] Conventional health checkup data analysis systems provide analysis results without considering the user's emotional state, which means they are unable to reflect the user's psychological factors.In addition, it is difficult to provide users with more reliable information because it is difficult to perform detailed analysis of diagnosis results or create reports that incorporate the opinions of specialists.

[1539] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1540] In this invention, the server includes means for inputting a user's health checkup data and emotional data, means for transmitting the input health checkup data and emotional data to the server, means for storing and preprocessing the health checkup data and emotional data received by the server and analyzing them using deep learning and multimodal AI technologies, means for providing individual health risks and solutions that take the user's emotional state into consideration based on the analysis results generated by the analysis means and means for notifying the user of the analysis results, and means for transmitting the analysis results to a specialist, if necessary, and generating and notifying the specialist of a final report including the specialist's opinion. This makes it possible to provide more accurate and individualized health risk assessments and solutions that take the user's emotional state into consideration.

[1541] A "user" is an individual who uses the system to input medical and emotional data.

[1542] "Health checkup data" refers to medical data used to evaluate a user's health condition, such as blood test results and image data.

[1543] "Emotional data" is data obtained through a user's voice and facial expressions that indicates the user's emotional state (e.g., stress, anxiety, relief).

[1544] A "server" is a computer system that receives, stores, pre-processes, analyzes, and notifies users of data submitted by the server.

[1545] "Preprocessing" refers to processing for improving the quality of received data, and includes, for example, noise removal and resizing of image data.

[1546] "Deep learning technology" is an advanced machine learning technology that uses neural networks to analyze data and recognize patterns.

[1547] "Multimodal AI technology" is an artificial intelligence technology that integrates and analyzes multiple data formats (e.g., numerical data, image data, audio data).

[1548] "Analysis means" refers to a means for analyzing stored health checkup data and emotion data using deep learning technology and multimodal AI technology.

[1549] "Analysis results" are results generated by the analysis means that indicate the user's health risks and abnormality patterns.

[1550] "Individual Health Risk" is a risk assessment based on a user's particular health condition and indicates a health problem specific to the user.

[1551] "Solutions" are methods or measures for reducing or improving health risks that are provided to users based on the analysis results.

[1552] A "specialist" is a medical professional with advanced knowledge and experience in the medical field, and is responsible for providing expert opinions on the analysis results.

[1553] "Final Report" means a detailed medical examination report provided to the user, including analysis results and expert opinions.

[1554] "Notification" is the act of informing the user that the analysis results or final report are ready.

[1555] This invention is a system that provides comprehensive health risk assessment and solutions by allowing users to input health checkup data and emotional data and send them to a server.The system uses deep learning and multimodal AI technologies to analyze the health checkup data and emotional data and provide individual countermeasures based on the analysis results.

[1556] User interaction

[1557] Users launch a dedicated application on their smartphone or computer and log in. Next, they follow the app's instructions to enter their health checkup results. This input method includes manually entering numerical data for blood test results and uploading image data of the checkup results. Voice input and facial expression recognition functions are also enabled to capture the user's emotional state.

[1558] Sending data

[1559] When the user presses the "Send" button, the entered health checkup data and emotion data are sent from the device to the server. This transmission is securely transmitted using the SSL / TLS protocol.

[1560] Server processing

[1561] The server stores the received data in a database and checks its integrity. If the data has defects, it is converted to a standard format. Next, the image data is preprocessed, such as noise removal and resizing. Libraries such as OpenCV can be used for this.

[1562] Once the data preprocessing is complete, the server analyzes the data using deep learning techniques (e.g., TensorFlow, PyTorch) and multimodal AI techniques. Specifically, it analyzes correlations between blood test values ​​and image data to detect abnormal patterns. It also evaluates the user's emotional state (e.g., stress, anxiety, relief) through analysis of voice and facial expression data.

[1563] Analysis results and notifications

[1564] Once the analysis results are generated, a detailed diagnostic report is created for the user, including any detected health risks, details of any abnormalities, and recommended actions (e.g., retesting or seeing a specialist) based on the user's emotional state. The server then notifies the user of the diagnostic report, typically via push notification or email.

[1565] Collaboration with specialists

[1566] If necessary, the server sends the analysis results to a specialist for further comment. A final report incorporating the specialist's feedback is generated and sent back to the user, allowing them to receive a more detailed and reliable diagnosis and treatment plan.

[1567] Specific examples

[1568] As a concrete example, consider the case where a 45-year-old male user undergoes a health checkup. Using a dedicated app, the user takes a blood test result and a chest X-ray image and inputs them into the app. The user also comments via voice on his / her feelings about the diagnosis (e.g., anxiety, stress), and facial expression data is also captured. The server receives this data and analyzes it using deep learning and multimodal AI technologies.

[1569] Through analysis, for example, high cholesterol blood test results and minor abnormalities in a chest X-ray can be detected as indicators of atherosclerosis risk. At the same time, high stress levels can be identified from emotional data. A detailed report that takes into account the analysis results and emotional state is provided to the user, suggesting necessary measures (e.g., dietary changes, recommended exercise, or consultation with a specialist). If serious abnormalities are found, the analysis results are sent to a specialist, and the user is notified of a final report including the specialist's opinion.

[1570] Prompt Sentence Examples

[1571] "Analyze the blood test results and chest X-ray images of a 45-year-old man after he undergoes a health checkup to assess his risk of arteriosclerosis. Also, take into account the user's emotional state (e.g., anxiety, stress) in response to the diagnosis, and suggest appropriate countermeasures."

[1572] The system will enable accurate and personalized health risk assessment and solutions that also take into account the user's emotional state.

[1573] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1574] Step 1: Entering User Data

[1575] How it works: The user launches a dedicated application on their smartphone or computer and logs in.

[1576] Input: Health checkup results (e.g., blood test numerical data, image data), voice data, facial expression data

[1577] Specific operation: The app displays a health check result input form. The user enters numerical data into the form and takes and uploads image data using the camera. The voice input and facial expression recognition functions are enabled, and the app captures voice comments and facial expression data in real time.

[1578] Output: Health check result data and emotion data are input into the app.

[1579] Step 2: Sending user data

[1580] Action: The user presses the "Submit" button, which sends the entered data to the server.

[1581] Input: Health check result data and emotion data

[1582] How it works: The app uses the SSL / TLS protocol to send health checkup data and emotion data to the server via secure communication.

[1583] Output: Health and emotion data are transmitted securely.

[1584] Step 3: Receiving and storing data on the server

[1585] How it works: The server receives the data and stores it in a database.

[1586] Input: Health check data and emotion data

[1587] What happens: The server checks the integrity of the data and converts any abnormal formatting into a standard format, for example, standardizing date and number formats.

[1588] Output: The integrity-checked health and emotion data are stored in a database.

[1589] Step 4: Data Preprocessing on the Server

[1590] Action: The server preprocesses the received data.

[1591] Input: Medical examination data (especially image data)

[1592] Specific operation: Uses the OpenCV library to remove noise and resize image data. Specifically, it removes noise using a Gaussian filter and unifies the image size.

[1593] Output: Preprocessed image data is generated.

[1594] Step 5: Server Data Analysis

[1595] How it works: The server uses deep learning and multimodal AI techniques to analyze health checkup data.

[1596] Input: Preprocessed health checkup data (blood test results, processed image data), emotion data

[1597] How it works: Using TensorFlow and PyTorch, the system analyzes the correlation between blood test values ​​and image data. For example, if blood test values ​​indicate high cholesterol and the image shows an abnormality, the system can link these to detect the risk of arteriosclerosis.

[1598] Output: Health risk assessment results and abnormal patterns are generated.

[1599] Step 6: Emotional Data Analysis on the Server

[1600] How it works: The server uses the emotion engine to analyze the emotion data.

[1601] Input: User's voice and facial expression data

[1602] How it works: Voice data is subjected to phonemic analysis, and facial expression data is analyzed using facial expression recognition algorithms (e.g., FaceNet). For example, stress or anxiety can be detected from the user's tone of voice and facial expressions.

[1603] Output: The user's emotional state assessment result is generated.

[1604] Step 7: Server result generation and notification

[1605] How it works: The server generates a detailed diagnostic report based on the analysis results and notifies the user.

[1606] Input: Health risk assessment results, abnormal patterns, emotional state assessment results

[1607] What it does: Creates a diagnostic report for the user based on the analysis results, including recommended actions based on detected anomalies, health risks, and emotional state. Sends a push notification to the user's device and allows them to view the report through the app.

[1608] Output: Detailed diagnostic reports and notifications

[1609] Step 8: Collaborate with a server specialist

[1610] Actions: If necessary, the server sends the analysis results to a specialist for further opinion.

[1611] Input: Analysis results

[1612] What it does: Receives feedback from the specialist and incorporates it into the final report, which is then communicated back to the user and made available for review through the app.

[1613] Output: Final report and notification including expert opinion

[1614] At each step, specific data processing and calculations are carried out, and the results are passed on to the next step, providing users with comprehensive and accurate health risk assessments and countermeasures.

[1615] (Application example 2)

[1616] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1617] In modern society, there are many systems that assess individual health risks and provide solutions, but few take the user's emotional state into account. Furthermore, most systems do not provide personalized meal plans that combine health and emotional data. Therefore, there is a need for a system that enables more effective health management by providing appropriate meal plans based on the user's health status.

[1618] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting a user's health checkup data and emotional data; means for transmitting the input health checkup data and emotional data to the server; means for storing the health checkup data and emotional data received by the server; analysis means including deep learning technology and multimodal AI technology for analyzing the stored health checkup data and emotional data; and means for providing the user with individual health risks, solutions, and a personalized meal plan based on the analysis results generated by the analysis means. This enables more accurate and personalized health risk assessments and solutions, as well as the provision of appropriate meal plans, based on the user's health and emotional state.

[1619] "User's health checkup data" refers to numerical data and image data obtained as a result of a user's health checkup, such as blood test results and X-ray images.

[1620] "Emotion data" is data that represents the user's emotional state, obtained from the user's voice input and facial expression data.

[1621] "Server" refers to the central system for receiving, storing and analyzing data sent from the User's device.

[1622] "Deep learning technology" is a technology that uses artificial neural networks to learn and analyze patterns and features from large amounts of data.

[1623] "Multimodal AI technology" is an artificial intelligence technology that integrates multiple different types of data (e.g., text data, audio data, image data), correlates them, and analyzes them.

[1624] "Analysis means" refers to a means for analyzing health checkup data and emotion data using deep learning technology and multimodal AI technology to identify abnormal patterns and health risks.

[1625] "Individual health risk" refers to the health risk that differs for each user and is assessed based on each user's health checkup data and emotional data.

[1626] "Solution" refers to a recommended measure or treatment for a health risk identified by the analytical means.

[1627] A "personalized meal plan" is a meal suggestion created based on a user's health and emotional data, taking into account their individual health and emotional state.

[1628] A "specialist" refers to a medical professional with advanced expertise in medicine and health.

[1629] The "final report" is a report that includes detailed diagnostic results and recommended measures, based on the analytical results obtained by the analytical means and the opinions of specialists.

[1630] The present invention is a system that analyzes a user's health checkup data and emotional data to provide an individualized health risk assessment and personalized meal plan. This system aims to improve the efficiency of health management and provide more accurate countermeasures. The overall configuration of the system and specific processing are described below.

[1631] System configuration

[1632] 1. User's Device

[1633] Hardware: Smartphones, tablets, computers

[1634] Software: Dedicated application, camera, microphone

[1635] Users can launch a dedicated application and input health checkup data (e.g., blood test results, image data), and can also use the application to collect emotional data from facial expressions and voice.

[1636] 2. Server System

[1637] Hardware: Server, GPU (Graphics Processing Unit)

[1638] Software: Database management systems (e.g., MySQL, PostgreSQL), deep learning analysis engines (e.g., TensorFlow), sentiment analysis engines (e.g., OpenAI)

[1639] It receives, stores, and analyzes data sent by users, including checking the integrity of the received data, preprocessing the data, and analyzing the data using deep learning technology and a sentiment analysis engine.

[1640] Processing flow

[1641] 1. Enter and submit data

[1642] Users input their health checkup data and emotion data through a dedicated application and send it to the server using a secure communication protocol (e.g., SSL / TLS).

[1643] 2. Receipt and storage of data

[1644] The server receives the data sent by the user and stores it in a database, where it undergoes consistency checks and format conversion.

[1645] 3. Data Analysis

[1646] The server sends the stored data to a deep learning analysis engine, which preprocesses the health check data and image data. The emotion analysis engine then analyzes the emotion data. Deep learning and multimodal AI technologies are integrated to assess health risks.

[1647] 4. Generating and notifying results

[1648] Based on the analysis results, the system provides users with an individual health risk assessment and solutions, and generates a personalized diet plan. The analysis results and diet plan are sent to the user's device, and a final report incorporating the opinion of a specialist may also be provided.

[1649] Specific examples

[1650] For example, consider the case of a 45-year-old male user undergoing a health checkup. The user obtains the results of a blood test and takes a photo of a chest X-ray with their smartphone, which they then enter into the app. At the same time, they record their emotions about the results (e.g., anxiety or stress) using audio and facial recording. This data is sent to a server and analyzed by a deep learning system. For example, if high cholesterol is detected and there are abnormalities in the X-ray image, the risk of arteriosclerosis is identified. At the same time, it is also analyzed that the user is in a state of high stress. Based on this, a meal plan with a relaxing effect, mainly consisting of low-fat foods, is suggested.

[1651] Prompt Sentence Examples

[1652] "45-year-old male, recent medical checkup revealed high cholesterol. High stress levels. Can you suggest a low-fat, relaxing diet plan?"

[1653] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1654] Step 1:

[1655] Users enter their health checkup data (blood test results and image data) through a dedicated application, and then record their emotional data. This data is collected using a specific input screen within the app. For example, they can use the camera function to take a photo of their checkup results, and record voice comments and facial expressions as emotional data. The entered data is sent to the server using a secure communication protocol (e.g., SSL / TLS).

[1656] Input: health check data, voice data, facial expression data

[1657] Output: User data sent to the server

[1658] Step 2:

[1659] The server receives the health checkup data and emotion data sent by the user. The received data is stored in a database, where it is checked for consistency and formatted for conversion. This allows inaccurate data or data with different formats to be stored in a unified format.

[1660] Input: Submitted user data

[1661] Output: Consistency-checked data stored in the database

[1662] Step 3:

[1663] The server uses a deep learning analysis engine to preprocess the health checkup data (blood test results and image data) based on the stored data. Specifically, the image is resized and noise is removed. Deep learning technology then begins analysis to assess health risks.

[1664] Input: Integrity checked data

[1665] Output: Preprocessed data

[1666] Step 4:

[1667] The server sends the stored emotional data to an emotion analysis engine for analysis. The user's emotional state (e.g., stress, anxiety, relief) is extracted from the voice and facial expression data. This allows the user's current psychological state to be understood.

[1668] Input: Emotion data that has been integrity checked

[1669] Output: Parsed emotional state data

[1670] Step 5:

[1671] The server combines the results from the deep learning analysis engine and the sentiment analysis engine to assess the user's health risks and generate a personalized diet plan based on this. For example, if a high cholesterol risk is identified, a low-fat diet plan will be suggested.

[1672] Input: Health risk analysis results, emotional state data

[1673] Output: Integrated health risk assessment and meal plan

[1674] Step 6:

[1675] The server notifies the user of the generated analysis results and meal plan. Notifications are made using the notification function within the application or push notifications, and the user can check them through the application.

[1676] Input: Integrated health risk assessment and meal plan

[1677] Output: User notification

[1678] Step 7:

[1679] If necessary, the server will share the generated analysis results with a specialist and generate a final report based on the specialist's opinion. This final report will also be notified to the user and can be viewed through the application.

[1680] Input: Integrated health risk assessment and meal plan

[1681] Output: Final report including expert opinion and notification to user

[1682] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1683] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1684] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1685] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1686] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1687] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1688] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1689] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1690] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1691] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1692] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1693] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1694] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1695] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1696] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1697] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1698] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1699] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1700] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1701] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1702] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1703] The following is further disclosed regarding the above embodiment.

[1704] (Claim 1)

[1705] a means for inputting the user's medical examination data;

[1706] means for transmitting the input health checkup data to a server;

[1707] a means for storing the health checkup data received by the server;

[1708] an analysis means including deep learning technology and multimodal AI technology for analyzing the stored health checkup data;

[1709] A system including a means for providing a user with individual health risks and solutions therefor based on the analysis results generated by the analysis means.

[1710] (Claim 2)

[1711] 2. The system according to claim 1, wherein the analyzing means includes means for analyzing correlation between blood test values ​​and image data.

[1712] (Claim 3)

[1713] The system of claim 1 further comprising means for sharing the analysis results with a specialist and generating a final report including the specialist's opinion.

[1714] "Example 1"

[1715] (Claim 1)

[1716] a means for a user to input medical examination data;

[1717] means for transmitting the input health checkup data from the terminal to a server;

[1718] a means for storing the medical examination data received by the server and checking the consistency of the format;

[1719] A means for analyzing the stored health checkup data using deep learning technology and multimodal AI technology;

[1720] means for generating a detailed diagnostic report for a user based on the analysis results generated by the analysis means and notifying the user of the report by push notification or email;

[1721] A means for sharing the analysis results with a specialist as necessary and reflecting the specialist's opinion in the final report;

[1722] The system includes means for notifying the user of the final report and making it available for viewing through an application.

[1723] (Claim 2)

[1724] The system of claim 1, wherein the deep learning technology detects abnormalities from blood test values ​​and image data, and the multimodal AI technology includes means for analyzing correlations between blood test values ​​and image data.

[1725] (Claim 3)

[1726] 10. The system of claim 1, further comprising means for including an assessment of health risks and recommended actions in a diagnostic report generated based on the analysis results.

[1727] "Application Example 1"

[1728] (Claim 1)

[1729] a means for inputting the user's medical examination data;

[1730] means for transmitting the input health checkup data to a server;

[1731] a means for storing the health checkup data received by the server;

[1732] an analysis means including deep learning technology and multimodal AI technology for analyzing the stored health checkup data;

[1733] A means for providing a user with individual health risks and solutions based on the analysis results generated by the analysis means;

[1734] The system includes a means for generating dietary recommendations according to the health data based on the analysis results and transmitting the recommendations to a delivery device.

[1735] (Claim 2)

[1736] 2. The system according to claim 1, wherein the analyzing means includes means for analyzing correlation between blood test values ​​and image data.

[1737] (Claim 3)

[1738] The system of claim 1 further comprising means for sharing the analysis results with a specialist and generating a final report including the specialist's opinion.

[1739] "Example 2: Combining Emotion Engines"

[1740] (Claim 1)

[1741] a means for inputting user health checkup data and emotional data;

[1742] means for transmitting the input health checkup data and emotion data to a server;

[1743] a means for storing the health checkup data and emotion data received by the server;

[1744] means for preprocessing the stored medical examination data;

[1745] an analysis means including deep learning technology and multimodal AI technology for analyzing the health checkup data and emotion data;

[1746] a means for providing an individual health risk and a solution therefor, taking into consideration the emotional state of the user, based on the analysis results generated by the analysis means, and a means for notifying the user of the analysis results;

[1747] A means to transmit the analysis results to a specialist, if necessary, and generate a final report including the specialist's opinion;

[1748] A system including:

[1749] (Claim 2)

[1750] 2. The system according to claim 1, wherein the analyzing means includes means for analyzing correlation between blood test values ​​and image data, and means for analyzing voice data and facial expression data.

[1751] (Claim 3)

[1752] 10. The system of claim 1, further comprising means for notifying a user of a final report generated based on the analysis results.

[1753] "Application example 2 when combining emotion engines"

[1754] (Claim 1)

[1755] a means for inputting user health checkup data and emotional data;

[1756] means for transmitting the input health checkup data and emotion data to a server;

[1757] a means for storing the health checkup data and emotion data received by the server;

[1758] an analysis means including deep learning technology and multimodal AI technology for analyzing the stored health checkup data and emotion data;

[1759] The system includes a means for providing a user with individual health risks and solutions therefor, as well as a personalized meal plan, based on the analysis results generated by the analysis means.

[1760] (Claim 2)

[1761] 2. The system according to claim 1, wherein the analyzing means includes means for analyzing correlations between blood test values, image data, and emotion data.

[1762] (Claim 3)

[1763] The system of claim 1, further comprising means for sharing the analysis results with a specialist and generating a final report including the specialist's opinion and a personalized diet plan based on the final report. [Explanation of symbols]

[1764] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for inputting the user's medical examination data; means for transmitting the input health checkup data to a server; a means for storing the health checkup data received by the server; an analysis means including deep learning technology and multimodal AI technology for analyzing the stored health checkup data; A system including a means for providing a user with individual health risks and solutions therefor based on the analysis results generated by the analysis means.

2. The system according to claim 1 , wherein the analyzing means includes means for analyzing correlation between blood test values ​​and image data.

3. The system of claim 1 , further comprising means for sharing the analysis results with a specialist and generating a final report including the specialist's opinion.

Images