System

Abstract

A system is provided.SOLUTION: A system comprising: means for uploading a medical examination report; means for receiving biological data from a wearable device of a user; means for receiving a picture of a meal taken by the user; means for collecting daily life data from a smart speaker; means for receiving online shopping and expenditure history; means for analyzing the medical examination report, the biological data, the picture of the meal, the daily life data, and the expenditure history and proposing life insurance and medical insurance optimized for the individual user; and means for optimizing financial planning, investment proposal, and budget allocation for leisure activities of the user based on the analysis result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Health issues are a major challenge in modern society, especially for middle-aged and elderly people. In addition, financial insecurity is a serious problem, and many people face difficulties in selecting appropriate life insurance and medical insurance. Furthermore, there is no clear guideline for how to best spend their limited income, leading to uncertainty about financial planning for retirement. Given this background, there is a need for a system that provides appropriate insurance selection, financial planning, investment suggestions, and budget allocation for leisure activities, taking into account each individual's health condition and lifestyle. [Means for solving the problem]

[0005] To solve the above-mentioned problems, the present invention provides the following means: a system including means for uploading health checkup reports, means for receiving biometric data from a user's wearable device, means for receiving photos of meals taken by the user, means for collecting daily life data from a smart speaker, and means for receiving online shopping and expenditure history. The system also includes means for analyzing this data and proposing life insurance and health insurance tailored to each individual user. The system further includes means for optimizing the user's financial planning, investment proposals, and budget allocation for leisure activities based on the analysis results. The system further includes means for receiving user feedback and adjusting the proposals, and means for accumulating biometric data and daily life data over the long term to continuously learn and update the user's health condition and lifestyle patterns.

[0006] A "health checkup report" is a document that describes the results of a diagnosis of a user's health condition made by a medical institution.

[0007] "Wearable devices" are various electronic devices that are worn by users and are primarily used to collect biometric data.

[0008] "Biometric data" refers to data that indicates the user's physical condition, such as heart rate, number of steps taken, and sleep patterns.

[0009] "Meal photos" are images of the meal contents taken by the user for each meal, and include date and time information.

[0010] A "smart speaker" is a speaker device that has a voice recognition function and can receive voice commands from a user.

[0011] "Daily life data" refers to data about a user's daily life, such as their daily activities, work hours, rest hours, and exercise frequency.

[0012] "Online shopping" is the act of purchasing goods and services over the Internet.

[0013] "Expense history" is data that records what expenses a user has incurred and when.

[0014] "Analysis" is the process of examining collected data in detail to find meaning and patterns.

[0015] "Life insurance" is an insurance contract that pays out benefits when the user dies or meets certain conditions.

[0016] "Medical insurance" is an insurance contract that pays compensation when medical expenses are incurred due to illness or injury.

[0017] "Financial planning" refers to making future economic plans that take into account the balance between the user's income and expenses.

[0018] "Investment proposals" refer to the proposal of appropriate investment methods and asset management plans based on the user's financial situation.

[0019] "Leisure activities" refers to the activities and entertainment that users engage in during their free time.

[0020] A budget is a plan for determining how to spend limited resources.

[0021] "Feedback" refers to the opinions and requests for improvement that users give in response to suggestions.

[0022] "Learning" is the process by which the system uses accumulated data to understand user patterns and tendencies and use this information to help it next time.

[0023] "Update" refers to bringing existing records or models up to date based on new information or data. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0025] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0026] First, the terms used in the following description will be explained.

[0027] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0028] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0029] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0030] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0031] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0032] [First embodiment]

[0033] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0034] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0035] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0036] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0037] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0038] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0039] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0040] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0041] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0042] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0043] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0044] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0045] This invention is a system that integrates health checkup reports collected over multiple years, biometric data from wearable devices, photographic data for each meal, lifestyle data from smart speakers, and online shopping and expenditure history to select life insurance and medical insurance suited to an individual's health condition, as well as financial planning, investment proposals, and budget allocation for leisure activities. This system collects, analyzes, and optimizes data through mutual communication between a server, terminals, and users.

[0046] Program processing explanation

[0047] Data collection

[0048] 1. The device provides an interface to the user, prompting them to upload the health checkup report. The user selects the PDF file and saves it on the device.

[0049] 2. The terminal works in conjunction with the user's wearable device to periodically receive biometric data (heart rate, number of steps, sleep patterns, etc.) and send it to the server.

[0050] 3. The device saves photos of each meal taken by the user and uploads them to the server. The photos contain date and time information.

[0051] 4. The smart speaker collects voice data from the user's daily life (e.g., "How much exercise time did you do today?") and sends it to a server via the device.

[0052] 5. The device periodically collects the user's online shopping and spending history and sends it to the server.

[0053] Data analysis

[0054] 1. The server analyzes the received health checkup report and extracts important health indicators (blood pressure, blood sugar level, cholesterol level, etc.).

[0055] 2. The server analyzes biometric data from the wearable device to understand the user's daily health patterns (heart rate variability, daily steps taken, sleep quality, etc.).

[0056] 3. The server analyzes photos of meals using image recognition technology and evaluates the calories and nutritional balance of each meal.

[0057] 4. The server analyzes the lifestyle data obtained from the smart speaker and evaluates the user's lifestyle patterns and stress levels.

[0058] 5. The server analyzes the user's online shopping history and spending history to evaluate the balance between income and expenditure and identify spending trends.

[0059] Proposal generation and refinement

[0060] 1. The server proposes optimal life and health insurance plans to the user based on the analysis results, including insurance selection to reduce the user's health risks.

[0061] 2. The server will provide financial planning and investment recommendations, taking into account the user's health status and lifestyle, including suitable investment options and financial planning for retirement.

[0062] 3. The server proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[0063] Feedback and Optimization

[0064] 1. The device displays the proposed insurance plan and financial plan to the user and accepts feedback. The user reviews the proposal and provides feedback on any necessary adjustments via the device.

[0065] 2. The server adjusts the proposal based on user feedback and re-proposes the optimal insurance plan and financial plan.

[0066] Specific examples

[0067] For example, if a 50-year-old male user is using this system, it will operate as follows:

[0068] Data collection

[0069] A user uploads last year's health checkup report using a device. Daily heart rate data and step count data are automatically sent from a wearable device to a server. A user takes a photo of their lunch and the photo is uploaded to a server. A smart speaker records that "today's jogging time was 30 minutes" and sends this data to a server. A user's credit card history is periodically uploaded to a server.

[0070] Data analysis

[0071] The server analyzes health checkup reports and extracts important indicators such as blood pressure of 130 / 85. It confirms from wearable device data that the user walks 7,000 steps per day. It analyzes food photos and evaluates the calorie intake, such as 700 calories. It determines from smart speaker data that the user jogs 150 minutes per week. It confirms from credit card history that the user's monthly food expenses are 30,000 yen.

[0072] Proposal generation and refinement

[0073] The server proposes a medical insurance plan tailored to the user's blood pressure risk, provides an exercise plan to encourage jogging, and suggests a monthly budget of 10,000 yen for leisure activities, taking into account food and entertainment costs.

[0074] Feedback and Optimization

[0075] The user reviews the suggestions and provides feedback on what they dislike. The server receives the feedback, adjusts the suggestions, and makes new suggestions to optimize them.

[0076] The processing flow will be explained below.

[0077] Step 1:

[0078] The device prompts the user to upload the health checkup report. The user saves the PDF file of the health checkup report on the device and uploads it.

[0079] Step 2:

[0080] The terminal periodically receives biometric data (heart rate, number of steps, sleep patterns, etc.) from the user's wearable device and transmits this data to a server.

[0081] Step 3:

[0082] The device stores photos of each meal taken by the user and uploads them to a server, with the photos containing date and time information.

[0083] Step 4:

[0084] The smart speaker collects voice data from the user's daily life (e.g., "How much time did you exercise today?") and sends it to the device, which then uploads the data to a server.

[0085] Step 5:

[0086] The terminal periodically collects the user's online shopping and spending history (such as credit card history) and sends it to a server.

[0087] Step 6:

[0088] The server analyzes the received health checkup report using OCR technology and extracts important health indicators such as blood pressure, blood sugar level, and cholesterol level.

[0089] Step 7:

[0090] The server analyzes the biometric data received from the wearable device to understand the user's health patterns, such as heart rate fluctuations, daily steps taken, and sleep quality.

[0091] Step 8:

[0092] The server analyzes photos of meals using image recognition technology, and calculates and evaluates the calories and nutritional balance of each meal.

[0093] Step 9:

[0094] The server analyzes lifestyle data obtained from the smart speaker and evaluates the user's activity patterns and stress levels.

[0095] Step 10:

[0096] The server analyzes the user's online shopping history and spending history, evaluates the balance between income and expenses, and identifies spending trends.

[0097] Step 11:

[0098] The server then uses the analysis results to generate optimal life and health insurance plans for the user, including selecting insurance to mitigate the user's health risks.

[0099] Step 12:

[0100] The server then provides optimal financial planning and investment recommendations based on the user's health status and lifestyle, including investment strategies to ensure financial security in retirement.

[0101] Step 13:

[0102] The server proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[0103] Step 14:

[0104] The terminal displays the insurance plans and financial plans proposed by the server to the user, who can then review the proposals and provide feedback if necessary.

[0105] Step 15:

[0106] The server receives feedback from the user, adjusts the proposal, and again provides the user with an optimized plan.

[0107] Example 1

[0108] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] There is currently no system that can centrally provide optimal life insurance, medical insurance, financial planning, investment proposals, and budget allocation for leisure activities based on diverse health and lifestyle data. There is also a lack of systems that can continuously learn and update a user's health status and lifestyle patterns over the long term. Furthermore, there is a need for a system that can automatically generate optimal proposals from data using generative AI models and provide an approach that meets the user's needs.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0111] In this invention, the server includes means for uploading health checkup reports, means for receiving biometric data from the user's wearable device, means for receiving meal photos taken by the user, means for collecting daily life data from a voice recognition device, means for receiving online shopping and expenditure history, means for analyzing the health checkup reports, biometric data, meal photos, daily life data, and expenditure history to propose life insurance and medical insurance tailored to each individual user, means for optimizing the user's financial plan, investment proposals, and budget allocation for leisure activities based on the analysis results, means for receiving feedback from the user and adjusting the proposals, means for accumulating the biometric data and daily life data over the long term and continuously learning and updating the user's health condition and lifestyle patterns, means for generating proposals based on the data analysis results using a generative AI model, and means for using prompt sentences at each step of the processing flow. This makes it possible to integrate various personal data and provide optimal advice and plans.

[0112] A "health checkup report" is a document that lists the results of a health checkup conducted at a hospital or clinic, and includes health indicators such as blood pressure, blood sugar level, and cholesterol level.

[0113] A "wearable device" is an electronic device that can be worn by a user and has the ability to collect biometric data such as heart rate, number of steps taken, and sleep patterns.

[0114] "Biometric data" refers to data that indicates a person's physiological state, such as heart rate, number of steps, and sleep patterns.

[0115] A "meal photo" is a digital image taken by a user to record the contents of a meal.

[0116] A "voice recognition device" is a device that collects and analyzes voice data and has the function of recognizing voice commands and comments made by users in their daily lives.

[0117] "Online shopping history" is a record of a user's purchasing activities on the Internet, including information such as purchase date, product, and amount.

[0118] "Expense history" is a record of expenses incurred by a user in their daily life, including credit card statements, bank transaction statements, etc.

[0119] "Means of proposing life insurance and medical insurance" refers to the process of analyzing the user's health condition and lifestyle data, selecting the most suitable insurance plan, and offering it to the user.

[0120] "Means for optimizing financial planning and investment proposals" refers to the process of analyzing a user's income, expenses, and asset information and proposing long-term financial and investment plans.

[0121] "Means for optimizing budget allocation for leisure activities" refers to a process that takes into account the user's daily life and spending history and suggests allocating an appropriate budget for leisure activities.

[0122] A "generative AI model" is a model that uses artificial intelligence technology to analyze data and automatically generate optimal suggestions.

[0123] A "prompt" is an instruction that can be input into a generative AI model to perform a specific analysis or suggestion.

[0124] This invention is a system that integrates health checkup reports collected over multiple years, biometric data from wearable devices, photographic data for each meal, lifestyle data from a voice recognition device, and online shopping and expenditure history to select life insurance and medical insurance suited to an individual's health condition, as well as financial planning, investment proposals, and budget allocation for leisure activities. This system collects, analyzes, and optimizes data through mutual communication between a server, terminals, and users.

[0125] Data collection

[0126] Uploading your medical checkup report

[0127] The terminal provides the user with an interface and prompts them to upload their health checkup report. The user selects a PDF file and saves it on the terminal. The terminal then sends this file to the server. Specifically, the user uploads last year's health checkup report.

[0128] Biometric data collection from wearable devices

[0129] The device connects to the user's wearable device and periodically collects biometric data (heart rate, steps, sleep patterns, etc.) via Bluetooth or Wi-Fi. This collected data is then sent from the device to a server. For example, the user's wearable device sends daily heart rate and step count data to the server.

[0130] Upload a meal photo

[0131] The device saves photos of each meal taken by the user and uploads them to the server. The photos are automatically dated and time stamped. The user takes photos of each meal and saves them on the device through the app. For example, the user takes a photo of their lunch and the photo is uploaded to the server.

[0132] Lifestyle data collection from voice recognition devices

[0133] The voice recognition device collects voice data from the user's daily life. For example, if the user says, "I'm going to jog for 30 minutes today," the voice recognition device recognizes the voice and sends the data to the server via the device.

[0134] Collecting online shopping and spending history

[0135] The device periodically collects the user's online shopping history and credit card details and sends them to the server. For example, when a user makes a purchase on a shopping site, the details are automatically recorded and uploaded to the server.

[0136] Data analysis

[0137] Analysis of medical examination reports

[0138] The server analyzes the uploaded health checkup report, extracting text data from the PDF using OCR technology and extracting important health indicators (blood pressure, blood sugar, cholesterol levels, etc.), which are then stored in a database.

[0139] Biological data analysis

[0140] The server analyzes the biometric data sent from the wearable device and uses time-series data analysis to understand the user's daily health patterns (heart rate variability, daily steps, sleep quality, etc.).

[0141] Food photo analysis

[0142] The server analyzes the photos of the meals using image recognition technology (such as YOLO or ResNet), evaluates the calories and nutritional balance of each meal, and stores that information in a database.

[0143] Lifestyle data analysis

[0144] The server analyzes the voice data obtained from the voice recognition device and uses voice recognition technology to evaluate the user's lifestyle patterns and stress levels. This data is used to analyze the user's lifestyle.

[0145] Analysis of shopping and spending history

[0146] The server analyzes online shopping and spending histories to assess the balance between income and expenditure, and uses machine learning algorithms to identify spending trends and build predictive models.

[0147] Proposal generation and refinement

[0148] Insurance plan proposals

[0149] The server then proposes the most suitable life and medical insurance plan for the user based on the analysis results. For example, if the user has high blood pressure, it will recommend an insurance plan that addresses the risk of high blood pressure.

[0150] Generate financial plans and investment proposals

[0151] The server will provide financial planning and investment recommendations, taking into account the user's health status and lifestyle patterns. For example, it will analyze the user's income and expenses and suggest mutual funds and pension plans.

[0152] Leisure activity budget proposal

[0153] The server then suggests budget allocations for leisure activities to the user. For example, it extracts entertainment expenses from the user's daily expenses and suggests spending 10,000 yen per month on leisure activities.

[0154] Feedback and Optimization

[0155] Receiving feedback from users

[0156] The device displays the proposed insurance and financial plans to the user and accepts their feedback, allowing them to submit comments and revisions to the proposals based on their preferences and needs.

[0157] Adjustment and resubmission of proposals

[0158] The server adjusts its proposals based on user feedback. For example, if a user gives feedback that the monthly premium for a proposed insurance plan is too high, the server will recalculate the premium and propose a more suitable plan.

[0159] Using generative AI models

[0160] The server generates proposals based on the results of data analysis using a generative AI model. The generative AI model in this system is designed to analyze information from various data sources and automatically generate optimal proposals.

[0161] Using prompt statements

[0162] At each step in the processing flow, the generative AI model is given a prompt like this:

[0163] "Please suggest an appropriate medical insurance plan for a 50-year-old man based on his medical examination results."

[0164] "Analyze your daily health patterns based on your heart rate and step count data."

[0165] "Evaluate the calories and nutritional balance from the food photos."

[0166] "Evaluate your income and expenditure balance based on your online shopping history and suggest the best financial plan for you."

[0167] This makes it possible to integrate various personal data and provide optimal advice and plans.

[0168] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0169] The processing flow (processing steps) of this system's program

[0170] Step 1: Data collection

[0171] The user uploads a health checkup report in PDF format. The terminal provides an interface to the user and saves the uploaded PDF file on the terminal. The input is the PDF file of the health checkup report, and the output is sending the PDF file to the server.

[0172] Step 2: Collecting biometric data from wearable devices

[0173] The terminal periodically collects biometric data (heart rate, number of steps, sleep patterns, etc.) from the user's wearable device via Bluetooth or Wi-Fi. The input is the biometric data from the wearable device, and the output is the transmission of the biometric data to the server.

[0174] Step 3: Upload your meal photos

[0175] The app stores photos of each meal taken by the user on their device. The device adds date information and uploads them to the server. The input is the meal photo and date information, and the output is the transmission of the meal photo to the server.

[0176] Step 4: Collecting lifestyle data from the voice recognition device

[0177] The speech recognition device collects voice data from the user's daily life and recognizes utterances such as "Today's jogging time is 30 minutes." The recognized data is sent to the server via the terminal. The input is the user's voice data, and the output is the transmission of the recognized data to the server.

[0178] Step 5: Collect your online shopping and spending history

[0179] The terminal periodically collects the user's online shopping history and credit card details and sends them to the server. The input is the online shopping history and spending history, and the output is the transmission of data to the server.

[0180] Step 6: Data analysis

[0181] The server uses OCR technology to extract text data from PDF health checkup reports and analyzes important health indicators such as blood pressure and blood sugar levels. It then performs time-series analysis of the biometric data to understand the user's daily health patterns. It uses image recognition technology to evaluate calories and nutritional balance from food photos, and analyzes lifestyle patterns and stress levels from audio data. It also evaluates the balance between income and expenditures from online shopping history and spending history. The input is various collected data, and the output is the analysis results.

[0182] Step 7: Propose an insurance plan

[0183] The server then proposes the most suitable life insurance or medical insurance plan to the user based on the analysis results. For example, if the user has high blood pressure, it will recommend an insurance plan that addresses the risk of high blood pressure. The input is the analysis results, and the output is a proposed insurance plan.

[0184] Step 8: Generate a financial plan and investment proposal

[0185] The server makes financial plans and investment recommendations, taking into account the user's health status and lifestyle patterns. It analyzes the user's income and expenses and suggests mutual funds and pension plans. The input is the analysis results, and the output is financial plans and investment plan recommendations.

[0186] Step 9: Propose a leisure activity budget

[0187] The server extracts entertainment expenses from the user's daily expenditures and suggests spending 10,000 yen per month on leisure activities. The input is the analysis result, and the output is a suggested leisure activity budget.

[0188] Step 10: Receive feedback from users

[0189] The terminal displays the proposed insurance plan or financial plan to the user and accepts feedback. The user can submit comments or requests for revisions to the proposal through the terminal. The input is the user's feedback, and the output is the collection of the feedback.

[0190] Step 11: Adjust and resubmit your proposal

[0191] The server adjusts the proposal based on user feedback. For example, if the user gives feedback that the monthly premium for a proposed insurance plan is too high, the server recalculates the premium and proposes a more suitable plan. The input is the feedback, and the output is the adjusted proposal.

[0192] Step 12: Use a generative AI model and prompts

[0193] The server uses a generative AI model to generate recommendations based on the results of data analysis. For example, a prompt such as "Please suggest an appropriate medical insurance plan based on the medical examination results of a 50-year-old male" can be input, and the AI ​​model will generate the optimal insurance plan based on that. The input is the prompt, and the output is recommendations for the optimal insurance plan, financial plan, leisure activity budget, etc.

[0194] (Application example 1)

[0195] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0196] In modern society, systems that efficiently collect and analyze diverse information are necessary to manage users' health and lifestyle patterns, and to provide individualized financial planning and security measures. However, while many systems exist that manage this information individually, there are still few systems that centrally integrate this information and provide optimal recommendations to users. Furthermore, there has been insufficient work done to evaluate security risks and propose countermeasures based on users' health information and lifestyle patterns. Therefore, there is a need for the development of a system that can provide comprehensive lifestyle support and security measures.

[0197] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0198] In this invention, the server includes means for uploading health checkup reports, means for receiving biometric data from the user's wearable device, means for receiving photos of meals taken by the user, means for collecting daily life data from a smart speaker, means for receiving online shopping and expenditure history, means for receiving video data from home surveillance cameras, means for analyzing the health checkup reports, biometric data, meal photos, daily life data, expenditure history, and home surveillance camera data to propose life insurance and medical insurance tailored to each individual user, means for optimizing the user's financial plan, investment proposals, and budget allocation for leisure activities based on the analysis results, and means for proposing security measures, means for receiving user feedback and adjusting the proposals, financial plan, investment proposals, budget allocation for leisure activities, and security measures, and means for accumulating the biometric data, daily life data, and home surveillance camera data over the long term and continuously learning and updating the user's health condition, lifestyle patterns, and security risks. This enables the integrated management and analysis of a user's diverse data to provide optimal proposals tailored to individual needs and risks.

[0199] A "health checkup report" is a document that describes the results of periodic examinations conducted to assess a User's health status.

[0200] A "wearable device" is an electronic device that can be worn by a user and collects biometric data such as heart rate, number of steps taken, and sleep patterns.

[0201] "Biometric data" refers to information such as a user's heart rate, number of steps, and sleep patterns obtained from wearable devices.

[0202] "Photos of meals" are image data of meals taken by the user and are used to evaluate the meal contents.

[0203] A "smart speaker" is an electronic device that receives commands from users via voice recognition and performs tasks such as playing music, providing information, and collecting daily life data.

[0204] "Daily Life Data" is data about a user's daily life collected from smart speakers and other home devices.

[0205] "Online shopping and spending history" refers to records of a user's purchasing activities and spending via the Internet.

[0206] A "home surveillance camera" is a video camera installed to monitor the interior and exterior of a home.

[0207] "Video data" refers to video information captured by a home surveillance camera.

[0208] "Analysis" refers to the process of processing information based on collected data using methods such as statistical analysis and machine learning to extract trends and patterns.

[0209] "Life insurance" is insurance in which an insurance benefit is paid based on the contract when the policyholder dies.

[0210] "Medical insurance" is insurance that covers medical expenses when you receive treatment for illness or injury.

[0211] "Financial planning" involves comprehensively analyzing a user's income, expenses, investments, etc. to plan their future economic situation.

[0212] "Investment Suggestions" provide optimal investment strategies based on a user's risk profile and goals.

[0213] "Leisure activity budget allocation" is a plan to effectively manage and appropriately allocate expenses for users' leisure activities.

[0214] The "security measures plan" is a plan that assesses security risks based on the user's individual health condition and lifestyle patterns and proposes countermeasures.

[0215] "Feedback" refers to the evaluations and opinions that users provide to the system.

[0216] "Long-term accumulation" means collecting data over a certain period of time and continuing to accumulate it.

[0217] "Learning" refers to the system's ability to recognize patterns and trends based on the data it collects and to improve its ability to adapt to new data.

[0218] "Updating" means keeping the contents of the system up to date by adding, correcting, or reflecting new data or information.

[0219] This invention is a system that comprehensively collects and analyzes users' health information, lifestyle patterns, expenditure history, and security-related data, and proposes optimal life insurance, medical insurance, financial planning, investment proposals, security measures plans, etc. This system collects, analyzes, and optimizes data through mutual communication between the server, terminals, and users.

[0220] Data collection

[0221] 1. Device: Provides a user interface and prompts the user to upload the health checkup report. The user selects the PDF file and saves it to the device.

[0222] 2. Device: Works with the user's wearable device to periodically receive biometric data (heart rate, steps, sleep patterns, etc.) and send it to the server.

[0223] 3. Device: The user takes photos of each meal and saves them to the server. The photos contain date and time information.

[0224] 4. Smart speaker: Collects voice data from the user's daily life (e.g., "How much exercise time did you do today?") and sends it to a server via the device.

[0225] 5. Device: Periodically collects the user's online shopping and spending history and sends it to the server.

[0226] 6. Home surveillance camera: Receives video data from home surveillance cameras as a standalone auxiliary device and transmits it to the server through the terminal.

[0227] Data analysis

[0228] 1. Server: Analyzes the received health checkup report and extracts important health indicators (blood pressure, blood sugar level, cholesterol level, etc.). This analysis uses machine learning models such as TENSORFLOW (registered trademark).

[0229] 2. Server: Analyzes biometric data from wearable devices to understand the user's daily health patterns (heart rate variability, daily steps, sleep quality, etc.).

[0230] 3. Server: Using photo analysis software (e.g., OpenCV), the server analyzes photos of meals and evaluates the calories and nutritional balance of each meal.

[0231] 4. Server: Analyzes lifestyle data obtained from the smart speaker and evaluates the user's lifestyle patterns and stress levels.

[0232] 5. Server: Analyzes the user's online shopping and spending history to assess the balance between income and expenditure and identify spending trends.

[0233] 6. Server: Analyzes video data from home surveillance cameras, detects anomalies, and assesses security risks.

[0234] Proposal generation and refinement

[0235] 1. Server: Based on the analysis results, the server proposes the best life and health insurance plan for the user. This proposal includes insurance selection to reduce the user's health risks.

[0236] 2. Server: Providing financial planning and investment recommendations taking into account the user's health status and lifestyle, including suitable investment options and retirement financial planning.

[0237] 3. Server: Proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[0238] 4. Server: Proposes a security plan based on the user's individual health condition and lifestyle patterns, including emergency contact information when a sudden change in heart rate is detected and countermeasures when an abnormal person is detected on security cameras.

[0239] Feedback and Optimization

[0240] 1. Device: Shows the user the proposed insurance plan, financial plan, or security plan and accepts their feedback. The user reviews the proposal and provides feedback on any necessary adjustments via the device.

[0241] 2. Server: Based on user feedback, the server adjusts its proposals and re-proposes optimal insurance, financial, and security plans.

[0242] Examples of concrete examples and prompts

[0243] Example: If the user detects a sudden change in heart rate, the system:

[0244] 1. The wearable device detects sudden changes in your heart rate and sends a notification to your smartphone.

[0245] 2. Your smartphone will automatically send a message to the people registered as emergency contacts.

[0246] 3. The home surveillance camera detects an abnormality and the surveillance footage can be viewed via a smartphone app.

[0247] 4. The app will provide users with an emergency response plan and the option to call emergency services if necessary.

[0248] Example prompt sentence:

[0249] "Your heart rate has increased dramatically. Would you like to notify your emergency contacts?"

[0250] "We've detected something unusual. Would you like us to check the surveillance footage?"

[0251] "Should I call emergency services?"

[0252] This system makes it possible to manage and analyze a wide range of user data in an integrated manner, and to make optimal proposals based on individual needs and risks.

[0253] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0254] Step 1:

[0255] Terminal: Provides a means for uploading the user's health checkup report. Specifically, it displays a health checkup report upload button on the terminal's user interface and saves the PDF file selected by the user on the terminal. The input is the health checkup report in PDF format, and the output is a PDF file saved on the terminal.

[0256] Step 2:

[0257] Terminal: Works with the user's wearable device and provides a means to receive biometric data. Specifically, it periodically receives heart rate, step count, sleep patterns, etc. from the wearable device using communication technologies such as Bluetooth, and sends this to a server. The input is the biometric data sent from the wearable device, and the output is the biometric data sent to the server.

[0258] Step 3:

[0259] Terminal: Provides a means to receive and store photos of meals taken by users. Specifically, the user takes a photo of their meal using the device's camera function, saves the image data on the device, and sends it to the server. The input is the photo of the meal taken by the user, and the output is the image data sent to the server.

[0260] Step 4:

[0261] Smart speaker: Collects voice data related to the user's daily life. Specifically, the smart speaker recognizes the voice commands given by the user (e.g., "I'll exercise for 30 minutes today") and sends them as text data to a server via the device. The input is voice data, and the output is the text data sent to the server.

[0262] Step 5:

[0263] Terminal: Provides a means to periodically collect a user's online shopping and spending history and send it to a server. Specifically, it connects with the user's credit card and bank account to obtain spending history and send it to the server. The input is spending history data, and the output is the data sent to the server.

[0264] Step 6:

[0265] Home surveillance cameras: Receives video data from home surveillance cameras. Specifically, cameras installed in the home capture video in real time and send the data to a server via a terminal. The input is video data, and the output is the video data sent to the server.

[0266] Step 7:

[0267] Server: Analyzes the received health checkup report. Specifically, it uses OCR (Optical Character Recognition) technology to extract data from the PDF format health checkup report and analyzes the health indicators using a machine learning model (e.g., TensorFlow). The input is the PDF file of the health checkup report, and the output is the analyzed health indicator data.

[0268] Step 8:

[0269] Server: Analyzes biometric data from wearable devices to understand the user's daily health patterns. Specifically, it analyzes biometric data over time to evaluate heart rate fluctuations, daily steps, sleep quality, etc. The input is biometric data, and the output is health pattern data.

[0270] Step 9:

[0271] Server: Analyzes photos of meals using image analysis technology to evaluate the calories and nutritional balance of each meal. Specifically, image analysis software (e.g., OpenCV) is used to analyze the photos and evaluate the meal content. The input is a photo of the meal, and the output is data on calories and nutritional balance.

[0272] Step 10:

[0273] Server: Analyzes lifestyle data obtained from the smart speaker and evaluates the user's lifestyle patterns and stress levels. Specifically, it analyzes voice data using natural language processing technology and evaluates things like exercise time. The input is voice data in text format, and the output is data on lifestyle patterns and stress levels.

[0274] Step 11:

[0275] Server: Analyzes the user's online shopping history and spending history, evaluates the balance between income and expenses, and identifies spending trends. Specifically, it performs statistical analysis of the collected spending history data and evaluates monthly expenses and trends. The input is spending history data, and the output is financial evaluation data.

[0276] Step 12:

[0277] Server: Analyzes video data from home surveillance cameras and detects abnormalities. Specifically, it uses video analysis software to recognize abnormal behavior and suspicious individuals and evaluate security risks. The input is video data, and the output is security risk assessment data.

[0278] Step 13:

[0279] Server: Based on the analysis results, the server proposes the most suitable life insurance or medical insurance plan to the user. Specifically, the server selects and proposes the most suitable insurance plan based on the analyzed health data. The input is health index data, and the output is insurance plan proposal data.

[0280] Step 14:

[0281] Server: Considers the user's health condition and lifestyle patterns and makes financial plans and investment proposals. Specifically, it proposes investment plans and financial plans based on analyzed biometric data and expenditure history. The input is health pattern data and financial evaluation data, and the output is financial plan proposal data.

[0282] Step 15:

[0283] Server: Proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of life. Specifically, it proposes the optimal budget allocation for leisure activities based on collected data. The input is expenditure history data, and the output is leisure activity budget allocation proposal data.

[0284] Step 16:

[0285] Server: Proposes a security plan based on the user's health condition and lifestyle patterns. Specifically, it provides emergency contact information when a sudden change in heart rate is detected, and countermeasures when an abnormal person is detected on a surveillance camera. The input is biometric data and video data, and the output is security plan data.

[0286] Step 17:

[0287] Terminal: Displays proposed insurance plans, financial plans, and security measures plans to the user and accepts feedback. Specifically, the proposals are displayed on the terminal screen and user feedback is received. The input is the proposal data and the output is the feedback data.

[0288] Step 18:

[0289] Server: Adjusts the proposal based on user feedback and re-proposes an optimized plan. Specifically, it analyzes the feedback and updates the proposal according to the user's requests. The input is the feedback data, and the output is the updated proposal data.

[0290] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0291] This invention is a system that integrates health checkup reports collected over multiple years, biometric data from wearable devices, photographic data for each meal, lifestyle data from smart speakers, and online shopping and expenditure history to select life insurance and medical insurance suited to an individual's health condition, as well as financial planning, investment proposals, and budget allocation for leisure activities, and also incorporates an emotion engine that recognizes the user's emotions. This system collects, analyzes, and optimizes data through mutual communication between the server, terminals, and users, and further improves the proposal content by taking emotional information into account.

[0292] Program processing explanation

[0293] Data collection

[0294] 1. The device prompts the user to upload the health checkup report. The user saves the PDF file of the health checkup report on the device and uploads it.

[0295] 2. The terminal periodically receives biometric data (heart rate, steps, sleep patterns, etc.) from the user's wearable device and transmits this data to the server.

[0296] 3. The device saves photos of each meal taken by the user and uploads them to the server. The photos contain date and time information.

[0297] 4. The smart speaker collects voice data from the user's daily life (e.g., "How much exercise time did you do today?") and sends it to the device, which then uploads the data to the server.

[0298] 5. The device periodically collects the user's online shopping and spending history (such as credit card history) and sends it to the server.

[0299] 6. The device sends the voice data and image data to the emotion engine to recognize the user's emotions and extracts the emotion information.

[0300] Data analysis

[0301] 1. The server analyzes the received health checkup report using OCR technology and extracts important health indicators such as blood pressure, blood sugar level, and cholesterol level.

[0302] 2. The server analyzes the received biometric data from the wearable device to understand the user's health patterns, such as heart rate fluctuations, daily steps taken, and sleep quality.

[0303] 3. The server analyzes the photos of the meals using image recognition technology, and calculates and evaluates the calories and nutritional balance of each meal.

[0304] 4. The server analyzes the lifestyle data obtained from the smart speaker and evaluates the user's activity patterns and stress levels.

[0305] 5. The server analyzes the user's online shopping history and spending history, evaluates the balance between income and expenses, and identifies spending trends.

[0306] 6. The emotion engine analyzes the user's emotional information from voice and image data to understand their current emotional state (e.g., joy, anger, sadness, surprise, etc.).

[0307] Proposal generation and refinement

[0308] 1. The server generates the optimal life and health insurance plan for the user based on the analysis results and emotional information, including selecting insurance that takes into account the user's health risks and emotional state.

[0309] 2. The server takes into account the user's health status, lifestyle patterns, and emotional information to provide optimal financial planning and investment recommendations, including investment strategies for financial security in retirement.

[0310] 3. The server proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[0311] Feedback and Optimization

[0312] 1. The terminal displays the insurance plan and financial plan proposed by the server to the user. The user reviews the proposal and provides feedback if necessary.

[0313] 2. The server receives feedback from the user, adjusts the proposal, and again provides the user with an optimized plan.

[0314] Specific examples

[0315] For example, if a 50-year-old male user is using this system, it will operate as follows:

[0316] Data collection

[0317] The user uses the device to upload last year's health checkup report. Daily heart rate and step count data are automatically sent from the wearable device to the server. The user takes a photo of their lunch, which is uploaded to the server. The smart speaker records that "today's jogging time was 30 minutes" and sends this data to the server. The user's credit card history is periodically uploaded to the server. Furthermore, the emotion engine analyzes the user's facial image and voice data to recognize the emotion of "joy."

[0318] Data analysis

[0319] The server analyzes health checkup reports and extracts important indicators such as blood pressure of 130 / 85. Data from wearable devices confirms that the person walks 7,000 steps per day. Food photos are analyzed and evaluated as 700 calories or less. Data from smart speakers reveals that the person jogs for 150 minutes per week. Credit card history confirms that monthly food expenses are 30,000 yen. An emotion engine recognizes the emotion of happiness and sends it to the server.

[0320] Proposal generation and refinement

[0321] The server proposes a medical insurance plan tailored to the user's blood pressure risk. It also presents an exercise plan to maintain a jogging habit. Taking into account food and entertainment expenses, it suggests a monthly budget of 10,000 yen for leisure activities. The emotion engine grasps the user's feelings of joy and makes suggestions that will further increase satisfaction.

[0322] Feedback and Optimization

[0323] The user reviews the suggestions and provides feedback on what they dislike. The server receives the feedback, adjusts the suggestions, and makes new suggestions to optimize them.

[0324] The processing flow will be explained below.

[0325] Step 1:

[0326] The device prompts the user to upload the health checkup report. The user saves the PDF file of the health checkup report on the device and uploads it.

[0327] Step 2:

[0328] The terminal periodically receives biometric data (e.g., heart rate, number of steps, sleep patterns) from the user's wearable device and transmits this data to a server.

[0329] Step 3:

[0330] The device stores photos of each meal taken by the user and uploads them to a server, with the photos containing date and time information.

[0331] Step 4:

[0332] The smart speaker collects voice data from the user's daily life (e.g., "How much time did you exercise today?") and sends it to the device, which then uploads the data to a server.

[0333] Step 5:

[0334] The terminal periodically collects the user's online shopping and spending history (such as credit card history) and sends it to a server.

[0335] Step 6:

[0336] The terminal transmits voice data and image data for recognizing the user's emotions to an emotion engine, and extracts emotion information.

[0337] Step 7:

[0338] The server analyzes the received health checkup report using OCR technology and extracts important health indicators such as blood pressure, blood sugar level, and cholesterol level.

[0339] Step 8:

[0340] The server analyzes the biometric data received from the wearable device to understand the user's health patterns, such as heart rate fluctuations, daily steps taken, and sleep quality.

[0341] Step 9:

[0342] The server analyzes photos of meals using image recognition technology, and calculates and evaluates the calories and nutritional balance of each meal.

[0343] Step 10:

[0344] The server analyzes lifestyle data obtained from the smart speaker (e.g., work hours, rest hours, and exercise frequency) to evaluate the user's activity patterns and stress levels.

[0345] Step 11:

[0346] The server analyzes the user's online shopping history and spending history, evaluates the balance between income and expenses, and identifies spending trends.

[0347] Step 12:

[0348] The emotion engine analyzes the user's emotional information from voice and image data to understand their current emotional state (e.g., joy, anger, sadness, surprise, etc.).

[0349] Step 13:

[0350] The server generates optimal life and health insurance plans for users based on the analysis results and emotional information, including insurance selection that takes into account the user's health risks and emotional state.

[0351] Step 14:

[0352] The server takes into account the user's health status, lifestyle patterns, and emotional information to provide optimal financial planning and investment recommendations, including investment strategies for financial security in retirement.

[0353] Step 15:

[0354] The server proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[0355] Step 16:

[0356] The terminal displays the insurance plans and financial plans proposed by the server to the user, who can then review the proposals and provide feedback if necessary.

[0357] Step 17:

[0358] The server receives feedback from the user, adjusts the proposal, and again provides the user with an optimized plan.

[0359] Specific examples

[0360] For example, if a 50-year-old male user wants to use this system to reduce his health risks and financial worries, the process would look like this:

[0361] Data collection

[0362] Step 1:

[0363] The user uploads their health checkup report from the previous year using their device, and the server then obtains basic data to understand the user's health condition.

[0364] Step 2:

[0365] The wearable device automatically sends daily heart rate and step count data to a server, providing daily activity and health indicators.

[0366] Step 3:

[0367] Users take photos of their lunch and upload them to a server. The image data includes date and time information, and is used to analyze daily meal content.

[0368] Step 4:

[0369] The smart speaker records "Today's jogging time is 30 minutes" and sends the data to a server, where it is used to evaluate the user's exercise level.

[0370] Step 5:

[0371] The user's credit card history is periodically uploaded to the server, which allows the company to understand the user's spending patterns and expenditures.

[0372] Step 6:

[0373] The emotion engine recognizes the emotion "happiness" from the user's voice and facial image data. This information is used to evaluate the user's daily life and stress level.

[0374] Data analysis

[0375] Step 7:

[0376] The server analyzes the health checkup report and extracts important indicators, such as blood pressure 130 / 85, to assess the user's health risk.

[0377] Step 8:

[0378] The wearable device data confirms that the user is walking 7,000 steps per day, allowing the user to understand their daily activity level.

[0379] Step 9:

[0380] It analyzes photos of meals and evaluates the calorie and nutritional balance of each meal, calculating, for example, 700 calories or the percentage of specific nutrients.

[0381] Step 10:

[0382] Data from the smart speaker reveals that the user jogs 150 minutes per week, helping to understand the user's exercise patterns.

[0383] Step 11:

[0384] Check the credit card history to see that the monthly food expense is 30,000 yen, thereby understanding the user's spending pattern.

[0385] Step 12:

[0386] The emotion engine recognizes the user's "happiness" emotion and sends it to the server, which uses it to evaluate the user's emotional feedback on the suggestions.

[0387] Proposal generation and refinement

[0388] Step 13:

[0389] The server proposes a medical insurance plan tailored to the user's blood pressure risk and also provides an exercise plan to help maintain the user's jogging habit.

[0390] Step 14:

[0391] Taking into account the user's health status, lifestyle patterns, and emotional information, the system makes investment suggestions to ensure financial stability in retirement.

[0392] Step 15:

[0393] Taking into account food and entertainment expenses, the system suggests a monthly budget of 10,000 yen for leisure activities. The emotion engine understands the user's feelings of joy and makes suggestions that will further increase satisfaction.

[0394] Feedback and Optimization

[0395] Step 16:

[0396] The user reviews the suggestions and provides feedback on what they don't like. The device receives the feedback and sends it to the server.

[0397] Step 17:

[0398] The server adjusts the proposal based on the feedback and makes new proposals to optimize the results.

[0399] Through the above specific processing steps, the system of the present invention provides individual users with optimized health management, insurance selection, financial planning, investment suggestions and leisure activity budget allocation, thereby improving the user's overall quality of life.

[0400] Example 2

[0401] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0402] Until now, there has been no system that efficiently selects life insurance and medical insurance suitable for individual users, provides financial planning, investment recommendations, and budget allocation for leisure activities. It has also been difficult to comprehensively analyze a user's health condition, lifestyle patterns, and even emotional state and make recommendations based on that analysis. Furthermore, it has also been difficult to integrate various data provided by users (health checkup reports, biometric data, food photos, daily life data, expenditure history, etc.), and accumulate and analyze them over the long term. The present invention aims to solve these problems.

[0403] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0404] In this invention, the server includes means for uploading health checkup reports, means for receiving biometric data from the user's wearable device, means for receiving photos of meals taken by the user, means for collecting daily life data from a voice recognition device, means for receiving online shopping and expenditure history, means for analyzing the user's emotional information from the voice data and image data using an emotion engine, means for analyzing the health checkup reports, biometric data, meal photos, daily life data, expenditure history, and emotional information to propose life insurance and medical insurance optimized for each user, means for optimizing the user's financial plan, investment proposals, and budget allocation for leisure activities based on the analysis results, means for receiving user feedback and adjusting the proposals, and means for accumulating the biometric data, daily life data, and emotional information over the long term and continuously learning and updating the user's health condition and lifestyle patterns, thereby enabling automatic and efficient proposals of insurance, financial plan, and budget allocation for leisure activities that are optimal for each user's condition.

[0405] A "health checkup report" is written or digital data indicating the user's health status obtained through a health checkup.

[0406] A "wearable device" is an electronic device worn by a user that measures biometric data (heart rate, number of steps, sleep patterns, etc.) and provides the data.

[0407] "Biometric data" refers to digital data that indicates a user's physical condition, including information such as heart rate, number of steps taken, sleep patterns, body temperature, and blood pressure.

[0408] A "meal photo" is image data taken by a user to record the contents of a meal, and indicates the type, amount, and visual characteristics of the meal.

[0409] A "voice recognition device" is a device that collects a user's voice and processes the content of that voice as digital data.

[0410] "Daily life data" refers to data related to the user's daily activities and behavior, including exercise time, sleep time, and speech content.

[0411] "Online shopping history" refers to a record of purchases made by a user online, including information such as the items purchased, the date and time of purchase, and the purchase amount.

[0412] "Expense history" refers to a record of a user's spending activities, including credit card statements, bank statements, cash usage records, etc.

[0413] An "emotion engine" is a system or software that analyzes a user's emotional state from voice and image data and identifies those emotions.

[0414] "Financial planning" is a plan to comprehensively manage a user's financial situation, including income, expenses, savings, and investments, in order to ensure future economic stability.

[0415] "Investment Suggestions" are suggestions to present appropriate investment products and investment strategies based on the User's financial situation and risk tolerance.

[0416] "Leisure Budget Allocation" refers to how the user allocates funds for leisure and entertainment activities, providing an appropriate spending plan.

[0417] "Feedback" refers to information such as opinions, impressions, and requests from users regarding presented proposals and plans.

[0418] This invention is a system that integrates health checkup reports collected over multiple years, biometric data from wearable devices, photographic data for each meal, lifestyle data from a voice recognition device, and online shopping and expenditure history to select life insurance and medical insurance suited to an individual's health condition, as well as financial planning, investment recommendations, and budget allocation for leisure activities. Furthermore, by incorporating an emotion engine, the system takes into account the user's emotional information and improves the content of its recommendations.

[0419] Hardware and software used

[0420] The system is implemented using the following hardware and software:

[0421] Terminal: A device such as a smartphone or computer operated by a user.

[0422] Wearable devices: Devices that collect biometric data such as heart rate, steps taken, and sleep patterns.

[0423] Voice recognition devices: Devices that collect daily life data (e.g., smart speakers).

[0424] Server: A computer system that analyzes data and generates recommendations.

[0425] Emotion engine: Software that analyzes a user's emotional information from voice and image data.

[0426] Data processing and calculation

[0427] The program in this system performs the following data processing and calculations:

[0428] 1. The device will notify the user through a dedicated application to prompt them to upload their health checkup report. Once the user uploads the report, the server will analyze the content using OCR technology and extract important health indicators (blood pressure, blood sugar level, cholesterol level, etc.).

[0429] 2. Biometric data from the wearable device is sent via the device to a server, which analyzes the data to understand the user's health patterns, such as heart rate fluctuations, daily steps, and sleep quality.

[0430] 3. After users take photos of their meals using their devices, they automatically upload them to a server, where they use image recognition technology to analyze the photos and evaluate the calorie and nutritional balance of each meal.

[0431] 4. Daily life data collected by voice recognition devices (such as smart speakers) is transmitted to a server via the device, where it is analyzed to assess activity patterns and stress levels.

[0432] 5. Online shopping history and spending history are automatically collected by the device and sent to a server, which analyzes this data to assess the balance between income and expenditure and identify spending trends.

[0433] 6. The emotion engine analyzes the user's emotional information from voice and image data to understand their current emotional state (e.g., joy, anger, sadness, surprise, etc.). This allows the server to generate optimal suggestions that take the emotional information into account.

[0434] Examples of concrete examples and prompts

[0435] For example, if a 50-year-old male user is using this system, the following will happen:

[0436] Data collection

[0437] The user uses the device to upload last year's health checkup report. Daily heart rate and step count data are automatically sent from the wearable device to the server. The user takes a photo of their lunch, which is uploaded to the server. The smart speaker records that "today's jogging time was 30 minutes" and sends this data to the server. The user's credit card history is periodically uploaded to the server. Furthermore, the emotion engine analyzes the user's facial image and voice data to recognize the emotion of "joy."

[0438] Data analysis

[0439] The server analyzes health checkup reports and extracts important indicators such as blood pressure of 130 / 85. Data from wearable devices confirms that the person walks 7,000 steps per day. Food photos are analyzed and evaluated as 700 calories or less. Data from smart speakers reveals that the person jogs for 150 minutes per week. Credit card history confirms that monthly food expenses are 30,000 yen. An emotion engine recognizes the emotion of happiness and sends it to the server.

[0440] Proposal generation and refinement

[0441] The server proposes a medical insurance plan tailored to the user's blood pressure risk. It also presents an exercise plan to maintain a jogging habit. Taking into account food and entertainment expenses, it suggests a monthly budget of 10,000 yen for leisure activities. The emotion engine grasps the user's feelings of joy and makes suggestions that will further increase satisfaction.

[0442] Feedback and Optimization

[0443] The user reviews the suggestions and provides feedback on what they dislike. The server receives the feedback, adjusts the suggestions, and makes new suggestions to optimize them.

[0444] Example prompts to input to the generative AI model

[0445] "A 50-year-old male user has provided us with his health checkup report and data from his wearable device. Please suggest the best medical insurance and exercise plan for this user, taking into account his health risks and emotional state."

[0446] This system will enable a comprehensive analysis of a user's health condition, lifestyle patterns, and emotional state, and provide optimal recommendations for each individual, which will not only improve the user's quality of life, but also provide efficient support for future risk management and financial planning.

[0447] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0448] Step 1:

[0449] The user uploads the medical checkup report.

[0450] Input: The user saves the PDF file of the medical examination report to the device.

[0451] Specific operation: The user clicks the "Upload" button in the dedicated application on the device.

[0452] Data processing: The terminal sends the health check report to the server.

[0453] Output: The health check report is saved on the server.

[0454] Step 2:

[0455] The server analyzes the health check report.

[0456] Input: PDF file of medical examination report.

[0457] Specific operation: The server uses OCR technology to extract text data from the PDF file.

[0458] Data processing: Extract important health indicators (blood pressure, blood sugar levels, cholesterol levels, etc.).

[0459] Output: The extracted health indicator data is stored in a database.

[0460] Step 3:

[0461] The terminal receives the biometric data from the wearable device.

[0462] Input: Biometric data obtained from wearable devices (heart rate, steps, sleep patterns, etc.).

[0463] Specific operation: The wearable device sends data to the terminal via Bluetooth or Wi-Fi.

[0464] Data processing: The device periodically sends this data to the server.

[0465] Output: Biometric data is stored on the server.

[0466] Step 4:

[0467] The server analyzes the biometric data.

[0468] Input: Biometric data received from a wearable device.

[0469] Specific operation: The server analyzes heart rate fluctuations, daily steps taken, and sleep quality.

[0470] Data processing: The analysis results are structured as time-series data.

[0471] Output: The analyzed biometric data is stored in a database.

[0472] Step 5:

[0473] The user takes a photo of the meal, which is received by the device.

[0474] Input: A photo file of the meal taken by the user.

[0475] Specific operation: The user takes a photo of a meal using the device's camera application.

[0476] Data processing: The device saves the photos in a dedicated folder and uploads them to the server.

[0477] Output: Photo data is saved on the server.

[0478] Step 6:

[0479] The server analyzes the food photos.

[0480] Input: Food photo data.

[0481] What it does: The server uses image recognition technology to analyze the photo and identify the ingredients.

[0482] Data processing: Calculate the calories and nutritional balance of each meal.

[0483] Output: The analysis results are saved in a database.

[0484] Step 7:

[0485] A voice recognition device collects daily life data and the data is received by the terminal.

[0486] Input: Voice data collected by a voice recognition device.

[0487] Specific operation: The voice recognition device records the user's daily activities (e.g., "How much time do I spend exercising today?") and converts them into text data.

[0488] Data processing: The terminal receives the text data and uploads it to the server.

[0489] Output: Daily life data is saved on the server.

[0490] Step 8:

[0491] The server analyzes daily life data.

[0492] Input: Text data received from a speech recognition device.

[0493] Specific operation: The server performs analysis to evaluate behavioral patterns and stress levels.

[0494] Data processing: Activity patterns and stress levels are assessed and stored as structured data.

[0495] Output: The analyzed daily life data is stored in a database.

[0496] Step 9:

[0497] The terminal collects online shopping and spending history.

[0498] Input: Spending data such as online shopping history, credit card history, etc.

[0499] What it does: The device retrieves spending history from online banking apps and emails.

[0500] Data processing: Upload expenditure data to the server periodically.

[0501] Output: The spending history data is saved on the server.

[0502] Step 10:

[0503] A server analyzes online shopping and spending history.

[0504] Input: Collected expenditure data.

[0505] What happens: The server categorizes spending data.

[0506] Data processing: Evaluate the balance between income and expenditure and understand expenditure trends.

[0507] Output: Parsed expenditure data is saved in a database.

[0508] Step 11:

[0509] The terminal transmits the voice data and image data to the emotion engine.

[0510] Input: Audio and image data collected by the camera and microphone.

[0511] Specific operation: The device sends voice data and image data to the emotion engine.

[0512] Data processing: The emotion engine analyzes the data and extracts emotional information.

[0513] Output: The analysis results are sent to the server and saved.

[0514] Step 12:

[0515] The server integrates the analysis results and generates optimal proposals.

[0516] Input: health checkup report, biometric data, food photos, daily life data, expenditure history and emotional information.

[0517] How it works: The server aggregates all the data and uses scoring algorithms to generate optimal life and health insurance plans, financial plans, investment suggestions, and budget allocations for leisure activities.

[0518] Data processing: Generate optimized proposals for each user.

[0519] Output: The generated proposal is sent to the device.

[0520] Step 13:

[0521] The terminal displays the suggestions to the user, who then provides feedback.

[0522] Input: Generated suggestions.

[0523] Specific operation: The device will display the suggestions to the user in a pop-up notification or within a dedicated application.

[0524] Data processing: The user clicks the feedback button and enters their opinion.

[0525] Output: The feedback is sent to the server and stored.

[0526] Step 14:

[0527] The server will incorporate the feedback and re-adjust the suggestions.

[0528] Input: User feedback.

[0529] Specific operation: The server analyzes the feedback and reflects it in the algorithm.

[0530] Data processing: Re-arranging data to generate new proposals.

[0531] Output: The optimized suggestions are sent back to the device.

[0532] (Application example 2)

[0533] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0534] Currently, there is no integrated system on the market that analyzes individual health checkup reports, biometric data from wearable devices, daily life data, and electronic payment history to provide optimal insurance, financial planning, investment recommendations, and even budget allocation for leisure activities. This requires users to manually collect and analyze a large amount of data, which is prone to errors and time-consuming. Furthermore, current systems do not take into account the user's emotional state when making recommendations, making it difficult to improve user satisfaction and quality of life.

[0535] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0536] In this invention, the server includes means for uploading health checkup reports, means for receiving biometric data from the user's wearable device, means for receiving images of meals taken by the user, means for collecting daily life data from a voice recognition device, means for receiving electronic payment history, means for analyzing the health checkup reports, biometric data, meal images, daily life data, and electronic payment history to provide insurance proposals optimized for each user, and means for optimizing the user's financial planning, investment proposals, and budget allocation for leisure activities based on the analysis results and emotion recognition technology. This allows users to centrally manage information from various data sources and receive comprehensive and personalized proposals, thereby improving their quality of life and saving time. Furthermore, emotion recognition technology can be used to provide more satisfying proposals that take the user's emotional state into account.

[0537] A "health checkup report" is a detailed record provided by a medical institution to assess a user's health condition.

[0538] A "wearable device" is an electronic device that can be worn by a user and is used to collect biometric data.

[0539] "Biometric data" refers to data that indicates the user's physical condition, such as heart rate, number of steps taken, and sleep patterns.

[0540] "Meal images" are photographic data of meals taken by the user, and are data whose contents can be analyzed.

[0541] A "voice recognition device" is a device for collecting and analyzing voice data from a user's daily life.

[0542] "Daily life data" is data that indicates the user's daily behavior and activity patterns.

[0543] "Electronic payment history" refers to recorded data relating to a user's online shopping and various payments.

[0544] "Emotion recognition technology" is a technology that analyzes voice data and image data to identify a user's emotional state.

[0545] "Insurance proposal" means proposing the most suitable life insurance or medical insurance plan to the user based on the analysis results.

[0546] A "financial plan" is a plan for managing money and achieving future financial security based on a user's income and expenses.

[0547] "Investment Proposal" means proposing the optimal investment plan to the User.

[0548] "Budget allocation for leisure activities" means allocating an appropriate budget for the user's daily leisure activities.

[0549] This invention provides a system that uses health checkup reports, biometric data from wearable devices, food images taken by the user, daily life data from a voice recognition device, and electronic payment history to make insurance proposals, financial plans, investment proposals, and budget allocations for leisure activities that are optimized for each individual user.

[0550] Data collection methods

[0551] The server and the terminal work together to collect the following data:

[0552] Health Checkup Report: The user uploads the health checkup report using the terminal. The report is uploaded in PDF format or image format and sent to the server.

[0553] Wearable devices: Biometric data such as heart rate, steps, and sleep patterns are periodically collected from a user's wearable device (e.g., Fitbit, Apple Watch). This data is then sent to a server using an API (e.g., Fitbit API, Apple HealthKit).

[0554] Meal image: The user takes a photo of the meal on their device and uploads it to the server. The image contains date and time information.

[0555] Daily Life Data: Daily life data obtained from a voice recognition device (e.g., Amazon Echo, GOOGLE HOME®) is sent to the server via the device. This data includes voice commands and daily activity logs.

[0556] Electronic payment history: A user's credit card history and online shopping history are periodically sent to a server via an API (e.g., Plaid API).

[0557] Data Analysis Methods

[0558] The server analyzes the collected data as follows:

[0559] Health checkup report: Using OCR technology (e.g., Tesseract OCR), the health checkup report is converted into text and health indicators such as blood pressure, blood sugar level, and cholesterol level are extracted.

[0560] Biometric data: Using statistical analysis techniques (e.g., Pandas, Numpy), we analyze heart rate fluctuations, daily steps, sleep quality, etc. to understand the user's health patterns.

[0561] Meal images: Calculate the calories and nutritional balance of meals using image recognition technology (e.g., Google (registered trademark) Cloud Vision API).

[0562] Daily Life Data: Convert voice data into text (e.g., Amazon Transcribe) and evaluate the user's activity log and stress level.

[0563] Electronic payment history: Use financial analysis techniques (e.g., FinancePy) to understand spending trends and evaluate the balance between income and expenses.

[0564] Emotion recognition: Analyze the user's emotional state from voice and image data using an emotion engine (e.g., Azure (registered trademark) Emotion API) to understand their current emotional state.

[0565] Suggestions and optimization measures

[0566] The server makes the following suggestions based on the analysis results and emotion data.

[0567] Insurance proposals: Based on the analyzed health data, the system generates the most suitable life and medical insurance plan for the user. In particular, by combining it with emotion recognition technology, it is possible to propose flexible plans tailored to the user's current condition.

[0568] Financial Planning and Investment Recommendations: Create a financial plan that takes into account your income, expenses, and emotional state, and then recommend an investment strategy based on that. Use optimization algorithms (e.g., Linear Programming) to ensure your financial future is secure.

[0569] Budget allocation for leisure activities: Based on the user's daily life data and emotional data, the system suggests optimal budget allocation and improves the user's quality of life.

[0570] Feedback and Resubmissions

[0571] Users can check the proposals and provide feedback via their devices. The server adjusts the proposals based on the received feedback and re-proposes an optimized plan. This ensures that users always receive the best proposals based on the latest situation.

[0572] Specific examples

[0573] For example, if a 50-year-old male user is using this system, the following specific behavior is expected:

[0574] Users upload their health checkup report from the previous year.

[0575] Data such as the number of steps taken and heart rate is automatically sent from the wearable device to a server every day.

[0576] The user takes a photo of their lunch, which is then uploaded to the server.

[0577] The smart speaker records that "Today's jogging time was 30 minutes" and sends the data to the server.

[0578] Credit card history is periodically uploaded to the server.

[0579] The emotion engine analyzes the user's facial image and voice data to recognize the emotion of "joy."

[0580] Prompt Sentence Examples

[0581] "A user uploaded yesterday's health check report. Please analyze it."

[0582] "Get data from wearable devices and analyze heart rate and step count."

[0583] "I took a photo of my lunch today. Please measure the calories and nutritional balance."

[0584] "I received data from my smart speaker saying, 'Today's jogging time was 30 minutes.' Please analyze it."

[0585] "Analyze your credit card history and calculate your monthly food expenses."

[0586] "The emotion engine has identified the user's emotional state as 'joy'. Please reflect this in your suggestions."

[0587] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0588] Processing steps of the system that realizes the application example

[0589] Step 1:

[0590] The user uploads a health checkup report using a terminal. The input is a PDF or image file of the health checkup report provided by the user. The terminal sends this file to the server, which converts it into text data using OCR technology (e.g., Tesseract OCR) and extracts health indicators such as blood pressure, blood glucose level, and cholesterol level. The output is the extracted health indicator data.

[0591] Step 2:

[0592] Biometric data is collected from the user's wearable device. The input is data such as heart rate, steps, and sleep patterns measured by the wearable device (e.g., Fitbit, Apple Watch). The device periodically sends this data to a server using an API (e.g., Fitbit API, Apple HealthKit). The server analyzes the data using statistical analysis techniques (e.g., Pandas, Numpy) to understand the user's health patterns. The output is the analyzed health pattern data.

[0593] Step 3:

[0594] The user takes a photo of their meal and uploads it to the server via their device. The input is the photo of the meal provided by the user. The server analyzes the photo using image recognition technology (e.g., Google Cloud Vision API) and calculates the calories and nutritional balance. The output is the analyzed data on calories and nutritional balance.

[0595] Step 4:

[0596] Daily life data is collected from a voice recognition device. The input is voice data from a voice recognition device (e.g., Amazon Echo, Google Home). The device sends this voice data to a server, which converts the voice into text (e.g., Amazon Transcribe) and evaluates the user's activity log and stress level. The output is the analyzed daily life data.

[0597] Step 5:

[0598] Collects electronic payment history. Input is the user's credit card history and online shopping history. The terminal periodically sends this data to a server using an API (e.g., Plaid API). The server uses financial analysis technology (e.g., FinancePy) to identify spending trends and evaluate the balance between income and expenses. The output is analyzed spending trend data.

[0599] Step 6:

[0600] Collects and analyzes emotional data. The input is the user's voice and facial image data. The device sends this data to a server, which uses an emotion engine (e.g., Azure Emotion API) to identify the user's emotional state. The output is analyzed emotional data.

[0601] Step 7:

[0602] Generate an insurance proposal. The input is all the analyzed data mentioned above. Based on this data, the server generates a life or health insurance plan optimized for the individual user. The output is an insurance plan proposal that is presented to the user.

[0603] Step 8:

[0604] Generate a financial plan and investment recommendations. The input is all the analyzed data mentioned above. The server uses optimization algorithms (e.g., Linear Programming) to generate a financial plan and investment recommendations based on the user's income, expenses, and emotional state. The output is a financial plan and investment recommendations presented to the user.

[0605] Step 9:

[0606] The system proposes budget allocation for leisure activities. The input is all the analysis data mentioned above. Based on this data, the server generates an optimal budget allocation that takes into account the user's daily life and emotional state. The output is a budget allocation proposal for leisure activities presented to the user.

[0607] Step 10:

[0608] Receives user feedback and adjusts the suggestions. The input is the feedback provided by the user through the device. The server receives this feedback, re-runs the analysis, and adjusts the suggestions. The output is new suggestions optimized based on the feedback.

[0609] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0610] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0611] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0612] [Second embodiment]

[0613] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0614] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0615] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0616] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0617] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0618] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0619] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0620] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0621] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0622] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0623] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0624] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0625] This invention is a system that integrates health checkup reports collected over multiple years, biometric data from wearable devices, photographic data for each meal, lifestyle data from smart speakers, and online shopping and expenditure history to select life insurance and medical insurance suited to an individual's health condition, as well as financial planning, investment proposals, and budget allocation for leisure activities. This system collects, analyzes, and optimizes data through mutual communication between a server, terminals, and users.

[0626] Program processing explanation

[0627] Data collection

[0628] 1. The device provides an interface to the user, prompting them to upload the health checkup report. The user selects the PDF file and saves it on the device.

[0629] 2. The terminal works in conjunction with the user's wearable device to periodically receive biometric data (heart rate, number of steps, sleep patterns, etc.) and send it to the server.

[0630] 3. The device saves photos of each meal taken by the user and uploads them to the server. The photos contain date and time information.

[0631] 4. The smart speaker collects voice data from the user's daily life (e.g., "How much exercise time did you do today?") and sends it to a server via the device.

[0632] 5. The device periodically collects the user's online shopping and spending history and sends it to the server.

[0633] Data analysis

[0634] 1. The server analyzes the received health checkup report and extracts important health indicators (blood pressure, blood sugar level, cholesterol level, etc.).

[0635] 2. The server analyzes biometric data from the wearable device to understand the user's daily health patterns (heart rate variability, daily steps taken, sleep quality, etc.).

[0636] 3. The server analyzes photos of meals using image recognition technology and evaluates the calories and nutritional balance of each meal.

[0637] 4. The server analyzes the lifestyle data obtained from the smart speaker and evaluates the user's lifestyle patterns and stress levels.

[0638] 5. The server analyzes the user's online shopping history and spending history to evaluate the balance between income and expenditure and identify spending trends.

[0639] Proposal generation and refinement

[0640] 1. The server proposes optimal life and health insurance plans to the user based on the analysis results, including insurance selection to reduce the user's health risks.

[0641] 2. The server will provide financial planning and investment recommendations, taking into account the user's health status and lifestyle, including suitable investment options and financial planning for retirement.

[0642] 3. The server proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[0643] Feedback and Optimization

[0644] 1. The device displays the proposed insurance plan and financial plan to the user and accepts feedback. The user reviews the proposal and provides feedback on any necessary adjustments via the device.

[0645] 2. The server adjusts the proposal based on user feedback and re-proposes the optimal insurance plan and financial plan.

[0646] Specific examples

[0647] For example, if a 50-year-old male user is using this system, it will operate as follows:

[0648] Data collection

[0649] A user uploads last year's health checkup report using a device. Daily heart rate data and step count data are automatically sent from a wearable device to a server. A user takes a photo of their lunch and the photo is uploaded to a server. A smart speaker records that "today's jogging time was 30 minutes" and sends this data to a server. A user's credit card history is periodically uploaded to a server.

[0650] Data analysis

[0651] The server analyzes health checkup reports and extracts important indicators such as blood pressure of 130 / 85. It confirms from wearable device data that the user walks 7,000 steps per day. It analyzes food photos and evaluates the calorie intake, such as 700 calories. It determines from smart speaker data that the user jogs 150 minutes per week. It confirms from credit card history that the user's monthly food expenses are 30,000 yen.

[0652] Proposal generation and refinement

[0653] The server proposes a medical insurance plan tailored to the user's blood pressure risk, provides an exercise plan to encourage jogging, and suggests a monthly budget of 10,000 yen for leisure activities, taking into account food and entertainment costs.

[0654] Feedback and Optimization

[0655] The user reviews the suggestions and provides feedback on what they dislike. The server receives the feedback, adjusts the suggestions, and makes new suggestions to optimize them.

[0656] The processing flow will be explained below.

[0657] Step 1:

[0658] The device prompts the user to upload the health checkup report. The user saves the PDF file of the health checkup report on the device and uploads it.

[0659] Step 2:

[0660] The terminal periodically receives biometric data (heart rate, number of steps, sleep patterns, etc.) from the user's wearable device and transmits this data to a server.

[0661] Step 3:

[0662] The device stores photos of each meal taken by the user and uploads them to a server, with the photos containing date and time information.

[0663] Step 4:

[0664] The smart speaker collects voice data from the user's daily life (e.g., "How much time did you exercise today?") and sends it to the device, which then uploads the data to a server.

[0665] Step 5:

[0666] The terminal periodically collects the user's online shopping and spending history (such as credit card history) and sends it to a server.

[0667] Step 6:

[0668] The server analyzes the received health checkup report using OCR technology and extracts important health indicators such as blood pressure, blood sugar level, and cholesterol level.

[0669] Step 7:

[0670] The server analyzes the biometric data received from the wearable device to understand the user's health patterns, such as heart rate fluctuations, daily steps taken, and sleep quality.

[0671] Step 8:

[0672] The server analyzes photos of meals using image recognition technology, and calculates and evaluates the calories and nutritional balance of each meal.

[0673] Step 9:

[0674] The server analyzes lifestyle data obtained from the smart speaker and evaluates the user's activity patterns and stress levels.

[0675] Step 10:

[0676] The server analyzes the user's online shopping history and spending history, evaluates the balance between income and expenses, and identifies spending trends.

[0677] Step 11:

[0678] The server then uses the analysis results to generate optimal life and health insurance plans for the user, including selecting insurance to mitigate the user's health risks.

[0679] Step 12:

[0680] The server then provides optimal financial planning and investment recommendations based on the user's health status and lifestyle, including investment strategies to ensure financial security in retirement.

[0681] Step 13:

[0682] The server proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[0683] Step 14:

[0684] The terminal displays the insurance plans and financial plans proposed by the server to the user, who can then review the proposals and provide feedback if necessary.

[0685] Step 15:

[0686] The server receives feedback from the user, adjusts the proposal, and again provides the user with an optimized plan.

[0687] Example 1

[0688] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0689] There is currently no system that can centrally provide optimal life insurance, medical insurance, financial planning, investment proposals, and budget allocation for leisure activities based on diverse health and lifestyle data. There is also a lack of systems that can continuously learn and update a user's health status and lifestyle patterns over the long term. Furthermore, there is a need for a system that can automatically generate optimal proposals from data using generative AI models and provide an approach that meets the user's needs.

[0690] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0691] In this invention, the server includes means for uploading health checkup reports, means for receiving biometric data from the user's wearable device, means for receiving meal photos taken by the user, means for collecting daily life data from a voice recognition device, means for receiving online shopping and expenditure history, means for analyzing the health checkup reports, biometric data, meal photos, daily life data, and expenditure history to propose life insurance and medical insurance tailored to each individual user, means for optimizing the user's financial plan, investment proposals, and budget allocation for leisure activities based on the analysis results, means for receiving feedback from the user and adjusting the proposals, means for accumulating the biometric data and daily life data over the long term and continuously learning and updating the user's health condition and lifestyle patterns, means for generating proposals based on the data analysis results using a generative AI model, and means for using prompt sentences at each step of the processing flow. This makes it possible to integrate various personal data and provide optimal advice and plans.

[0692] A "health checkup report" is a document that lists the results of a health checkup conducted at a hospital or clinic, and includes health indicators such as blood pressure, blood sugar level, and cholesterol level.

[0693] A "wearable device" is an electronic device that can be worn by a user and has the ability to collect biometric data such as heart rate, number of steps taken, and sleep patterns.

[0694] "Biometric data" refers to data that indicates a person's physiological state, such as heart rate, number of steps, and sleep patterns.

[0695] A "meal photo" is a digital image taken by a user to record the contents of a meal.

[0696] A "voice recognition device" is a device that collects and analyzes voice data and has the function of recognizing voice commands and comments made by users in their daily lives.

[0697] "Online shopping history" is a record of a user's purchasing activities on the Internet, including information such as purchase date, product, and amount.

[0698] "Expense history" is a record of expenses incurred by a user in their daily life, including credit card statements, bank transaction statements, etc.

[0699] "Means of proposing life insurance and medical insurance" refers to the process of analyzing the user's health condition and lifestyle data, selecting the most suitable insurance plan, and offering it to the user.

[0700] "Means for optimizing financial planning and investment proposals" refers to the process of analyzing a user's income, expenses, and asset information and proposing long-term financial and investment plans.

[0701] "Means for optimizing budget allocation for leisure activities" refers to a process that takes into account the user's daily life and spending history and suggests allocating an appropriate budget for leisure activities.

[0702] A "generative AI model" is a model that uses artificial intelligence technology to analyze data and automatically generate optimal suggestions.

[0703] A "prompt" is an instruction that can be input into a generative AI model to perform a specific analysis or suggestion.

[0704] This invention is a system that integrates health checkup reports collected over multiple years, biometric data from wearable devices, photographic data for each meal, lifestyle data from a voice recognition device, and online shopping and expenditure history to select life insurance and medical insurance suited to an individual's health condition, as well as financial planning, investment proposals, and budget allocation for leisure activities. This system collects, analyzes, and optimizes data through mutual communication between a server, terminals, and users.

[0705] Data collection

[0706] Uploading your medical checkup report

[0707] The terminal provides the user with an interface and prompts them to upload their health checkup report. The user selects a PDF file and saves it on the terminal. The terminal then sends this file to the server. Specifically, the user uploads last year's health checkup report.

[0708] Biometric data collection from wearable devices

[0709] The device connects to the user's wearable device and periodically collects biometric data (heart rate, steps, sleep patterns, etc.) via Bluetooth or Wi-Fi. This collected data is then sent from the device to a server. For example, the user's wearable device sends daily heart rate and step count data to the server.

[0710] Upload a meal photo

[0711] The device saves photos of each meal taken by the user and uploads them to the server. The photos are automatically dated and time stamped. The user takes photos of each meal and saves them on the device through the app. For example, the user takes a photo of their lunch and the photo is uploaded to the server.

[0712] Lifestyle data collection from voice recognition devices

[0713] The voice recognition device collects voice data from the user's daily life. For example, if the user says, "I'm going to jog for 30 minutes today," the voice recognition device recognizes the voice and sends the data to the server via the device.

[0714] Collecting online shopping and spending history

[0715] The device periodically collects the user's online shopping history and credit card details and sends them to the server. For example, when a user makes a purchase on a shopping site, the details are automatically recorded and uploaded to the server.

[0716] Data analysis

[0717] Analysis of medical examination reports

[0718] The server analyzes the uploaded health checkup report, extracting text data from the PDF using OCR technology and extracting important health indicators (blood pressure, blood sugar, cholesterol levels, etc.), which are then stored in a database.

[0719] Biological data analysis

[0720] The server analyzes the biometric data sent from the wearable device and uses time-series data analysis to understand the user's daily health patterns (heart rate variability, daily steps, sleep quality, etc.).

[0721] Food photo analysis

[0722] The server analyzes the photos of the meals using image recognition technology (such as YOLO or ResNet), evaluates the calories and nutritional balance of each meal, and stores that information in a database.

[0723] Lifestyle data analysis

[0724] The server analyzes the voice data obtained from the voice recognition device and uses voice recognition technology to evaluate the user's lifestyle patterns and stress levels. This data is used to analyze the user's lifestyle.

[0725] Analysis of shopping and spending history

[0726] The server analyzes online shopping and spending histories to assess the balance between income and expenditure, and uses machine learning algorithms to identify spending trends and build predictive models.

[0727] Proposal generation and refinement

[0728] Insurance plan proposals

[0729] The server then proposes the most suitable life and medical insurance plan for the user based on the analysis results. For example, if the user has high blood pressure, it will recommend an insurance plan that addresses the risk of high blood pressure.

[0730] Generate financial plans and investment proposals

[0731] The server will provide financial planning and investment recommendations, taking into account the user's health status and lifestyle patterns. For example, it will analyze the user's income and expenses and suggest mutual funds and pension plans.

[0732] Leisure activity budget proposal

[0733] The server then suggests budget allocations for leisure activities to the user. For example, it extracts entertainment expenses from the user's daily expenses and suggests spending 10,000 yen per month on leisure activities.

[0734] Feedback and Optimization

[0735] Receiving feedback from users

[0736] The device displays the proposed insurance and financial plans to the user and accepts their feedback, allowing them to submit comments and revisions to the proposals based on their preferences and needs.

[0737] Adjustment and resubmission of proposals

[0738] The server adjusts its proposals based on user feedback. For example, if a user gives feedback that the monthly premium for a proposed insurance plan is too high, the server will recalculate the premium and propose a more suitable plan.

[0739] Using generative AI models

[0740] The server generates proposals based on the results of data analysis using a generative AI model. The generative AI model in this system is designed to analyze information from various data sources and automatically generate optimal proposals.

[0741] Using prompt statements

[0742] At each step in the processing flow, the generative AI model is given a prompt like this:

[0743] "Please suggest an appropriate medical insurance plan for a 50-year-old man based on his medical examination results."

[0744] "Analyze your daily health patterns based on your heart rate and step count data."

[0745] "Evaluate the calories and nutritional balance from the food photos."

[0746] "Evaluate your income and expenditure balance based on your online shopping history and suggest the best financial plan for you."

[0747] This makes it possible to integrate various personal data and provide optimal advice and plans.

[0748] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0749] The processing flow (processing steps) of this system's program

[0750] Step 1: Data collection

[0751] The user uploads a health checkup report in PDF format. The terminal provides an interface to the user and saves the uploaded PDF file on the terminal. The input is the PDF file of the health checkup report, and the output is sending the PDF file to the server.

[0752] Step 2: Collecting biometric data from wearable devices

[0753] The terminal periodically collects biometric data (heart rate, number of steps, sleep patterns, etc.) from the user's wearable device via Bluetooth or Wi-Fi. The input is the biometric data from the wearable device, and the output is the transmission of the biometric data to the server.

[0754] Step 3: Upload your meal photos

[0755] The app stores photos of each meal taken by the user on their device. The device adds date information and uploads them to the server. The input is the meal photo and date information, and the output is the transmission of the meal photo to the server.

[0756] Step 4: Collecting lifestyle data from the voice recognition device

[0757] The speech recognition device collects voice data from the user's daily life and recognizes utterances such as "Today's jogging time is 30 minutes." The recognized data is sent to the server via the terminal. The input is the user's voice data, and the output is the transmission of the recognized data to the server.

[0758] Step 5: Collect your online shopping and spending history

[0759] The terminal periodically collects the user's online shopping history and credit card details and sends them to the server. The input is the online shopping history and spending history, and the output is the transmission of data to the server.

[0760] Step 6: Data analysis

[0761] The server uses OCR technology to extract text data from PDF health checkup reports and analyzes important health indicators such as blood pressure and blood sugar levels. It then performs time-series analysis of the biometric data to understand the user's daily health patterns. It uses image recognition technology to evaluate calories and nutritional balance from food photos, and analyzes lifestyle patterns and stress levels from audio data. It also evaluates the balance between income and expenditures from online shopping history and spending history. The input is various collected data, and the output is the analysis results.

[0762] Step 7: Propose an insurance plan

[0763] The server then proposes the most suitable life insurance or medical insurance plan to the user based on the analysis results. For example, if the user has high blood pressure, it will recommend an insurance plan that addresses the risk of high blood pressure. The input is the analysis results, and the output is a proposed insurance plan.

[0764] Step 8: Generate a financial plan and investment proposal

[0765] The server makes financial plans and investment recommendations, taking into account the user's health status and lifestyle patterns. It analyzes the user's income and expenses and suggests mutual funds and pension plans. The input is the analysis results, and the output is financial plans and investment plan recommendations.

[0766] Step 9: Propose a leisure activity budget

[0767] The server extracts entertainment expenses from the user's daily expenditures and suggests spending 10,000 yen per month on leisure activities. The input is the analysis result, and the output is a suggested leisure activity budget.

[0768] Step 10: Receive feedback from users

[0769] The terminal displays the proposed insurance plan or financial plan to the user and accepts feedback. The user can submit comments or requests for revisions to the proposal through the terminal. The input is the user's feedback, and the output is the collection of the feedback.

[0770] Step 11: Adjust and resubmit your proposal

[0771] The server adjusts the proposal based on user feedback. For example, if the user gives feedback that the monthly premium for a proposed insurance plan is too high, the server recalculates the premium and proposes a more suitable plan. The input is the feedback, and the output is the adjusted proposal.

[0772] Step 12: Use a generative AI model and prompts

[0773] The server uses a generative AI model to generate recommendations based on the results of data analysis. For example, a prompt such as "Please suggest an appropriate medical insurance plan based on the medical examination results of a 50-year-old male" can be input, and the AI ​​model will generate the optimal insurance plan based on that. The input is the prompt, and the output is recommendations for the optimal insurance plan, financial plan, leisure activity budget, etc.

[0774] (Application example 1)

[0775] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0776] In modern society, systems that efficiently collect and analyze diverse information are necessary to manage users' health and lifestyle patterns, and to provide individualized financial planning and security measures. However, while many systems exist that manage this information individually, there are still few systems that centrally integrate this information and provide optimal recommendations to users. Furthermore, there has been insufficient work done to evaluate security risks and propose countermeasures based on users' health information and lifestyle patterns. Therefore, there is a need for the development of a system that can provide comprehensive lifestyle support and security measures.

[0777] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0778] In this invention, the server includes means for uploading health checkup reports, means for receiving biometric data from the user's wearable device, means for receiving photos of meals taken by the user, means for collecting daily life data from a smart speaker, means for receiving online shopping and expenditure history, means for receiving video data from home surveillance cameras, means for analyzing the health checkup reports, biometric data, meal photos, daily life data, expenditure history, and home surveillance camera data to propose life insurance and medical insurance tailored to each individual user, means for optimizing the user's financial plan, investment proposals, and budget allocation for leisure activities based on the analysis results, and means for proposing security measures, means for receiving user feedback and adjusting the proposals, financial plan, investment proposals, budget allocation for leisure activities, and security measures, and means for accumulating the biometric data, daily life data, and home surveillance camera data over the long term and continuously learning and updating the user's health condition, lifestyle patterns, and security risks. This enables the integrated management and analysis of a user's diverse data to provide optimal proposals tailored to individual needs and risks.

[0779] A "health checkup report" is a document that describes the results of periodic examinations conducted to assess a User's health status.

[0780] A "wearable device" is an electronic device that can be worn by a user and collects biometric data such as heart rate, number of steps taken, and sleep patterns.

[0781] "Biometric data" refers to information such as a user's heart rate, number of steps, and sleep patterns obtained from wearable devices.

[0782] "Photos of meals" are image data of meals taken by the user and are used to evaluate the meal contents.

[0783] A "smart speaker" is an electronic device that receives commands from users via voice recognition and performs tasks such as playing music, providing information, and collecting daily life data.

[0784] "Daily Life Data" is data about a user's daily life collected from smart speakers and other home devices.

[0785] "Online shopping and spending history" refers to records of a user's purchasing activities and spending via the Internet.

[0786] A "home surveillance camera" is a video camera installed to monitor the interior and exterior of a home.

[0787] "Video data" refers to video information captured by a home surveillance camera.

[0788] "Analysis" refers to the process of processing information based on collected data using methods such as statistical analysis and machine learning to extract trends and patterns.

[0789] "Life insurance" is insurance in which an insurance benefit is paid based on the contract when the policyholder dies.

[0790] "Medical insurance" is insurance that covers medical expenses when you receive treatment for illness or injury.

[0791] "Financial planning" involves comprehensively analyzing a user's income, expenses, investments, etc. to plan their future economic situation.

[0792] "Investment Suggestions" provide optimal investment strategies based on a user's risk profile and goals.

[0793] "Leisure activity budget allocation" is a plan to effectively manage and appropriately allocate expenses for users' leisure activities.

[0794] The "security measures plan" is a plan that assesses security risks based on the user's individual health condition and lifestyle patterns and proposes countermeasures.

[0795] "Feedback" refers to the evaluations and opinions that users provide to the system.

[0796] "Long-term accumulation" means collecting data over a certain period of time and continuing to accumulate it.

[0797] "Learning" refers to the system's ability to recognize patterns and trends based on the data it collects and to improve its ability to adapt to new data.

[0798] "Updating" means keeping the contents of the system up to date by adding, correcting, or reflecting new data or information.

[0799] This invention is a system that comprehensively collects and analyzes users' health information, lifestyle patterns, expenditure history, and security-related data, and proposes optimal life insurance, medical insurance, financial planning, investment proposals, security measures plans, etc. This system collects, analyzes, and optimizes data through mutual communication between the server, terminals, and users.

[0800] Data collection

[0801] 1. Device: Provides a user interface and prompts the user to upload the health checkup report. The user selects the PDF file and saves it to the device.

[0802] 2. Device: Works with the user's wearable device to periodically receive biometric data (heart rate, steps, sleep patterns, etc.) and send it to the server.

[0803] 3. Device: The user takes photos of each meal and saves them to the server. The photos contain date and time information.

[0804] 4. Smart speaker: Collects voice data from the user's daily life (e.g., "How much exercise time did you do today?") and sends it to a server via the device.

[0805] 5. Device: Periodically collects the user's online shopping and spending history and sends it to the server.

[0806] 6. Home surveillance camera: Receives video data from home surveillance cameras as a standalone auxiliary device and transmits it to the server through the terminal.

[0807] Data analysis

[0808] 1. Server: Analyzes the received health checkup report and extracts important health indicators (blood pressure, blood sugar level, cholesterol level, etc.). This analysis is performed using machine learning models such as TensorFlow.

[0809] 2. Server: Analyzes biometric data from wearable devices to understand the user's daily health patterns (heart rate variability, daily steps, sleep quality, etc.).

[0810] 3. Server: Using photo analysis software (e.g., OpenCV), the server analyzes photos of meals and evaluates the calories and nutritional balance of each meal.

[0811] 4. Server: Analyzes lifestyle data obtained from the smart speaker and evaluates the user's lifestyle patterns and stress levels.

[0812] 5. Server: Analyzes the user's online shopping and spending history to assess the balance between income and expenditure and identify spending trends.

[0813] 6. Server: Analyzes video data from home surveillance cameras, detects anomalies, and assesses security risks.

[0814] Proposal generation and refinement

[0815] 1. Server: Based on the analysis results, the server proposes the best life and health insurance plan for the user. This proposal includes insurance selection to reduce the user's health risks.

[0816] 2. Server: Providing financial planning and investment recommendations taking into account the user's health status and lifestyle, including suitable investment options and retirement financial planning.

[0817] 3. Server: Proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[0818] 4. Server: Proposes a security plan based on the user's individual health condition and lifestyle patterns, including emergency contact information when a sudden change in heart rate is detected and countermeasures when an abnormal person is detected on security cameras.

[0819] Feedback and Optimization

[0820] 1. Device: Shows the user the proposed insurance plan, financial plan, or security plan and accepts their feedback. The user reviews the proposal and provides feedback on any necessary adjustments via the device.

[0821] 2. Server: Based on user feedback, the server adjusts its proposals and re-proposes optimal insurance, financial, and security plans.

[0822] Examples of concrete examples and prompts

[0823] Example: If the user detects a sudden change in heart rate, the system:

[0824] 1. The wearable device detects sudden changes in your heart rate and sends a notification to your smartphone.

[0825] 2. Your smartphone will automatically send a message to the people registered as emergency contacts.

[0826] 3. The home surveillance camera detects an abnormality and the surveillance footage can be viewed via a smartphone app.

[0827] 4. The app will provide users with an emergency response plan and the option to call emergency services if necessary.

[0828] Example prompt sentence:

[0829] "Your heart rate has increased dramatically. Would you like to notify your emergency contacts?"

[0830] "We've detected something unusual. Would you like us to check the surveillance footage?"

[0831] "Should I call emergency services?"

[0832] This system makes it possible to manage and analyze a wide range of user data in an integrated manner, and to make optimal proposals based on individual needs and risks.

[0833] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0834] Step 1:

[0835] Terminal: Provides a means for uploading the user's health checkup report. Specifically, it displays a health checkup report upload button on the terminal's user interface and saves the PDF file selected by the user on the terminal. The input is the health checkup report in PDF format, and the output is a PDF file saved on the terminal.

[0836] Step 2:

[0837] Terminal: Works with the user's wearable device and provides a means to receive biometric data. Specifically, it periodically receives heart rate, step count, sleep patterns, etc. from the wearable device using communication technologies such as Bluetooth, and sends this to a server. The input is the biometric data sent from the wearable device, and the output is the biometric data sent to the server.

[0838] Step 3:

[0839] Terminal: Provides a means to receive and store photos of meals taken by users. Specifically, the user takes a photo of their meal using the device's camera function, saves the image data on the device, and sends it to the server. The input is the photo of the meal taken by the user, and the output is the image data sent to the server.

[0840] Step 4:

[0841] Smart speaker: Collects voice data related to the user's daily life. Specifically, the smart speaker recognizes the voice commands given by the user (e.g., "I'll exercise for 30 minutes today") and sends them as text data to a server via the device. The input is voice data, and the output is the text data sent to the server.

[0842] Step 5:

[0843] Terminal: Provides a means to periodically collect a user's online shopping and spending history and send it to a server. Specifically, it connects with the user's credit card and bank account to obtain spending history and send it to the server. The input is spending history data, and the output is the data sent to the server.

[0844] Step 6:

[0845] Home surveillance cameras: Receives video data from home surveillance cameras. Specifically, cameras installed in the home capture video in real time and send the data to a server via a terminal. The input is video data, and the output is the video data sent to the server.

[0846] Step 7:

[0847] Server: Analyzes the received health checkup report. Specifically, it uses OCR (Optical Character Recognition) technology to extract data from the PDF format health checkup report and analyzes the health indicators using a machine learning model (e.g., TensorFlow). The input is the PDF file of the health checkup report, and the output is the analyzed health indicator data.

[0848] Step 8:

[0849] Server: Analyzes biometric data from wearable devices to understand the user's daily health patterns. Specifically, it analyzes biometric data over time to evaluate heart rate fluctuations, daily steps, sleep quality, etc. The input is biometric data, and the output is health pattern data.

[0850] Step 9:

[0851] Server: Analyzes photos of meals using image analysis technology to evaluate the calories and nutritional balance of each meal. Specifically, image analysis software (e.g., OpenCV) is used to analyze the photos and evaluate the meal content. The input is a photo of the meal, and the output is data on calories and nutritional balance.

[0852] Step 10:

[0853] Server: Analyzes lifestyle data obtained from the smart speaker and evaluates the user's lifestyle patterns and stress levels. Specifically, it analyzes voice data using natural language processing technology and evaluates things like exercise time. The input is voice data in text format, and the output is data on lifestyle patterns and stress levels.

[0854] Step 11:

[0855] Server: Analyzes the user's online shopping history and spending history, evaluates the balance between income and expenses, and identifies spending trends. Specifically, it performs statistical analysis of the collected spending history data and evaluates monthly expenses and trends. The input is spending history data, and the output is financial evaluation data.

[0856] Step 12:

[0857] Server: Analyzes video data from home surveillance cameras and detects abnormalities. Specifically, it uses video analysis software to recognize abnormal behavior and suspicious individuals and evaluate security risks. The input is video data, and the output is security risk assessment data.

[0858] Step 13:

[0859] Server: Based on the analysis results, the server proposes the most suitable life insurance or medical insurance plan to the user. Specifically, the server selects and proposes the most suitable insurance plan based on the analyzed health data. The input is health index data, and the output is insurance plan proposal data.

[0860] Step 14:

[0861] Server: Considers the user's health condition and lifestyle patterns and makes financial plans and investment proposals. Specifically, it proposes investment plans and financial plans based on analyzed biometric data and expenditure history. The input is health pattern data and financial evaluation data, and the output is financial plan proposal data.

[0862] Step 15:

[0863] Server: Proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of life. Specifically, it proposes the optimal budget allocation for leisure activities based on collected data. The input is expenditure history data, and the output is leisure activity budget allocation proposal data.

[0864] Step 16:

[0865] Server: Proposes a security plan based on the user's health condition and lifestyle patterns. Specifically, it provides emergency contact information when a sudden change in heart rate is detected, and countermeasures when an abnormal person is detected on a surveillance camera. The input is biometric data and video data, and the output is security plan data.

[0866] Step 17:

[0867] Terminal: Displays proposed insurance plans, financial plans, and security measures plans to the user and accepts feedback. Specifically, the proposals are displayed on the terminal screen and user feedback is received. The input is the proposal data and the output is the feedback data.

[0868] Step 18:

[0869] Server: Adjusts the proposal based on user feedback and re-proposes an optimized plan. Specifically, it analyzes the feedback and updates the proposal according to the user's requests. The input is the feedback data, and the output is the updated proposal data.

[0870] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0871] This invention is a system that integrates health checkup reports collected over multiple years, biometric data from wearable devices, photographic data for each meal, lifestyle data from smart speakers, and online shopping and expenditure history to select life insurance and medical insurance suited to an individual's health condition, as well as financial planning, investment proposals, and budget allocation for leisure activities, and also incorporates an emotion engine that recognizes the user's emotions. This system collects, analyzes, and optimizes data through mutual communication between the server, terminals, and users, and further improves the proposal content by taking emotional information into account.

[0872] Program processing explanation

[0873] Data collection

[0874] 1. The device prompts the user to upload the health checkup report. The user saves the PDF file of the health checkup report on the device and uploads it.

[0875] 2. The terminal periodically receives biometric data (heart rate, steps, sleep patterns, etc.) from the user's wearable device and transmits this data to the server.

[0876] 3. The device saves photos of each meal taken by the user and uploads them to the server. The photos contain date and time information.

[0877] 4. The smart speaker collects voice data from the user's daily life (e.g., "How much exercise time did you do today?") and sends it to the device, which then uploads the data to the server.

[0878] 5. The device periodically collects the user's online shopping and spending history (such as credit card history) and sends it to the server.

[0879] 6. The device sends the voice data and image data to the emotion engine to recognize the user's emotions and extracts the emotion information.

[0880] Data analysis

[0881] 1. The server analyzes the received health checkup report using OCR technology and extracts important health indicators such as blood pressure, blood sugar level, and cholesterol level.

[0882] 2. The server analyzes the received biometric data from the wearable device to understand the user's health patterns, such as heart rate fluctuations, daily steps taken, and sleep quality.

[0883] 3. The server analyzes the photos of the meals using image recognition technology, and calculates and evaluates the calories and nutritional balance of each meal.

[0884] 4. The server analyzes the lifestyle data obtained from the smart speaker and evaluates the user's activity patterns and stress levels.

[0885] 5. The server analyzes the user's online shopping history and spending history, evaluates the balance between income and expenses, and identifies spending trends.

[0886] 6. The emotion engine analyzes the user's emotional information from voice and image data to understand their current emotional state (e.g., joy, anger, sadness, surprise, etc.).

[0887] Proposal generation and refinement

[0888] 1. The server generates the optimal life and health insurance plan for the user based on the analysis results and emotional information, including selecting insurance that takes into account the user's health risks and emotional state.

[0889] 2. The server takes into account the user's health status, lifestyle patterns, and emotional information to provide optimal financial planning and investment recommendations, including investment strategies for financial security in retirement.

[0890] 3. The server proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[0891] Feedback and Optimization

[0892] 1. The terminal displays the insurance plan and financial plan proposed by the server to the user. The user reviews the proposal and provides feedback if necessary.

[0893] 2. The server receives feedback from the user, adjusts the proposal, and again provides the user with an optimized plan.

[0894] Specific examples

[0895] For example, if a 50-year-old male user is using this system, it will operate as follows:

[0896] Data collection

[0897] The user uses the device to upload last year's health checkup report. Daily heart rate and step count data are automatically sent from the wearable device to the server. The user takes a photo of their lunch, which is uploaded to the server. The smart speaker records that "today's jogging time was 30 minutes" and sends this data to the server. The user's credit card history is periodically uploaded to the server. Furthermore, the emotion engine analyzes the user's facial image and voice data to recognize the emotion of "joy."

[0898] Data analysis

[0899] The server analyzes health checkup reports and extracts important indicators such as blood pressure of 130 / 85. Data from wearable devices confirms that the person walks 7,000 steps per day. Food photos are analyzed and evaluated as 700 calories or less. Data from smart speakers reveals that the person jogs for 150 minutes per week. Credit card history confirms that monthly food expenses are 30,000 yen. An emotion engine recognizes the emotion of happiness and sends it to the server.

[0900] Proposal generation and refinement

[0901] The server proposes a medical insurance plan tailored to the user's blood pressure risk. It also presents an exercise plan to maintain a jogging habit. Taking into account food and entertainment expenses, it suggests a monthly budget of 10,000 yen for leisure activities. The emotion engine grasps the user's feelings of joy and makes suggestions that will further increase satisfaction.

[0902] Feedback and Optimization

[0903] The user reviews the suggestions and provides feedback on what they dislike. The server receives the feedback, adjusts the suggestions, and makes new suggestions to optimize them.

[0904] The processing flow will be explained below.

[0905] Step 1:

[0906] The device prompts the user to upload the health checkup report. The user saves the PDF file of the health checkup report on the device and uploads it.

[0907] Step 2:

[0908] The terminal periodically receives biometric data (e.g., heart rate, number of steps, sleep patterns) from the user's wearable device and transmits this data to a server.

[0909] Step 3:

[0910] The device stores photos of each meal taken by the user and uploads them to a server, with the photos containing date and time information.

[0911] Step 4:

[0912] The smart speaker collects voice data from the user's daily life (e.g., "How much time did you exercise today?") and sends it to the device, which then uploads the data to a server.

[0913] Step 5:

[0914] The terminal periodically collects the user's online shopping and spending history (such as credit card history) and sends it to a server.

[0915] Step 6:

[0916] The terminal transmits voice data and image data for recognizing the user's emotions to an emotion engine, and extracts emotion information.

[0917] Step 7:

[0918] The server analyzes the received health checkup report using OCR technology and extracts important health indicators such as blood pressure, blood sugar level, and cholesterol level.

[0919] Step 8:

[0920] The server analyzes the biometric data received from the wearable device to understand the user's health patterns, such as heart rate fluctuations, daily steps taken, and sleep quality.

[0921] Step 9:

[0922] The server analyzes photos of meals using image recognition technology, and calculates and evaluates the calories and nutritional balance of each meal.

[0923] Step 10:

[0924] The server analyzes lifestyle data obtained from the smart speaker (e.g., work hours, rest hours, and exercise frequency) to evaluate the user's activity patterns and stress levels.

[0925] Step 11:

[0926] The server analyzes the user's online shopping history and spending history, evaluates the balance between income and expenses, and identifies spending trends.

[0927] Step 12:

[0928] The emotion engine analyzes the user's emotional information from voice and image data to understand their current emotional state (e.g., joy, anger, sadness, surprise, etc.).

[0929] Step 13:

[0930] The server generates optimal life and health insurance plans for users based on the analysis results and emotional information, including insurance selection that takes into account the user's health risks and emotional state.

[0931] Step 14:

[0932] The server takes into account the user's health status, lifestyle patterns, and emotional information to provide optimal financial planning and investment recommendations, including investment strategies for financial security in retirement.

[0933] Step 15:

[0934] The server proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[0935] Step 16:

[0936] The terminal displays the insurance plans and financial plans proposed by the server to the user, who can then review the proposals and provide feedback if necessary.

[0937] Step 17:

[0938] The server receives feedback from the user, adjusts the proposal, and again provides the user with an optimized plan.

[0939] Specific examples

[0940] For example, if a 50-year-old male user wants to use this system to reduce his health risks and financial worries, the process would look like this:

[0941] Data collection

[0942] Step 1:

[0943] The user uploads their health checkup report from the previous year using their device, and the server then obtains basic data to understand the user's health condition.

[0944] Step 2:

[0945] The wearable device automatically sends daily heart rate and step count data to a server, providing daily activity and health indicators.

[0946] Step 3:

[0947] Users take photos of their lunch and upload them to a server. The image data includes date and time information, and is used to analyze daily meal content.

[0948] Step 4:

[0949] The smart speaker records "Today's jogging time is 30 minutes" and sends the data to a server, where it is used to evaluate the user's exercise level.

[0950] Step 5:

[0951] The user's credit card history is periodically uploaded to the server, which allows the company to understand the user's spending patterns and expenditures.

[0952] Step 6:

[0953] The emotion engine recognizes the emotion "happiness" from the user's voice and facial image data. This information is used to evaluate the user's daily life and stress level.

[0954] Data analysis

[0955] Step 7:

[0956] The server analyzes the health checkup report and extracts important indicators, such as blood pressure 130 / 85, to assess the user's health risk.

[0957] Step 8:

[0958] The wearable device data confirms that the user is walking 7,000 steps per day, allowing the user to understand their daily activity level.

[0959] Step 9:

[0960] It analyzes photos of meals and evaluates the calorie and nutritional balance of each meal, calculating, for example, 700 calories or the percentage of specific nutrients.

[0961] Step 10:

[0962] Data from the smart speaker reveals that the user jogs 150 minutes per week, helping to understand the user's exercise patterns.

[0963] Step 11:

[0964] Check the credit card history to see that the monthly food expense is 30,000 yen, thereby understanding the user's spending pattern.

[0965] Step 12:

[0966] The emotion engine recognizes the user's "happiness" emotion and sends it to the server, which uses it to evaluate the user's emotional feedback on the suggestions.

[0967] Proposal generation and refinement

[0968] Step 13:

[0969] The server proposes a medical insurance plan tailored to the user's blood pressure risk and also provides an exercise plan to help maintain the user's jogging habit.

[0970] Step 14:

[0971] Taking into account the user's health status, lifestyle patterns, and emotional information, the system makes investment suggestions to ensure financial stability in retirement.

[0972] Step 15:

[0973] Taking into account food and entertainment expenses, the system suggests a monthly budget of 10,000 yen for leisure activities. The emotion engine understands the user's feelings of joy and makes suggestions that will further increase satisfaction.

[0974] Feedback and Optimization

[0975] Step 16:

[0976] The user reviews the suggestions and provides feedback on what they don't like. The device receives the feedback and sends it to the server.

[0977] Step 17:

[0978] The server adjusts the proposal based on the feedback and makes new proposals to optimize the results.

[0979] Through the above specific processing steps, the system of the present invention provides individual users with optimized health management, insurance selection, financial planning, investment suggestions and leisure activity budget allocation, thereby improving the user's overall quality of life.

[0980] Example 2

[0981] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0982] Until now, there has been no system that efficiently selects life insurance and medical insurance suitable for individual users, provides financial planning, investment recommendations, and budget allocation for leisure activities. It has also been difficult to comprehensively analyze a user's health condition, lifestyle patterns, and even emotional state and make recommendations based on that analysis. Furthermore, it has also been difficult to integrate various data provided by users (health checkup reports, biometric data, food photos, daily life data, expenditure history, etc.), and accumulate and analyze them over the long term. The present invention aims to solve these problems.

[0983] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0984] In this invention, the server includes means for uploading health checkup reports, means for receiving biometric data from the user's wearable device, means for receiving photos of meals taken by the user, means for collecting daily life data from a voice recognition device, means for receiving online shopping and expenditure history, means for analyzing the user's emotional information from the voice data and image data using an emotion engine, means for analyzing the health checkup reports, biometric data, meal photos, daily life data, expenditure history, and emotional information to propose life insurance and medical insurance optimized for each user, means for optimizing the user's financial plan, investment proposals, and budget allocation for leisure activities based on the analysis results, means for receiving user feedback and adjusting the proposals, and means for accumulating the biometric data, daily life data, and emotional information over the long term and continuously learning and updating the user's health condition and lifestyle patterns, thereby enabling automatic and efficient proposals of insurance, financial plan, and budget allocation for leisure activities that are optimal for each user's condition.

[0985] A "health checkup report" is written or digital data indicating the user's health status obtained through a health checkup.

[0986] A "wearable device" is an electronic device worn by a user that measures biometric data (heart rate, number of steps, sleep patterns, etc.) and provides the data.

[0987] "Biometric data" refers to digital data that indicates a user's physical condition, including information such as heart rate, number of steps taken, sleep patterns, body temperature, and blood pressure.

[0988] A "meal photo" is image data taken by a user to record the contents of a meal, and indicates the type, amount, and visual characteristics of the meal.

[0989] A "voice recognition device" is a device that collects a user's voice and processes the content of that voice as digital data.

[0990] "Daily life data" refers to data related to the user's daily activities and behavior, including exercise time, sleep time, and speech content.

[0991] "Online shopping history" refers to a record of purchases made by a user online, including information such as the items purchased, the date and time of purchase, and the purchase amount.

[0992] "Expense history" refers to a record of a user's spending activities, including credit card statements, bank statements, cash usage records, etc.

[0993] An "emotion engine" is a system or software that analyzes a user's emotional state from voice and image data and identifies those emotions.

[0994] "Financial planning" is a plan to comprehensively manage a user's financial situation, including income, expenses, savings, and investments, in order to ensure future economic stability.

[0995] "Investment Suggestions" are suggestions to present appropriate investment products and investment strategies based on the User's financial situation and risk tolerance.

[0996] "Leisure Budget Allocation" refers to how the user allocates funds for leisure and entertainment activities, providing an appropriate spending plan.

[0997] "Feedback" refers to information such as opinions, impressions, and requests from users regarding presented proposals and plans.

[0998] This invention is a system that integrates health checkup reports collected over multiple years, biometric data from wearable devices, photographic data for each meal, lifestyle data from a voice recognition device, and online shopping and expenditure history to select life insurance and medical insurance suited to an individual's health condition, as well as financial planning, investment recommendations, and budget allocation for leisure activities. Furthermore, by incorporating an emotion engine, the system takes into account the user's emotional information and improves the content of its recommendations.

[0999] Hardware and software used

[1000] The system is implemented using the following hardware and software:

[1001] Terminal: A device such as a smartphone or computer operated by a user.

[1002] Wearable devices: Devices that collect biometric data such as heart rate, steps taken, and sleep patterns.

[1003] Voice recognition devices: Devices that collect daily life data (e.g., smart speakers).

[1004] Server: A computer system that analyzes data and generates recommendations.

[1005] Emotion engine: Software that analyzes a user's emotional information from voice and image data.

[1006] Data processing and calculation

[1007] The program in this system performs the following data processing and calculations:

[1008] 1. The device will notify the user through a dedicated application to prompt them to upload their health checkup report. Once the user uploads the report, the server will analyze the content using OCR technology and extract important health indicators (blood pressure, blood sugar level, cholesterol level, etc.).

[1009] 2. Biometric data from the wearable device is sent via the device to a server, which analyzes the data to understand the user's health patterns, such as heart rate fluctuations, daily steps, and sleep quality.

[1010] 3. After users take photos of their meals using their devices, they automatically upload them to a server, where they use image recognition technology to analyze the photos and evaluate the calorie and nutritional balance of each meal.

[1011] 4. Daily life data collected by voice recognition devices (such as smart speakers) is transmitted to a server via the device, where it is analyzed to assess activity patterns and stress levels.

[1012] 5. Online shopping history and spending history are automatically collected by the device and sent to a server, which analyzes this data to assess the balance between income and expenditure and identify spending trends.

[1013] 6. The emotion engine analyzes the user's emotional information from voice and image data to understand their current emotional state (e.g., joy, anger, sadness, surprise, etc.). This allows the server to generate optimal suggestions that take the emotional information into account.

[1014] Examples of concrete examples and prompts

[1015] For example, if a 50-year-old male user is using this system, the following will happen:

[1016] Data collection

[1017] The user uses the device to upload last year's health checkup report. Daily heart rate and step count data are automatically sent from the wearable device to the server. The user takes a photo of their lunch, which is uploaded to the server. The smart speaker records that "today's jogging time was 30 minutes" and sends this data to the server. The user's credit card history is periodically uploaded to the server. Furthermore, the emotion engine analyzes the user's facial image and voice data to recognize the emotion of "joy."

[1018] Data analysis

[1019] The server analyzes health checkup reports and extracts important indicators such as blood pressure of 130 / 85. Data from wearable devices confirms that the person walks 7,000 steps per day. Food photos are analyzed and evaluated as 700 calories or less. Data from smart speakers reveals that the person jogs for 150 minutes per week. Credit card history confirms that monthly food expenses are 30,000 yen. An emotion engine recognizes the emotion of happiness and sends it to the server.

[1020] Proposal generation and refinement

[1021] The server proposes a medical insurance plan tailored to the user's blood pressure risk. It also presents an exercise plan to maintain a jogging habit. Taking into account food and entertainment expenses, it suggests a monthly budget of 10,000 yen for leisure activities. The emotion engine grasps the user's feelings of joy and makes suggestions that will further increase satisfaction.

[1022] Feedback and Optimization

[1023] The user reviews the suggestions and provides feedback on what they dislike. The server receives the feedback, adjusts the suggestions, and makes new suggestions to optimize them.

[1024] Example prompts to input to the generative AI model

[1025] "A 50-year-old male user has provided us with his health checkup report and data from his wearable device. Please suggest the best medical insurance and exercise plan for this user, taking into account his health risks and emotional state."

[1026] This system will enable a comprehensive analysis of a user's health condition, lifestyle patterns, and emotional state, and provide optimal recommendations for each individual, which will not only improve the user's quality of life, but also provide efficient support for future risk management and financial planning.

[1027] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1028] Step 1:

[1029] The user uploads the medical checkup report.

[1030] Input: The user saves the PDF file of the medical examination report to the device.

[1031] Specific operation: The user clicks the "Upload" button in the dedicated application on the device.

[1032] Data processing: The terminal sends the health check report to the server.

[1033] Output: The health check report is saved on the server.

[1034] Step 2:

[1035] The server analyzes the health check report.

[1036] Input: PDF file of medical examination report.

[1037] Specific operation: The server uses OCR technology to extract text data from the PDF file.

[1038] Data processing: Extract important health indicators (blood pressure, blood sugar levels, cholesterol levels, etc.).

[1039] Output: The extracted health indicator data is stored in a database.

[1040] Step 3:

[1041] The terminal receives the biometric data from the wearable device.

[1042] Input: Biometric data obtained from wearable devices (heart rate, steps, sleep patterns, etc.).

[1043] Specific operation: The wearable device sends data to the terminal via Bluetooth or Wi-Fi.

[1044] Data processing: The device periodically sends this data to the server.

[1045] Output: Biometric data is stored on the server.

[1046] Step 4:

[1047] The server analyzes the biometric data.

[1048] Input: Biometric data received from a wearable device.

[1049] Specific operation: The server analyzes heart rate fluctuations, daily steps taken, and sleep quality.

[1050] Data processing: The analysis results are structured as time-series data.

[1051] Output: The analyzed biometric data is stored in a database.

[1052] Step 5:

[1053] The user takes a photo of the meal, which is received by the device.

[1054] Input: A photo file of the meal taken by the user.

[1055] Specific operation: The user takes a photo of a meal using the device's camera application.

[1056] Data processing: The device saves the photos in a dedicated folder and uploads them to the server.

[1057] Output: Photo data is saved on the server.

[1058] Step 6:

[1059] The server analyzes the food photos.

[1060] Input: Food photo data.

[1061] What it does: The server uses image recognition technology to analyze the photo and identify the ingredients.

[1062] Data processing: Calculate the calories and nutritional balance of each meal.

[1063] Output: The analysis results are saved in a database.

[1064] Step 7:

[1065] A voice recognition device collects daily life data and the data is received by the terminal.

[1066] Input: Voice data collected by a voice recognition device.

[1067] Specific operation: The voice recognition device records the user's daily activities (e.g., "How much time do I spend exercising today?") and converts them into text data.

[1068] Data processing: The terminal receives the text data and uploads it to the server.

[1069] Output: Daily life data is saved on the server.

[1070] Step 8:

[1071] The server analyzes daily life data.

[1072] Input: Text data received from a speech recognition device.

[1073] Specific operation: The server performs analysis to evaluate behavioral patterns and stress levels.

[1074] Data processing: Activity patterns and stress levels are assessed and stored as structured data.

[1075] Output: The analyzed daily life data is stored in a database.

[1076] Step 9:

[1077] The terminal collects online shopping and spending history.

[1078] Input: Spending data such as online shopping history, credit card history, etc.

[1079] What it does: The device retrieves spending history from online banking apps and emails.

[1080] Data processing: Upload expenditure data to the server periodically.

[1081] Output: The spending history data is saved on the server.

[1082] Step 10:

[1083] A server analyzes online shopping and spending history.

[1084] Input: Collected expenditure data.

[1085] What happens: The server categorizes spending data.

[1086] Data processing: Evaluate the balance between income and expenditure and understand expenditure trends.

[1087] Output: Parsed expenditure data is saved in a database.

[1088] Step 11:

[1089] The terminal transmits the voice data and image data to the emotion engine.

[1090] Input: Audio and image data collected by the camera and microphone.

[1091] Specific operation: The device sends voice data and image data to the emotion engine.

[1092] Data processing: The emotion engine analyzes the data and extracts emotional information.

[1093] Output: The analysis results are sent to the server and saved.

[1094] Step 12:

[1095] The server integrates the analysis results and generates optimal proposals.

[1096] Input: health checkup report, biometric data, food photos, daily life data, expenditure history and emotional information.

[1097] How it works: The server aggregates all the data and uses scoring algorithms to generate optimal life and health insurance plans, financial plans, investment suggestions, and budget allocations for leisure activities.

[1098] Data processing: Generate optimized proposals for each user.

[1099] Output: The generated proposal is sent to the device.

[1100] Step 13:

[1101] The terminal displays the suggestions to the user, who then provides feedback.

[1102] Input: Generated suggestions.

[1103] Specific operation: The device will display the suggestions to the user in a pop-up notification or within a dedicated application.

[1104] Data processing: The user clicks the feedback button and enters their opinion.

[1105] Output: The feedback is sent to the server and stored.

[1106] Step 14:

[1107] The server will incorporate the feedback and re-adjust the suggestions.

[1108] Input: User feedback.

[1109] Specific operation: The server analyzes the feedback and reflects it in the algorithm.

[1110] Data processing: Re-arranging data to generate new proposals.

[1111] Output: The optimized suggestions are sent back to the device.

[1112] (Application example 2)

[1113] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1114] Currently, there is no integrated system on the market that analyzes individual health checkup reports, biometric data from wearable devices, daily life data, and electronic payment history to provide optimal insurance, financial planning, investment recommendations, and even budget allocation for leisure activities. This requires users to manually collect and analyze a large amount of data, which is prone to errors and time-consuming. Furthermore, current systems do not take into account the user's emotional state when making recommendations, making it difficult to improve user satisfaction and quality of life.

[1115] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1116] In this invention, the server includes means for uploading health checkup reports, means for receiving biometric data from the user's wearable device, means for receiving images of meals taken by the user, means for collecting daily life data from a voice recognition device, means for receiving electronic payment history, means for analyzing the health checkup reports, biometric data, meal images, daily life data, and electronic payment history to provide insurance proposals optimized for each user, and means for optimizing the user's financial planning, investment proposals, and budget allocation for leisure activities based on the analysis results and emotion recognition technology. This allows users to centrally manage information from various data sources and receive comprehensive and personalized proposals, thereby improving their quality of life and saving time. Furthermore, emotion recognition technology can be used to provide more satisfying proposals that take the user's emotional state into account.

[1117] A "health checkup report" is a detailed record provided by a medical institution to assess a user's health condition.

[1118] A "wearable device" is an electronic device that can be worn by a user and is used to collect biometric data.

[1119] "Biometric data" refers to data that indicates the user's physical condition, such as heart rate, number of steps taken, and sleep patterns.

[1120] "Meal images" are photographic data of meals taken by the user, and are data whose contents can be analyzed.

[1121] A "voice recognition device" is a device for collecting and analyzing voice data from a user's daily life.

[1122] "Daily life data" is data that indicates the user's daily behavior and activity patterns.

[1123] "Electronic payment history" refers to recorded data relating to a user's online shopping and various payments.

[1124] "Emotion recognition technology" is a technology that analyzes voice data and image data to identify a user's emotional state.

[1125] "Insurance proposal" means proposing the most suitable life insurance or medical insurance plan to the user based on the analysis results.

[1126] A "financial plan" is a plan for managing money and achieving future financial security based on a user's income and expenses.

[1127] "Investment Proposal" means proposing the optimal investment plan to the User.

[1128] "Budget allocation for leisure activities" means allocating an appropriate budget for the user's daily leisure activities.

[1129] This invention provides a system that uses health checkup reports, biometric data from wearable devices, food images taken by the user, daily life data from a voice recognition device, and electronic payment history to make insurance proposals, financial plans, investment proposals, and budget allocations for leisure activities that are optimized for each individual user.

[1130] Data collection methods

[1131] The server and the terminal work together to collect the following data:

[1132] Health Checkup Report: The user uploads the health checkup report using the terminal. The report is uploaded in PDF format or image format and sent to the server.

[1133] Wearable devices: Biometric data such as heart rate, steps, and sleep patterns are periodically collected from a user's wearable device (e.g., Fitbit, Apple Watch). This data is then sent to a server using an API (e.g., Fitbit API, Apple HealthKit).

[1134] Meal image: The user takes a photo of the meal on their device and uploads it to the server. The image contains date and time information.

[1135] Daily Life Data: Daily life data obtained from a voice recognition device (e.g., Amazon Echo, Google Home) is sent to the server via the device. This data includes voice commands and daily activity logs.

[1136] Electronic payment history: A user's credit card history and online shopping history are periodically sent to a server via an API (e.g., Plaid API).

[1137] Data Analysis Methods

[1138] The server analyzes the collected data as follows:

[1139] Health checkup report: Using OCR technology (e.g., Tesseract OCR), the health checkup report is converted into text and health indicators such as blood pressure, blood sugar level, and cholesterol level are extracted.

[1140] Biometric data: Using statistical analysis techniques (e.g., Pandas, Numpy), we analyze heart rate fluctuations, daily steps, sleep quality, etc. to understand the user's health patterns.

[1141] Meal images: Calculate the calories and nutritional balance of meals using image recognition technology (e.g., Google Cloud Vision API).

[1142] Daily Life Data: Convert voice data into text (e.g., Amazon Transcribe) and evaluate the user's activity log and stress level.

[1143] Electronic payment history: Use financial analysis techniques (e.g., FinancePy) to understand spending trends and evaluate the balance between income and expenses.

[1144] Emotion recognition: Analyze the user's emotional state from voice and image data using an emotion engine (e.g., Azure Emotion API) to understand their current emotional state.

[1145] Suggestions and optimization measures

[1146] The server makes the following suggestions based on the analysis results and emotion data.

[1147] Insurance proposals: Based on the analyzed health data, the system generates the most suitable life and medical insurance plan for the user. In particular, by combining it with emotion recognition technology, it is possible to propose flexible plans tailored to the user's current condition.

[1148] Financial Planning and Investment Recommendations: Create a financial plan that takes into account your income, expenses, and emotional state, and then recommend an investment strategy based on that. Use optimization algorithms (e.g., Linear Programming) to ensure your financial future is secure.

[1149] Budget allocation for leisure activities: Based on the user's daily life data and emotional data, the system suggests optimal budget allocation and improves the user's quality of life.

[1150] Feedback and Resubmissions

[1151] Users can check the proposals and provide feedback via their devices. The server adjusts the proposals based on the received feedback and re-proposes an optimized plan. This ensures that users always receive the best proposals based on the latest situation.

[1152] Specific examples

[1153] For example, if a 50-year-old male user is using this system, the following specific behavior is expected:

[1154] Users upload their health checkup report from the previous year.

[1155] Data such as the number of steps taken and heart rate is automatically sent from the wearable device to a server every day.

[1156] The user takes a photo of their lunch, which is then uploaded to the server.

[1157] The smart speaker records that "Today's jogging time was 30 minutes" and sends the data to the server.

[1158] Credit card history is periodically uploaded to the server.

[1159] The emotion engine analyzes the user's facial image and voice data to recognize the emotion of "joy."

[1160] Prompt Sentence Examples

[1161] "A user uploaded yesterday's health check report. Please analyze it."

[1162] "Get data from wearable devices and analyze heart rate and step count."

[1163] "I took a photo of my lunch today. Please measure the calories and nutritional balance."

[1164] "I received data from my smart speaker saying, 'Today's jogging time was 30 minutes.' Please analyze it."

[1165] "Analyze your credit card history and calculate your monthly food expenses."

[1166] "The emotion engine has identified the user's emotional state as 'joy'. Please reflect this in your suggestions."

[1167] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1168] Processing steps of the system that realizes the application example

[1169] Step 1:

[1170] The user uploads a health checkup report using a terminal. The input is a PDF or image file of the health checkup report provided by the user. The terminal sends this file to the server, which converts it into text data using OCR technology (e.g., Tesseract OCR) and extracts health indicators such as blood pressure, blood glucose level, and cholesterol level. The output is the extracted health indicator data.

[1171] Step 2:

[1172] Biometric data is collected from the user's wearable device. The input is data such as heart rate, steps, and sleep patterns measured by the wearable device (e.g., Fitbit, Apple Watch). The device periodically sends this data to a server using an API (e.g., Fitbit API, Apple HealthKit). The server analyzes the data using statistical analysis techniques (e.g., Pandas, Numpy) to understand the user's health patterns. The output is the analyzed health pattern data.

[1173] Step 3:

[1174] The user takes a photo of their meal and uploads it to the server via their device. The input is the photo of the meal provided by the user. The server analyzes the photo using image recognition technology (e.g., Google Cloud Vision API) and calculates the calories and nutritional balance. The output is the analyzed data on calories and nutritional balance.

[1175] Step 4:

[1176] Daily life data is collected from a voice recognition device. The input is voice data from a voice recognition device (e.g., Amazon Echo, Google Home). The device sends this voice data to a server, which converts the voice into text (e.g., Amazon Transcribe) and evaluates the user's activity log and stress level. The output is the analyzed daily life data.

[1177] Step 5:

[1178] Collects electronic payment history. Input is the user's credit card history and online shopping history. The terminal periodically sends this data to a server using an API (e.g., Plaid API). The server uses financial analysis technology (e.g., FinancePy) to identify spending trends and evaluate the balance between income and expenses. The output is analyzed spending trend data.

[1179] Step 6:

[1180] Collects and analyzes emotional data. The input is the user's voice and facial image data. The device sends this data to a server, which uses an emotion engine (e.g., Azure Emotion API) to identify the user's emotional state. The output is analyzed emotional data.

[1181] Step 7:

[1182] Generate an insurance proposal. The input is all the analyzed data mentioned above. Based on this data, the server generates a life or health insurance plan optimized for the individual user. The output is an insurance plan proposal that is presented to the user.

[1183] Step 8:

[1184] Generate a financial plan and investment recommendations. The input is all the analyzed data mentioned above. The server uses optimization algorithms (e.g., Linear Programming) to generate a financial plan and investment recommendations based on the user's income, expenses, and emotional state. The output is a financial plan and investment recommendations presented to the user.

[1185] Step 9:

[1186] The system proposes budget allocation for leisure activities. The input is all the analysis data mentioned above. Based on this data, the server generates an optimal budget allocation that takes into account the user's daily life and emotional state. The output is a budget allocation proposal for leisure activities presented to the user.

[1187] Step 10:

[1188] Receives user feedback and adjusts the suggestions. The input is the feedback provided by the user through the device. The server receives this feedback, re-runs the analysis, and adjusts the suggestions. The output is new suggestions optimized based on the feedback.

[1189] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1190] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1191] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1192] [Third embodiment]

[1193] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1194] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1195] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1196] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1197] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1198] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1199] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1200] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1201] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1202] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1203] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1204] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1205] This invention is a system that integrates health checkup reports collected over multiple years, biometric data from wearable devices, photographic data for each meal, lifestyle data from smart speakers, and online shopping and expenditure history to select life insurance and medical insurance suited to an individual's health condition, as well as financial planning, investment proposals, and budget allocation for leisure activities. This system collects, analyzes, and optimizes data through mutual communication between a server, terminals, and users.

[1206] Program processing explanation

[1207] Data collection

[1208] 1. The device provides an interface to the user, prompting them to upload the health checkup report. The user selects the PDF file and saves it on the device.

[1209] 2. The terminal works in conjunction with the user's wearable device to periodically receive biometric data (heart rate, number of steps, sleep patterns, etc.) and send it to the server.

[1210] 3. The device saves photos of each meal taken by the user and uploads them to the server. The photos contain date and time information.

[1211] 4. The smart speaker collects voice data from the user's daily life (e.g., "How much exercise time did you do today?") and sends it to a server via the device.

[1212] 5. The device periodically collects the user's online shopping and spending history and sends it to the server.

[1213] Data analysis

[1214] 1. The server analyzes the received health checkup report and extracts important health indicators (blood pressure, blood sugar level, cholesterol level, etc.).

[1215] 2. The server analyzes biometric data from the wearable device to understand the user's daily health patterns (heart rate variability, daily steps taken, sleep quality, etc.).

[1216] 3. The server analyzes photos of meals using image recognition technology and evaluates the calories and nutritional balance of each meal.

[1217] 4. The server analyzes the lifestyle data obtained from the smart speaker and evaluates the user's lifestyle patterns and stress levels.

[1218] 5. The server analyzes the user's online shopping history and spending history to evaluate the balance between income and expenditure and identify spending trends.

[1219] Proposal generation and refinement

[1220] 1. The server proposes optimal life and health insurance plans to the user based on the analysis results, including insurance selection to reduce the user's health risks.

[1221] 2. The server will provide financial planning and investment recommendations, taking into account the user's health status and lifestyle, including suitable investment options and financial planning for retirement.

[1222] 3. The server proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[1223] Feedback and Optimization

[1224] 1. The device displays the proposed insurance plan and financial plan to the user and accepts feedback. The user reviews the proposal and provides feedback on any necessary adjustments via the device.

[1225] 2. The server adjusts the proposal based on user feedback and re-proposes the optimal insurance plan and financial plan.

[1226] Specific examples

[1227] For example, if a 50-year-old male user is using this system, it will operate as follows:

[1228] Data collection

[1229] A user uploads last year's health checkup report using a device. Daily heart rate data and step count data are automatically sent from a wearable device to a server. A user takes a photo of their lunch and the photo is uploaded to a server. A smart speaker records that "today's jogging time was 30 minutes" and sends this data to a server. A user's credit card history is periodically uploaded to a server.

[1230] Data analysis

[1231] The server analyzes health checkup reports and extracts important indicators such as blood pressure of 130 / 85. It confirms from wearable device data that the user walks 7,000 steps per day. It analyzes food photos and evaluates the calorie intake, such as 700 calories. It determines from smart speaker data that the user jogs 150 minutes per week. It confirms from credit card history that the user's monthly food expenses are 30,000 yen.

[1232] Proposal generation and refinement

[1233] The server proposes a medical insurance plan tailored to the user's blood pressure risk, provides an exercise plan to encourage jogging, and suggests a monthly budget of 10,000 yen for leisure activities, taking into account food and entertainment costs.

[1234] Feedback and Optimization

[1235] The user reviews the suggestions and provides feedback on what they dislike. The server receives the feedback, adjusts the suggestions, and makes new suggestions to optimize them.

[1236] The processing flow will be explained below.

[1237] Step 1:

[1238] The device prompts the user to upload the health checkup report. The user saves the PDF file of the health checkup report on the device and uploads it.

[1239] Step 2:

[1240] The terminal periodically receives biometric data (heart rate, number of steps, sleep patterns, etc.) from the user's wearable device and transmits this data to a server.

[1241] Step 3:

[1242] The device stores photos of each meal taken by the user and uploads them to a server, with the photos containing date and time information.

[1243] Step 4:

[1244] The smart speaker collects voice data from the user's daily life (e.g., "How much time did you exercise today?") and sends it to the device, which then uploads the data to a server.

[1245] Step 5:

[1246] The terminal periodically collects the user's online shopping and spending history (such as credit card history) and sends it to a server.

[1247] Step 6:

[1248] The server analyzes the received health checkup report using OCR technology and extracts important health indicators such as blood pressure, blood sugar level, and cholesterol level.

[1249] Step 7:

[1250] The server analyzes the biometric data received from the wearable device to understand the user's health patterns, such as heart rate fluctuations, daily steps taken, and sleep quality.

[1251] Step 8:

[1252] The server analyzes photos of meals using image recognition technology, and calculates and evaluates the calories and nutritional balance of each meal.

[1253] Step 9:

[1254] The server analyzes lifestyle data obtained from the smart speaker and evaluates the user's activity patterns and stress levels.

[1255] Step 10:

[1256] The server analyzes the user's online shopping history and spending history, evaluates the balance between income and expenses, and identifies spending trends.

[1257] Step 11:

[1258] The server then uses the analysis results to generate optimal life and health insurance plans for the user, including selecting insurance to mitigate the user's health risks.

[1259] Step 12:

[1260] The server then provides optimal financial planning and investment recommendations based on the user's health status and lifestyle, including investment strategies to ensure financial security in retirement.

[1261] Step 13:

[1262] The server proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[1263] Step 14:

[1264] The terminal displays the insurance plans and financial plans proposed by the server to the user, who can then review the proposals and provide feedback if necessary.

[1265] Step 15:

[1266] The server receives feedback from the user, adjusts the proposal, and again provides the user with an optimized plan.

[1267] Example 1

[1268] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1269] There is currently no system that can centrally provide optimal life insurance, medical insurance, financial planning, investment proposals, and budget allocation for leisure activities based on diverse health and lifestyle data. There is also a lack of systems that can continuously learn and update a user's health status and lifestyle patterns over the long term. Furthermore, there is a need for a system that can automatically generate optimal proposals from data using generative AI models and provide an approach that meets the user's needs.

[1270] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1271] In this invention, the server includes means for uploading health checkup reports, means for receiving biometric data from the user's wearable device, means for receiving meal photos taken by the user, means for collecting daily life data from a voice recognition device, means for receiving online shopping and expenditure history, means for analyzing the health checkup reports, biometric data, meal photos, daily life data, and expenditure history to propose life insurance and medical insurance tailored to each individual user, means for optimizing the user's financial plan, investment proposals, and budget allocation for leisure activities based on the analysis results, means for receiving feedback from the user and adjusting the proposals, means for accumulating the biometric data and daily life data over the long term and continuously learning and updating the user's health condition and lifestyle patterns, means for generating proposals based on the data analysis results using a generative AI model, and means for using prompt sentences at each step of the processing flow. This makes it possible to integrate various personal data and provide optimal advice and plans.

[1272] A "health checkup report" is a document that lists the results of a health checkup conducted at a hospital or clinic, and includes health indicators such as blood pressure, blood sugar level, and cholesterol level.

[1273] A "wearable device" is an electronic device that can be worn by a user and has the ability to collect biometric data such as heart rate, number of steps taken, and sleep patterns.

[1274] "Biometric data" refers to data that indicates a person's physiological state, such as heart rate, number of steps, and sleep patterns.

[1275] A "meal photo" is a digital image taken by a user to record the contents of a meal.

[1276] A "voice recognition device" is a device that collects and analyzes voice data and has the function of recognizing voice commands and comments made by users in their daily lives.

[1277] "Online shopping history" is a record of a user's purchasing activities on the Internet, including information such as purchase date, product, and amount.

[1278] "Expense history" is a record of expenses incurred by a user in their daily life, including credit card statements, bank transaction statements, etc.

[1279] "Means of proposing life insurance and medical insurance" refers to the process of analyzing the user's health condition and lifestyle data, selecting the most suitable insurance plan, and offering it to the user.

[1280] "Means for optimizing financial planning and investment proposals" refers to the process of analyzing a user's income, expenses, and asset information and proposing long-term financial and investment plans.

[1281] "Means for optimizing budget allocation for leisure activities" refers to a process that takes into account the user's daily life and spending history and suggests allocating an appropriate budget for leisure activities.

[1282] A "generative AI model" is a model that uses artificial intelligence technology to analyze data and automatically generate optimal suggestions.

[1283] A "prompt" is an instruction that can be input into a generative AI model to perform a specific analysis or suggestion.

[1284] This invention is a system that integrates health checkup reports collected over multiple years, biometric data from wearable devices, photographic data for each meal, lifestyle data from a voice recognition device, and online shopping and expenditure history to select life insurance and medical insurance suited to an individual's health condition, as well as financial planning, investment proposals, and budget allocation for leisure activities. This system collects, analyzes, and optimizes data through mutual communication between a server, terminals, and users.

[1285] Data collection

[1286] Uploading your medical checkup report

[1287] The terminal provides the user with an interface and prompts them to upload their health checkup report. The user selects a PDF file and saves it on the terminal. The terminal then sends this file to the server. Specifically, the user uploads last year's health checkup report.

[1288] Biometric data collection from wearable devices

[1289] The device connects to the user's wearable device and periodically collects biometric data (heart rate, steps, sleep patterns, etc.) via Bluetooth or Wi-Fi. This collected data is then sent from the device to a server. For example, the user's wearable device sends daily heart rate and step count data to the server.

[1290] Upload a meal photo

[1291] The device saves photos of each meal taken by the user and uploads them to the server. The photos are automatically dated and time stamped. The user takes photos of each meal and saves them on the device through the app. For example, the user takes a photo of their lunch and the photo is uploaded to the server.

[1292] Lifestyle data collection from voice recognition devices

[1293] The voice recognition device collects voice data from the user's daily life. For example, if the user says, "I'm going to jog for 30 minutes today," the voice recognition device recognizes the voice and sends the data to the server via the device.

[1294] Collecting online shopping and spending history

[1295] The device periodically collects the user's online shopping history and credit card details and sends them to the server. For example, when a user makes a purchase on a shopping site, the details are automatically recorded and uploaded to the server.

[1296] Data analysis

[1297] Analysis of medical examination reports

[1298] The server analyzes the uploaded health checkup report, extracting text data from the PDF using OCR technology and extracting important health indicators (blood pressure, blood sugar, cholesterol levels, etc.), which are then stored in a database.

[1299] Biological data analysis

[1300] The server analyzes the biometric data sent from the wearable device and uses time-series data analysis to understand the user's daily health patterns (heart rate variability, daily steps, sleep quality, etc.).

[1301] Food photo analysis

[1302] The server analyzes the photos of the meals using image recognition technology (such as YOLO or ResNet), evaluates the calories and nutritional balance of each meal, and stores that information in a database.

[1303] Lifestyle data analysis

[1304] The server analyzes the voice data obtained from the voice recognition device and uses voice recognition technology to evaluate the user's lifestyle patterns and stress levels. This data is used to analyze the user's lifestyle.

[1305] Analysis of shopping and spending history

[1306] The server analyzes online shopping and spending histories to assess the balance between income and expenditure, and uses machine learning algorithms to identify spending trends and build predictive models.

[1307] Proposal generation and refinement

[1308] Insurance plan proposals

[1309] The server then proposes the most suitable life and medical insurance plan for the user based on the analysis results. For example, if the user has high blood pressure, it will recommend an insurance plan that addresses the risk of high blood pressure.

[1310] Generate financial plans and investment proposals

[1311] The server will provide financial planning and investment recommendations, taking into account the user's health status and lifestyle patterns. For example, it will analyze the user's income and expenses and suggest mutual funds and pension plans.

[1312] Leisure activity budget proposal

[1313] The server then suggests budget allocations for leisure activities to the user. For example, it extracts entertainment expenses from the user's daily expenses and suggests spending 10,000 yen per month on leisure activities.

[1314] Feedback and Optimization

[1315] Receiving feedback from users

[1316] The device displays the proposed insurance and financial plans to the user and accepts their feedback, allowing them to submit comments and revisions to the proposals based on their preferences and needs.

[1317] Adjustment and resubmission of proposals

[1318] The server adjusts its proposals based on user feedback. For example, if a user gives feedback that the monthly premium for a proposed insurance plan is too high, the server will recalculate the premium and propose a more suitable plan.

[1319] Using generative AI models

[1320] The server generates proposals based on the results of data analysis using a generative AI model. The generative AI model in this system is designed to analyze information from various data sources and automatically generate optimal proposals.

[1321] Using prompt statements

[1322] At each step in the processing flow, the generative AI model is given a prompt like this:

[1323] "Please suggest an appropriate medical insurance plan for a 50-year-old man based on his medical examination results."

[1324] "Analyze your daily health patterns based on your heart rate and step count data."

[1325] "Evaluate the calories and nutritional balance from the food photos."

[1326] "Evaluate your income and expenditure balance based on your online shopping history and suggest the best financial plan for you."

[1327] This makes it possible to integrate various personal data and provide optimal advice and plans.

[1328] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1329] The processing flow (processing steps) of this system's program

[1330] Step 1: Data collection

[1331] The user uploads a health checkup report in PDF format. The terminal provides an interface to the user and saves the uploaded PDF file on the terminal. The input is the PDF file of the health checkup report, and the output is sending the PDF file to the server.

[1332] Step 2: Collecting biometric data from wearable devices

[1333] The terminal periodically collects biometric data (heart rate, number of steps, sleep patterns, etc.) from the user's wearable device via Bluetooth or Wi-Fi. The input is the biometric data from the wearable device, and the output is the transmission of the biometric data to the server.

[1334] Step 3: Upload your meal photos

[1335] The app stores photos of each meal taken by the user on their device. The device adds date information and uploads them to the server. The input is the meal photo and date information, and the output is the transmission of the meal photo to the server.

[1336] Step 4: Collecting lifestyle data from the voice recognition device

[1337] The speech recognition device collects voice data from the user's daily life and recognizes utterances such as "Today's jogging time is 30 minutes." The recognized data is sent to the server via the terminal. The input is the user's voice data, and the output is the transmission of the recognized data to the server.

[1338] Step 5: Collect your online shopping and spending history

[1339] The terminal periodically collects the user's online shopping history and credit card details and sends them to the server. The input is the online shopping history and spending history, and the output is the transmission of data to the server.

[1340] Step 6: Data analysis

[1341] The server uses OCR technology to extract text data from PDF health checkup reports and analyzes important health indicators such as blood pressure and blood sugar levels. It then performs time-series analysis of the biometric data to understand the user's daily health patterns. It uses image recognition technology to evaluate calories and nutritional balance from food photos, and analyzes lifestyle patterns and stress levels from audio data. It also evaluates the balance between income and expenditures from online shopping history and spending history. The input is various collected data, and the output is the analysis results.

[1342] Step 7: Propose an insurance plan

[1343] The server then proposes the most suitable life insurance or medical insurance plan to the user based on the analysis results. For example, if the user has high blood pressure, it will recommend an insurance plan that addresses the risk of high blood pressure. The input is the analysis results, and the output is a proposed insurance plan.

[1344] Step 8: Generate a financial plan and investment proposal

[1345] The server makes financial plans and investment recommendations, taking into account the user's health status and lifestyle patterns. It analyzes the user's income and expenses and suggests mutual funds and pension plans. The input is the analysis results, and the output is financial plans and investment plan recommendations.

[1346] Step 9: Propose a leisure activity budget

[1347] The server extracts entertainment expenses from the user's daily expenditures and suggests spending 10,000 yen per month on leisure activities. The input is the analysis result, and the output is a suggested leisure activity budget.

[1348] Step 10: Receive feedback from users

[1349] The terminal displays the proposed insurance plan or financial plan to the user and accepts feedback. The user can submit comments or requests for revisions to the proposal through the terminal. The input is the user's feedback, and the output is the collection of the feedback.

[1350] Step 11: Adjust and resubmit your proposal

[1351] The server adjusts the proposal based on user feedback. For example, if the user gives feedback that the monthly premium for a proposed insurance plan is too high, the server recalculates the premium and proposes a more suitable plan. The input is the feedback, and the output is the adjusted proposal.

[1352] Step 12: Use a generative AI model and prompts

[1353] The server uses a generative AI model to generate recommendations based on the results of data analysis. For example, a prompt such as "Please suggest an appropriate medical insurance plan based on the medical examination results of a 50-year-old male" can be input, and the AI ​​model will generate the optimal insurance plan based on that. The input is the prompt, and the output is recommendations for the optimal insurance plan, financial plan, leisure activity budget, etc.

[1354] (Application example 1)

[1355] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1356] In modern society, systems that efficiently collect and analyze diverse information are necessary to manage users' health and lifestyle patterns, and to provide individualized financial planning and security measures. However, while many systems exist that manage this information individually, there are still few systems that centrally integrate this information and provide optimal recommendations to users. Furthermore, there has been insufficient work done to evaluate security risks and propose countermeasures based on users' health information and lifestyle patterns. Therefore, there is a need for the development of a system that can provide comprehensive lifestyle support and security measures.

[1357] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1358] In this invention, the server includes means for uploading health checkup reports, means for receiving biometric data from the user's wearable device, means for receiving photos of meals taken by the user, means for collecting daily life data from a smart speaker, means for receiving online shopping and expenditure history, means for receiving video data from home surveillance cameras, means for analyzing the health checkup reports, biometric data, meal photos, daily life data, expenditure history, and home surveillance camera data to propose life insurance and medical insurance tailored to each individual user, means for optimizing the user's financial plan, investment proposals, and budget allocation for leisure activities based on the analysis results, and means for proposing security measures, means for receiving user feedback and adjusting the proposals, financial plan, investment proposals, budget allocation for leisure activities, and security measures, and means for accumulating the biometric data, daily life data, and home surveillance camera data over the long term and continuously learning and updating the user's health condition, lifestyle patterns, and security risks. This enables the integrated management and analysis of a user's diverse data to provide optimal proposals tailored to individual needs and risks.

[1359] A "health checkup report" is a document that describes the results of periodic examinations conducted to assess a User's health status.

[1360] A "wearable device" is an electronic device that can be worn by a user and collects biometric data such as heart rate, number of steps taken, and sleep patterns.

[1361] "Biometric data" refers to information such as a user's heart rate, number of steps, and sleep patterns obtained from wearable devices.

[1362] "Photos of meals" are image data of meals taken by the user and are used to evaluate the meal contents.

[1363] A "smart speaker" is an electronic device that receives commands from users via voice recognition and performs tasks such as playing music, providing information, and collecting daily life data.

[1364] "Daily Life Data" is data about a user's daily life collected from smart speakers and other home devices.

[1365] "Online shopping and spending history" refers to records of a user's purchasing activities and spending via the Internet.

[1366] A "home surveillance camera" is a video camera installed to monitor the interior and exterior of a home.

[1367] "Video data" refers to video information captured by a home surveillance camera.

[1368] "Analysis" refers to the process of processing information based on collected data using methods such as statistical analysis and machine learning to extract trends and patterns.

[1369] "Life insurance" is insurance in which an insurance benefit is paid based on the contract when the policyholder dies.

[1370] "Medical insurance" is insurance that covers medical expenses when you receive treatment for illness or injury.

[1371] "Financial planning" involves comprehensively analyzing a user's income, expenses, investments, etc. to plan their future economic situation.

[1372] "Investment Suggestions" provide optimal investment strategies based on a user's risk profile and goals.

[1373] "Leisure activity budget allocation" is a plan to effectively manage and appropriately allocate expenses for users' leisure activities.

[1374] The "security measures plan" is a plan that assesses security risks based on the user's individual health condition and lifestyle patterns and proposes countermeasures.

[1375] "Feedback" refers to the evaluations and opinions that users provide to the system.

[1376] "Long-term accumulation" means collecting data over a certain period of time and continuing to accumulate it.

[1377] "Learning" refers to the system's ability to recognize patterns and trends based on the data it collects and to improve its ability to adapt to new data.

[1378] "Updating" means keeping the contents of the system up to date by adding, correcting, or reflecting new data or information.

[1379] This invention is a system that comprehensively collects and analyzes users' health information, lifestyle patterns, expenditure history, and security-related data, and proposes optimal life insurance, medical insurance, financial planning, investment proposals, security measures plans, etc. This system collects, analyzes, and optimizes data through mutual communication between the server, terminals, and users.

[1380] Data collection

[1381] 1. Device: Provides a user interface and prompts the user to upload the health checkup report. The user selects the PDF file and saves it to the device.

[1382] 2. Device: Works with the user's wearable device to periodically receive biometric data (heart rate, steps, sleep patterns, etc.) and send it to the server.

[1383] 3. Device: The user takes photos of each meal and saves them to the server. The photos contain date and time information.

[1384] 4. Smart speaker: Collects voice data from the user's daily life (e.g., "How much exercise time did you do today?") and sends it to a server via the device.

[1385] 5. Device: Periodically collects the user's online shopping and spending history and sends it to the server.

[1386] 6. Home surveillance camera: Receives video data from home surveillance cameras as a standalone auxiliary device and transmits it to the server through the terminal.

[1387] Data analysis

[1388] 1. Server: Analyzes the received health checkup report and extracts important health indicators (blood pressure, blood sugar level, cholesterol level, etc.). This analysis is performed using machine learning models such as TensorFlow.

[1389] 2. Server: Analyzes biometric data from wearable devices to understand the user's daily health patterns (heart rate variability, daily steps, sleep quality, etc.).

[1390] 3. Server: Using photo analysis software (e.g., OpenCV), the server analyzes photos of meals and evaluates the calories and nutritional balance of each meal.

[1391] 4. Server: Analyzes lifestyle data obtained from the smart speaker and evaluates the user's lifestyle patterns and stress levels.

[1392] 5. Server: Analyzes the user's online shopping and spending history to assess the balance between income and expenditure and identify spending trends.

[1393] 6. Server: Analyzes video data from home surveillance cameras, detects anomalies, and assesses security risks.

[1394] Proposal generation and refinement

[1395] 1. Server: Based on the analysis results, the server proposes the best life and health insurance plan for the user. This proposal includes insurance selection to reduce the user's health risks.

[1396] 2. Server: Providing financial planning and investment recommendations taking into account the user's health status and lifestyle, including suitable investment options and retirement financial planning.

[1397] 3. Server: Proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[1398] 4. Server: Proposes a security plan based on the user's individual health condition and lifestyle patterns, including emergency contact information when a sudden change in heart rate is detected and countermeasures when an abnormal person is detected on security cameras.

[1399] Feedback and Optimization

[1400] 1. Device: Shows the user the proposed insurance plan, financial plan, or security plan and accepts their feedback. The user reviews the proposal and provides feedback on any necessary adjustments via the device.

[1401] 2. Server: Based on user feedback, the server adjusts its proposals and re-proposes optimal insurance, financial, and security plans.

[1402] Examples of concrete examples and prompts

[1403] Example: If the user detects a sudden change in heart rate, the system:

[1404] 1. The wearable device detects sudden changes in your heart rate and sends a notification to your smartphone.

[1405] 2. Your smartphone will automatically send a message to the people registered as emergency contacts.

[1406] 3. The home surveillance camera detects an abnormality and the surveillance footage can be viewed via a smartphone app.

[1407] 4. The app will provide users with an emergency response plan and the option to call emergency services if necessary.

[1408] Example prompt sentence:

[1409] "Your heart rate has increased dramatically. Would you like to notify your emergency contacts?"

[1410] "We've detected something unusual. Would you like us to check the surveillance footage?"

[1411] "Should I call emergency services?"

[1412] This system makes it possible to manage and analyze a wide range of user data in an integrated manner, and to make optimal proposals based on individual needs and risks.

[1413] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1414] Step 1:

[1415] Terminal: Provides a means for uploading the user's health checkup report. Specifically, it displays a health checkup report upload button on the terminal's user interface and saves the PDF file selected by the user on the terminal. The input is the health checkup report in PDF format, and the output is a PDF file saved on the terminal.

[1416] Step 2:

[1417] Terminal: Works with the user's wearable device and provides a means to receive biometric data. Specifically, it periodically receives heart rate, step count, sleep patterns, etc. from the wearable device using communication technologies such as Bluetooth, and sends this to a server. The input is the biometric data sent from the wearable device, and the output is the biometric data sent to the server.

[1418] Step 3:

[1419] Terminal: Provides a means to receive and store photos of meals taken by users. Specifically, the user takes a photo of their meal using the device's camera function, saves the image data on the device, and sends it to the server. The input is the photo of the meal taken by the user, and the output is the image data sent to the server.

[1420] Step 4:

[1421] Smart speaker: Collects voice data related to the user's daily life. Specifically, the smart speaker recognizes the voice commands given by the user (e.g., "I'll exercise for 30 minutes today") and sends them as text data to a server via the device. The input is voice data, and the output is the text data sent to the server.

[1422] Step 5:

[1423] Terminal: Provides a means to periodically collect a user's online shopping and spending history and send it to a server. Specifically, it connects with the user's credit card and bank account to obtain spending history and send it to the server. The input is spending history data, and the output is the data sent to the server.

[1424] Step 6:

[1425] Home surveillance cameras: Receives video data from home surveillance cameras. Specifically, cameras installed in the home capture video in real time and send the data to a server via a terminal. The input is video data, and the output is the video data sent to the server.

[1426] Step 7:

[1427] Server: Analyzes the received health checkup report. Specifically, it uses OCR (Optical Character Recognition) technology to extract data from the PDF format health checkup report and analyzes the health indicators using a machine learning model (e.g., TensorFlow). The input is the PDF file of the health checkup report, and the output is the analyzed health indicator data.

[1428] Step 8:

[1429] Server: Analyzes biometric data from wearable devices to understand the user's daily health patterns. Specifically, it analyzes biometric data over time to evaluate heart rate fluctuations, daily steps, sleep quality, etc. The input is biometric data, and the output is health pattern data.

[1430] Step 9:

[1431] Server: Analyzes photos of meals using image analysis technology to evaluate the calories and nutritional balance of each meal. Specifically, image analysis software (e.g., OpenCV) is used to analyze the photos and evaluate the meal content. The input is a photo of the meal, and the output is data on calories and nutritional balance.

[1432] Step 10:

[1433] Server: Analyzes lifestyle data obtained from the smart speaker and evaluates the user's lifestyle patterns and stress levels. Specifically, it analyzes voice data using natural language processing technology and evaluates things like exercise time. The input is voice data in text format, and the output is data on lifestyle patterns and stress levels.

[1434] Step 11:

[1435] Server: Analyzes the user's online shopping history and spending history, evaluates the balance between income and expenses, and identifies spending trends. Specifically, it performs statistical analysis of the collected spending history data and evaluates monthly expenses and trends. The input is spending history data, and the output is financial evaluation data.

[1436] Step 12:

[1437] Server: Analyzes video data from home surveillance cameras and detects abnormalities. Specifically, it uses video analysis software to recognize abnormal behavior and suspicious individuals and evaluate security risks. The input is video data, and the output is security risk assessment data.

[1438] Step 13:

[1439] Server: Based on the analysis results, the server proposes the most suitable life insurance or medical insurance plan to the user. Specifically, the server selects and proposes the most suitable insurance plan based on the analyzed health data. The input is health index data, and the output is insurance plan proposal data.

[1440] Step 14:

[1441] Server: Considers the user's health condition and lifestyle patterns and makes financial plans and investment proposals. Specifically, it proposes investment plans and financial plans based on analyzed biometric data and expenditure history. The input is health pattern data and financial evaluation data, and the output is financial plan proposal data.

[1442] Step 15:

[1443] Server: Proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of life. Specifically, it proposes the optimal budget allocation for leisure activities based on collected data. The input is expenditure history data, and the output is leisure activity budget allocation proposal data.

[1444] Step 16:

[1445] Server: Proposes a security plan based on the user's health condition and lifestyle patterns. Specifically, it provides emergency contact information when a sudden change in heart rate is detected, and countermeasures when an abnormal person is detected on a surveillance camera. The input is biometric data and video data, and the output is security plan data.

[1446] Step 17:

[1447] Terminal: Displays proposed insurance plans, financial plans, and security measures plans to the user and accepts feedback. Specifically, the proposals are displayed on the terminal screen and user feedback is received. The input is the proposal data and the output is the feedback data.

[1448] Step 18:

[1449] Server: Adjusts the proposal based on user feedback and re-proposes an optimized plan. Specifically, it analyzes the feedback and updates the proposal according to the user's requests. The input is the feedback data, and the output is the updated proposal data.

[1450] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1451] This invention is a system that integrates health checkup reports collected over multiple years, biometric data from wearable devices, photographic data for each meal, lifestyle data from smart speakers, and online shopping and expenditure history to select life insurance and medical insurance suited to an individual's health condition, as well as financial planning, investment proposals, and budget allocation for leisure activities, and also incorporates an emotion engine that recognizes the user's emotions. This system collects, analyzes, and optimizes data through mutual communication between the server, terminals, and users, and further improves the proposal content by taking emotional information into account.

[1452] Program processing explanation

[1453] Data collection

[1454] 1. The device prompts the user to upload the health checkup report. The user saves the PDF file of the health checkup report on the device and uploads it.

[1455] 2. The terminal periodically receives biometric data (heart rate, steps, sleep patterns, etc.) from the user's wearable device and transmits this data to the server.

[1456] 3. The device saves photos of each meal taken by the user and uploads them to the server. The photos contain date and time information.

[1457] 4. The smart speaker collects voice data from the user's daily life (e.g., "How much exercise time did you do today?") and sends it to the device, which then uploads the data to the server.

[1458] 5. The device periodically collects the user's online shopping and spending history (such as credit card history) and sends it to the server.

[1459] 6. The device sends the voice data and image data to the emotion engine to recognize the user's emotions and extracts the emotion information.

[1460] Data analysis

[1461] 1. The server analyzes the received health checkup report using OCR technology and extracts important health indicators such as blood pressure, blood sugar level, and cholesterol level.

[1462] 2. The server analyzes the received biometric data from the wearable device to understand the user's health patterns, such as heart rate fluctuations, daily steps taken, and sleep quality.

[1463] 3. The server analyzes the photos of the meals using image recognition technology, and calculates and evaluates the calories and nutritional balance of each meal.

[1464] 4. The server analyzes the lifestyle data obtained from the smart speaker and evaluates the user's activity patterns and stress levels.

[1465] 5. The server analyzes the user's online shopping history and spending history, evaluates the balance between income and expenses, and identifies spending trends.

[1466] 6. The emotion engine analyzes the user's emotional information from voice and image data to understand their current emotional state (e.g., joy, anger, sadness, surprise, etc.).

[1467] Proposal generation and refinement

[1468] 1. The server generates the optimal life and health insurance plan for the user based on the analysis results and emotional information, including selecting insurance that takes into account the user's health risks and emotional state.

[1469] 2. The server takes into account the user's health status, lifestyle patterns, and emotional information to provide optimal financial planning and investment recommendations, including investment strategies for financial security in retirement.

[1470] 3. The server proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[1471] Feedback and Optimization

[1472] 1. The terminal displays the insurance plan and financial plan proposed by the server to the user. The user reviews the proposal and provides feedback if necessary.

[1473] 2. The server receives feedback from the user, adjusts the proposal, and again provides the user with an optimized plan.

[1474] Specific examples

[1475] For example, if a 50-year-old male user is using this system, it will operate as follows:

[1476] Data collection

[1477] The user uses the device to upload last year's health checkup report. Daily heart rate and step count data are automatically sent from the wearable device to the server. The user takes a photo of their lunch, which is uploaded to the server. The smart speaker records that "today's jogging time was 30 minutes" and sends this data to the server. The user's credit card history is periodically uploaded to the server. Furthermore, the emotion engine analyzes the user's facial image and voice data to recognize the emotion of "joy."

[1478] Data analysis

[1479] The server analyzes health checkup reports and extracts important indicators such as blood pressure of 130 / 85. Data from wearable devices confirms that the person walks 7,000 steps per day. Food photos are analyzed and evaluated as 700 calories or less. Data from smart speakers reveals that the person jogs for 150 minutes per week. Credit card history confirms that monthly food expenses are 30,000 yen. An emotion engine recognizes the emotion of happiness and sends it to the server.

[1480] Proposal generation and refinement

[1481] The server proposes a medical insurance plan tailored to the user's blood pressure risk. It also presents an exercise plan to maintain a jogging habit. Taking into account food and entertainment expenses, it suggests a monthly budget of 10,000 yen for leisure activities. The emotion engine grasps the user's feelings of joy and makes suggestions that will further increase satisfaction.

[1482] Feedback and Optimization

[1483] The user reviews the suggestions and provides feedback on what they dislike. The server receives the feedback, adjusts the suggestions, and makes new suggestions to optimize them.

[1484] The processing flow will be explained below.

[1485] Step 1:

[1486] The device prompts the user to upload the health checkup report. The user saves the PDF file of the health checkup report on the device and uploads it.

[1487] Step 2:

[1488] The terminal periodically receives biometric data (e.g., heart rate, number of steps, sleep patterns) from the user's wearable device and transmits this data to a server.

[1489] Step 3:

[1490] The device stores photos of each meal taken by the user and uploads them to a server, with the photos containing date and time information.

[1491] Step 4:

[1492] The smart speaker collects voice data from the user's daily life (e.g., "How much time did you exercise today?") and sends it to the device, which then uploads the data to a server.

[1493] Step 5:

[1494] The terminal periodically collects the user's online shopping and spending history (such as credit card history) and sends it to a server.

[1495] Step 6:

[1496] The terminal transmits voice data and image data for recognizing the user's emotions to an emotion engine, and extracts emotion information.

[1497] Step 7:

[1498] The server analyzes the received health checkup report using OCR technology and extracts important health indicators such as blood pressure, blood sugar level, and cholesterol level.

[1499] Step 8:

[1500] The server analyzes the biometric data received from the wearable device to understand the user's health patterns, such as heart rate fluctuations, daily steps taken, and sleep quality.

[1501] Step 9:

[1502] The server analyzes photos of meals using image recognition technology, and calculates and evaluates the calories and nutritional balance of each meal.

[1503] Step 10:

[1504] The server analyzes lifestyle data obtained from the smart speaker (e.g., work hours, rest hours, and exercise frequency) to evaluate the user's activity patterns and stress levels.

[1505] Step 11:

[1506] The server analyzes the user's online shopping history and spending history, evaluates the balance between income and expenses, and identifies spending trends.

[1507] Step 12:

[1508] The emotion engine analyzes the user's emotional information from voice and image data to understand their current emotional state (e.g., joy, anger, sadness, surprise, etc.).

[1509] Step 13:

[1510] The server generates optimal life and health insurance plans for users based on the analysis results and emotional information, including insurance selection that takes into account the user's health risks and emotional state.

[1511] Step 14:

[1512] The server takes into account the user's health status, lifestyle patterns, and emotional information to provide optimal financial planning and investment recommendations, including investment strategies for financial security in retirement.

[1513] Step 15:

[1514] The server proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[1515] Step 16:

[1516] The terminal displays the insurance plans and financial plans proposed by the server to the user, who can then review the proposals and provide feedback if necessary.

[1517] Step 17:

[1518] The server receives feedback from the user, adjusts the proposal, and again provides the user with an optimized plan.

[1519] Specific examples

[1520] For example, if a 50-year-old male user wants to use this system to reduce his health risks and financial worries, the process would look like this:

[1521] Data collection

[1522] Step 1:

[1523] The user uploads their health checkup report from the previous year using their device, and the server then obtains basic data to understand the user's health condition.

[1524] Step 2:

[1525] The wearable device automatically sends daily heart rate and step count data to a server, providing daily activity and health indicators.

[1526] Step 3:

[1527] Users take photos of their lunch and upload them to a server. The image data includes date and time information, and is used to analyze daily meal content.

[1528] Step 4:

[1529] The smart speaker records "Today's jogging time is 30 minutes" and sends the data to a server, where it is used to evaluate the user's exercise level.

[1530] Step 5:

[1531] The user's credit card history is periodically uploaded to the server, which allows the company to understand the user's spending patterns and expenditures.

[1532] Step 6:

[1533] The emotion engine recognizes the emotion "happiness" from the user's voice and facial image data. This information is used to evaluate the user's daily life and stress level.

[1534] Data analysis

[1535] Step 7:

[1536] The server analyzes the health checkup report and extracts important indicators, such as blood pressure 130 / 85, to assess the user's health risk.

[1537] Step 8:

[1538] The wearable device data confirms that the user is walking 7,000 steps per day, allowing the user to understand their daily activity level.

[1539] Step 9:

[1540] It analyzes photos of meals and evaluates the calorie and nutritional balance of each meal, calculating, for example, 700 calories or the percentage of specific nutrients.

[1541] Step 10:

[1542] Data from the smart speaker reveals that the user jogs 150 minutes per week, helping to understand the user's exercise patterns.

[1543] Step 11:

[1544] Check the credit card history to see that the monthly food expense is 30,000 yen, thereby understanding the user's spending pattern.

[1545] Step 12:

[1546] The emotion engine recognizes the user's "happiness" emotion and sends it to the server, which uses it to evaluate the user's emotional feedback on the suggestions.

[1547] Proposal generation and refinement

[1548] Step 13:

[1549] The server proposes a medical insurance plan tailored to the user's blood pressure risk and also provides an exercise plan to help maintain the user's jogging habit.

[1550] Step 14:

[1551] Taking into account the user's health status, lifestyle patterns, and emotional information, the system makes investment suggestions to ensure financial stability in retirement.

[1552] Step 15:

[1553] Taking into account food and entertainment expenses, the system suggests a monthly budget of 10,000 yen for leisure activities. The emotion engine understands the user's feelings of joy and makes suggestions that will further increase satisfaction.

[1554] Feedback and Optimization

[1555] Step 16:

[1556] The user reviews the suggestions and provides feedback on what they don't like. The device receives the feedback and sends it to the server.

[1557] Step 17:

[1558] The server adjusts the proposal based on the feedback and makes new proposals to optimize the results.

[1559] Through the above specific processing steps, the system of the present invention provides individual users with optimized health management, insurance selection, financial planning, investment suggestions and leisure activity budget allocation, thereby improving the user's overall quality of life.

[1560] Example 2

[1561] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1562] Until now, there has been no system that efficiently selects life insurance and medical insurance suitable for individual users, provides financial planning, investment recommendations, and budget allocation for leisure activities. It has also been difficult to comprehensively analyze a user's health condition, lifestyle patterns, and even emotional state and make recommendations based on that analysis. Furthermore, it has also been difficult to integrate various data provided by users (health checkup reports, biometric data, food photos, daily life data, expenditure history, etc.), and accumulate and analyze them over the long term. The present invention aims to solve these problems.

[1563] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1564] In this invention, the server includes means for uploading health checkup reports, means for receiving biometric data from the user's wearable device, means for receiving photos of meals taken by the user, means for collecting daily life data from a voice recognition device, means for receiving online shopping and expenditure history, means for analyzing the user's emotional information from the voice data and image data using an emotion engine, means for analyzing the health checkup reports, biometric data, meal photos, daily life data, expenditure history, and emotional information to propose life insurance and medical insurance optimized for each user, means for optimizing the user's financial plan, investment proposals, and budget allocation for leisure activities based on the analysis results, means for receiving user feedback and adjusting the proposals, and means for accumulating the biometric data, daily life data, and emotional information over the long term and continuously learning and updating the user's health condition and lifestyle patterns, thereby enabling automatic and efficient proposals of insurance, financial plan, and budget allocation for leisure activities that are optimal for each user's condition.

[1565] A "health checkup report" is written or digital data indicating the user's health status obtained through a health checkup.

[1566] A "wearable device" is an electronic device worn by a user that measures biometric data (heart rate, number of steps, sleep patterns, etc.) and provides the data.

[1567] "Biometric data" refers to digital data that indicates a user's physical condition, including information such as heart rate, number of steps taken, sleep patterns, body temperature, and blood pressure.

[1568] A "meal photo" is image data taken by a user to record the contents of a meal, and indicates the type, amount, and visual characteristics of the meal.

[1569] A "voice recognition device" is a device that collects a user's voice and processes the content of that voice as digital data.

[1570] "Daily life data" refers to data related to the user's daily activities and behavior, including exercise time, sleep time, and speech content.

[1571] "Online shopping history" refers to a record of purchases made by a user online, including information such as the items purchased, the date and time of purchase, and the purchase amount.

[1572] "Expense history" refers to a record of a user's spending activities, including credit card statements, bank statements, cash usage records, etc.

[1573] An "emotion engine" is a system or software that analyzes a user's emotional state from voice and image data and identifies those emotions.

[1574] "Financial planning" is a plan to comprehensively manage a user's financial situation, including income, expenses, savings, and investments, in order to ensure future economic stability.

[1575] "Investment Suggestions" are suggestions to present appropriate investment products and investment strategies based on the User's financial situation and risk tolerance.

[1576] "Leisure Budget Allocation" refers to how the user allocates funds for leisure and entertainment activities, providing an appropriate spending plan.

[1577] "Feedback" refers to information such as opinions, impressions, and requests from users regarding presented proposals and plans.

[1578] This invention is a system that integrates health checkup reports collected over multiple years, biometric data from wearable devices, photographic data for each meal, lifestyle data from a voice recognition device, and online shopping and expenditure history to select life insurance and medical insurance suited to an individual's health condition, as well as financial planning, investment recommendations, and budget allocation for leisure activities. Furthermore, by incorporating an emotion engine, the system takes into account the user's emotional information and improves the content of its recommendations.

[1579] Hardware and software used

[1580] The system is implemented using the following hardware and software:

[1581] Terminal: A device such as a smartphone or computer operated by a user.

[1582] Wearable devices: Devices that collect biometric data such as heart rate, steps taken, and sleep patterns.

[1583] Voice recognition devices: Devices that collect daily life data (e.g., smart speakers).

[1584] Server: A computer system that analyzes data and generates recommendations.

[1585] Emotion engine: Software that analyzes a user's emotional information from voice and image data.

[1586] Data processing and calculation

[1587] The program in this system performs the following data processing and calculations:

[1588] 1. The device will notify the user through a dedicated application to prompt them to upload their health checkup report. Once the user uploads the report, the server will analyze the content using OCR technology and extract important health indicators (blood pressure, blood sugar level, cholesterol level, etc.).

[1589] 2. Biometric data from the wearable device is sent via the device to a server, which analyzes the data to understand the user's health patterns, such as heart rate fluctuations, daily steps, and sleep quality.

[1590] 3. After users take photos of their meals using their devices, they automatically upload them to a server, where they use image recognition technology to analyze the photos and evaluate the calorie and nutritional balance of each meal.

[1591] 4. Daily life data collected by voice recognition devices (such as smart speakers) is transmitted to a server via the device, where it is analyzed to assess activity patterns and stress levels.

[1592] 5. Online shopping history and spending history are automatically collected by the device and sent to a server, which analyzes this data to assess the balance between income and expenditure and identify spending trends.

[1593] 6. The emotion engine analyzes the user's emotional information from voice and image data to understand their current emotional state (e.g., joy, anger, sadness, surprise, etc.). This allows the server to generate optimal suggestions that take the emotional information into account.

[1594] Examples of concrete examples and prompts

[1595] For example, if a 50-year-old male user is using this system, the following will happen:

[1596] Data collection

[1597] The user uses the device to upload last year's health checkup report. Daily heart rate and step count data are automatically sent from the wearable device to the server. The user takes a photo of their lunch, which is uploaded to the server. The smart speaker records that "today's jogging time was 30 minutes" and sends this data to the server. The user's credit card history is periodically uploaded to the server. Furthermore, the emotion engine analyzes the user's facial image and voice data to recognize the emotion of "joy."

[1598] Data analysis

[1599] The server analyzes health checkup reports and extracts important indicators such as blood pressure of 130 / 85. Data from wearable devices confirms that the person walks 7,000 steps per day. Food photos are analyzed and evaluated as 700 calories or less. Data from smart speakers reveals that the person jogs for 150 minutes per week. Credit card history confirms that monthly food expenses are 30,000 yen. An emotion engine recognizes the emotion of happiness and sends it to the server.

[1600] Proposal generation and refinement

[1601] The server proposes a medical insurance plan tailored to the user's blood pressure risk. It also presents an exercise plan to maintain a jogging habit. Taking into account food and entertainment expenses, it suggests a monthly budget of 10,000 yen for leisure activities. The emotion engine grasps the user's feelings of joy and makes suggestions that will further increase satisfaction.

[1602] Feedback and Optimization

[1603] The user reviews the suggestions and provides feedback on what they dislike. The server receives the feedback, adjusts the suggestions, and makes new suggestions to optimize them.

[1604] Example prompts to input to the generative AI model

[1605] "A 50-year-old male user has provided us with his health checkup report and data from his wearable device. Please suggest the best medical insurance and exercise plan for this user, taking into account his health risks and emotional state."

[1606] This system will enable a comprehensive analysis of a user's health condition, lifestyle patterns, and emotional state, and provide optimal recommendations for each individual, which will not only improve the user's quality of life, but also provide efficient support for future risk management and financial planning.

[1607] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1608] Step 1:

[1609] The user uploads the medical checkup report.

[1610] Input: The user saves the PDF file of the medical examination report to the device.

[1611] Specific operation: The user clicks the "Upload" button in the dedicated application on the device.

[1612] Data processing: The terminal sends the health check report to the server.

[1613] Output: The health check report is saved on the server.

[1614] Step 2:

[1615] The server analyzes the health check report.

[1616] Input: PDF file of medical examination report.

[1617] Specific operation: The server uses OCR technology to extract text data from the PDF file.

[1618] Data processing: Extract important health indicators (blood pressure, blood sugar levels, cholesterol levels, etc.).

[1619] Output: The extracted health indicator data is stored in a database.

[1620] Step 3:

[1621] The terminal receives the biometric data from the wearable device.

[1622] Input: Biometric data obtained from wearable devices (heart rate, steps, sleep patterns, etc.).

[1623] Specific operation: The wearable device sends data to the terminal via Bluetooth or Wi-Fi.

[1624] Data processing: The device periodically sends this data to the server.

[1625] Output: Biometric data is stored on the server.

[1626] Step 4:

[1627] The server analyzes the biometric data.

[1628] Input: Biometric data received from a wearable device.

[1629] Specific operation: The server analyzes heart rate fluctuations, daily steps taken, and sleep quality.

[1630] Data processing: The analysis results are structured as time-series data.

[1631] Output: The analyzed biometric data is stored in a database.

[1632] Step 5:

[1633] The user takes a photo of the meal, which is received by the device.

[1634] Input: A photo file of the meal taken by the user.

[1635] Specific operation: The user takes a photo of a meal using the device's camera application.

[1636] Data processing: The device saves the photos in a dedicated folder and uploads them to the server.

[1637] Output: Photo data is saved on the server.

[1638] Step 6:

[1639] The server analyzes the food photos.

[1640] Input: Food photo data.

[1641] What it does: The server uses image recognition technology to analyze the photo and identify the ingredients.

[1642] Data processing: Calculate the calories and nutritional balance of each meal.

[1643] Output: The analysis results are saved in a database.

[1644] Step 7:

[1645] A voice recognition device collects daily life data and the data is received by the terminal.

[1646] Input: Voice data collected by a voice recognition device.

[1647] Specific operation: The voice recognition device records the user's daily activities (e.g., "How much time do I spend exercising today?") and converts them into text data.

[1648] Data processing: The terminal receives the text data and uploads it to the server.

[1649] Output: Daily life data is saved on the server.

[1650] Step 8:

[1651] The server analyzes daily life data.

[1652] Input: Text data received from a speech recognition device.

[1653] Specific operation: The server performs analysis to evaluate behavioral patterns and stress levels.

[1654] Data processing: Activity patterns and stress levels are assessed and stored as structured data.

[1655] Output: The analyzed daily life data is stored in a database.

[1656] Step 9:

[1657] The terminal collects online shopping and spending history.

[1658] Input: Spending data such as online shopping history, credit card history, etc.

[1659] What it does: The device retrieves spending history from online banking apps and emails.

[1660] Data processing: Upload expenditure data to the server periodically.

[1661] Output: The spending history data is saved on the server.

[1662] Step 10:

[1663] A server analyzes online shopping and spending history.

[1664] Input: Collected expenditure data.

[1665] What happens: The server categorizes spending data.

[1666] Data processing: Evaluate the balance between income and expenditure and understand expenditure trends.

[1667] Output: Parsed expenditure data is saved in a database.

[1668] Step 11:

[1669] The terminal transmits the voice data and image data to the emotion engine.

[1670] Input: Audio and image data collected by the camera and microphone.

[1671] Specific operation: The device sends voice data and image data to the emotion engine.

[1672] Data processing: The emotion engine analyzes the data and extracts emotional information.

[1673] Output: The analysis results are sent to the server and saved.

[1674] Step 12:

[1675] The server integrates the analysis results and generates optimal proposals.

[1676] Input: health checkup report, biometric data, food photos, daily life data, expenditure history and emotional information.

[1677] How it works: The server aggregates all the data and uses scoring algorithms to generate optimal life and health insurance plans, financial plans, investment suggestions, and budget allocations for leisure activities.

[1678] Data processing: Generate optimized proposals for each user.

[1679] Output: The generated proposal is sent to the device.

[1680] Step 13:

[1681] The terminal displays the suggestions to the user, who then provides feedback.

[1682] Input: Generated suggestions.

[1683] Specific operation: The device will display the suggestions to the user in a pop-up notification or within a dedicated application.

[1684] Data processing: The user clicks the feedback button and enters their opinion.

[1685] Output: The feedback is sent to the server and stored.

[1686] Step 14:

[1687] The server will incorporate the feedback and re-adjust the suggestions.

[1688] Input: User feedback.

[1689] Specific operation: The server analyzes the feedback and reflects it in the algorithm.

[1690] Data processing: Re-arranging data to generate new proposals.

[1691] Output: The optimized suggestions are sent back to the device.

[1692] (Application example 2)

[1693] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1694] Currently, there is no integrated system on the market that analyzes individual health checkup reports, biometric data from wearable devices, daily life data, and electronic payment history to provide optimal insurance, financial planning, investment recommendations, and even budget allocation for leisure activities. This requires users to manually collect and analyze a large amount of data, which is prone to errors and time-consuming. Furthermore, current systems do not take into account the user's emotional state when making recommendations, making it difficult to improve user satisfaction and quality of life.

[1695] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1696] In this invention, the server includes means for uploading health checkup reports, means for receiving biometric data from the user's wearable device, means for receiving images of meals taken by the user, means for collecting daily life data from a voice recognition device, means for receiving electronic payment history, means for analyzing the health checkup reports, biometric data, meal images, daily life data, and electronic payment history to provide insurance proposals optimized for each user, and means for optimizing the user's financial planning, investment proposals, and budget allocation for leisure activities based on the analysis results and emotion recognition technology. This allows users to centrally manage information from various data sources and receive comprehensive and personalized proposals, thereby improving their quality of life and saving time. Furthermore, emotion recognition technology can be used to provide more satisfying proposals that take the user's emotional state into account.

[1697] A "health checkup report" is a detailed record provided by a medical institution to assess a user's health condition.

[1698] A "wearable device" is an electronic device that can be worn by a user and is used to collect biometric data.

[1699] "Biometric data" refers to data that indicates the user's physical condition, such as heart rate, number of steps taken, and sleep patterns.

[1700] "Meal images" are photographic data of meals taken by the user, and are data whose contents can be analyzed.

[1701] A "voice recognition device" is a device for collecting and analyzing voice data from a user's daily life.

[1702] "Daily life data" is data that indicates the user's daily behavior and activity patterns.

[1703] "Electronic payment history" refers to recorded data relating to a user's online shopping and various payments.

[1704] "Emotion recognition technology" is a technology that analyzes voice data and image data to identify a user's emotional state.

[1705] "Insurance proposal" means proposing the most suitable life insurance or medical insurance plan to the user based on the analysis results.

[1706] A "financial plan" is a plan for managing money and achieving future financial security based on a user's income and expenses.

[1707] "Investment Proposal" means proposing the optimal investment plan to the User.

[1708] "Budget allocation for leisure activities" means allocating an appropriate budget for the user's daily leisure activities.

[1709] This invention provides a system that uses health checkup reports, biometric data from wearable devices, food images taken by the user, daily life data from a voice recognition device, and electronic payment history to make insurance proposals, financial plans, investment proposals, and budget allocations for leisure activities that are optimized for each individual user.

[1710] Data collection methods

[1711] The server and the terminal work together to collect the following data:

[1712] Health Checkup Report: The user uploads the health checkup report using the terminal. The report is uploaded in PDF format or image format and sent to the server.

[1713] Wearable devices: Biometric data such as heart rate, steps, and sleep patterns are periodically collected from a user's wearable device (e.g., Fitbit, Apple Watch). This data is then sent to a server using an API (e.g., Fitbit API, Apple HealthKit).

[1714] Meal image: The user takes a photo of the meal on their device and uploads it to the server. The image contains date and time information.

[1715] Daily Life Data: Daily life data obtained from a voice recognition device (e.g., Amazon Echo, Google Home) is sent to the server via the device. This data includes voice commands and daily activity logs.

[1716] Electronic payment history: A user's credit card history and online shopping history are periodically sent to a server via an API (e.g., Plaid API).

[1717] Data Analysis Methods

[1718] The server analyzes the collected data as follows:

[1719] Health checkup report: Using OCR technology (e.g., Tesseract OCR), the health checkup report is converted into text and health indicators such as blood pressure, blood sugar level, and cholesterol level are extracted.

[1720] Biometric data: Using statistical analysis techniques (e.g., Pandas, Numpy), we analyze heart rate fluctuations, daily steps, sleep quality, etc. to understand the user's health patterns.

[1721] Meal images: Calculate the calories and nutritional balance of meals using image recognition technology (e.g., Google Cloud Vision API).

[1722] Daily Life Data: Convert voice data into text (e.g., Amazon Transcribe) and evaluate the user's activity log and stress level.

[1723] Electronic payment history: Use financial analysis techniques (e.g., FinancePy) to understand spending trends and evaluate the balance between income and expenses.

[1724] Emotion recognition: Analyze the user's emotional state from voice and image data using an emotion engine (e.g., Azure Emotion API) to understand their current emotional state.

[1725] Suggestions and optimization measures

[1726] The server makes the following suggestions based on the analysis results and emotion data.

[1727] Insurance proposals: Based on the analyzed health data, the system generates the most suitable life and medical insurance plan for the user. In particular, by combining it with emotion recognition technology, it is possible to propose flexible plans tailored to the user's current condition.

[1728] Financial Planning and Investment Recommendations: Create a financial plan that takes into account your income, expenses, and emotional state, and then recommend an investment strategy based on that. Use optimization algorithms (e.g., Linear Programming) to ensure your financial future is secure.

[1729] Budget allocation for leisure activities: Based on the user's daily life data and emotional data, the system suggests optimal budget allocation and improves the user's quality of life.

[1730] Feedback and Resubmissions

[1731] Users can check the proposals and provide feedback via their devices. The server adjusts the proposals based on the received feedback and re-proposes an optimized plan. This ensures that users always receive the best proposals based on the latest situation.

[1732] Specific examples

[1733] For example, if a 50-year-old male user is using this system, the following specific behavior is expected:

[1734] Users upload their health checkup report from the previous year.

[1735] Data such as the number of steps taken and heart rate is automatically sent from the wearable device to a server every day.

[1736] The user takes a photo of their lunch, which is then uploaded to the server.

[1737] The smart speaker records that "Today's jogging time was 30 minutes" and sends the data to the server.

[1738] Credit card history is periodically uploaded to the server.

[1739] The emotion engine analyzes the user's facial image and voice data to recognize the emotion of "joy."

[1740] Prompt Sentence Examples

[1741] "A user uploaded yesterday's health check report. Please analyze it."

[1742] "Get data from wearable devices and analyze heart rate and step count."

[1743] "I took a photo of my lunch today. Please measure the calories and nutritional balance."

[1744] "I received data from my smart speaker saying, 'Today's jogging time was 30 minutes.' Please analyze it."

[1745] "Analyze your credit card history and calculate your monthly food expenses."

[1746] "The emotion engine has identified the user's emotional state as 'joy'. Please reflect this in your suggestions."

[1747] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1748] Processing steps of the system that realizes the application example

[1749] Step 1:

[1750] The user uploads a health checkup report using a terminal. The input is a PDF or image file of the health checkup report provided by the user. The terminal sends this file to the server, which converts it into text data using OCR technology (e.g., Tesseract OCR) and extracts health indicators such as blood pressure, blood glucose level, and cholesterol level. The output is the extracted health indicator data.

[1751] Step 2:

[1752] Biometric data is collected from the user's wearable device. The input is data such as heart rate, steps, and sleep patterns measured by the wearable device (e.g., Fitbit, Apple Watch). The device periodically sends this data to a server using an API (e.g., Fitbit API, Apple HealthKit). The server analyzes the data using statistical analysis techniques (e.g., Pandas, Numpy) to understand the user's health patterns. The output is the analyzed health pattern data.

[1753] Step 3:

[1754] The user takes a photo of their meal and uploads it to the server via their device. The input is the photo of the meal provided by the user. The server analyzes the photo using image recognition technology (e.g., Google Cloud Vision API) and calculates the calories and nutritional balance. The output is the analyzed data on calories and nutritional balance.

[1755] Step 4:

[1756] Daily life data is collected from a voice recognition device. The input is voice data from a voice recognition device (e.g., Amazon Echo, Google Home). The device sends this voice data to a server, which converts the voice into text (e.g., Amazon Transcribe) and evaluates the user's activity log and stress level. The output is the analyzed daily life data.

[1757] Step 5:

[1758] Collects electronic payment history. Input is the user's credit card history and online shopping history. The terminal periodically sends this data to a server using an API (e.g., Plaid API). The server uses financial analysis technology (e.g., FinancePy) to identify spending trends and evaluate the balance between income and expenses. The output is analyzed spending trend data.

[1759] Step 6:

[1760] Collects and analyzes emotional data. The input is the user's voice and facial image data. The device sends this data to a server, which uses an emotion engine (e.g., Azure Emotion API) to identify the user's emotional state. The output is analyzed emotional data.

[1761] Step 7:

[1762] Generate an insurance proposal. The input is all the analyzed data mentioned above. Based on this data, the server generates a life or health insurance plan optimized for the individual user. The output is an insurance plan proposal that is presented to the user.

[1763] Step 8:

[1764] Generate a financial plan and investment recommendations. The input is all the analyzed data mentioned above. The server uses optimization algorithms (e.g., Linear Programming) to generate a financial plan and investment recommendations based on the user's income, expenses, and emotional state. The output is a financial plan and investment recommendations presented to the user.

[1765] Step 9:

[1766] The system proposes budget allocation for leisure activities. The input is all the analysis data mentioned above. Based on this data, the server generates an optimal budget allocation that takes into account the user's daily life and emotional state. The output is a budget allocation proposal for leisure activities presented to the user.

[1767] Step 10:

[1768] Receives user feedback and adjusts the suggestions. The input is the feedback provided by the user through the device. The server receives this feedback, re-runs the analysis, and adjusts the suggestions. The output is new suggestions optimized based on the feedback.

[1769] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1770] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1771] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1772] [Fourth embodiment]

[1773] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1774] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1775] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1776] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1777] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1778] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1779] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1780] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1781] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1782] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1783] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1784] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1785] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1786] This invention is a system that integrates health checkup reports collected over multiple years, biometric data from wearable devices, photographic data for each meal, lifestyle data from smart speakers, and online shopping and expenditure history to select life insurance and medical insurance suited to an individual's health condition, as well as financial planning, investment proposals, and budget allocation for leisure activities. This system collects, analyzes, and optimizes data through mutual communication between a server, terminals, and users.

[1787] Program processing explanation

[1788] Data collection

[1789] 1. The device provides an interface to the user, prompting them to upload the health checkup report. The user selects the PDF file and saves it on the device.

[1790] 2. The terminal works in conjunction with the user's wearable device to periodically receive biometric data (heart rate, number of steps, sleep patterns, etc.) and send it to the server.

[1791] 3. The device saves photos of each meal taken by the user and uploads them to the server. The photos contain date and time information.

[1792] 4. The smart speaker collects voice data from the user's daily life (e.g., "How much exercise time did you do today?") and sends it to a server via the device.

[1793] 5. The device periodically collects the user's online shopping and spending history and sends it to the server.

[1794] Data analysis

[1795] 1. The server analyzes the received health checkup report and extracts important health indicators (blood pressure, blood sugar level, cholesterol level, etc.).

[1796] 2. The server analyzes biometric data from the wearable device to understand the user's daily health patterns (heart rate variability, daily steps taken, sleep quality, etc.).

[1797] 3. The server analyzes photos of meals using image recognition technology and evaluates the calories and nutritional balance of each meal.

[1798] 4. The server analyzes the lifestyle data obtained from the smart speaker and evaluates the user's lifestyle patterns and stress levels.

[1799] 5. The server analyzes the user's online shopping history and spending history to evaluate the balance between income and expenditure and identify spending trends.

[1800] Proposal generation and refinement

[1801] 1. The server proposes optimal life and health insurance plans to the user based on the analysis results, including insurance selection to reduce the user's health risks.

[1802] 2. The server will provide financial planning and investment recommendations, taking into account the user's health status and lifestyle, including suitable investment options and financial planning for retirement.

[1803] 3. The server proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[1804] Feedback and Optimization

[1805] 1. The device displays the proposed insurance plan and financial plan to the user and accepts feedback. The user reviews the proposal and provides feedback on any necessary adjustments via the device.

[1806] 2. The server adjusts the proposal based on user feedback and re-proposes the optimal insurance plan and financial plan.

[1807] Specific examples

[1808] For example, if a 50-year-old male user is using this system, it will operate as follows:

[1809] Data collection

[1810] A user uploads last year's health checkup report using a device. Daily heart rate data and step count data are automatically sent from a wearable device to a server. A user takes a photo of their lunch and the photo is uploaded to a server. A smart speaker records that "today's jogging time was 30 minutes" and sends this data to a server. A user's credit card history is periodically uploaded to a server.

[1811] Data analysis

[1812] The server analyzes health checkup reports and extracts important indicators such as blood pressure of 130 / 85. It confirms from wearable device data that the user walks 7,000 steps per day. It analyzes food photos and evaluates the calorie intake, such as 700 calories. It determines from smart speaker data that the user jogs 150 minutes per week. It confirms from credit card history that the user's monthly food expenses are 30,000 yen.

[1813] Proposal generation and refinement

[1814] The server proposes a medical insurance plan tailored to the user's blood pressure risk, provides an exercise plan to encourage jogging, and suggests a monthly budget of 10,000 yen for leisure activities, taking into account food and entertainment costs.

[1815] Feedback and Optimization

[1816] The user reviews the suggestions and provides feedback on what they dislike. The server receives the feedback, adjusts the suggestions, and makes new suggestions to optimize them.

[1817] The processing flow will be explained below.

[1818] Step 1:

[1819] The device prompts the user to upload the health checkup report. The user saves the PDF file of the health checkup report on the device and uploads it.

[1820] Step 2:

[1821] The terminal periodically receives biometric data (heart rate, number of steps, sleep patterns, etc.) from the user's wearable device and transmits this data to a server.

[1822] Step 3:

[1823] The device stores photos of each meal taken by the user and uploads them to a server, with the photos containing date and time information.

[1824] Step 4:

[1825] The smart speaker collects voice data from the user's daily life (e.g., "How much time did you exercise today?") and sends it to the device, which then uploads the data to a server.

[1826] Step 5:

[1827] The terminal periodically collects the user's online shopping and spending history (such as credit card history) and sends it to a server.

[1828] Step 6:

[1829] The server analyzes the received health checkup report using OCR technology and extracts important health indicators such as blood pressure, blood sugar level, and cholesterol level.

[1830] Step 7:

[1831] The server analyzes the biometric data received from the wearable device to understand the user's health patterns, such as heart rate fluctuations, daily steps taken, and sleep quality.

[1832] Step 8:

[1833] The server analyzes photos of meals using image recognition technology, and calculates and evaluates the calories and nutritional balance of each meal.

[1834] Step 9:

[1835] The server analyzes lifestyle data obtained from the smart speaker and evaluates the user's activity patterns and stress levels.

[1836] Step 10:

[1837] The server analyzes the user's online shopping history and spending history, evaluates the balance between income and expenses, and identifies spending trends.

[1838] Step 11:

[1839] The server then uses the analysis results to generate optimal life and health insurance plans for the user, including selecting insurance to mitigate the user's health risks.

[1840] Step 12:

[1841] The server then provides optimal financial planning and investment recommendations based on the user's health status and lifestyle, including investment strategies to ensure financial security in retirement.

[1842] Step 13:

[1843] The server proposes budget allocation for the user's leisure activities and creates a plan to improve the quality of daily life.

[1844] Step 14:

[1845] The terminal displays the insurance plans and financial plans proposed by the server to the user, who can then review the proposals and provide feedback if necessary.

[1846] Step 15:

[1847] The server receives feedback from the user, adjusts the proposal, and again provides the user with an optimized plan.

[1848] Example 1

[1849] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1850] There is currently no system that can centrally provide optimal life insurance, medical insurance, financial planning, investment proposals, and budget allocation for leisure activities based on diverse health and lifestyle data. There is also a lack of systems that can continuously learn and update a user's health status and lifestyle patterns over the long term. Furthermore, there is a need for a system that can automatically generate optimal proposals from data using generative AI models and provide an approach that meets the user's needs.

[1851] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1852] In this invention, the server includes means for uploading health checkup reports, means for receiving biometric data from the user's wearable device, means for receiving meal photos taken by the user, means for collecting daily life data from a voice recognition device, means for receiving online shopping and expenditure history, means for analyzing the health checkup reports, biometric data, meal photos, daily life data, and expenditure history to propose life insurance and medical insurance tailored to each individual user, means for optimizing the user's financial plan, investment proposals, and budget allocation for leisure activities based on the analysis results, means for receiving feedback from the user and adjusting the proposals, means for accumulating the biometric data and daily life data over the long term and continuously learning and updating the user's health condition and lifestyle patterns, means for generating proposals based on the data analysis results using a generative AI model, and means for using prompt sentences at each step of the processing flow. This makes it possible to integrate various personal data and provide optimal advice and plans.

[1853] A "health checkup report" is a document that lists the results of a health checkup conducted at a hospital or clinic, and includes health indicators such as blood pressure, blood sugar level, and cholesterol level.

[1854] A "wearable device" is an electronic device that can be worn by a user and has the ability to collect biometric data such as heart rate, number of steps taken, and sleep patterns.

[1855] "Biometric data" refers to data that indicates a person's physiological state, such as heart rate, number of steps, and sleep patterns.

[1856] A "meal photo" is a digital image taken by a user to record the contents of a meal.

[1857] A "voice recognition device" is a device that collects and analyzes voice data and has the function of recognizing voice commands and comments made by users in their daily lives.

[1858] "Online shopping history" is a record of a user's purchasing activities on the Internet, including information such as purchase date, product, and amount.

[1859] "Expense history" is a record of expenses incurred by a user in their daily life, including credit card statements, bank transaction statements, etc.

[1860] "Means of proposing life insurance and medical insurance" refers to the process of analyzing the user's health condition and lifestyle data, selecting the most suitable insurance plan, and offering it to the user.

[1861] "Means for optimizing financial planning and investment proposals" refers to the process of analyzing a user's income, expenses, and asset information and proposing long-term financial and investment plans.

[1862] "Means for optimizing budget allocation for leisure activities" refers to a process that takes into account the user's daily life and spending history and suggests allocating an appropriate budget for leisure activities.

[1863] A "generative AI model" is a model that uses artificial intelligence technology to analyze data and automatically generate optimal suggestions.

[1864] A "prompt" is an instruction that can be input into a generative AI model to perform a specific analysis or suggestion.

[1865] This invention is a system that integrates health checkup reports collected over multiple years, biometric data from wearable devices, photographic data for each meal, lifestyle data from a voice recognition device, and online shopping and expenditure history to select life insurance and medical insurance suited to an individual's health condition, as well as financial planning, investment proposals, and budget allocation for leisure activities. This system collects, analyzes, and optimizes data through mutual communication between a server, terminals, and users.

[1866] Data collection

[1867] Uploading your medical checkup report

[1868] The terminal provides the user with an interface and prompts them to upload their health checkup report. The user selects a PDF file and saves it on the terminal. The terminal then sends this file to the server. Specifically, the user uploads last year's health checkup report.

[1869] Biometric data collection from wearable devices

[1870] The device connects to the user's wearable device and periodically collects biometric data (heart rate, steps, sleep patterns, etc.) via Bluetooth or Wi-Fi. This collected data is then sent from the device to a server. For example, the user's wearable device sends daily heart rate and step count data to the server.

[1871] Upload a meal photo

[1872] The device saves photos of each meal taken by the user and uploads them to the server. The photos are automatically dated and time stamped. The user takes photos of each meal and saves them on the device through the app. For example, the user takes a photo of their lunch and the photo is uploaded to the server.

[1873] Lifestyle data collection from voice recognition devices

[1874] The voice recognition device collects voice data from the user's daily life. For example, if the user says, "I'm going to jog for 30 minutes today," the voice recognition device recognizes the voice and sends the data to the server via the device.

[1875] Collecting online shopping and spending history

[1876] The device periodically collects the user's online shopping history and credit card details and sends them to the server. For example, when a user makes a purchase on a shopping site, the details are automatically recorded and uploaded to the server.

[1877] Data analysis

[1878] Analysis of medical examination reports

[1879] The server analyzes the uploaded health checkup report, extracting text data from the PDF using OCR technology and extracting important health indicators (blood pressure, blood sugar, cholesterol levels, etc.), wh...

Claims

1. A means to upload medical examination reports; means for receiving biometric data from a user's wearable device; means for receiving a photograph of a meal taken by a user; A means of collecting daily life data from smart speakers, means for receiving online shopping and spending history; A means for analyzing the health checkup report, biometric data, food photos, daily life data, and expenditure history to propose life insurance and medical insurance optimized for each individual user; The system includes a means for optimizing the user's financial planning, investment proposals, and budget allocation for leisure activities based on the analysis results.

2. The system of claim 1 further comprising means for receiving user feedback and adjusting said suggestions.

3. The system according to claim 1, further comprising means for storing the biometric data and daily life data over a long period of time, and continuously learning and updating the user's health condition and lifestyle patterns.

Images