system

Abstract

To provide a diet support system adjusted to needs of a user.SOLUTION: The system includes: means for recording the current figure of a user in image data; means for presenting ideal figures in a selectable manner; means for registering information on the exercise routine of the user; means for generating information on at least one of a meal menu and an exercise program for losing weight on the basis of the current figure of the user, an image of an ideal figure selected by the user, and the exercise routine of the user, and also generating information for that period of time; means for collecting exercise data; means for analyzing the collected exercise data; generating a prompt sentence which generates a voice instruction including an exercise instruction to the user in real time on the basis of the result of the analysis; means for generating a voice instruction by using the generated prompt sentence and a generative AI model; and means for regenerating the generated voice instruction to a terminal of the user.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a conventional diet support system, it is difficult to provide an individualized diet menu and exercise program based on a user's body shape, exercise habits, and target body image, and the support during exercise is insufficient.

Means for Solving the Problems

[0005] This invention registers the user's body shape as an image, uses a generation AI to select an image that closely resembles the desired body shape, and registers the user's exercise habits to generate a diet plan including mealtimes, exercises, and duration. Furthermore, the generated plan is editable by the user, and voice support is provided during exercise. Specifically, during strength training, the next exercise is read aloud, and during running, the pace is monitored and suggestions are made to increase or decrease the pace. This enables diet support tailored to the individual needs of each user. [Brief explanation of the drawing]

[0006] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Embodiment 1 of Example 1. [Figure 12]This is a sequence diagram showing the processing flow of the data processing system in Application Example 1 of Form Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2 of Embodiment 2. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2 of Form Example 2. [Figure 15] This is a sequence diagram showing the processing flow of the data processing system in Embodiment 3 of Example 3. [Figure 16] This is a sequence diagram showing the processing flow of the data processing system in Application Example 3 of Form Example 3. [Figure 17] This is a sequence diagram showing the processing flow of the data processing system in Example 1 of the Form 1 when an emotion engine is combined. [Figure 18] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Modes for carrying out the invention]

[0007] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0008] First, let's explain the terminology used in the following explanation.

[0009] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)), etc.

[0010] In the following embodiments, the labeled RAM (Random Access Memory) is a memory where information is temporarily stored and is used as a work memory by the processor.

[0011] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0012] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0014] [First Embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of the "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. Also, the reception device 38, the output device 40, and the camera 42 are connected to the bus 52.

[0019] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0022] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0026] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0027] "Example of form 1"

[0028] The system of this invention allows the user to take a picture of their body shape using a digital camera or smartphone camera and register the image in the system. Next, the system selects the image that best matches the user's desired body shape from a set of body shape images provided within the system. Furthermore, the user registers their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day) in the system. Based on this information, the system uses a generative AI to generate an optimal diet plan, exercise program, and duration for the user. The generated menu can be edited by the user as needed.

[0029] "Example of form 2"

[0030] The system of this invention provides voice support when a user is exercising. For example, when performing strength training, it reads out the next exercise to be done (e.g., "Next, please do 20 sit-ups"). When running, it checks the user's pace and suggests increasing or decreasing the pace as needed (e.g., "At your current pace, you may not reach your target time. Try increasing your pace a little"). This enables the user to continue exercising appropriately.

[0031] The following describes the processing flow for each example of the form.

[0032] "Example of form 1"

[0033] Step 1: The user takes a picture of their body shape using a digital camera or smartphone camera. The captured image is uploaded to the system.

[0034] Step 2: From the multiple body shape images provided within the system, the user selects the one that best matches their desired body shape. This selection is made through the system's user interface.

[0035] Step 3: The user registers their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day) in the system. This registration is also done through the system's user interface.

[0036] Step 4: Based on this information, the system uses AI to generate an optimal diet plan, exercise program, and duration for the user. The generated plan can be edited by the user as needed.

[0037] "Example of form 2"

[0038] Step 1: When the user starts exercising, the system provides voice support.

[0039] Step 2: When performing strength training, the system will announce the next exercise to do (e.g., "Next, do 20 sit-ups").

[0040] Step 3: When running, the system monitors the user's pace and suggests increasing or decreasing the pace as needed (e.g., "At your current pace, you may not reach your target time. Try increasing your pace a little").

[0041] (Example 1)

[0042] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0043] Traditional diet systems have struggled to generate personalized diet plans based on users' body types and exercise habits. Furthermore, the lack of editing capabilities for the generated plans and insufficient support during exercise made it difficult for users to consistently follow their diets.

[0044] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0045] In this invention, the server includes means for registering the user's body shape as an image, means for presenting multiple body shape images and allowing the user to select their desired body shape, means for registering the user's exercise habits, means for generating a diet meal plan, exercise program, and duration using an AI model based on the user's body shape image, desired body shape, and exercise habit information, and means for editing the generated menu. This makes it possible to generate and edit a diet plan tailored to the user's individual needs.

[0046] "Method for registering user body shape with images" refers to a function that allows users to take pictures of their own body shape using a digital camera or smartphone camera and upload those images to the system.

[0047] "A means of presenting multiple body shape images and allowing the user to select the body shape they aspire to" refers to a function that presents the user with multiple body shape images prepared within the system and allows the user to select the one that most closely matches their desired body shape.

[0048] "Means for registering users' exercise habits" refers to a function that allows users to input their own exercise habits (e.g., jogging three times a week, walking for 30 minutes every day, etc.) into the system and register that information.

[0049] "A means of generating diet meal plans, exercise programs, and their duration using a generative AI model" refers to a function that uses a generative AI model to generate optimal diet meal plans, exercise programs, and their duration based on the user's body shape image, desired physique, and exercise habits.

[0050] "Means for editing generated menus" refers to a function that allows users to review the generated diet plan and edit meal menus and exercise programs as needed.

[0051] This invention begins with a user taking a picture of their own body shape using a digital camera or smartphone camera and registering that image in the system. The user launches the camera app on their smartphone and takes a picture of their body shape. The captured image is uploaded from the terminal to the system, and the server receives the uploaded image and saves it to a database. At this time, the image quality is checked using an image processing library (e.g., OpenCV).

[0052] Next, the server presents the user with several body shape images prepared within the system. The user selects the image that most closely matches their desired body shape from the presented images. The ID of the selected body shape image is sent from the terminal to the server, and the server stores that ID in its database.

[0053] Users input their exercise habits from their device. For example, they might enter information such as "I jog three times a week" or "I walk for 30 minutes every day." The entered exercise habit data is sent from the device to the server, which stores that data in a database.

[0054] The server acquires information on the user's body shape image, desired physique, and exercise habits, and prompts a generation AI model (e.g., GPT-4®) to generate a diet plan. An example of a prompt is, "Based on the user's body shape image, desired physique, and exercise habits, please generate an optimal diet meal plan, exercise program, and duration." The generation AI model generates the optimal meal plan, exercise program, and duration based on the prompt. The generated diet plan is then saved to a database by the server.

[0055] Users can view the generated diet plan on their device and edit the meal menu and exercise program as needed. For example, they can change breakfast from oatmeal to yogurt. The edited information is sent from the device to the server, which then updates the database.

[0056] In this way, the system allows users to easily create a diet plan tailored to their own goals and edit it as needed.

[0057] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0058] Step 1: Register the user's body shape image.

[0059] The user launches the camera app on their smartphone and takes a picture of their own body shape.

[0060] Input: Body shape image taken by the user

[0061] The device displays the captured image on the system's upload screen, and the user presses the "Upload" button.

[0062] The server receives the uploaded image and uses an image processing library (e.g., OpenCV) to check the image quality.

[0063] Output: Quality-verified body shape image

[0064] The server saves verified images to the database.

[0065] Step 2: Choosing your desired body image

[0066] The server retrieves multiple body shape images from the database and displays them on the user's device.

[0067] Input: Multiple body shape images stored in a database

[0068] Users tap to select their desired body shape from the displayed images.

[0069] The device sends the ID of the selected body shape image to the server.

[0070] Output: ID of the body shape image selected by the user

[0071] The server saves the ID of the selected body shape image to the database.

[0072] Step 3: Register your exercise habit

[0073] The user enters their exercise habits into the input form on their device (e.g., "I jog three times a week").

[0074] Input: Exercise habit data entered by the user.

[0075] The device sends the entered exercise habit data to the server.

[0076] Output: Exercise habit data

[0077] The server stores data on exercise habits in a database.

[0078] Step 4: Generating an optimal diet plan

[0079] The server retrieves information from the database regarding the user's body shape image, desired physique, and exercise habits.

[0080] Input: User's body shape image, desired physique, and exercise habits.

[0081] The server will input the following prompt into the generated AI model (e.g., GPT-4):

[0082] "Based on the user's body shape image, desired physique, and exercise habits, please generate an optimal diet plan, exercise program, and duration."

[0083] The generative AI model generates optimal meal plans, exercise programs, and their durations based on the prompt text.

[0084] Output: Generated diet plan (meal menu, exercise program, duration)

[0085] The server saves the generated diet plan to the database.

[0086] Step 5: Edit the menu

[0087] Users can view the generated diet plan on their device.

[0088] Input: Generated diet plan

[0089] Users can edit their meal plans and exercise programs as needed (e.g., change oatmeal for breakfast to yogurt).

[0090] The terminal sends the edited content to the server.

[0091] Output: Edited diet plan

[0092] The server updates the database with the edited diet plan.

[0093] (Application Example 1)

[0094] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0095] Traditional diet and fitness programs often offer generic menus and exercise plans, making it difficult to provide programs optimized for individual users' body types and exercise habits. Furthermore, fitness facilities faced the challenge of placing a heavy burden on trainers to provide individually optimized training programs and meal plans. Additionally, the lack of real-time feedback made it difficult for users to train effectively.

[0096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0097] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, and means for registering the user's exercise habits. This makes it possible to provide individually optimized training programs and meal plans.

[0098] "Method for registering a user's body shape using an image" refers to a method for users to take a picture of their own body shape using a digital camera or smartphone camera and register that image in the system.

[0099] "A method of selecting an image that closely resembles the desired body shape using generative AI" refers to a method of using generative AI to select the image that most closely matches the user's desired body shape from multiple body shape images provided within the system.

[0100] "Means for registering users' exercise habits" refers to the means by which users register their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day, etc.) in the system.

[0101] "Methods for generating diet meal plans, exercises, and durations" refers to methods that use a generating AI to create optimal diet meal plans, exercise programs, and their durations for the user, based on the user's body shape image and exercise habits.

[0102] "Means for editing generated menus" refers to means that users can edit the generated diet meal plans and exercise programs as needed.

[0103] "A means of providing individually optimized training programs and meal plans by having fitness facility members take photos of their own physique, select their desired body shape, and register their exercise habits" refers to a means of providing individually optimized training programs and meal plans by having fitness facility members take photos of their own physique, select their desired body shape, and register their exercise habits.

[0104] "A means of receiving real-time feedback in collaboration with trainers within the facility" refers to a method of receiving real-time feedback on training and diet in collaboration with trainers within the fitness facility.

[0105] A system for carrying out this invention includes means for registering the user's body shape with an image, means for selecting an image that closely resembles the desired body shape using a generating AI, means for registering the user's exercise habits, means for generating a diet meal plan, exercise routines, and duration, means for editing the generated menu, means for providing individually optimized training programs and meal plans by allowing fitness facility members to take a picture of their body shape, select their desired body shape, and register their exercise habits, and means for receiving real-time feedback in cooperation with trainers within the facility.

[0106] 1. System Program

[0107] The program in this system performs the following operations:

[0108] 2. Explanation of the program's processing

[0109] The server receives body shape images taken by the user using their smartphone camera and performs image processing. Specifically, it uses the Python PIL library to load the images and preprocess them as needed. Next, it uses a generative AI model to select the image that most closely resembles the user's desired body shape. This generative AI model uses a pre-trained Keras model.

[0110] The user's exercise habits are registered on the server in text format. This includes information such as how many times a week the user exercises and what kind of exercise they do. Based on this information, the generative AI model generates an optimal diet plan, exercise program, and duration for the user.

[0111] The generated menu can be viewed by the user via a smartphone app and edited as needed. Furthermore, users can collaborate with trainers at fitness facilities to receive real-time feedback. This feedback includes training progress and dietary advice.

[0112] 3. Specific Examples and Examples of Prompt Statements

[0113] As a concrete example, consider a user who jogs three times a week. Based on this information, the generative AI model generates an optimal meal plan and exercise program for the user. The following is an example of a prompt to input to the generative AI model.

[0114] User's body shape image: user_image.jpg

[0115] Target body shape: target_shape.jpg

[0116] Exercise habits: I jog three times a week.

[0117] Based on this information, please generate an optimal diet plan and exercise program.

[0118] By using this prompt message as input to the generative AI model, it is possible to generate a menu that is optimal for the user.

[0119] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0120] Step 1:

[0121] The user takes a picture of their body shape using their smartphone camera and saves the image to their device.

[0122] Input: User's body shape image

[0123] Output: Body shape image saved on the device

[0124] Specific actions: The user launches the camera app on their smartphone and takes a picture of their own body shape. The captured image is then saved to the device.

[0125] Step 2:

[0126] The device uploads the saved body shape image to the server.

[0127] Input: Body shape image saved on the device

[0128] Output: Body shape images uploaded to the server

[0129] Specific operation: The device uploads the saved body shape images to the server via an internet connection. HTTP requests are used for the upload.

[0130] Step 3:

[0131] The server receives the uploaded body shape image and performs image processing.

[0132] Input: Body shape image uploaded to the server

[0133] Output: Preprocessed body shape image

[0134] Specific operation: The server uses the Python PIL library to read images and performs preprocessing such as resizing and denoising as needed.

[0135] Step 4:

[0136] The server uses a generated AI model to select the image that most closely matches the user's desired body shape.

[0137] Input: Preprocessed body shape image

[0138] Output: The image that most closely matches the desired body shape.

[0139] Specific operation: The server uses a pre-trained Keras model to compare the user's body shape image with the image of the target body shape and selects the closest image.

[0140] Step 5:

[0141] The user enters their exercise habits into their device and sends them to the server.

[0142] Input: User's exercise habit information

[0143] Output: Exercise habit information sent to the server

[0144] Specific operation: The user inputs their exercise habits (e.g., jogging three times a week) through a smartphone app and sends that information to the server.

[0145] Step 6:

[0146] The server uses a generative AI model to generate optimal diet meal plans and exercise programs based on the user's body shape image and exercise habit information.

[0147] Input: User's body shape image, exercise habit information

[0148] Output: Optimal diet meal plans and exercise programs

[0149] Specific operation: The server inputs prompt messages into the generating AI model, which then generates an optimal diet meal plan and exercise program for the user. Examples of prompt messages are as follows:

[0150] User's body shape image: user_image.jpg

[0151] Target body shape: target_shape.jpg

[0152] Exercise habits: I jog three times a week.

[0153] Based on this information, please generate an optimal diet plan and exercise program.

[0154] Step 7:

[0155] The server sends the generated menu to the user's terminal, and the user edits it as needed.

[0156] Input: Optimal diet meal plans and exercise programs

[0157] Output: Menu sent to the user's terminal

[0158] Specific operation: The server sends the generated menu to the user's device, and the user views the menu through a smartphone app and edits it as needed.

[0159] Step 8:

[0160] Users train within a fitness facility and receive real-time feedback in collaboration with trainers.

[0161] Input: Training progress, dietary advice

[0162] Output: Real-time feedback

[0163] Specific operation: Users train within a fitness facility, and trainers monitor the user's progress and provide real-time feedback through a dedicated application.

[0164] (Example 2)

[0165] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0166] Conventional exercise support systems have struggled to collect and analyze user exercise data in real time and provide appropriate voice guidance. Furthermore, they lacked sufficient support in generating specific exercise and meal plans to help users achieve their desired physique, and in providing adequate guidance during exercise. This made it difficult for users to effectively maintain a consistent exercise routine.

[0167] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body using a generation AI model, means for registering the user's exercise habits, means for generating a diet meal menu, exercises, and duration, means for editing the generated menu, means for collecting the user's exercise data, means for analyzing the collected exercise data, means for generating voice instructions based on the analysis results, means for transmitting the generated voice instructions to the user's terminal, and means for playing the voice instructions on the user's terminal. As a result, the user can effectively continue exercising to get closer to their desired body while receiving appropriate exercise instructions in real time.

[0168] "Method for registering a user's body shape with images" refers to a function that records a user's current body shape as image data using devices such as cameras or scanners.

[0169] "A method for selecting images that closely resemble the desired body shape using a generative AI model" refers to a function that utilizes a generative AI model to present images that closely resemble the user's ideal body shape, allowing the user to select from among them.

[0170] "Methods for registering users' exercise habits" refers to a function that records information such as the type, frequency, and duration of exercise that users perform on a daily basis in a database.

[0171] "Methods for generating diet meal plans, exercises, and durations" refers to a function that automatically creates appropriate meal plans, exercise plans, and durations based on the user's goals, current body shape, and exercise habits.

[0172] "Means for editing generated menus" refers to a function that allows users to manually modify and adjust automatically generated meal plans and exercise plans.

[0173] "Means of collecting user exercise data" refers to a function that uses wearable devices or smartphones to acquire exercise-related data such as the user's heart rate, speed, and distance in real time.

[0174] "Means for analyzing collected exercise data" refers to a function that evaluates and analyzes the user's exercise status and performance based on the collected exercise data.

[0175] "Means for generating voice instructions based on analysis results" refers to a function that generates instructions to provide users with appropriate exercise instructions and advice via voice, based on the results of data analysis.

[0176] "Means for sending generated voice commands to the user's device" refers to a function that sends generated voice commands to the user's smartphone or tablet via communication means such as the internet or Bluetooth.

[0177] "Means for playing voice commands on the user's device" refers to a function that plays voice commands received by the user's smartphone or tablet through a speaker or earphones.

[0178] This invention is a system that provides voice support to users while they exercise. The system registers the user's body shape as an image, uses a generative AI model to select an image that closely resembles the desired body shape, and registers the user's exercise habits. Furthermore, it includes a function to generate a diet plan, exercise routine, and duration, and to edit the generated plan.

[0179] The server collects and analyzes the user's exercise data in real time. The hardware used includes wearable devices (e.g., smartwatches and fitness trackers) for collecting the user's exercise data. The software includes algorithms for data analysis and generative AI models (e.g., OpenAI®'s GPT-4) for speech synthesis.

[0180] The device (the user's smartphone or tablet) relays voice instructions sent from the server to the user. Specifically, the device plays the voice data received from the server, providing the user with advice on the next exercise routine and pacing.

[0181] Users receive voice instructions from the device while exercising and continue exercising according to those instructions. For example, when doing strength training, they might receive the instruction, "Next, do 20 sit-ups," and then perform sit-ups. When running, if their current pace is not reaching their target time, they might receive advice such as, "Try to pick up the pace a little," and adjust their pace accordingly.

[0182] As a concrete example, consider a scenario where a user is running. The user puts on a smartwatch and starts running with their smartphone. The smartwatch collects data such as the user's heart rate and speed and sends it to a server. The server analyzes this data and detects that the user's current pace is not enough to reach their target time. The server uses a generative AI model to generate a voice command saying, "At your current pace, you may not reach your target time. Try to pick up the pace a little," and sends it to the device. The device plays this voice command and communicates it to the user.

[0183] Example of a prompt:

[0184] "When a user is running and their current pace is not reaching their target time, please generate voice prompts suggesting they increase their pace."

[0185] This system allows users to consistently engage in appropriate exercise.

[0186] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0187] Step 1:

[0188] Before starting exercise, the user puts on a wearable device (e.g., a smartwatch) and launches a dedicated app on their smartphone. The wearable device collects exercise data such as the user's heart rate, speed, and distance in real time. This data is transmitted to the user's smartphone via Bluetooth or Wi-Fi. The input is the user's exercise data, and the output is the exercise data transmitted to the smartphone.

[0189] Step 2:

[0190] The server receives exercise data transmitted from the user's smartphone. The server uses data analysis algorithms to evaluate the user's exercise status in real time. For example, it can determine whether the user's pace during a run is reaching their target time. The input is the exercise data transmitted from the smartphone, and the output is the analysis result.

[0191] Step 3:

[0192] The server uses a generative AI model (e.g., a generative AI model) to generate appropriate voice instructions for the user. For example, if the user is moving too slowly, it might generate a voice instruction such as, "At your current pace, you may not reach your target time. Try to pick up the pace a little." The input is the analysis result, and the output is the generated voice instruction.

[0193] Step 4:

[0194] The server sends the generated voice commands to the user's smartphone. An internet connection is required for transmission. The input is the generated voice command, and the output is the voice command sent to the smartphone.

[0195] Step 5:

[0196] The device (the user's smartphone) plays the voice instructions received from the server. The user can hear the voice instructions through the smartphone's speaker or earphones. The input is the voice instructions received from the server, and the output is the played voice instructions.

[0197] Step 6:

[0198] The user continues exercising according to the voice instructions from the device. For example, if instructed to "do 20 sit-ups next," the user will perform 20 sit-ups. Similarly, if instructed to "try to increase your pace," the user will increase their running pace. The input is the played voice instructions, and the output is the user's exercise behavior.

[0199] (Application Example 2)

[0200] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0201] While conventional exercise support systems can provide appropriate menus based on a user's exercise habits and physique, they have limitations in terms of efficient route guidance and time management during delivery. Furthermore, there was a lack of means to provide appropriate advice when delivery delays were anticipated, highlighting the need for increased efficiency in delivery operations.

[0202] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0203] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, means for registering the user's exercise habits, means for generating a diet meal menu, exercise routines, and duration, means for editing the generated menu, means for registering delivery destination information, means for generating a delivery route, means for managing delivery time, means for guiding the delivery destination and route by voice, and means for prompting the user to speed up if the delivery time is likely to be delayed. This enables efficient route guidance and time management in delivery operations, and allows for the provision of appropriate advice if the delivery time is likely to be delayed.

[0204] "Methods for registering a user's body shape using images" refers to methods for saving a user's body shape and size as image data.

[0205] "A method of selecting images that closely resemble the desired body using generation AI" refers to a method of using artificial intelligence to select images of the body that the user aims for.

[0206] "Methods for registering users' exercise habits" refer to methods for recording the types and frequency of exercises that users perform on a daily basis.

[0207] "Methods for generating diet meal plans, exercises, and durations" refers to methods for creating meal plans, exercise programs, and implementation periods tailored to the user's goals.

[0208] "Means for editing generated menus" refers to methods for modifying created meal plans and exercise programs according to the user's requests.

[0209] "Methods for registering delivery address information" refers to methods for recording information such as the address and contact details of the place where the delivery will be made.

[0210] "Methods for generating delivery routes" refer to methods for calculating the optimal route for efficiently visiting multiple delivery destinations.

[0211] "Means of managing delivery times" refers to methods for recording and managing the estimated arrival time and actual arrival time at each delivery destination.

[0212] "A method for guiding delivery destinations and routes via voice" refers to a method of instructing delivery personnel by voice to the next delivery destination or the optimal route.

[0213] "Methods to encourage speeding up when delivery is likely to be delayed" refers to methods of using voice commands to instruct delivery personnel to increase their speed when deliveries are behind schedule.

[0214] The system for implementing this invention has the function of registering the user's body shape with an image, selecting an image that closely resembles the desired body shape using a generation AI, and registering the user's exercise habits. It also includes the function of generating a diet meal menu, exercise plan, and duration, and editing the generated menu. Furthermore, it has the function of registering delivery destination information, generating a delivery route, and managing delivery times. It also includes a function to provide voice guidance for the delivery destination and route, and to encourage the user to speed up if the delivery time is likely to be delayed.

[0215] System program processing

[0216] The server acquires image data using a smartphone or camera to register the user's body shape and stores it in a database. Using a generative AI model, it generates prompts for the user to select an image that closely resembles their desired physique and presents them to the user. To register the user's exercise habits, it collects exercise data from a smartphone app or wearable device and stores it in a database.

[0217] To generate a diet plan including meal menus, exercise routines, and duration, the server creates an optimal plan based on the user's goals, current physique, and exercise habits. Users can edit the generated menu using a smartphone app.

[0218] To register delivery information, the server stores the delivery address and contact information in a database. To generate delivery routes, the server uses map data to calculate the optimal route. To manage delivery times, the server records and manages the estimated and actual arrival times for each delivery location.

[0219] To guide delivery drivers to their destinations and routes via voice, the server uses speech synthesis technology to issue instructions. If a delivery is likely to be delayed, the server will also instruct the driver to speed up via voice.

[0220] Specific example

[0221] For example, if a delivery person delivers to "1-1-1 Shibuya-ku, Tokyo", the server will operate as follows:

[0222] 1. Retrieve "1-1-1 Shibuya-ku, Tokyo" from the list of delivery addresses.

[0223] 2. The voice will announce, "Your next delivery address is 1-1-1 Shibuya-ku, Tokyo. We will guide you to the optimal route."

[0224] 3. If it appears that the delivery time will be delayed, a voice message will advise, "At the current pace, the delivery time will be delayed. Please increase your pace slightly."

[0225] Examples of prompts to input into a generative AI model:

[0226] "Your next delivery address is 1-1-1 Shibuya-ku, Tokyo. I will guide you to the optimal route. At your current pace, the delivery time will likely be delayed. Please pick up the pace a little."

[0227] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0228] Step 1:

[0229] The server registers the user's body shape as an image. The user takes a picture of their body shape using a smartphone or camera and sends the image data to the server. The server stores the received image data in a database. The input is the user's body shape image, and the output is the image data stored in the database.

[0230] Step 2:

[0231] The server generates prompts to select an image that closely resembles the user's desired physique using a generative AI model. When the user inputs the desired physical characteristics, the server uses the generative AI model to generate an image based on those characteristics and presents it to the user. The input is the user's desired physical characteristics, and the output is the generated image.

[0232] Step 3:

[0233] The server registers the user's exercise habits. Users input daily exercise data using a smartphone app or wearable device and send that data to the server. The server stores the received exercise data in a database. The input is the user's exercise data, and the output is the exercise data stored in the database.

[0234] Step 4:

[0235] The server generates meal plans, exercise routines, and timelines for weight loss. Based on the user's goals, current physique, and exercise habits, the server generates and presents an optimal meal plan and exercise program to the user. The input is the user's goals, physique, and exercise habits, and the output is the generated meal plan and exercise program.

[0236] Step 5:

[0237] The user edits the generated menu. Using a smartphone app, the user modifies the generated meal plan and exercise program and sends the modified data to the server. The server stores the received modified data in a database. The input is the user's modified data, and the output is the modified data stored in the database.

[0238] Step 6:

[0239] The server registers delivery address information. The user enters the delivery address and contact information and sends this information to the server. The server stores the received delivery address information in a database. The input is the delivery address and contact information, and the output is the delivery address information stored in the database.

[0240] Step 7:

[0241] The server generates delivery routes. Based on the delivery destination information, the server uses map data to calculate the optimal route and presents it to the delivery person. The input is the delivery destination information, and the output is the optimal delivery route.

[0242] Step 8:

[0243] The server manages delivery times. It records and manages estimated and actual arrival times for each delivery destination. Inputs are delivery destination information and current location information, and outputs are estimated and actual arrival times.

[0244] Step 9:

[0245] The server provides voice guidance for delivery locations and routes. Using speech synthesis technology, it instructs delivery personnel verbally on the next delivery location and the optimal route. The input is delivery route information, and the output is voice guidance.

[0246] Step 10:

[0247] The server prompts the delivery person to speed up if the delivery is likely to be delayed. If the delivery is behind schedule, the server will verbally instruct the delivery person to increase their speed. The input is the current location and estimated arrival time, and the output is a verbal instruction to speed up.

[0248] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0249] "Example of form 1"

[0250] In one embodiment of the present invention, an emotion engine that recognizes the user's emotions is incorporated into the system. This emotion engine estimates emotions from the user's facial expressions, tone of voice, and movements during exercise. For example, if the user shows a pained expression during exercise, the emotion engine captures this information and determines that the user is experiencing excessive stress. This information is fed back to the system, and the content and tone of the voice support are adjusted accordingly. Specifically, the user may be provided with encouraging messages, or the intensity of the exercise may be automatically reduced.

[0251] "Example of form 2"

[0252] Furthermore, the emotion engine adjusts diet meal plans and exercise programs according to the user's emotional state. For example, if a user shows a joyful expression after exercise, the emotion engine picks up on this information and determines that the user likes that exercise. This information is fed back into the system, and that exercise is frequently incorporated into recommended exercise programs. Similarly, if a user shows a satisfied expression after a meal, the system determines that the meal plan is favorable to the user and recommends similar meal plans.

[0253] The following describes the processing flow for each example of the form.

[0254] "Example of form 1"

[0255] Step 1: The user begins exercising.

[0256] Step 2: The emotion engine estimates the user's emotions from their facial expressions, tone of voice, and movements during exercise.

[0257] Step 3: If the emotion engine determines that the user is experiencing excessive stress, it feeds that information back into the system.

[0258] Step 4: The system adjusts the content and tone of the voice support. Specifically, it may provide encouraging messages to the user or automatically lower the intensity of the exercise.

[0259] "Example of form 2"

[0260] Step 1: The user finishes exercising.

[0261] Step 2: The emotion engine estimates the user's emotions from their facial expressions.

[0262] Step 3: If the emotion engine determines that the user likes the exercise, it feeds that information back into the system.

[0263] Step 4: Adjust the system so that the exercise is frequently incorporated into the recommended exercise program.

[0264] Step 5: The user finishes their meal.

[0265] Step 6: The emotion engine estimates the user's emotions from their facial expressions.

[0266] Step 7: If the emotion engine determines that the meal menu is favorable to the user, it feeds that information back into the system.

[0267] Step 8: Adjust the system so that it recommends similar meal menus.

[0268] (Example 1)

[0269] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0270] Conventional diet support systems have faced challenges in providing effective diet support because they struggle to offer personalized menus based on the user's body shape and exercise habits, and they lack feedback that takes into account the user's emotions and stress levels. Furthermore, they cannot detect the stress and fatigue the user experiences during exercise in real time and provide support accordingly, making it difficult to maintain the user's motivation.

[0271] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0272] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, means for registering the user's exercise habits, means for generating a diet menu, exercises, and duration, means for editing the generated menu, means including an emotion engine that recognizes the user's emotions, means for the emotion engine to analyze the user's facial expressions, tone of voice, and movements during exercise to estimate emotions, means for adjusting the content and tone of voice support based on emotions, and means for automatically adjusting the intensity of exercise. This makes it possible not only to provide a personalized menu based on the user's body shape and exercise habits, but also to detect the user's emotions and stress levels in real time and provide support accordingly.

[0273] "Method for registering user body shape with images" refers to a function that allows users to take pictures of their own body shape using a digital camera or smartphone camera and upload those images to the system.

[0274] "The method of selecting an image that closely resembles the desired body shape using generative AI" refers to a function that uses generative AI to select the image that most closely matches the user's desired body shape from multiple body shape images provided within the system.

[0275] "Means for registering users' exercise habits" refers to a function that allows users to input their own exercise habits (e.g., jogging three times a week, walking for 30 minutes every day, etc.) into the system and save that information.

[0276] "Method for generating diet meal plans, exercises, and durations" refers to a function that uses a generative AI model to create optimal diet meal plans, exercise programs, and their durations based on the user's body shape image, desired physique, and exercise habits.

[0277] "Means for editing generated menus" refers to a function that allows users to review generated diet meal plans and exercise programs and make changes or additions as needed.

[0278] "Means including an emotion engine that recognizes user emotions" refers to a function that incorporates an emotion engine into the system to estimate emotions from the user's facial expressions, tone of voice, movements during exercise, etc.

[0279] "The emotion engine's method of analyzing the user's facial expressions, tone of voice, and movements during exercise to estimate emotions" refers to the function of the emotion engine that analyzes data collected through the camera and microphone to estimate the user's emotional state.

[0280] The means for adjusting the content and tone of voice support based on emotions refers to the function of changing the content and tone of voice support according to the emotional state of the user estimated by the emotion engine and providing messages to encourage the user.

[0281] The means for automatically adjusting the intensity of exercise refers to the function of appropriately changing the intensity of exercise based on the emotional state of the user by the emotion engine to reduce the stress and fatigue of the user.

[0282] Mode for implementing the invention

[0283] The present invention starts with the user taking a picture of their body shape using a digital camera or the camera of a smartphone and registering the image in the system. Next, the user selects the body image that is closest to the body image they aim for from a plurality of body shape images prepared in the system. Furthermore, the user registers their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day, etc.) in the system. Based on this information, the system generates an optimal diet menu, exercise program, and its period for the user using a generation AI model. The generated menu can be edited by the user as needed.

[0284] In one embodiment of the present invention, an emotion engine for recognizing the emotions of the user is incorporated into the system. This emotion engine estimates emotions from the user's facial expressions, tone of voice, movements during exercise, etc. For example, when the user shows a painful expression during exercise, the emotion engine captures this information and determines that the user is feeling excessive stress. This information is fed back to the system, and the content and tone of voice support are adjusted. Specifically, messages to encourage the user are provided, or the intensity of exercise is automatically decreased.

[0285] Hardware and software to be used

[0286] This system uses the following hardware and software.

[0287] Digital camera or smartphone camera: Used to photograph the user's body shape.

[0288] Devices (PCs, smartphones, tablets, etc.): Used by users to access the system and register images and exercise habits.

[0289] Server: Stores data sent by users and generates diet menus and exercise programs using a generation AI model.

[0290] Generative AI Model: Based on the user's body shape image, desired physique, and exercise habits, it generates an optimal diet plan and exercise program.

[0291] Emotion Engine: Analyzes the user's facial expressions, voice tone, and movements during exercise to estimate their emotions.

[0292] Specific example

[0293] Example 1: Registration of body shape images

[0294] Users take photos of their body shape with their smartphone cameras and upload the images to the system. The server receives the images and saves them to the user's profile.

[0295] Specific example 2: Choosing your desired body image

[0296] The user selects their desired body shape from body shape images within the system and sends that information to the server. The server saves the selected image as the user's target.

[0297] Example 3: Registering an exercise habit

[0298] The user enters their exercise habits, such as "I jog three times a week," into the system and sends it to the server. The server then adds this information to the user's profile.

[0299] Specific Example 4: Generation of Diet Menu

[0300] Based on the user's body shape image, target body image, and exercise habit information, the server uses a generation AI model to generate a "one-week diet menu". The generated menu is sent to the user's terminal, and the user can edit it as needed.

[0301] Specific Example 5: Feedback by Emotion Engine

[0302] When the user shows a pained expression during exercise, the emotion engine captures this information and provides feedback to the server. The server generates a message to encourage the user and sends it to the terminal. Also, the intensity of the exercise is automatically reduced.

[0303] Examples of Prompt Sentences

[0304] "Please upload the user's body shape image, select the target body image. Next, enter your exercise habits, and the optimal diet menu will be generated. Emotions during exercise will also be recognized and feedback will be provided."

[0305] The flow of the specific process in Example 1 will be described using FIG. 15.

[0306] Step 1:

[0307] The user takes a photo of their body shape using a digital camera or the camera of a smartphone.

[0308] Input: User's body shape image

[0309] Output: Captured body shape image file

[0310] Specific operation: The user launches the camera app and takes a photo from the front so that the whole body is shown. After taking the photo, save the image file.

[0311] Step 2:

[0312] The user logs into the system from their device and uploads the images they have taken.

[0313] Input: Captured body shape image file

[0314] Output: Body shape image saved on the server

[0315] Specific steps: The user accesses the system's login screen, enters their user ID and password, and logs in. After logging in, they move to the image upload screen, select the saved image file, and click the upload button.

[0316] Step 3:

[0317] The server saves the received images and associates them with the user's profile.

[0318] Input: Uploaded body shape image file

[0319] Output: Body shape image associated with the user's profile

[0320] Specific operation: The server receives the uploaded image file and saves it to the database. The saved image is linked to the user ID.

[0321] Step 4:

[0322] The user views multiple body shape images provided within the system.

[0323] Input: Body shape image data within the system

[0324] Output: List of body shape images viewed by the user

[0325] Specific operation: The user accesses a body shape image selection screen and views body shape images displayed in a slideshow format.

[0326] Step 5:

[0327] Select the image that best matches the user's desired physique.

[0328] Input: User selects body shape image

[0329] Output: Selected body shape image data

[0330] Specific action: The user selects the image that best represents their desired body shape and clicks the select button.

[0331] Step 6:

[0332] The server saves the selected image as the user's target.

[0333] Input: Selected body shape image data

[0334] Output: Target body shape image saved in the user's profile

[0335] Specific operation: The server receives the selected image data and saves it to the user's profile as a target body shape image.

[0336] Step 7:

[0337] Users input their exercise habits into the system via their device.

[0338] Input: User's exercise habits (e.g., jogs 3 times a week)

[0339] Output: Input exercise habit data

[0340] Specific actions: The user accesses the exercise habit input screen and enters their exercise habits into the text box. After entering the information, they click the save button.

[0341] Step 8:

[0342] The server receives the entered information about exercise habits and adds it to the user's profile.

[0343] Input: Entered exercise habit data

[0344] Output: Exercise habit information added to the user's profile

[0345] Specific operation: The server receives the entered exercise habit data and saves it to the database. The saved data is linked to the user ID.

[0346] Step 9:

[0347] Based on the user's body shape image, desired physique, and exercise habits, the server uses a generative AI model to generate an optimal diet plan, exercise program, and duration.

[0348] Input: User's body shape image, desired body shape, exercise habit information

[0349] Output: Generated diet meal plans and exercise programs

[0350] Specific operation: The server runs the generated AI model and creates a menu based on the input data, taking into account calorie calculations and nutritional balance.

[0351] Step 10:

[0352] The server sends the generated menu to the user's terminal.

[0353] Input: Generated diet meal plans and exercise programs

[0354] Output: Menu sent to the user's terminal

[0355] Specific operation: The server sends the generated menu to the user's terminal in PDF format or similar.

[0356] Step 11:

[0357] Review the menu received by the user and edit it as needed.

[0358] Input: Generated diet meal plans and exercise programs

[0359] Output: Edited menu

[0360] Specific actions: The user accesses the menu editing screen and makes changes to ingredients or adds exercises. After editing, they click the save button.

[0361] Step 12:

[0362] The user uses the system while exercising.

[0363] Input: User's exercise data (e.g., facial expressions, voice tone, movements during exercise)

[0364] Output: Real-time collected motion data

[0365] Specific operation: The user exercises while holding a smartphone, and data is collected through the camera and microphone.

[0366] Step 13:

[0367] The emotion engine analyzes the user's facial expressions, voice tone, and movements during exercise in real time.

[0368] Input: Real-time collected exercise data

[0369] Output: Estimated user emotional state

[0370] Specific operation: The emotion engine analyzes the collected data and estimates the user's emotional state.

[0371] Step 14:

[0372] The emotion engine estimates the user's emotions, and if it determines that the user is experiencing excessive stress, it feeds that information back to the server.

[0373] Input: Estimated user's emotional state

[0374] Output: Sentiment information fed back to the server

[0375] Specific operation: The emotion engine determines the user's emotional state and, if it detects excessive stress, sends that information to the server.

[0376] Step 15:

[0377] The server adjusts the content and tone of the voice support based on feedback and generates encouraging messages for the user.

[0378] Input: Feedback on emotional information

[0379] Output: Generated encouraging message

[0380] Specific operation: The server modifies the content and tone of the voice support based on emotional information, generating messages such as, "Keep going! You're almost there!"

[0381] Step 16:

[0382] The server automatically adjusts the exercise intensity and sends the information to the user's device.

[0383] Input: Feedback on emotional information

[0384] Output: Instructions for adjusted exercise intensity

[0385] Specific operation: The server adjusts the exercise intensity appropriately based on emotional information and sends instructions to the user's device. For example, it might instruct the user to slightly slow down their exercise pace.

[0386] (Application Example 1)

[0387] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0388] Traditional diet support systems can provide personalized menus based on a user's body shape and exercise habits, but they have the drawback of not being able to provide feedback that takes into account the user's emotional state. This has led to problems such as users experiencing excessive stress or being unable to maintain motivation.

[0389] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0390] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, means for registering the user's exercise habits, means for generating a diet menu, exercises, and duration, means for editing the generated menu, means for recognizing the user's emotions, and means for providing feedback based on the user's emotional state. This makes it possible not only to provide an optimal menu based on the user's individual body shape and exercise habits, but also to provide feedback according to the emotional state.

[0391] "Method for registering a user's body shape using an image" refers to a method for users to take a picture of their own body shape using a digital camera or smartphone camera and register that image in the system.

[0392] "A method for selecting an image that closely resembles the desired body shape using a generating AI" refers to a method for selecting the image that most closely matches the user's desired body shape from multiple body shape images provided within the system, using a generating AI.

[0393] "Means for registering users' exercise habits" refers to the means by which users register their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day, etc.) in the system.

[0394] "Methods for generating diet meal plans, exercises, and durations" refers to methods for generating optimal diet meal plans, exercise programs, and their durations for a user, using a generating AI based on the user's body shape image and exercise habits.

[0395] "Means for editing generated menus" refers to means that users can edit the generated diet meal plans and exercise programs as needed.

[0396] "Means of recognizing user emotions" refers to methods for estimating emotions from the user's facial expressions, tone of voice, and movements during exercise.

[0397] "Means of providing feedback based on the user's emotional state" refers to methods for adjusting the content and tone of voice support according to the user's emotional state, providing encouraging messages to the user, or automatically reducing the intensity of exercise.

[0398] A system for carrying out this invention includes means for registering the user's body shape with images, means for selecting images that closely resemble the desired body shape using a generating AI, means for registering the user's exercise habits, means for generating a diet menu, exercises, and duration, means for editing the generated menu, means for recognizing the user's emotions, and means for providing feedback based on the user's emotional state.

[0399] Hardware and software configuration

[0400] Hardware:

[0401] Smartphone (camera, microphone, sensors)

[0402] Server (data processing, execution of generational AI models)

[0403] software:

[0404] Image processing library (OpenCV)

[0405] Emotion recognition engine (Microsoft® Azure® Cognitive Services)

[0406] Generative AI model (GPT-4)

[0407] Database (MySQL(registered trademark))

[0408] Data processing and data calculation

[0409] User body shape image registration:

[0410] Users take photos of their body shape using their smartphone cameras and upload the images to the system. The system then uses an image processing library (OpenCV) to analyze body shape data from the images.

[0411] Select an image that closely resembles the body you want to achieve:

[0412] Using a generative AI model (GPT-4), the system selects the image that most closely matches the user's desired body shape from multiple body shape images provided within the system.

[0413] Register your exercise habits:

[0414] Users input their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day) into the system as text data.

[0415] Generating diet meal plans and exercise programs:

[0416] Enter the following prompts into the generating AI model to generate an optimal diet plan and exercise program.

[0417] Based on the user's body shape image and exercise habits, please provide the following information:

[0418] 1. Body shape image that most closely matches the user's desired physique.

[0419] 2. Optimal diet meal plan (breakfast, lunch, dinner)

[0420] 3. An optimal exercise program (e.g., jogging three times a week, walking for 30 minutes daily)

[0421] 4. Recommended period

[0422] Editing the generated menu:

[0423] Users can edit the generated diet meal plans and exercise programs as needed.

[0424] Emotional recognition and feedback:

[0425] Using an emotion recognition engine (Microsoft Azure Cognitive Services), the system analyzes the user's facial expressions and tone of voice to estimate their emotional state. If the user shows signs of pain during exercise, the system captures this information and adjusts the content and tone of the voice support. Specifically, it may provide encouraging messages or automatically reduce the intensity of the exercise.

[0426] Specific example

[0427] The user launches the app and takes a picture of their body shape with their smartphone camera. The app analyzes the image and obtains the user's body shape data. The user inputs their exercise habits (e.g., jogging 3 times a week). An emotion recognition engine analyzes the user's facial expressions and tone of voice. A generative AI model generates an optimal meal plan and exercise program based on the prompt text. The app displays the generated menu to the user, who can edit it as needed. If the user shows signs of pain during exercise, the emotion engine detects this and provides feedback (e.g., displays an encouraging message, reduces the exercise intensity).

[0428] In this way, users receive individually customized diet plans and support tailored to their emotional state.

[0429] The flow of a specific process in Application Example 1 will be explained using Figure 16.

[0430] Step 1:

[0431] The user takes a picture of their own body shape using their smartphone camera.

[0432] Input: User's body shape image

[0433] Output: Body shape image data

[0434] Specific operation: The user activates the smartphone camera and takes a full-body photo. The captured image is saved within the application.

[0435] Step 2:

[0436] The device uses an image processing library (OpenCV) to analyze the body shape image.

[0437] Input: Body shape image data

[0438] Output: Body shape analysis data

[0439] Specific operation: The device uses OpenCV to analyze the image and extract the user's body shape characteristics (e.g., height, weight, body fat percentage, etc.).

[0440] Step 3:

[0441] Users input their exercise habits as text data.

[0442] Input: Exercise habit data (e.g., jogging 3 times a week)

[0443] Output: Exercise habit text data

[0444] Specific action: The user enters their exercise habits into the application's input form and presses the submit button.

[0445] Step 4:

[0446] The server uses a generation AI model (GPT-4) to select images that closely resemble the desired body shape.

[0447] Input: Body shape analysis data, exercise habit text data

[0448] Output: Image data that closely resembles the desired physique.

[0449] Specific operation: The server inputs body shape analysis data and exercise habit text data into a generation AI model based on prompt messages, and generates an image that closely resembles the desired body shape.

[0450] Step 5:

[0451] The server uses an AI model to generate meal plans and exercise programs for weight loss.

[0452] Input: Body shape analysis data, exercise habit text data, image data that closely resembles the desired physique.

[0453] Output: Diet meal plan, exercise program, recommended duration

[0454] Specific operation: The server generates the following prompt messages and inputs them into the AI ​​model to generate an optimal meal plan and exercise program.

[0455] Based on the user's body shape image and exercise habits, please provide the following information:

[0456] 1. Body shape image that most closely matches the user's desired physique.

[0457] 2. Optimal diet meal plan (breakfast, lunch, dinner)

[0458] 3. An optimal exercise program (e.g., jogging three times a week, walking for 30 minutes daily)

[0459] 4. Recommended period

[0460] Step 6:

[0461] The user reviews the generated menu and edits it as needed.

[0462] Input: Diet meal plan, exercise program, recommended duration

[0463] Output: Edited menu

[0464] Specific actions: The user reviews the menu generated within the application and edits meal plans and exercise programs as needed.

[0465] Step 7:

[0466] The device uses an emotion recognition engine (Microsoft Azure Cognitive Services) to recognize the user's emotions.

[0467] Input: User facial image, voice tone data

[0468] Output: Emotional state data

[0469] Specific operation: The device uses its camera and microphone to capture the user's facial expressions and voice tone, which are then analyzed by an emotion recognition engine.

[0470] Step 8:

[0471] The server provides feedback based on the user's emotional state.

[0472] Input: Emotional state data

[0473] Output: Feedback message, exercise intensity adjustment

[0474] Specific operation: The server generates encouraging messages for the user based on emotional state data and automatically reduces the exercise intensity as needed.

[0475] (Example 2)

[0476] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0477] Conventional exercise support systems have struggled to provide personalized exercise programs that fully consider users' exercise habits and emotional states. Furthermore, the lack of real-time voice support during exercise and the absence of emotionally-based feedback made it difficult for users to maintain their motivation to exercise consistently.

[0478] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0479] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, means for registering the user's exercise habits, means for generating a diet menu, exercises, and duration, means for editing the generated menu, means for collecting the user's exercise data, means for analyzing the collected exercise data, means for collecting the user's emotional data, means for analyzing the collected emotional data and adjusting the exercise program, and means for giving voice instructions for the exercise menu. This makes it possible to provide an exercise program based on the user's individual exercise habits and emotional state, and helps the user maintain motivation to exercise continuously.

[0480] "Means of registering a user's body shape using images" refers to devices or software that record the shape and size of a user's body as image data.

[0481] "Generative AI" refers to a system that uses artificial intelligence technology to generate data and information tailored to specific purposes.

[0482] "Means for selecting images that closely resemble the desired physique" refers to interfaces and algorithms that allow users to select images that closely match their target body image.

[0483] "Means for registering a user's exercise habits" refers to devices or software used to record a user's daily exercise patterns and frequency.

[0484] "Means for generating diet meal plans, exercises, and timelines" refers to a system that automatically creates meal plans and exercise programs tailored to the user's goals.

[0485] "Means for editing generated menus" refers to an interface that allows users to manually modify and adjust automatically generated meal plans and exercise programs.

[0486] "Means for collecting user exercise data" refers to devices or software used to record data such as the type, number of repetitions, and intensity of exercise performed by the user.

[0487] "Means for analyzing collected exercise data" refers to algorithms and systems used to analyze collected exercise data and evaluate the user's exercise performance and progress.

[0488] "Means of collecting user emotional data" refers to devices and software that record emotional states from users' facial expressions and voice.

[0489] "Means for analyzing collected emotional data and adjusting exercise programs" refers to algorithms and systems that analyze collected emotional data and optimize exercise programs according to the user's emotional state.

[0490] "Means of providing voice instructions for exercise menus" refers to devices or software that provide voice guidance to users on the next exercise menu they should perform.

[0491] This invention is a system that provides voice support to users while they exercise and further adjusts the exercise program based on the user's emotional state. Specific embodiments of this system are described below.

[0492] First, the user launches an exercise app using their smartphone or wearable device. The smartphone's camera or the wearable device's sensors are used to register the user's body shape as images. This records the user's body shape and size as image data.

[0493] Next, the user selects an image that closely resembles the body they aspire to achieve using a generative AI. The generative AI model generates images that closely match the user's desired body image, allowing the user to choose from among them.

[0494] To register users' exercise habits, applications on smartphones or wearable devices are used. This records the user's daily exercise patterns and frequency.

[0495] To generate diet meal plans, exercise routines, and timelines, the server uses data analysis tools such as Python and Tensorflow®. This automatically creates meal plans and exercise programs tailored to the user's goals.

[0496] To edit the generated menus, users use a smartphone application. This allows users to manually modify and adjust the automatically generated meal plans and exercise programs.

[0497] Sensors from smartphones and wearable devices are used to collect user exercise data. For example, accelerometers and heart rate monitors are used to record data such as the type, frequency, and intensity of exercise.

[0498] To analyze the collected exercise data, the server uses Python and TensorFlow. This allows the collected exercise data to be analyzed and the user's exercise performance and progress to be evaluated.

[0499] The smartphone's camera and microphone are used to collect user emotion data. This records the user's emotional state from their facial expressions and voice.

[0500] The server uses an emotion engine to analyze collected emotional data and adjust the exercise program. This allows the collected emotional data to be analyzed, and the exercise program to be optimized according to the user's emotional state.

[0501] To provide voice instructions for exercise routines, the device uses the Google® Text-to-Speech API. This allows the user to receive voice guidance on the next exercise they should perform.

[0502] Specific example:

[0503] The user launches a strength training app on their smartphone. The app gives a voice command saying, "Next, do 20 sit-ups." After the user performs the sit-ups, the device uses its accelerometer to record the number of repetitions. The server analyzes this data and determines the next exercise to do, "Next, do 15 squats," and sends this command to the device. The device reads this command aloud. If the user smiles after the exercise, the device uses its camera to record the expression and sends it to the server. The server analyzes this data and incorporates sit-ups more frequently into future exercise programs.

[0504] Example of a prompt:

[0505] "Please create a program that provides voice instructions for the next exercise a user should do during their strength training. For example, it should say, 'Next, do 20 sit-ups.' Also, analyze the user's emotional state and, if they show a joyful expression, frequently incorporate that exercise into future programs."

[0506] The flow of the specific processing in Example 2 will be explained using Figure 17.

[0507] Step 1:

[0508] The user starts exercising.

[0509] Input: The user launches an exercise app using a smartphone or wearable device.

[0510] Specific action: The user taps the app on their smartphone and selects "Strength Training".

[0511] Output: A signal to initiate movement is sent to the terminal.

[0512] Step 2:

[0513] The device provides voice instructions for the exercise routine.

[0514] Input: The signal to start exercise and the exercise menu sent from the server.

[0515] Specific action: The device uses the Google Text-to-Speech API to read aloud, "Next, do 20 sit-ups."

[0516] Output: Recognizes the exercise routine the user should perform next.

[0517] Step 3:

[0518] The user performs exercise.

[0519] Input: Voice commands from the device.

[0520] Specific action: The user performs 20 sit-ups.

[0521] Output: Performing exercise.

[0522] Step 4:

[0523] The device collects the user's exercise data.

[0524] Input: User's exercise performance.

[0525] Specific operation: The device uses an accelerometer and heart rate monitor to record the number of exercises and their intensity.

[0526] Output: Exercise data is saved to the device.

[0527] Step 5:

[0528] The server analyzes the exercise data.

[0529] Input: Exercise data transmitted from the device.

[0530] Specific operation: The server uses Python and TensorFlow to analyze exercise data and evaluate the user's exercise performance.

[0531] Output: Analysis results are generated.

[0532] Step 6:

[0533] The server determines the next exercise menu and sends it to the terminal.

[0534] Input: Analysis results of exercise data.

[0535] Specific operation: Based on the analysis results, the server determines the next exercise to perform and sends that information to the terminal. For example, it might decide, "Next, please do 15 squats."

[0536] Output: The next exercise menu will be sent to the terminal.

[0537] Step 7:

[0538] The device will provide voice instructions for the next exercise routine.

[0539] Input: The following exercise menu sent from the server.

[0540] Specific action: The device will again use the Google Text-to-Speech API to read aloud, "Now, do 15 squats."

[0541] Output: Recognizes the exercise routine the user should perform next.

[0542] Step 8:

[0543] The user finishes their exercise.

[0544] Input: User indicates their intention to end the exercise.

[0545] Specific action: The user taps the "Exit" button in the app.

[0546] Output: A signal indicating the end of the exercise is sent to the terminal.

[0547] Step 9:

[0548] The device collects user emotion data.

[0549] Input: Signal to end exercise.

[0550] Specific operation: The device uses its camera and microphone to collect emotional data from the user's facial expressions and voice.

[0551] Output: Emotional data is saved to the device.

[0552] Step 10:

[0553] The server analyzes emotional data and adjusts the exercise program accordingly.

[0554] Input: Emotional data sent from the device.

[0555] Specific operation: The server uses an emotion engine to analyze the collected emotion data and optimize the exercise program according to the user's emotional state.

[0556] Output: A modified exercise program is generated.

[0557] (Application Example 2)

[0558] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0559] Conventional exercise and diet support systems only provide menus based on the user's exercise habits and body shape, but they lack consideration for the user's emotional state when suggesting meal plans and integration with delivery services for the suggested meals. Therefore, there is a challenge in that it is difficult for users to maintain the motivation to continue exercising and eating consistently.

[0560] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0561] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, means for registering the user's exercise habits, means for generating a diet meal plan, exercise routines, and duration, means for editing the generated menu, means for analyzing the user's emotional state, means for suggesting a meal plan based on the emotional state, and means for delivering the suggested meal plan in cooperation with a delivery service. This enables the suggestion of an optimal meal plan according to the user's emotional state and the rapid delivery of that menu.

[0562] "Means for registering a user's body shape using images" refers to devices or software for recording a user's body shape and dimensions as image data.

[0563] "A method for selecting images that closely resemble the desired body using generation AI" refers to a device or software that uses artificial intelligence technology to select images of the body that the user is aiming for.

[0564] "Means for registering a user's exercise habits" refers to devices or software used to record the type and frequency of a user's daily exercise.

[0565] "Means for generating diet meal plans, exercises, and durations" refers to devices or software that automatically create appropriate meal plans, exercise programs, and durations based on the user's goals and current condition.

[0566] "Means for editing generated menus" refers to devices or software that allow users to manually modify and adjust generated meal plans and exercise programs.

[0567] "Means for analyzing a user's emotional state" refers to devices or software that analyze a user's facial expressions and feedback to determine their emotional state.

[0568] "Means for suggesting meal menus based on emotional state" refers to devices or software that suggest the optimal meal menu considering the user's emotional state.

[0569] "Methods for delivering suggested meal menus in conjunction with delivery services" refers to devices or software that deliver suggested meal menus to users in conjunction with delivery services.

[0570] A system for carrying out this invention includes means for registering the user's body shape with an image, means for selecting an image that closely resembles the desired body shape using a generating AI, means for registering the user's exercise habits, means for generating a diet meal menu, exercises, and duration, means for editing the generated menu, means for analyzing the user's emotional state, means for suggesting a meal menu based on the emotional state, and means for delivering the suggested meal menu in cooperation with a delivery service.

[0571] Hardware and software to be used

[0572] hardware

[0573] smartphone

[0574] server

[0575] Delivery service terminal

[0576] software

[0577] Python

[0578] Speech recognition libraries (e.g., Google Speech-to-Text)

[0579] Sentiment analysis libraries (e.g., IBM Watson®)

[0580] Generative AI models (e.g., GPT-3(registered trademark))

[0581] Data processing and data calculation

[0582] A method for registering a user's body shape using an image.

[0583] The user uses their smartphone camera to take a picture of their body shape and sends the image data to the server. The server stores this image data and uses it for later analysis.

[0584] A method of selecting images that closely resemble the desired body shape using generation AI.

[0585] The server uses a generative AI model to generate images of the body the user desires. The user then selects their target body from the generated images through a smartphone application.

[0586] A means of registering a user's exercise habits.

[0587] Users use a smartphone application to input the type and frequency of their daily exercise. This data is sent to and stored on a server.

[0588] Means for generating diet meal plans, exercise routines, and timeframes.

[0589] The server generates appropriate meal plans, exercise programs, and durations based on the user's body shape data, target body image, and exercise habit data. These plans are automatically created using a generation AI model.

[0590] Means for editing the generated menu

[0591] Users can manually modify and adjust the generated meal plans and exercise programs using a smartphone application. The edited data is stored on the server.

[0592] A means of analyzing a user's emotional state

[0593] The system collects user feedback after exercise or meals using the smartphone's camera and microphone. The server uses an emotion analysis library to analyze this feedback and determine the user's emotional state.

[0594] A method for suggesting meal menus based on emotional state

[0595] Based on the results of the emotion analysis, the server suggests the optimal meal menu tailored to the user's emotional state. It uses a generative AI model to automatically generate the suggested menu.

[0596] A method of delivering the suggested meal menu in cooperation with a delivery service.

[0597] The server sends the suggested meal menu to the delivery service terminal and instructs it to deliver it to the user. The delivery service then promptly delivers the suggested menu.

[0598] Examples of specific cases and prompt statements

[0599] Specific example

[0600] When a user is running, the system will notify them by voice if their current pace is not enough to reach their target time and will suggest increasing their pace.

[0601] If a user gives feedback that they "had fun" after exercising, the emotion engine will determine that they are "happy" and suggest a salad and grilled chicken.

[0602] Example of a prompt

[0603] "When a user is running, analyze their current pace and suggest increasing their pace if they are not reaching their target time."

[0604] "Analyze user feedback after exercise and suggest appropriate meal plans based on their emotional state."

[0605] The above describes the embodiments for carrying out this invention.

[0606] The flow of a specific process in Application Example 2 will be explained using Figure 18.

[0607] Step 1:

[0608] The user uses their smartphone camera to take a picture of their body shape and sends the image data to the server. The server stores this image data and uses it for later analysis. The input is the user's body shape image, and the output is the image data stored on the server.

[0609] Step 2:

[0610] The server uses a generative AI model to generate images of the body the user desires. The user selects their target body from the generated images via a smartphone application. The input is the user's body shape image and the characteristics of the target body, and the output is the image of the target body selected by the user.

[0611] Step 3:

[0612] Users input the type and frequency of their daily exercise using a smartphone application. This data is sent to and stored on a server. The input is the user's exercise habit data, and the output is the exercise habit data stored on the server.

[0613] Step 4:

[0614] The server generates appropriate meal plans, exercise programs, and durations based on the user's body shape data, target body image, and exercise habit data. These plans are automatically created using a generation AI model. The input is the user's body shape data, target body image, and exercise habit data, and the output is the generated meal plan and exercise program.

[0615] Step 5:

[0616] Users can manually modify and adjust the generated meal plans and exercise programs using a smartphone application. The edited data is saved on the server. The input is the user's edits, and the output is the edited menu saved on the server.

[0617] Step 6:

[0618] The system collects user feedback after exercise or meals using the smartphone's camera and microphone. A server uses an emotion analysis library to analyze this feedback and determine the user's emotional state. The input is the user's feedback data, and the output is the analyzed emotional state.

[0619] Step 7:

[0620] The server suggests the optimal meal menu based on the user's emotional state, using the results of the emotion analysis. A generative AI model is used to automatically generate the suggestions. The input is the analyzed emotional state, and the output is the suggested meal menu.

[0621] Step 8:

[0622] The server sends the suggested meal menu to the delivery service terminal and instructs it to deliver it to the user. The delivery service quickly delivers the suggested menu. The input is the suggested meal menu, and the output is the meal delivered to the user.

[0623] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0624] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0625] Other examples of generative AI include Gemini® (registered trademark) (Internet search). <url: https: gemini.google.com ?hl="ja">) are some examples.

[0626] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0627] [Second Embodiment]

[0628] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0629] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0630] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0631] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0632] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0633] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0634] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0635] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0636] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0637] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0638] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0639] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0640] "Example of form 1"

[0641] The system of this invention allows the user to take a picture of their body shape using a digital camera or smartphone camera and register the image in the system. Next, the system selects the image that best matches the user's desired body shape from a set of body shape images provided within the system. Furthermore, the user registers their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day) in the system. Based on this information, the system uses a generative AI to generate an optimal diet plan, exercise program, and duration for the user. The generated menu can be edited by the user as needed.

[0642] "Example of form 2"

[0643] The system of this invention provides voice support when a user is exercising. For example, when performing strength training, it reads out the next exercise to be done (e.g., "Next, please do 20 sit-ups"). When running, it checks the user's pace and suggests increasing or decreasing the pace as needed (e.g., "At your current pace, you may not reach your target time. Try increasing your pace a little"). This enables the user to continue exercising appropriately.

[0644] The following describes the processing flow for each example of the form.

[0645] "Example of form 1"

[0646] Step 1: The user takes a picture of their body shape using a digital camera or smartphone camera. The captured image is uploaded to the system.

[0647] Step 2: From the multiple body shape images provided within the system, the user selects the one that best matches their desired body shape. This selection is made through the system's user interface.

[0648] Step 3: The user registers their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day) in the system. This registration is also done through the system's user interface.

[0649] Step 4: Based on this information, the system uses AI to generate an optimal diet plan, exercise program, and duration for the user. The generated plan can be edited by the user as needed.

[0650] "Example of form 2"

[0651] Step 1: When the user starts exercising, the system provides voice support.

[0652] Step 2: When performing strength training, the system will announce the next exercise to do (e.g., "Next, do 20 sit-ups").

[0653] Step 3: When running, the system monitors the user's pace and suggests increasing or decreasing the pace as needed (e.g., "At your current pace, you may not reach your target time. Try increasing your pace a little").

[0654] (Example 1)

[0655] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0656] Traditional diet systems have struggled to generate personalized diet plans based on users' body types and exercise habits. Furthermore, the lack of editing capabilities for the generated plans and insufficient support during exercise made it difficult for users to consistently follow their diets.

[0657] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0658] In this invention, the server includes means for registering the user's body shape as an image, means for presenting multiple body shape images and allowing the user to select their desired body shape, means for registering the user's exercise habits, means for generating a diet meal plan, exercise program, and duration using an AI model based on the user's body shape image, desired body shape, and exercise habit information, and means for editing the generated menu. This makes it possible to generate and edit a diet plan tailored to the user's individual needs.

[0659] "Method for registering user body shape with images" refers to a function that allows users to take pictures of their own body shape using a digital camera or smartphone camera and upload those images to the system.

[0660] "A means of presenting multiple body shape images and allowing the user to select the body shape they aspire to" refers to a function that presents the user with multiple body shape images prepared within the system and allows the user to select the one that most closely matches their desired body shape.

[0661] "Means for registering users' exercise habits" refers to a function that allows users to input their own exercise habits (e.g., jogging three times a week, walking for 30 minutes every day, etc.) into the system and register that information.

[0662] "A means of generating diet meal plans, exercise programs, and their duration using a generative AI model" refers to a function that uses a generative AI model to generate optimal diet meal plans, exercise programs, and their duration based on the user's body shape image, desired physique, and exercise habits.

[0663] "Means for editing generated menus" refers to a function that allows users to review the generated diet plan and edit meal menus and exercise programs as needed.

[0664] This invention begins with a user taking a picture of their own body shape using a digital camera or smartphone camera and registering that image in the system. The user launches the camera app on their smartphone and takes a picture of their body shape. The captured image is uploaded from the terminal to the system, and the server receives the uploaded image and saves it to a database. At this time, the image quality is checked using an image processing library (e.g., OpenCV).

[0665] Next, the server presents the user with several body shape images prepared within the system. The user selects the image that most closely matches their desired body shape from the presented images. The ID of the selected body shape image is sent from the terminal to the server, and the server stores that ID in its database.

[0666] Users input their exercise habits from their device. For example, they might enter information such as "I jog three times a week" or "I walk for 30 minutes every day." The entered exercise habit data is sent from the device to the server, which stores that data in a database.

[0667] The server obtains information on the user's body shape image, desired physique, and exercise habits, and prompts a generation AI model (e.g., GPT-4) to generate a diet plan. An example of a prompt is, "Based on the user's body shape image, desired physique, and exercise habits, please generate an optimal diet meal plan, exercise program, and duration." The generation AI model generates the optimal meal plan, exercise program, and duration based on the prompt. The generated diet plan is then saved to a database by the server.

[0668] Users can view the generated diet plan on their device and edit the meal menu and exercise program as needed. For example, they can change breakfast from oatmeal to yogurt. The edited information is sent from the device to the server, which then updates the database.

[0669] In this way, the system allows users to easily create a diet plan tailored to their own goals and edit it as needed.

[0670] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0671] Step 1: Register the user's body shape image.

[0672] The user launches the camera app on their smartphone and takes a picture of their own body shape.

[0673] Input: Body shape image taken by the user

[0674] The device displays the captured image on the system's upload screen, and the user presses the "Upload" button.

[0675] The server receives the uploaded image and uses an image processing library (e.g., OpenCV) to check the image quality.

[0676] Output: Quality-verified body shape image

[0677] The server saves verified images to the database.

[0678] Step 2: Choosing your desired body image

[0679] The server retrieves multiple body shape images from the database and displays them on the user's device.

[0680] Input: Multiple body shape images stored in a database

[0681] Users tap to select their desired body shape from the displayed images.

[0682] The device sends the ID of the selected body shape image to the server.

[0683] Output: ID of the body shape image selected by the user

[0684] The server saves the ID of the selected body shape image to the database.

[0685] Step 3: Register your exercise habit

[0686] The user enters their exercise habits into the input form on their device (e.g., "I jog three times a week").

[0687] Input: Exercise habit data entered by the user.

[0688] The device sends the entered exercise habit data to the server.

[0689] Output: Exercise habit data

[0690] The server stores data on exercise habits in a database.

[0691] Step 4: Generating an optimal diet plan

[0692] The server retrieves information from the database regarding the user's body shape image, desired physique, and exercise habits.

[0693] Input: User's body shape image, desired physique, and exercise habits.

[0694] The server will input the following prompt into the generated AI model (e.g., GPT-4):

[0695] "Based on the user's body shape image, desired physique, and exercise habits, please generate an optimal diet plan, exercise program, and duration."

[0696] The generative AI model generates optimal meal plans, exercise programs, and their durations based on the prompt text.

[0697] Output: Generated diet plan (meal menu, exercise program, duration)

[0698] The server saves the generated diet plan to the database.

[0699] Step 5: Edit the menu

[0700] Users can view the generated diet plan on their device.

[0701] Input: Generated diet plan

[0702] Users can edit their meal plans and exercise programs as needed (e.g., change oatmeal for breakfast to yogurt).

[0703] The terminal sends the edited content to the server.

[0704] Output: Edited diet plan

[0705] The server updates the database with the edited diet plan.

[0706] (Application Example 1)

[0707] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0708] Traditional diet and fitness programs often offer generic menus and exercise plans, making it difficult to provide programs optimized for individual users' body types and exercise habits. Furthermore, fitness facilities faced the challenge of placing a heavy burden on trainers to provide individually optimized training programs and meal plans. Additionally, the lack of real-time feedback made it difficult for users to train effectively.

[0709] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0710] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, and means for registering the user's exercise habits. This makes it possible to provide individually optimized training programs and meal plans.

[0711] "Method for registering a user's body shape using an image" refers to a method for users to take a picture of their own body shape using a digital camera or smartphone camera and register that image in the system.

[0712] "A method of selecting an image that closely resembles the desired body shape using generative AI" refers to a method of using generative AI to select the image that most closely matches the user's desired body shape from multiple body shape images provided within the system.

[0713] "Means for registering users' exercise habits" refers to the means by which users register their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day, etc.) in the system.

[0714] "Methods for generating diet meal plans, exercises, and durations" refers to methods that use a generating AI to create optimal diet meal plans, exercise programs, and their durations for the user, based on the user's body shape image and exercise habits.

[0715] "Means for editing generated menus" refers to means that users can edit the generated diet meal plans and exercise programs as needed.

[0716] "A means of providing individually optimized training programs and meal plans by having fitness facility members take photos of their own physique, select their desired body shape, and register their exercise habits" refers to a means of providing individually optimized training programs and meal plans by having fitness facility members take photos of their own physique, select their desired body shape, and register their exercise habits.

[0717] "A means of receiving real-time feedback in collaboration with trainers within the facility" refers to a method of receiving real-time feedback on training and diet in collaboration with trainers within the fitness facility.

[0718] A system for carrying out this invention includes means for registering the user's body shape with an image, means for selecting an image that closely resembles the desired body shape using a generating AI, means for registering the user's exercise habits, means for generating a diet meal plan, exercise routines, and duration, means for editing the generated menu, means for providing individually optimized training programs and meal plans by allowing fitness facility members to take a picture of their body shape, select their desired body shape, and register their exercise habits, and means for receiving real-time feedback in cooperation with trainers within the facility.

[0719] 1. System Program

[0720] The program in this system performs the following operations:

[0721] 2. Explanation of the program's processing

[0722] The server receives body shape images taken by the user using their smartphone camera and performs image processing. Specifically, it uses the Python PIL library to load the images and preprocess them as needed. Next, it uses a generative AI model to select the image that most closely resembles the user's desired body shape. This generative AI model uses a pre-trained Keras model.

[0723] The user's exercise habits are registered on the server in text format. This includes information such as how many times a week the user exercises and what kind of exercise they do. Based on this information, the generative AI model generates an optimal diet plan, exercise program, and duration for the user.

[0724] The generated menu can be viewed by the user via a smartphone app and edited as needed. Furthermore, users can collaborate with trainers at fitness facilities to receive real-time feedback. This feedback includes training progress and dietary advice.

[0725] 3. Specific Examples and Examples of Prompt Statements

[0726] As a concrete example, consider a user who jogs three times a week. Based on this information, the generative AI model generates an optimal meal plan and exercise program for the user. The following is an example of a prompt to input to the generative AI model.

[0727] User's body shape image: user_image.jpg

[0728] Target body shape: target_shape.jpg

[0729] Exercise habits: I jog three times a week.

[0730] Based on this information, please generate an optimal diet plan and exercise program.

[0731] By using this prompt message as input to the generative AI model, it is possible to generate a menu that is optimal for the user.

[0732] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0733] Step 1:

[0734] The user takes a picture of their body shape using their smartphone camera and saves the image to their device.

[0735] Input: User's body shape image

[0736] Output: Body shape image saved on the device

[0737] Specific actions: The user launches the camera app on their smartphone and takes a picture of their own body shape. The captured image is then saved to the device.

[0738] Step 2:

[0739] The device uploads the saved body shape image to the server.

[0740] Input: Body shape image saved on the device

[0741] Output: Body shape images uploaded to the server

[0742] Specific operation: The device uploads the saved body shape images to the server via an internet connection. HTTP requests are used for the upload.

[0743] Step 3:

[0744] The server receives the uploaded body shape image and performs image processing.

[0745] Input: Body shape image uploaded to the server

[0746] Output: Preprocessed body shape image

[0747] Specific operation: The server uses the Python PIL library to read images and performs preprocessing such as resizing and denoising as needed.

[0748] Step 4:

[0749] The server uses a generated AI model to select the image that most closely matches the user's desired body shape.

[0750] Input: Preprocessed body shape image

[0751] Output: The image that most closely matches the desired body shape.

[0752] Specific operation: The server uses a pre-trained Keras model to compare the user's body shape image with the image of the target body shape and selects the closest image.

[0753] Step 5:

[0754] The user enters their exercise habits into their device and sends them to the server.

[0755] Input: User's exercise habit information

[0756] Output: Exercise habit information sent to the server

[0757] Specific operation: The user inputs their exercise habits (e.g., jogging three times a week) through a smartphone app and sends that information to the server.

[0758] Step 6:

[0759] The server uses a generative AI model to generate optimal diet meal plans and exercise programs based on the user's body shape image and exercise habit information.

[0760] Input: User's body shape image, exercise habit information

[0761] Output: Optimal diet meal plans and exercise programs

[0762] Specific operation: The server inputs prompt messages into the generating AI model, which then generates an optimal diet meal plan and exercise program for the user. Examples of prompt messages are as follows:

[0763] User's body shape image: user_image.jpg

[0764] Target body shape: target_shape.jpg

[0765] Exercise habits: I jog three times a week.

[0766] Based on this information, please generate an optimal diet plan and exercise program.

[0767] Step 7:

[0768] The server sends the generated menu to the user's terminal, and the user edits it as needed.

[0769] Input: Optimal diet meal plans and exercise programs

[0770] Output: Menu sent to the user's terminal

[0771] Specific operation: The server sends the generated menu to the user's device, and the user views the menu through a smartphone app and edits it as needed.

[0772] Step 8:

[0773] Users train within a fitness facility and receive real-time feedback in collaboration with trainers.

[0774] Input: Training progress, dietary advice

[0775] Output: Real-time feedback

[0776] Specific operation: Users train within a fitness facility, and trainers monitor the user's progress and provide real-time feedback through a dedicated application.

[0777] (Example 2)

[0778] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0779] Conventional exercise support systems have struggled to collect and analyze user exercise data in real time and provide appropriate voice guidance. Furthermore, they lacked sufficient support in generating specific exercise and meal plans to help users achieve their desired physique, and in guiding users through the exercise process based on these plans. This made it difficult for users to effectively maintain a consistent exercise routine.

[0780] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body using a generation AI model, means for registering the user's exercise habits, means for generating a diet meal menu, exercises, and duration, means for editing the generated menu, means for collecting the user's exercise data, means for analyzing the collected exercise data, means for generating voice instructions based on the analysis results, means for transmitting the generated voice instructions to the user's terminal, and means for playing the voice instructions on the user's terminal. As a result, the user can effectively continue exercising to get closer to their desired body while receiving appropriate exercise instructions in real time.

[0781] "Method for registering a user's body shape with images" refers to a function that records a user's current body shape as image data using devices such as cameras or scanners.

[0782] "A method for selecting images that closely resemble the desired body shape using a generative AI model" refers to a function that utilizes a generative AI model to present images that closely resemble the user's ideal body shape, allowing the user to select from among them.

[0783] "Methods for registering users' exercise habits" refers to a function that records information such as the type, frequency, and duration of exercise that users perform on a daily basis in a database.

[0784] "Methods for generating diet meal plans, exercises, and durations" refers to a function that automatically creates appropriate meal plans, exercise plans, and durations based on the user's goals, current body shape, and exercise habits.

[0785] "Means for editing generated menus" refers to a function that allows users to manually modify and adjust automatically generated meal plans and exercise plans.

[0786] "Means of collecting user exercise data" refers to a function that uses wearable devices or smartphones to acquire exercise-related data such as the user's heart rate, speed, and distance in real time.

[0787] "Means for analyzing collected exercise data" refers to a function that evaluates and analyzes the user's exercise status and performance based on the collected exercise data.

[0788] "Means for generating voice instructions based on analysis results" refers to a function that generates instructions to provide users with appropriate exercise instructions and advice via voice, based on the results of data analysis.

[0789] "Means for sending generated voice commands to the user's device" refers to a function that sends generated voice commands to the user's smartphone or tablet via communication means such as the internet or Bluetooth.

[0790] "Means for playing voice commands on the user's device" refers to a function that plays voice commands received by the user's smartphone or tablet through a speaker or earphones.

[0791] This invention is a system that provides voice support to users while they exercise. The system registers the user's body shape as an image, uses a generative AI model to select an image that closely resembles the desired body shape, and registers the user's exercise habits. Furthermore, it includes a function to generate a diet plan, exercise routine, and duration, and to edit the generated plan.

[0792] The server collects and analyzes user exercise data in real time. The hardware used includes wearable devices (e.g., smartwatches and fitness trackers) for collecting user exercise data. The software includes algorithms for data analysis and generative AI models (e.g., OpenAI's GPT-4) for speech synthesis.

[0793] The device (the user's smartphone or tablet) relays voice instructions sent from the server to the user. Specifically, the device plays the voice data received from the server, providing the user with advice on the next exercise routine and pacing.

[0794] Users receive voice instructions from the device while exercising and continue exercising according to those instructions. For example, when doing strength training, they might receive the instruction, "Next, do 20 sit-ups," and then perform sit-ups. When running, if their current pace is not reaching their target time, they might receive advice such as, "Try to pick up the pace a little," and adjust their pace accordingly.

[0795] As a concrete example, consider a scenario where a user is running. The user puts on a smartwatch and starts running with their smartphone. The smartwatch collects data such as the user's heart rate and speed and sends it to a server. The server analyzes this data and detects that the user's current pace is not enough to reach their target time. The server uses a generative AI model to generate a voice command saying, "At your current pace, you may not reach your target time. Try to pick up the pace a little," and sends it to the device. The device plays this voice command and communicates it to the user.

[0796] Example of a prompt:

[0797] "When a user is running and their current pace is not reaching their target time, please generate voice prompts suggesting they increase their pace."

[0798] This system allows users to consistently engage in appropriate exercise.

[0799] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0800] Step 1:

[0801] Before starting exercise, the user puts on a wearable device (e.g., a smartwatch) and launches a dedicated app on their smartphone. The wearable device collects exercise data such as the user's heart rate, speed, and distance in real time. This data is transmitted to the user's smartphone via Bluetooth or Wi-Fi. The input is the user's exercise data, and the output is the exercise data transmitted to the smartphone.

[0802] Step 2:

[0803] The server receives exercise data transmitted from the user's smartphone. The server uses data analysis algorithms to evaluate the user's exercise status in real time. For example, it can determine whether the user's pace during a run is reaching their target time. The input is the exercise data transmitted from the smartphone, and the output is the analysis result.

[0804] Step 3:

[0805] The server uses a generative AI model (e.g., a generative AI model) to generate appropriate voice instructions for the user. For example, if the user is moving too slowly, it might generate a voice instruction such as, "At your current pace, you may not reach your target time. Try to pick up the pace a little." The input is the analysis result, and the output is the generated voice instruction.

[0806] Step 4:

[0807] The server sends the generated voice commands to the user's smartphone. An internet connection is required for transmission. The input is the generated voice command, and the output is the voice command sent to the smartphone.

[0808] Step 5:

[0809] The device (the user's smartphone) plays the voice instructions received from the server. The user can hear the voice instructions through the smartphone's speaker or earphones. The input is the voice instructions received from the server, and the output is the played voice instructions.

[0810] Step 6:

[0811] The user continues exercising according to the voice instructions from the device. For example, if instructed to "do 20 sit-ups next," the user will perform 20 sit-ups. Similarly, if instructed to "try to increase your pace," the user will increase their running pace. The input is the played voice instructions, and the output is the user's exercise behavior.

[0812] (Application Example 2)

[0813] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0814] While conventional exercise support systems can provide appropriate menus based on a user's exercise habits and physique, they have limitations in terms of efficient route guidance and time management during delivery. Furthermore, there was a lack of means to provide appropriate advice when delivery delays were anticipated, highlighting the need for increased efficiency in delivery operations.

[0815] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0816] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, means for registering the user's exercise habits, means for generating a diet meal menu, exercise routines, and duration, means for editing the generated menu, means for registering delivery destination information, means for generating a delivery route, means for managing delivery time, means for guiding the delivery destination and route by voice, and means for prompting the user to speed up if the delivery time is likely to be delayed. This enables efficient route guidance and time management in delivery operations, and allows for the provision of appropriate advice if the delivery time is likely to be delayed.

[0817] "Methods for registering a user's body shape using images" refers to methods for saving a user's body shape and size as image data.

[0818] "A method of selecting images that closely resemble the desired body using generation AI" refers to a method of using artificial intelligence to select images of the body that the user aims for.

[0819] "Methods for registering users' exercise habits" refer to methods for recording the types and frequency of exercises that users perform on a daily basis.

[0820] "Methods for generating diet meal plans, exercises, and durations" refers to methods for creating meal plans, exercise programs, and implementation periods tailored to the user's goals.

[0821] "Means for editing generated menus" refers to methods for modifying created meal plans and exercise programs according to the user's requests.

[0822] "Methods for registering delivery address information" refers to methods for recording information such as the address and contact details of the place where the delivery will be made.

[0823] "Methods for generating delivery routes" refer to methods for calculating the optimal route for efficiently visiting multiple delivery destinations.

[0824] "Means of managing delivery times" refers to methods for recording and managing the estimated arrival time and actual arrival time at each delivery destination.

[0825] "A method for guiding delivery destinations and routes via voice" refers to a method of instructing delivery personnel by voice to the next delivery destination or the optimal route.

[0826] "Methods to encourage speeding up when delivery is likely to be delayed" refers to methods of using voice commands to instruct delivery personnel to increase their speed when deliveries are behind schedule.

[0827] The system for implementing this invention has the function of registering the user's body shape with an image, selecting an image that closely resembles the desired body shape using a generation AI, and registering the user's exercise habits. It also includes the function of generating a diet meal menu, exercise plan, and duration, and editing the generated menu. Furthermore, it has the function of registering delivery destination information, generating a delivery route, and managing delivery times. It also includes a function to provide voice guidance for the delivery destination and route, and to encourage the user to speed up if the delivery time is likely to be delayed.

[0828] System program processing

[0829] The server acquires image data using a smartphone or camera to register the user's body shape and stores it in a database. Using a generative AI model, it generates prompts for the user to select an image that closely resembles their desired physique and presents them to the user. To register the user's exercise habits, it collects exercise data from a smartphone app or wearable device and stores it in a database.

[0830] To generate a diet plan including meal menus, exercise routines, and duration, the server creates an optimal plan based on the user's goals, current physique, and exercise habits. Users can edit the generated menu using a smartphone app.

[0831] To register delivery information, the server stores the delivery address and contact information in a database. To generate delivery routes, the server uses map data to calculate the optimal route. To manage delivery times, the server records and manages the estimated and actual arrival times for each delivery location.

[0832] To guide delivery drivers to their destinations and routes via voice, the server uses speech synthesis technology to issue instructions. If a delivery is likely to be delayed, the server will also instruct the driver to speed up via voice.

[0833] Specific example

[0834] For example, if a delivery person delivers to "1-1-1 Shibuya-ku, Tokyo", the server will operate as follows:

[0835] 1. Retrieve "1-1-1 Shibuya-ku, Tokyo" from the list of delivery addresses.

[0836] 2. The voice will announce, "Your next delivery address is 1-1-1 Shibuya-ku, Tokyo. We will guide you to the optimal route."

[0837] 3. If it appears that the delivery time will be delayed, a voice message will advise, "At the current pace, the delivery time will be delayed. Please increase your pace slightly."

[0838] Examples of prompts to input into a generative AI model:

[0839] "Your next delivery address is 1-1-1 Shibuya-ku, Tokyo. I will guide you to the optimal route. At your current pace, the delivery time will likely be delayed. Please pick up the pace a little."

[0840] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0841] Step 1:

[0842] The server registers the user's body shape as an image. The user takes a picture of their body shape using a smartphone or camera and sends the image data to the server. The server stores the received image data in a database. The input is the user's body shape image, and the output is the image data stored in the database.

[0843] Step 2:

[0844] The server generates prompts to select an image that closely resembles the user's desired physique using a generative AI model. When the user inputs the desired physical characteristics, the server uses the generative AI model to generate an image based on those characteristics and presents it to the user. The input is the user's desired physical characteristics, and the output is the generated image.

[0845] Step 3:

[0846] The server registers the user's exercise habits. Users input daily exercise data using a smartphone app or wearable device and send that data to the server. The server stores the received exercise data in a database. The input is the user's exercise data, and the output is the exercise data stored in the database.

[0847] Step 4:

[0848] The server generates meal plans, exercise routines, and timelines for weight loss. Based on the user's goals, current physique, and exercise habits, the server generates and presents an optimal meal plan and exercise program to the user. The input is the user's goals, physique, and exercise habits, and the output is the generated meal plan and exercise program.

[0849] Step 5:

[0850] The user edits the generated menu. Using a smartphone app, the user modifies the generated meal plan and exercise program and sends the modified data to the server. The server stores the received modified data in a database. The input is the user's modified data, and the output is the modified data stored in the database.

[0851] Step 6:

[0852] The server registers delivery address information. The user enters the delivery address and contact information and sends this information to the server. The server stores the received delivery address information in a database. The input is the delivery address and contact information, and the output is the delivery address information stored in the database.

[0853] Step 7:

[0854] The server generates delivery routes. Based on the delivery destination information, the server uses map data to calculate the optimal route and presents it to the delivery person. The input is the delivery destination information, and the output is the optimal delivery route.

[0855] Step 8:

[0856] The server manages delivery times. It records and manages estimated and actual arrival times for each delivery destination. Inputs are delivery destination information and current location information, and outputs are estimated and actual arrival times.

[0857] Step 9:

[0858] The server provides voice guidance for delivery locations and routes. Using speech synthesis technology, it instructs delivery personnel verbally on the next delivery location and the optimal route. The input is delivery route information, and the output is voice guidance.

[0859] Step 10:

[0860] The server prompts the delivery person to speed up if the delivery is likely to be delayed. If the delivery is behind schedule, the server will verbally instruct the delivery person to increase their speed. The input is the current location and estimated arrival time, and the output is a verbal instruction to speed up.

[0861] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0862] "Example of form 1"

[0863] In one embodiment of the present invention, an emotion engine that recognizes the user's emotions is incorporated into the system. This emotion engine estimates emotions from the user's facial expressions, tone of voice, and movements during exercise. For example, if the user shows a pained expression during exercise, the emotion engine captures this information and determines that the user is experiencing excessive stress. This information is fed back to the system, and the content and tone of the voice support are adjusted accordingly. Specifically, the user may be provided with encouraging messages, or the intensity of the exercise may be automatically reduced.

[0864] "Example of form 2"

[0865] Furthermore, the emotion engine adjusts diet meal plans and exercise programs according to the user's emotional state. For example, if a user shows a joyful expression after exercise, the emotion engine picks up on this information and determines that the user likes that exercise. This information is fed back into the system, and that exercise is frequently incorporated into recommended exercise programs. Similarly, if a user shows a satisfied expression after a meal, the system determines that the meal plan is favorable to the user and recommends similar meal plans.

[0866] The following describes the processing flow for each example of the form.

[0867] "Example of form 1"

[0868] Step 1: The user begins exercising.

[0869] Step 2: The emotion engine estimates the user's emotions from their facial expressions, tone of voice, and movements during exercise.

[0870] Step 3: If the emotion engine determines that the user is experiencing excessive stress, it feeds that information back into the system.

[0871] Step 4: The system adjusts the content and tone of the voice support. Specifically, it may provide encouraging messages to the user or automatically lower the intensity of the exercise.

[0872] "Example of form 2"

[0873] Step 1: The user finishes exercising.

[0874] Step 2: The emotion engine estimates the user's emotions from their facial expressions.

[0875] Step 3: If the emotion engine determines that the user likes the exercise, it feeds that information back into the system.

[0876] Step 4: Adjust the system so that the exercise is frequently incorporated into the recommended exercise program.

[0877] Step 5: The user finishes their meal.

[0878] Step 6: The emotion engine estimates the user's emotions from their facial expressions.

[0879] Step 7: If the emotion engine determines that the meal menu is favorable to the user, it feeds that information back into the system.

[0880] Step 8: Adjust the system so that it recommends similar meal menus.

[0881] (Example 1)

[0882] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0883] Conventional diet support systems have faced challenges in providing effective diet support because they struggle to offer personalized menus based on the user's body shape and exercise habits, and they lack feedback that takes into account the user's emotions and stress levels. Furthermore, they cannot detect the stress and fatigue the user experiences during exercise in real time and provide support accordingly, making it difficult to maintain the user's motivation.

[0884] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0885] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, means for registering the user's exercise habits, means for generating a diet menu, exercises, and duration, means for editing the generated menu, means including an emotion engine that recognizes the user's emotions, means for the emotion engine to analyze the user's facial expressions, tone of voice, and movements during exercise to estimate emotions, means for adjusting the content and tone of voice support based on emotions, and means for automatically adjusting the intensity of exercise. This makes it possible not only to provide a personalized menu based on the user's body shape and exercise habits, but also to detect the user's emotions and stress levels in real time and provide support accordingly.

[0886] "Method for registering user body shape with images" refers to a function that allows users to take pictures of their own body shape using a digital camera or smartphone camera and upload those images to the system.

[0887] "The method of selecting an image that closely resembles the desired body shape using generative AI" refers to a function that uses generative AI to select the image that most closely matches the user's desired body shape from multiple body shape images provided within the system.

[0888] "Means for registering users' exercise habits" refers to a function that allows users to input their own exercise habits (e.g., jogging three times a week, walking for 30 minutes every day, etc.) into the system and save that information.

[0889] "Method for generating diet meal plans, exercises, and durations" refers to a function that uses a generative AI model to create optimal diet meal plans, exercise programs, and their durations based on the user's body shape image, desired physique, and exercise habits.

[0890] "Means for editing generated menus" refers to a function that allows users to review generated diet meal plans and exercise programs and make changes or additions as needed.

[0891] "Means including an emotion engine that recognizes user emotions" refers to a function that incorporates an emotion engine into the system to estimate emotions from the user's facial expressions, tone of voice, movements during exercise, etc.

[0892] "The emotion engine's method of analyzing the user's facial expressions, tone of voice, and movements during exercise to estimate emotions" refers to the function of the emotion engine that analyzes data collected through the camera and microphone to estimate the user's emotional state.

[0893] "Means of adjusting the content and tone of voice support based on emotions" refers to a function that changes the content and tone of voice support according to the user's emotional state estimated by the emotion engine, and provides encouraging messages to the user.

[0894] "Means for automatically adjusting exercise intensity" refers to a function in which the emotional engine appropriately changes the exercise intensity based on the user's emotional state, thereby reducing the user's stress and fatigue.

[0895] Modes for carrying out the invention

[0896] This invention begins with the user taking a picture of their body shape using a digital camera or smartphone camera and registering the image in the system. Next, the user selects the image that best matches their desired body shape from several body shape images provided in the system. Furthermore, the user registers their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day) in the system. Based on this information, the system uses a generative AI model to generate an optimal diet plan, exercise program, and duration for the user. The generated plan can be edited by the user as needed.

[0897] In one embodiment of the present invention, an emotion engine that recognizes the user's emotions is incorporated into the system. This emotion engine estimates emotions from the user's facial expressions, tone of voice, and movements during exercise. For example, if the user shows a pained expression during exercise, the emotion engine captures this information and determines that the user is experiencing excessive stress. This information is fed back to the system, and the content and tone of the voice support are adjusted accordingly. Specifically, the user may be provided with encouraging messages, or the intensity of the exercise may be automatically reduced.

[0898] Hardware and software to be used

[0899] This system uses the following hardware and software.

[0900] Digital camera or smartphone camera: Used to photograph the user's body shape.

[0901] Devices (PCs, smartphones, tablets, etc.): Used by users to access the system and register images and exercise habits.

[0902] Server: Stores data sent by users and generates diet menus and exercise programs using a generation AI model.

[0903] Generative AI Model: Based on the user's body shape image, desired physique, and exercise habits, it generates an optimal diet plan and exercise program.

[0904] Emotion Engine: Analyzes the user's facial expressions, voice tone, and movements during exercise to estimate their emotions.

[0905] Specific example

[0906] Example 1: Registration of body shape images

[0907] Users take photos of their body shape with their smartphone cameras and upload the images to the system. The server receives the images and saves them to the user's profile.

[0908] Specific example 2: Choosing your desired body image

[0909] The user selects their desired body shape from body shape images within the system and sends that information to the server. The server saves the selected image as the user's target.

[0910] Example 3: Registering an exercise habit

[0911] The user enters their exercise habits, such as "I jog three times a week," into the system and sends it to the server. The server then adds this information to the user's profile.

[0912] Example 4: Creating a diet menu

[0913] The server uses an AI model to generate a "one-week diet plan" based on the user's body shape image, desired physique, and exercise habits. The generated plan is sent to the user's device, and the user can edit it as needed.

[0914] Example 5: Feedback from an emotional engine

[0915] If a user shows signs of pain during exercise, the emotion engine detects this information and sends it back to the server. The server generates an encouraging message and sends it to the user's device. Additionally, the exercise intensity is automatically reduced.

[0916] Example of a prompt

[0917] "Upload a photo of your body shape and select your desired physique. Next, input your exercise habits, and we'll generate an optimal diet plan. We'll also recognize your emotions during exercise and provide feedback."

[0918] The flow of the specific processing in Example 1 will be explained using Figure 15.

[0919] Step 1:

[0920] Users take photos of their own body shape using a digital camera or smartphone camera.

[0921] Input: User's body shape image

[0922] Output: Captured body shape image file

[0923] Specific actions: The user launches the camera app and takes a photo from the front, capturing their entire body. After taking the photo, the image file is saved.

[0924] Step 2:

[0925] The user logs into the system from their device and uploads the images they have taken.

[0926] Input: Captured body shape image file

[0927] Output: Body shape image saved on the server

[0928] Specific steps: The user accesses the system's login screen, enters their user ID and password, and logs in. After logging in, they move to the image upload screen, select the saved image file, and click the upload button.

[0929] Step 3:

[0930] The server saves the received images and associates them with the user's profile.

[0931] Input: Uploaded body shape image file

[0932] Output: Body shape image associated with the user's profile

[0933] Specific operation: The server receives the uploaded image file and saves it to the database. The saved image is linked to the user ID.

[0934] Step 4:

[0935] The user views multiple body shape images provided within the system.

[0936] Input: Body shape image data within the system

[0937] Output: List of body shape images viewed by the user

[0938] Specific operation: The user accesses a body shape image selection screen and views body shape images displayed in a slideshow format.

[0939] Step 5:

[0940] Select the image that best matches the user's desired physique.

[0941] Input: User selects body shape image

[0942] Output: Selected body shape image data

[0943] Specific action: The user selects the image that best represents their desired body shape and clicks the select button.

[0944] Step 6:

[0945] The server saves the selected image as the user's target.

[0946] Input: Selected body shape image data

[0947] Output: Target body shape image saved in the user's profile

[0948] Specific operation: The server receives the selected image data and saves it to the user's profile as a target body shape image.

[0949] Step 7:

[0950] Users input their exercise habits into the system via their device.

[0951] Input: User's exercise habits (e.g., jogs 3 times a week)

[0952] Output: Input exercise habit data

[0953] Specific actions: The user accesses the exercise habit input screen and enters their exercise habits into the text box. After entering the information, they click the save button.

[0954] Step 8:

[0955] The server receives the entered information about exercise habits and adds it to the user's profile.

[0956] Input: Entered exercise habit data

[0957] Output: Exercise habit information added to the user's profile

[0958] Specific operation: The server receives the entered exercise habit data and saves it to the database. The saved data is linked to the user ID.

[0959] Step 9:

[0960] Based on the user's body shape image, desired physique, and exercise habits, the server uses a generative AI model to generate an optimal diet plan, exercise program, and duration.

[0961] Input: User's body shape image, desired body shape, exercise habit information

[0962] Output: Generated diet meal plans and exercise programs

[0963] Specific operation: The server runs the generated AI model and creates a menu based on the input data, taking into account calorie calculations and nutritional balance.

[0964] Step 10:

[0965] The server sends the generated menu to the user's terminal.

[0966] Input: Generated diet meal plans and exercise programs

[0967] Output: Menu sent to the user's terminal

[0968] Specific operation: The server sends the generated menu to the user's terminal in PDF format or similar.

[0969] Step 11:

[0970] Review the menu received by the user and edit it as needed.

[0971] Input: Generated diet meal plans and exercise programs

[0972] Output: Edited menu

[0973] Specific actions: The user accesses the menu editing screen and makes changes to ingredients or adds exercises. After editing, they click the save button.

[0974] Step 12:

[0975] The user uses the system while exercising.

[0976] Input: User's exercise data (e.g., facial expressions, voice tone, movements during exercise)

[0977] Output: Real-time collected motion data

[0978] Specific operation: The user exercises while holding a smartphone, and data is collected through the camera and microphone.

[0979] Step 13:

[0980] The emotion engine analyzes the user's facial expressions, voice tone, and movements during exercise in real time.

[0981] Input: Real-time collected exercise data

[0982] Output: Estimated user emotional state

[0983] Specific operation: The emotion engine analyzes the collected data and estimates the user's emotional state.

[0984] Step 14:

[0985] The emotion engine estimates the user's emotions, and if it determines that the user is experiencing excessive stress, it feeds that information back to the server.

[0986] Input: Estimated user's emotional state

[0987] Output: Sentiment information fed back to the server

[0988] Specific operation: The emotion engine determines the user's emotional state and, if it detects excessive stress, sends that information to the server.

[0989] Step 15:

[0990] The server adjusts the content and tone of the voice support based on feedback and generates encouraging messages for the user.

[0991] Input: Feedback on emotional information

[0992] Output: Generated encouraging message

[0993] Specific operation: The server modifies the content and tone of the voice support based on emotional information, generating messages such as, "Keep going! You're almost there!"

[0994] Step 16:

[0995] The server automatically adjusts the exercise intensity and sends the information to the user's device.

[0996] Input: Feedback on emotional information

[0997] Output: Instructions for adjusted exercise intensity

[0998] Specific operation: The server adjusts the exercise intensity appropriately based on emotional information and sends instructions to the user's device. For example, it might instruct the user to slightly slow down their exercise pace.

[0999] (Application Example 1)

[1000] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1001] Traditional diet support systems can provide personalized menus based on a user's body shape and exercise habits, but they have the drawback of not being able to provide feedback that takes into account the user's emotional state. This has led to problems such as users experiencing excessive stress or being unable to maintain motivation.

[1002] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1003] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, means for registering the user's exercise habits, means for generating a diet menu, exercises, and duration, means for editing the generated menu, means for recognizing the user's emotions, and means for providing feedback based on the user's emotional state. This makes it possible not only to provide an optimal menu based on the user's individual body shape and exercise habits, but also to provide feedback according to the emotional state.

[1004] "Method for registering a user's body shape using an image" refers to a method for users to take a picture of their own body shape using a digital camera or smartphone camera and register that image in the system.

[1005] "A method for selecting an image that closely resembles the desired body shape using a generating AI" refers to a method for selecting the image that most closely matches the user's desired body shape from multiple body shape images provided within the system, using a generating AI.

[1006] "Means for registering users' exercise habits" refers to the means by which users register their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day, etc.) in the system.

[1007] "Methods for generating diet meal plans, exercises, and durations" refers to methods for generating optimal diet meal plans, exercise programs, and their durations for a user, using a generating AI based on the user's body shape image and exercise habits.

[1008] "Means for editing generated menus" refers to means that users can edit the generated diet meal plans and exercise programs as needed.

[1009] "Means of recognizing user emotions" refers to methods for estimating emotions from the user's facial expressions, tone of voice, and movements during exercise.

[1010] "Means of providing feedback based on the user's emotional state" refers to methods for adjusting the content and tone of voice support according to the user's emotional state, providing encouraging messages to the user, or automatically reducing the intensity of exercise.

[1011] A system for carrying out this invention includes means for registering the user's body shape with images, means for selecting images that closely resemble the desired body shape using a generating AI, means for registering the user's exercise habits, means for generating a diet menu, exercises, and duration, means for editing the generated menu, means for recognizing the user's emotions, and means for providing feedback based on the user's emotional state.

[1012] Hardware and software configuration

[1013] Hardware:

[1014] Smartphone (camera, microphone, sensors)

[1015] Server (data processing, execution of generational AI models)

[1016] software:

[1017] Image processing library (OpenCV)

[1018] Emotion recognition engine (Microsoft Azure Cognitive Services)

[1019] Generative AI model (GPT-4)

[1020] Database (MySQL)

[1021] Data processing and data calculation

[1022] User body shape image registration:

[1023] Users take photos of their body shape using their smartphone cameras and upload the images to the system. The system then uses an image processing library (OpenCV) to analyze body shape data from the images.

[1024] Select an image that closely resembles the body you want to achieve:

[1025] Using a generative AI model (GPT-4), the system selects the image that most closely matches the user's desired body shape from multiple body shape images provided within the system.

[1026] Register your exercise habits:

[1027] Users input their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day) into the system as text data.

[1028] Generating diet meal plans and exercise programs:

[1029] Enter the following prompts into the generating AI model to generate an optimal diet plan and exercise program.

[1030] Based on the user's body shape image and exercise habits, please provide the following information:

[1031] 1. Body shape image that most closely matches the user's desired physique.

[1032] 2. Optimal diet meal plan (breakfast, lunch, dinner)

[1033] 3. An optimal exercise program (e.g., jogging three times a week, walking for 30 minutes daily)

[1034] 4. Recommended period

[1035] Editing the generated menu:

[1036] Users can edit the generated diet meal plans and exercise programs as needed.

[1037] Emotional recognition and feedback:

[1038] Using an emotion recognition engine (Microsoft Azure Cognitive Services), the system analyzes the user's facial expressions and tone of voice to estimate their emotional state. If the user shows signs of pain during exercise, the system captures this information and adjusts the content and tone of the voice support. Specifically, it may provide encouraging messages or automatically reduce the intensity of the exercise.

[1039] Specific example

[1040] The user launches the app and takes a picture of their body shape with their smartphone camera. The app analyzes the image and obtains the user's body shape data. The user inputs their exercise habits (e.g., jogging 3 times a week). An emotion recognition engine analyzes the user's facial expressions and tone of voice. A generative AI model generates an optimal meal plan and exercise program based on the prompt text. The app displays the generated menu to the user, who can edit it as needed. If the user shows signs of pain during exercise, the emotion engine detects this and provides feedback (e.g., displays an encouraging message, reduces the exercise intensity).

[1041] In this way, users receive individually customized diet plans and support tailored to their emotional state.

[1042] The flow of a specific process in Application Example 1 will be explained using Figure 16.

[1043] Step 1:

[1044] The user takes a picture of their own body shape using their smartphone camera.

[1045] Input: User's body shape image

[1046] Output: Body shape image data

[1047] Specific operation: The user activates the smartphone camera and takes a full-body photo. The captured image is saved within the application.

[1048] Step 2:

[1049] The device uses an image processing library (OpenCV) to analyze the body shape image.

[1050] Input: Body shape image data

[1051] Output: Body shape analysis data

[1052] Specific operation: The device uses OpenCV to analyze the image and extract the user's body shape characteristics (e.g., height, weight, body fat percentage, etc.).

[1053] Step 3:

[1054] Users input their exercise habits as text data.

[1055] Input: Exercise habit data (e.g., jogging 3 times a week)

[1056] Output: Exercise habit text data

[1057] Specific action: The user enters their exercise habits into the application's input form and presses the submit button.

[1058] Step 4:

[1059] The server uses a generation AI model (GPT-4) to select images that closely resemble the desired body shape.

[1060] Input: Body shape analysis data, exercise habit text data

[1061] Output: Image data that closely resembles the desired physique.

[1062] Specific operation: The server inputs body shape analysis data and exercise habit text data into a generation AI model based on prompt messages, and generates an image that closely resembles the desired body shape.

[1063] Step 5:

[1064] The server uses an AI model to generate meal plans and exercise programs for weight loss.

[1065] Input: Body shape analysis data, exercise habit text data, image data that closely resembles the desired physique.

[1066] Output: Diet meal plan, exercise program, recommended duration

[1067] Specific operation: The server generates the following prompt messages and inputs them into the AI ​​model to generate an optimal meal plan and exercise program.

[1068] Based on the user's body shape image and exercise habits, please provide the following information:

[1069] 1. Body shape image that most closely matches the user's desired physique.

[1070] 2. Optimal diet meal plan (breakfast, lunch, dinner)

[1071] 3. An optimal exercise program (e.g., jogging three times a week, walking for 30 minutes daily)

[1072] 4. Recommended period

[1073] Step 6:

[1074] The user reviews the generated menu and edits it as needed.

[1075] Input: Diet meal plan, exercise program, recommended duration

[1076] Output: Edited menu

[1077] Specific actions: The user reviews the menu generated within the application and edits meal plans and exercise programs as needed.

[1078] Step 7:

[1079] The device uses an emotion recognition engine (Microsoft Azure Cognitive Services) to recognize the user's emotions.

[1080] Input: User facial image, voice tone data

[1081] Output: Emotional state data

[1082] Specific operation: The device uses its camera and microphone to capture the user's facial expressions and voice tone, which are then analyzed by an emotion recognition engine.

[1083] Step 8:

[1084] The server provides feedback based on the user's emotional state.

[1085] Input: Emotional state data

[1086] Output: Feedback message, exercise intensity adjustment

[1087] Specific operation: The server generates encouraging messages for the user based on emotional state data and automatically reduces the exercise intensity as needed.

[1088] (Example 2)

[1089] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1090] Conventional exercise support systems have struggled to provide personalized exercise programs that fully consider users' exercise habits and emotional states. Furthermore, the lack of real-time voice support during exercise and the absence of emotionally-based feedback made it difficult for users to maintain their motivation to exercise consistently.

[1091] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1092] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, means for registering the user's exercise habits, means for generating a diet menu, exercises, and duration, means for editing the generated menu, means for collecting the user's exercise data, means for analyzing the collected exercise data, means for collecting the user's emotional data, means for analyzing the collected emotional data and adjusting the exercise program, and means for giving voice instructions for the exercise menu. This makes it possible to provide an exercise program based on the user's individual exercise habits and emotional state, and helps the user maintain motivation to exercise continuously.

[1093] "Means of registering a user's body shape using images" refers to devices or software that record the shape and size of a user's body as image data.

[1094] "Generative AI" refers to a system that uses artificial intelligence technology to generate data and information tailored to specific purposes.

[1095] "Means for selecting images that closely resemble the desired physique" refers to interfaces and algorithms that allow users to select images that closely match their target body image.

[1096] "Means for registering a user's exercise habits" refers to devices or software used to record a user's daily exercise patterns and frequency.

[1097] "Means for generating diet meal plans, exercises, and timelines" refers to a system that automatically creates meal plans and exercise programs tailored to the user's goals.

[1098] "Means for editing generated menus" refers to an interface that allows users to manually modify and adjust automatically generated meal plans and exercise programs.

[1099] "Means for collecting user exercise data" refers to devices or software used to record data such as the type, number of repetitions, and intensity of exercise performed by the user.

[1100] "Means for analyzing collected exercise data" refers to algorithms and systems used to analyze collected exercise data and evaluate the user's exercise performance and progress.

[1101] "Means of collecting user emotional data" refers to devices and software that record emotional states from users' facial expressions and voice.

[1102] "Means for analyzing collected emotional data and adjusting exercise programs" refers to algorithms and systems that analyze collected emotional data and optimize exercise programs according to the user's emotional state.

[1103] "Means of providing voice instructions for exercise menus" refers to devices or software that provide voice guidance to users on the next exercise menu they should perform.

[1104] This invention is a system that provides voice support to users while they exercise and further adjusts the exercise program based on the user's emotional state. Specific embodiments of this system are described below.

[1105] First, the user launches an exercise app using their smartphone or wearable device. The smartphone's camera or the wearable device's sensors are used to register the user's body shape as images. This records the user's body shape and size as image data.

[1106] Next, the user selects an image that closely resembles the body they aspire to achieve using a generative AI. The generative AI model generates images that closely match the user's desired body image, allowing the user to choose from among them.

[1107] To register users' exercise habits, applications on smartphones or wearable devices are used. This records the user's daily exercise patterns and frequency.

[1108] To generate diet meal plans, exercise routines, and timelines, the server uses data analysis tools such as Python and TensorFlow. This automatically creates meal plans and exercise programs tailored to the user's goals.

[1109] To edit the generated menus, users use a smartphone application. This allows users to manually modify and adjust the automatically generated meal plans and exercise programs.

[1110] Sensors from smartphones and wearable devices are used to collect user exercise data. For example, accelerometers and heart rate monitors are used to record data such as the type, frequency, and intensity of exercise.

[1111] To analyze the collected exercise data, the server uses Python and TensorFlow. This allows the collected exercise data to be analyzed and the user's exercise performance and progress to be evaluated.

[1112] The smartphone's camera and microphone are used to collect user emotion data. This records the user's emotional state from their facial expressions and voice.

[1113] The server uses an emotion engine to analyze collected emotional data and adjust the exercise program. This allows the collected emotional data to be analyzed, and the exercise program to be optimized according to the user's emotional state.

[1114] To provide voice instructions for exercise routines, the device uses the Google Text-to-Speech API. This allows the user to receive voice guidance on the next exercise they should perform.

[1115] Specific example:

[1116] The user launches a strength training app on their smartphone. The app gives a voice command saying, "Next, do 20 sit-ups." After the user performs the sit-ups, the device uses its accelerometer to record the number of repetitions. The server analyzes this data and determines the next exercise to do, "Next, do 15 squats," and sends this command to the device. The device reads this command aloud. If the user smiles after the exercise, the device uses its camera to record the expression and sends it to the server. The server analyzes this data and incorporates sit-ups more frequently into future exercise programs.

[1117] Example of a prompt:

[1118] "Please create a program that provides voice instructions for the next exercise a user should do during their strength training. For example, it should say, 'Next, do 20 sit-ups.' Also, analyze the user's emotional state and, if they show a joyful expression, frequently incorporate that exercise into future programs."

[1119] The flow of the specific processing in Example 2 will be explained using Figure 17.

[1120] Step 1:

[1121] The user starts exercising.

[1122] Input: The user launches an exercise app using a smartphone or wearable device.

[1123] Specific action: The user taps the app on their smartphone and selects "Strength Training".

[1124] Output: A signal to initiate movement is sent to the terminal.

[1125] Step 2:

[1126] The device provides voice instructions for the exercise routine.

[1127] Input: The signal to start exercise and the exercise menu sent from the server.

[1128] Specific action: The device uses the Google Text-to-Speech API to read aloud, "Next, do 20 sit-ups."

[1129] Output: Recognizes the exercise routine the user should perform next.

[1130] Step 3:

[1131] The user performs exercise.

[1132] Input: Voice commands from the device.

[1133] Specific action: The user performs 20 sit-ups.

[1134] Output: Performing exercise.

[1135] Step 4:

[1136] The device collects the user's exercise data.

[1137] Input: User's exercise performance.

[1138] Specific operation: The device uses an accelerometer and heart rate monitor to record the number of exercises and their intensity.

[1139] Output: Exercise data is saved to the device.

[1140] Step 5:

[1141] The server analyzes the exercise data.

[1142] Input: Exercise data transmitted from the device.

[1143] Specific operation: The server uses Python and TensorFlow to analyze exercise data and evaluate the user's exercise performance.

[1144] Output: Analysis results are generated.

[1145] Step 6:

[1146] The server determines the next exercise menu and sends it to the terminal.

[1147] Input: Analysis results of exercise data.

[1148] Specific operation: Based on the analysis results, the server determines the next exercise to perform and sends that information to the terminal. For example, it might decide, "Next, please do 15 squats."

[1149] Output: The next exercise menu will be sent to the terminal.

[1150] Step 7:

[1151] The device will provide voice instructions for the next exercise routine.

[1152] Input: The following exercise menu sent from the server.

[1153] Specific action: The device will again use the Google Text-to-Speech API to read aloud, "Now, do 15 squats."

[1154] Output: Recognizes the exercise routine the user should perform next.

[1155] Step 8:

[1156] The user finishes their exercise.

[1157] Input: User indicates their intention to end the exercise.

[1158] Specific action: The user taps the "Exit" button in the app.

[1159] Output: A signal indicating the end of the exercise is sent to the terminal.

[1160] Step 9:

[1161] The device collects user emotion data.

[1162] Input: Signal to end exercise.

[1163] Specific operation: The device uses its camera and microphone to collect emotional data from the user's facial expressions and voice.

[1164] Output: Emotional data is saved to the device.

[1165] Step 10:

[1166] The server analyzes emotional data and adjusts the exercise program accordingly.

[1167] Input: Emotional data sent from the device.

[1168] Specific operation: The server uses an emotion engine to analyze the collected emotion data and optimize the exercise program according to the user's emotional state.

[1169] Output: A modified exercise program is generated.

[1170] (Application Example 2)

[1171] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[1172] Conventional exercise and diet support systems only provide menus based on the user's exercise habits and body shape, but they lack consideration for the user's emotional state when suggesting meal plans and integration with delivery services for the suggested meals. Therefore, there is a challenge in that it is difficult for users to maintain the motivation to continue exercising and eating consistently.

[1173] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1174] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, means for registering the user's exercise habits, means for generating a diet meal plan, exercise routines, and duration, means for editing the generated menu, means for analyzing the user's emotional state, means for suggesting a meal plan based on the emotional state, and means for delivering the suggested meal plan in cooperation with a delivery service. This enables the suggestion of an optimal meal plan according to the user's emotional state and the rapid delivery of that menu.

[1175] "Means for registering a user's body shape using images" refers to devices or software for recording a user's body shape and dimensions as image data.

[1176] "A method for selecting images that closely resemble the desired body using generation AI" refers to a device or software that uses artificial intelligence technology to select images of the body that the user is aiming for.

[1177] "Means for registering a user's exercise habits" refers to devices or software used to record the type and frequency of a user's daily exercise.

[1178] "Means for generating diet meal plans, exercises, and durations" refers to devices or software that automatically create appropriate meal plans, exercise programs, and durations based on the user's goals and current condition.

[1179] "Means for editing generated menus" refers to devices or software that allow users to manually modify and adjust generated meal plans and exercise programs.

[1180] "Means for analyzing a user's emotional state" refers to devices or software that analyze a user's facial expressions and feedback to determine their emotional state.

[1181] "Means for suggesting meal menus based on emotional state" refers to devices or software that suggest the optimal meal menu considering the user's emotional state.

[1182] "Methods for delivering suggested meal menus in conjunction with delivery services" refers to devices or software that deliver suggested meal menus to users in conjunction with delivery services.

[1183] A system for carrying out this invention includes means for registering the user's body shape with an image, means for selecting an image that closely resembles the desired body shape using a generating AI, means for registering the user's exercise habits, means for generating a diet meal menu, exercises, and duration, means for editing the generated menu, means for analyzing the user's emotional state, means for suggesting a meal menu based on the emotional state, and means for delivering the suggested meal menu in cooperation with a delivery service.

[1184] Hardware and software to be used

[1185] hardware

[1186] smartphone

[1187] server

[1188] Delivery service terminal

[1189] software

[1190] Python

[1191] Speech recognition libraries (e.g., Google Speech-to-Text)

[1192] Sentiment analysis libraries (e.g., IBM Watson)

[1193] Generative AI models (e.g., GPT-3)

[1194] Data processing and data calculation

[1195] A method for registering a user's body shape using an image.

[1196] The user uses their smartphone camera to take a picture of their body shape and sends the image data to the server. The server stores this image data and uses it for later analysis.

[1197] A method of selecting images that closely resemble the desired body shape using generation AI.

[1198] The server uses a generative AI model to generate images of the body the user desires. The user then selects their target body from the generated images through a smartphone application.

[1199] A means of registering a user's exercise habits.

[1200] Users use a smartphone application to input the type and frequency of their daily exercise. This data is sent to and stored on a server.

[1201] Means for generating diet meal plans, exercise routines, and timeframes.

[1202] The server generates appropriate meal plans, exercise programs, and durations based on the user's body shape data, target body image, and exercise habit data. These plans are automatically created using a generation AI model.

[1203] Means for editing the generated menu

[1204] Users can manually modify and adjust the generated meal plans and exercise programs using a smartphone application. The edited data is stored on the server.

[1205] A means of analyzing a user's emotional state

[1206] The system collects user feedback after exercise or meals using the smartphone's camera and microphone. The server uses an emotion analysis library to analyze this feedback and determine the user's emotional state.

[1207] A method for suggesting meal menus based on emotional state

[1208] Based on the results of the emotion analysis, the server suggests the optimal meal menu tailored to the user's emotional state. It uses a generative AI model to automatically generate the suggested menu.

[1209] A method of delivering the suggested meal menu in cooperation with a delivery service.

[1210] The server sends the suggested meal menu to the delivery service terminal and instructs it to deliver it to the user. The delivery service then promptly delivers the suggested menu.

[1211] Examples of specific cases and prompt statements

[1212] Specific example

[1213] When a user is running, the system will notify them by voice if their current pace is not enough to reach their target time and will suggest increasing their pace.

[1214] If a user gives feedback that they "had fun" after exercising, the emotion engine will determine that they are "happy" and suggest a salad and grilled chicken.

[1215] Example of a prompt

[1216] "When a user is running, analyze their current pace and suggest increasing their pace if they are not reaching their target time."

[1217] "Analyze user feedback after exercise and suggest appropriate meal plans based on their emotional state."

[1218] The above describes the embodiments for carrying out this invention.

[1219] The flow of a specific process in Application Example 2 will be explained using Figure 18.

[1220] Step 1:

[1221] The user uses their smartphone camera to take a picture of their body shape and sends the image data to the server. The server stores this image data and uses it for later analysis. The input is the user's body shape image, and the output is the image data stored on the server.

[1222] Step 2:

[1223] The server uses a generative AI model to generate images of the body the user desires. The user selects their target body from the generated images via a smartphone application. The input is the user's body shape image and the characteristics of the target body, and the output is the image of the target body selected by the user.

[1224] Step 3:

[1225] Users input the type and frequency of their daily exercise using a smartphone application. This data is sent to and stored on a server. The input is the user's exercise habit data, and the output is the exercise habit data stored on the server.

[1226] Step 4:

[1227] The server generates appropriate meal plans, exercise programs, and durations based on the user's body shape data, target body image, and exercise habit data. These plans are automatically created using a generation AI model. The input is the user's body shape data, target body image, and exercise habit data, and the output is the generated meal plan and exercise program.

[1228] Step 5:

[1229] Users can manually modify and adjust the generated meal plans and exercise programs using a smartphone application. The edited data is saved on the server. The input is the user's edits, and the output is the edited menu saved on the server.

[1230] Step 6:

[1231] The system collects user feedback after exercise or meals using the smartphone's camera and microphone. A server uses an emotion analysis library to analyze this feedback and determine the user's emotional state. The input is the user's feedback data, and the output is the analyzed emotional state.

[1232] Step 7:

[1233] The server suggests the optimal meal menu based on the user's emotional state, using the results of the emotion analysis. A generative AI model is used to automatically generate the suggestions. The input is the analyzed emotional state, and the output is the suggested meal menu.

[1234] Step 8:

[1235] The server sends the suggested meal menu to the delivery service terminal and instructs it to deliver it to the user. The delivery service quickly delivers the suggested menu. The input is the suggested meal menu, and the output is the meal delivered to the user.

[1236] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1237] The data generation model 58 is a form of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1238] Other examples of generative AI include Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are some examples.

[1239] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1240] [Third Embodiment]

[1241] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1242] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1243] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1244] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1245] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1246] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1247] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1248] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1249] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1250] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1251] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1252] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[1253] "Example of form 1"

[1254] The system of this invention allows the user to take a picture of their body shape using a digital camera or smartphone camera and register the image in the system. Next, the system selects the image that best matches the user's desired body shape from a set of body shape images provided within the system. Furthermore, the user registers their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day) in the system. Based on this information, the system uses a generative AI to generate an optimal diet plan, exercise program, and duration for the user. The generated menu can be edited by the user as needed.

[1255] "Example of form 2"

[1256] The system of this invention provides voice support when a user is exercising. For example, when performing strength training, it reads out the next exercise to be done (e.g., "Next, please do 20 sit-ups"). When running, it checks the user's pace and suggests increasing or decreasing the pace as needed (e.g., "At your current pace, you may not reach your target time. Try increasing your pace a little"). This enables the user to continue exercising appropriately.

[1257] The following describes the processing flow for each example of the form.

[1258] "Example of form 1"

[1259] Step 1: The user takes a picture of their body shape using a digital camera or smartphone camera. The captured image is uploaded to the system.

[1260] Step 2: From the multiple body shape images provided within the system, the user selects the one that best matches their desired body shape. This selection is made through the system's user interface.

[1261] Step 3: The user registers their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day) in the system. This registration is also done through the system's user interface.

[1262] Step 4: Based on this information, the system uses AI to generate an optimal diet plan, exercise program, and duration for the user. The generated plan can be edited by the user as needed.

[1263] "Example of form 2"

[1264] Step 1: When the user starts exercising, the system provides voice support.

[1265] Step 2: When performing strength training, the system will announce the next exercise to do (e.g., "Next, do 20 sit-ups").

[1266] Step 3: When running, the system monitors the user's pace and suggests increasing or decreasing the pace as needed (e.g., "At your current pace, you may not reach your target time. Try increasing your pace a little").

[1267] (Example 1)

[1268] Next, we will describe Embodiment 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1269] Traditional diet systems have struggled to generate personalized diet plans based on users' body types and exercise habits. Furthermore, the lack of editing capabilities for the generated plans and insufficient support during exercise made it difficult for users to consistently follow their diets.

[1270] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1271] In this invention, the server includes means for registering the user's body shape as an image, means for presenting multiple body shape images and allowing the user to select their desired body shape, means for registering the user's exercise habits, means for generating a diet meal plan, exercise program, and duration using an AI model based on the user's body shape image, desired body shape, and exercise habit information, and means for editing the generated menu. This makes it possible to generate and edit a diet plan tailored to the user's individual needs.

[1272] "Method for registering user body shape with images" refers to a function that allows users to take pictures of their own body shape using a digital camera or smartphone camera and upload those images to the system.

[1273] "A means of presenting multiple body shape images and allowing the user to select the body shape they aspire to" refers to a function that presents the user with multiple body shape images prepared within the system and allows the user to select the one that most closely matches their desired body shape.

[1274] "Means for registering users' exercise habits" refers to a function that allows users to input their own exercise habits (e.g., jogging three times a week, walking for 30 minutes every day, etc.) into the system and register that information.

[1275] "A means of generating diet meal plans, exercise programs, and their duration using a generative AI model" refers to a function that uses a generative AI model to generate optimal diet meal plans, exercise programs, and their duration based on the user's body shape image, desired physique, and exercise habits.

[1276] "Means for editing generated menus" refers to a function that allows users to review the generated diet plan and edit meal menus and exercise programs as needed.

[1277] This invention begins with a user taking a picture of their own body shape using a digital camera or smartphone camera and registering that image in the system. The user launches the camera app on their smartphone and takes a picture of their body shape. The captured image is uploaded from the terminal to the system, and the server receives the uploaded image and saves it to a database. At this time, the image quality is checked using an image processing library (e.g., OpenCV).

[1278] Next, the server presents the user with several body shape images prepared within the system. The user selects the image that most closely matches their desired body shape from the presented images. The ID of the selected body shape image is sent from the terminal to the server, and the server stores that ID in its database.

[1279] Users input their exercise habits from their device. For example, they might enter information such as "I jog three times a week" or "I walk for 30 minutes every day." The entered exercise habit data is sent from the device to the server, which stores that data in a database.

[1280] The server obtains information on the user's body shape image, desired physique, and exercise habits, and prompts a generation AI model (e.g., GPT-4) to generate a diet plan. An example of a prompt is, "Based on the user's body shape image, desired physique, and exercise habits, please generate an optimal diet meal plan, exercise program, and duration." The generation AI model generates the optimal meal plan, exercise program, and duration based on the prompt. The generated diet plan is then saved to a database by the server.

[1281] Users can view the generated diet plan on their device and edit the meal menu and exercise program as needed. For example, they can change breakfast from oatmeal to yogurt. The edited information is sent from the device to the server, which then updates the database.

[1282] In this way, the system allows users to easily create a diet plan tailored to their own goals and edit it as needed.

[1283] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1284] Step 1: Register the user's body shape image.

[1285] The user launches the camera app on their smartphone and takes a picture of their own body shape.

[1286] Input: Body shape image taken by the user

[1287] The device displays the captured image on the system's upload screen, and the user presses the "Upload" button.

[1288] The server receives the uploaded image and uses an image processing library (e.g., OpenCV) to check the image quality.

[1289] Output: Quality-verified body shape image

[1290] The server saves verified images to the database.

[1291] Step 2: Choosing your desired body image

[1292] The server retrieves multiple body shape images from the database and displays them on the user's device.

[1293] Input: Multiple body shape images stored in a database

[1294] Users tap to select their desired body shape from the displayed images.

[1295] The device sends the ID of the selected body shape image to the server.

[1296] Output: ID of the body shape image selected by the user

[1297] The server saves the ID of the selected body shape image to the database.

[1298] Step 3: Register your exercise habit

[1299] The user enters their exercise habits into the input form on their device (e.g., "I jog three times a week").

[1300] Input: Exercise habit data entered by the user.

[1301] The device sends the entered exercise habit data to the server.

[1302] Output: Exercise habit data

[1303] The server stores data on exercise habits in a database.

[1304] Step 4: Generating an optimal diet plan

[1305] The server retrieves information from the database regarding the user's body shape image, desired physique, and exercise habits.

[1306] Input: User's body shape image, desired physique, and exercise habits.

[1307] The server will input the following prompt into the generated AI model (e.g., GPT-4):

[1308] "Based on the user's body shape image, desired physique, and exercise habits, please generate an optimal diet plan, exercise program, and duration."

[1309] The generative AI model generates optimal meal plans, exercise programs, and their durations based on the prompt text.

[1310] Output: Generated diet plan (meal menu, exercise program, duration)

[1311] The server saves the generated diet plan to the database.

[1312] Step 5: Edit the menu

[1313] Users can view the generated diet plan on their device.

[1314] Input: Generated diet plan

[1315] Users can edit their meal plans and exercise programs as needed (e.g., change oatmeal for breakfast to yogurt).

[1316] The terminal sends the edited content to the server.

[1317] Output: Edited diet plan

[1318] The server updates the database with the edited diet plan.

[1319] (Application Example 1)

[1320] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1321] Traditional diet and fitness programs often offer generic menus and exercise plans, making it difficult to provide programs optimized for individual users' body types and exercise habits. Furthermore, fitness facilities faced the challenge of placing a heavy burden on trainers to provide individually optimized training programs and meal plans. Additionally, the lack of real-time feedback made it difficult for users to train effectively.

[1322] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1323] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, and means for registering the user's exercise habits. This makes it possible to provide individually optimized training programs and meal plans.

[1324] "Method for registering a user's body shape using an image" refers to a method for users to take a picture of their own body shape using a digital camera or smartphone camera and register that image in the system.

[1325] "A method of selecting an image that closely resembles the desired body shape using generative AI" refers to a method of using generative AI to select the image that most closely matches the user's desired body shape from multiple body shape images provided within the system.

[1326] "Means for registering users' exercise habits" refers to the means by which users register their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day, etc.) in the system.

[1327] "Methods for generating diet meal plans, exercises, and durations" refers to methods that use a generating AI to create optimal diet meal plans, exercise programs, and their durations for the user, based on the user's body shape image and exercise habits.

[1328] "Means for editing generated menus" refers to means that users can edit the generated diet meal plans and exercise programs as needed.

[1329] "A means of providing individually optimized training programs and meal plans by having fitness facility members take photos of their own physique, select their desired body shape, and register their exercise habits" refers to a means of providing individually optimized training programs and meal plans by having fitness facility members take photos of their own physique, select their desired body shape, and register their exercise habits.

[1330] "A means of receiving real-time feedback in collaboration with trainers within the facility" refers to a method of receiving real-time feedback on training and diet in collaboration with trainers within the fitness facility.

[1331] A system for carrying out this invention includes means for registering the user's body shape with an image, means for selecting an image that closely resembles the desired body shape using a generating AI, means for registering the user's exercise habits, means for generating a diet meal plan, exercise routines, and duration, means for editing the generated menu, means for providing individually optimized training programs and meal plans by allowing fitness facility members to take a picture of their body shape, select their desired body shape, and register their exercise habits, and means for receiving real-time feedback in cooperation with trainers within the facility.

[1332] 1. System Program

[1333] The program in this system performs the following operations:

[1334] 2. Explanation of the program's processing

[1335] The server receives body shape images taken by the user using their smartphone camera and performs image processing. Specifically, it uses the Python PIL library to load the images and preprocess them as needed. Next, it uses a generative AI model to select the image that most closely resembles the user's desired body shape. This generative AI model uses a pre-trained Keras model.

[1336] The user's exercise habits are registered on the server in text format. This includes information such as how many times a week the user exercises and what kind of exercise they do. Based on this information, the generative AI model generates an optimal diet plan, exercise program, and duration for the user.

[1337] The generated menu can be viewed by the user via a smartphone app and edited as needed. Furthermore, users can collaborate with trainers at fitness facilities to receive real-time feedback. This feedback includes training progress and dietary advice.

[1338] 3. Specific Examples and Examples of Prompt Statements

[1339] As a concrete example, consider a user who jogs three times a week. Based on this information, the generative AI model generates an optimal meal plan and exercise program for the user. The following is an example of a prompt to input to the generative AI model.

[1340] User's body shape image: user_image.jpg

[1341] Target body shape: target_shape.jpg

[1342] Exercise habits: I jog three times a week.

[1343] Based on this information, please generate an optimal diet plan and exercise program.

[1344] By using this prompt message as input to the generative AI model, it is possible to generate a menu that is optimal for the user.

[1345] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1346] Step 1:

[1347] The user takes a picture of their body shape using their smartphone camera and saves the image to their device.

[1348] Input: User's body shape image

[1349] Output: Body shape image saved on the device

[1350] Specific actions: The user launches the camera app on their smartphone and takes a picture of their own body shape. The captured image is then saved to the device.

[1351] Step 2:

[1352] The device uploads the saved body shape image to the server.

[1353] Input: Body shape image saved on the device

[1354] Output: Body shape images uploaded to the server

[1355] Specific operation: The device uploads the saved body shape images to the server via an internet connection. HTTP requests are used for the upload.

[1356] Step 3:

[1357] The server receives the uploaded body shape image and performs image processing.

[1358] Input: Body shape image uploaded to the server

[1359] Output: Preprocessed body shape image

[1360] Specific operation: The server uses the Python PIL library to read images and performs preprocessing such as resizing and denoising as needed.

[1361] Step 4:

[1362] The server uses a generated AI model to select the image that most closely matches the user's desired body shape.

[1363] Input: Preprocessed body shape image

[1364] Output: The image that most closely matches the desired body shape.

[1365] Specific operation: The server uses a pre-trained Keras model to compare the user's body shape image with the image of the target body shape and selects the closest image.

[1366] Step 5:

[1367] The user enters their exercise habits into their device and sends them to the server.

[1368] Input: User's exercise habit information

[1369] Output: Exercise habit information sent to the server

[1370] Specific operation: The user inputs their exercise habits (e.g., jogging three times a week) through a smartphone app and sends that information to the server.

[1371] Step 6:

[1372] The server uses a generative AI model to generate optimal diet meal plans and exercise programs based on the user's body shape image and exercise habit information.

[1373] Input: User's body shape image, exercise habit information

[1374] Output: Optimal diet meal plans and exercise programs

[1375] Specific operation: The server inputs prompt messages into the generating AI model, which then generates an optimal diet meal plan and exercise program for the user. Examples of prompt messages are as follows:

[1376] User's body shape image: user_image.jpg

[1377] Target body shape: target_shape.jpg

[1378] Exercise habits: I jog three times a week.

[1379] Based on this information, please generate an optimal diet plan and exercise program.

[1380] Step 7:

[1381] The server sends the generated menu to the user's terminal, and the user edits it as needed.

[1382] Input: Optimal diet meal plans and exercise programs

[1383] Output: Menu sent to the user's terminal

[1384] Specific operation: The server sends the generated menu to the user's device, and the user views the menu through a smartphone app and edits it as needed.

[1385] Step 8:

[1386] Users train within a fitness facility and receive real-time feedback in collaboration with trainers.

[1387] Input: Training progress, dietary advice

[1388] Output: Real-time feedback

[1389] Specific operation: Users train within a fitness facility, and trainers monitor the user's progress and provide real-time feedback through a dedicated application.

[1390] (Example 2)

[1391] Next, we will describe Example 2 of the morphological example. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1392] Conventional exercise support systems have struggled to collect and analyze user exercise data in real time and provide appropriate voice guidance. Furthermore, they lacked sufficient support in generating specific exercise and meal plans to help users achieve their desired physique, and in guiding users through the exercise process based on these plans. This made it difficult for users to effectively maintain a consistent exercise routine.

[1393] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body using a generation AI model, means for registering the user's exercise habits, means for generating a diet meal menu, exercises, and duration, means for editing the generated menu, means for collecting the user's exercise data, means for analyzing the collected exercise data, means for generating voice instructions based on the analysis results, means for transmitting the generated voice instructions to the user's terminal, and means for playing the voice instructions on the user's terminal. As a result, the user can effectively continue exercising to get closer to their desired body while receiving appropriate exercise instructions in real time.

[1394] "Method for registering a user's body shape with images" refers to a function that records a user's current body shape as image data using devices such as cameras or scanners.

[1395] "A method for selecting images that closely resemble the desired body shape using a generative AI model" refers to a function that utilizes a generative AI model to present images that closely resemble the user's ideal body shape, allowing the user to select from among them.

[1396] "Methods for registering users' exercise habits" refers to a function that records information such as the type, frequency, and duration of exercise that users perform on a daily basis in a database.

[1397] "Methods for generating diet meal plans, exercises, and durations" refers to a function that automatically creates appropriate meal plans, exercise plans, and durations based on the user's goals, current body shape, and exercise habits.

[1398] "Means for editing generated menus" refers to a function that allows users to manually modify and adjust automatically generated meal plans and exercise plans.

[1399] "Means of collecting user exercise data" refers to a function that uses wearable devices or smartphones to acquire exercise-related data such as the user's heart rate, speed, and distance in real time.

[1400] "Means for analyzing collected exercise data" refers to a function that evaluates and analyzes the user's exercise status and performance based on the collected exercise data.

[1401] "Means for generating voice instructions based on analysis results" refers to a function that generates instructions to provide users with appropriate exercise instructions and advice via voice, based on the results of data analysis.

[1402] "Means for sending generated voice commands to the user's device" refers to a function that sends generated voice commands to the user's smartphone or tablet via communication means such as the internet or Bluetooth.

[1403] "Means for playing voice commands on the user's device" refers to a function that plays voice commands received by the user's smartphone or tablet through a speaker or earphones.

[1404] This invention is a system that provides voice support to users while they exercise. The system registers the user's body shape as an image, uses a generative AI model to select an image that closely resembles the desired body shape, and registers the user's exercise habits. Furthermore, it includes a function to generate a diet plan, exercise routine, and duration, and to edit the generated plan.

[1405] The server collects and analyzes user exercise data in real time. The hardware used includes wearable devices (e.g., smartwatches and fitness trackers) for collecting user exercise data. The software includes algorithms for data analysis and generative AI models (e.g., OpenAI's GPT-4) for speech synthesis.

[1406] The device (the user's smartphone or tablet) relays voice instructions sent from the server to the user. Specifically, the device plays the voice data received from the server, providing the user with advice on the next exercise routine and pacing.

[1407] Users receive voice instructions from the device while exercising and continue exercising according to those instructions. For example, when doing strength training, they might receive the instruction, "Next, do 20 sit-ups," and then perform sit-ups. When running, if their current pace is not reaching their target time, they might receive advice such as, "Try to pick up the pace a little," and adjust their pace accordingly.

[1408] As a concrete example, consider a scenario where a user is running. The user puts on a smartwatch and starts running with their smartphone. The smartwatch collects data such as the user's heart rate and speed and sends it to a server. The server analyzes this data and detects that the user's current pace is not enough to reach their target time. The server uses a generative AI model to generate a voice command saying, "At your current pace, you may not reach your target time. Try to pick up the pace a little," and sends it to the device. The device plays this voice command and communicates it to the user.

[1409] Example of a prompt:

[1410] "When a user is running and their current pace is not reaching their target time, please generate voice prompts suggesting they increase their pace."

[1411] This system allows users to consistently engage in appropriate exercise.

[1412] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1413] Step 1:

[1414] Before starting exercise, the user puts on a wearable device (e.g., a smartwatch) and launches a dedicated app on their smartphone. The wearable device collects exercise data such as the user's heart rate, speed, and distance in real time. This data is transmitted to the user's smartphone via Bluetooth or Wi-Fi. The input is the user's exercise data, and the output is the exercise data transmitted to the smartphone.

[1415] Step 2:

[1416] The server receives exercise data transmitted from the user's smartphone. The server uses data analysis algorithms to evaluate the user's exercise status in real time. For example, it can determine whether the user's pace during a run is reaching their target time. The input is the exercise data transmitted from the smartphone, and the output is the analysis result.

[1417] Step 3:

[1418] The server uses a generative AI model (e.g., a generative AI model) to generate appropriate voice instructions for the user. For example, if the user is moving too slowly, it might generate a voice instruction such as, "At your current pace, you may not reach your target time. Try to pick up the pace a little." The input is the analysis result, and the output is the generated voice instruction.

[1419] Step 4:

[1420] The server sends the generated voice commands to the user's smartphone. An internet connection is required for transmission. The input is the generated voice command, and the output is the voice command sent to the smartphone.

[1421] Step 5:

[1422] The device (the user's smartphone) plays the voice instructions received from the server. The user can hear the voice instructions through the smartphone's speaker or earphones. The input is the voice instructions received from the server, and the output is the played voice instructions.

[1423] Step 6:

[1424] The user continues exercising according to the voice instructions from the device. For example, if instructed to "do 20 sit-ups next," the user will perform 20 sit-ups. Similarly, if instructed to "try to increase your pace," the user will increase their running pace. The input is the played voice instructions, and the output is the user's exercise behavior.

[1425] (Application Example 2)

[1426] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1427] While conventional exercise support systems can provide appropriate menus based on a user's exercise habits and physique, they have limitations in terms of efficient route guidance and time management during delivery. Furthermore, there was a lack of means to provide appropriate advice when delivery delays were anticipated, highlighting the need for increased efficiency in delivery operations.

[1428] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1429] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, means for registering the user's exercise habits, means for generating a diet meal menu, exercise routines, and duration, means for editing the generated menu, means for registering delivery destination information, means for generating a delivery route, means for managing delivery time, means for guiding the delivery destination and route by voice, and means for prompting the user to speed up if the delivery time is likely to be delayed. This enables efficient route guidance and time management in delivery operations, and allows for the provision of appropriate advice if the delivery time is likely to be delayed.

[1430] "Methods for registering a user's body shape using images" refers to methods for saving a user's body shape and size as image data.

[1431] "A method of selecting images that closely resemble the desired body using generation AI" refers to a method of using artificial intelligence to select images of the body that the user aims for.

[1432] "Methods for registering users' exercise habits" refer to methods for recording the types and frequency of exercises that users perform on a daily basis.

[1433] "Methods for generating diet meal plans, exercises, and durations" refers to methods for creating meal plans, exercise programs, and implementation periods tailored to the user's goals.

[1434] "Means for editing generated menus" refers to methods for modifying created meal plans and exercise programs according to the user's requests.

[1435] "Methods for registering delivery address information" refers to methods for recording information such as the address and contact details of the place where the delivery will be made.

[1436] "Methods for generating delivery routes" refer to methods for calculating the optimal route for efficiently visiting multiple delivery destinations.

[1437] "Means of managing delivery times" refers to methods for recording and managing the estimated arrival time and actual arrival time at each delivery destination.

[1438] "A method for guiding delivery destinations and routes via voice" refers to a method of instructing delivery personnel by voice to the next delivery destination or the optimal route.

[1439] "Methods to encourage speeding up when delivery is likely to be delayed" refers to methods of using voice commands to instruct delivery personnel to increase their speed when deliveries are behind schedule.

[1440] The system for implementing this invention has the function of registering the user's body shape with an image, selecting an image that closely resembles the desired body shape using a generation AI, and registering the user's exercise habits. It also includes the function of generating a diet meal menu, exercise plan, and duration, and editing the generated menu. Furthermore, it has the function of registering delivery destination information, generating a delivery route, and managing delivery times. It also includes a function to provide voice guidance for the delivery destination and route, and to encourage the user to speed up if the delivery time is likely to be delayed.

[1441] System program processing

[1442] The server acquires image data using a smartphone or camera to register the user's body shape and stores it in a database. Using a generative AI model, it generates prompts for the user to select an image that closely resembles their desired physique and presents them to the user. To register the user's exercise habits, it collects exercise data from a smartphone app or wearable device and stores it in a database.

[1443] To generate a diet plan including meal menus, exercise routines, and duration, the server creates an optimal plan based on the user's goals, current physique, and exercise habits. Users can edit the generated menu using a smartphone app.

[1444] To register delivery information, the server stores the delivery address and contact information in a database. To generate delivery routes, the server uses map data to calculate the optimal route. To manage delivery times, the server records and manages the estimated and actual arrival times for each delivery location.

[1445] To guide delivery drivers to their destinations and routes via voice, the server uses speech synthesis technology to issue instructions. If a delivery is likely to be delayed, the server will also instruct the driver to speed up via voice.

[1446] Specific example

[1447] For example, if a delivery person delivers to "1-1-1 Shibuya-ku, Tokyo", the server will operate as follows:

[1448] 1. Retrieve "1-1-1 Shibuya-ku, Tokyo" from the list of delivery addresses.

[1449] 2. The voice will announce, "Your next delivery address is 1-1-1 Shibuya-ku, Tokyo. We will guide you to the optimal route."

[1450] 3. If it appears that the delivery time will be delayed, a voice message will advise, "At the current pace, the delivery time will be delayed. Please increase your pace slightly."

[1451] Examples of prompts to input into a generative AI model:

[1452] "Your next delivery address is 1-1-1 Shibuya-ku, Tokyo. I will guide you to the optimal route. At your current pace, the delivery time will likely be delayed. Please pick up the pace a little."

[1453] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1454] Step 1:

[1455] The server registers the user's body shape as an image. The user takes a picture of their body shape using a smartphone or camera and sends the image data to the server. The server stores the received image data in a database. The input is the user's body shape image, and the output is the image data stored in the database.

[1456] Step 2:

[1457] The server generates prompts to select an image that closely resembles the user's desired physique using a generative AI model. When the user inputs the desired physical characteristics, the server uses the generative AI model to generate an image based on those characteristics and presents it to the user. The input is the user's desired physical characteristics, and the output is the generated image.

[1458] Step 3:

[1459] The server registers the user's exercise habits. Users input daily exercise data using a smartphone app or wearable device and send that data to the server. The server stores the received exercise data in a database. The input is the user's exercise data, and the output is the exercise data stored in the database.

[1460] Step 4:

[1461] The server generates meal plans, exercise routines, and timelines for weight loss. Based on the user's goals, current physique, and exercise habits, the server generates and presents an optimal meal plan and exercise program to the user. The input is the user's goals, physique, and exercise habits, and the output is the generated meal plan and exercise program.

[1462] Step 5:

[1463] The user edits the generated menu. Using a smartphone app, the user modifies the generated meal plan and exercise program and sends the modified data to the server. The server stores the received modified data in a database. The input is the user's modified data, and the output is the modified data stored in the database.

[1464] Step 6:

[1465] The server registers delivery address information. The user enters the delivery address and contact information and sends this information to the server. The server stores the received delivery address information in a database. The input is the delivery address and contact information, and the output is the delivery address information stored in the database.

[1466] Step 7:

[1467] The server generates delivery routes. Based on the delivery destination information, the server uses map data to calculate the optimal route and presents it to the delivery person. The input is the delivery destination information, and the output is the optimal delivery route.

[1468] Step 8:

[1469] The server manages delivery times. It records and manages estimated and actual arrival times for each delivery destination. Inputs are delivery destination information and current location information, and outputs are estimated and actual arrival times.

[1470] Step 9:

[1471] The server provides voice guidance for delivery locations and routes. Using speech synthesis technology, it instructs delivery personnel verbally on the next delivery location and the optimal route. The input is delivery route information, and the output is voice guidance.

[1472] Step 10:

[1473] The server prompts the delivery person to speed up if the delivery is likely to be delayed. If the delivery is behind schedule, the server will verbally instruct the delivery person to increase their speed. The input is the current location and estimated arrival time, and the output is a verbal instruction to speed up.

[1474] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1475] "Example of form 1"

[1476] In one embodiment of the present invention, an emotion engine that recognizes the user's emotions is incorporated into the system. This emotion engine estimates emotions from the user's facial expressions, tone of voice, and movements during exercise. For example, if the user shows a pained expression during exercise, the emotion engine captures this information and determines that the user is experiencing excessive stress. This information is fed back to the system, and the content and tone of the voice support are adjusted accordingly. Specifically, the user may be provided with encouraging messages, or the intensity of the exercise may be automatically reduced.

[1477] "Example of form 2"

[1478] Furthermore, the emotion engine adjusts diet meal plans and exercise programs according to the user's emotional state. For example, if a user shows a joyful expression after exercise, the emotion engine picks up on this information and determines that the user likes that exercise. This information is fed back into the system, and that exercise is frequently incorporated into recommended exercise programs. Similarly, if a user shows a satisfied expression after a meal, the system determines that the meal plan is favorable to the user and recommends similar meal plans.

[1479] The following describes the processing flow for each example of the form.

[1480] "Example of form 1"

[1481] Step 1: The user begins exercising.

[1482] Step 2: The emotion engine estimates the user's emotions from their facial expressions, tone of voice, and movements during exercise.

[1483] Step 3: If the emotion engine determines that the user is experiencing excessive stress, it feeds that information back into the system.

[1484] Step 4: The system adjusts the content and tone of the voice support. Specifically, it may provide encouraging messages to the user or automatically lower the intensity of the exercise.

[1485] "Example of form 2"

[1486] Step 1: The user finishes exercising.

[1487] Step 2: The emotion engine estimates the user's emotions from their facial expressions.

[1488] Step 3: If the emotion engine determines that the user likes the exercise, it feeds that information back into the system.

[1489] Step 4: Adjust the system so that the exercise is frequently incorporated into the recommended exercise program.

[1490] Step 5: The user finishes their meal.

[1491] Step 6: The emotion engine estimates the user's emotions from their facial expressions.

[1492] Step 7: If the emotion engine determines that the meal menu is favorable to the user, it feeds that information back into the system.

[1493] Step 8: Adjust the system so that it recommends similar meal menus.

[1494] (Example 1)

[1495] Next, we will describe Embodiment 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1496] Conventional diet support systems have faced challenges in providing effective diet support because they struggle to offer personalized menus based on the user's body shape and exercise habits, and they lack feedback that takes into account the user's emotions and stress levels. Furthermore, they cannot detect the stress and fatigue the user experiences during exercise in real time and provide support accordingly, making it difficult to maintain the user's motivation.

[1497] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1498] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, means for registering the user's exercise habits, means for generating a diet menu, exercises, and duration, means for editing the generated menu, means including an emotion engine that recognizes the user's emotions, means for the emotion engine to analyze the user's facial expressions, tone of voice, and movements during exercise to estimate emotions, means for adjusting the content and tone of voice support based on emotions, and means for automatically adjusting the intensity of exercise. This makes it possible not only to provide a personalized menu based on the user's body shape and exercise habits, but also to detect the user's emotions and stress levels in real time and provide support accordingly.

[1499] "Method for registering user body shape with images" refers to a function that allows users to take pictures of their own body shape using a digital camera or smartphone camera and upload those images to the system.

[1500] "The method of selecting an image that closely resembles the desired body shape using generative AI" refers to a function that uses generative AI to select the image that most closely matches the user's desired body shape from multiple body shape images provided within the system.

[1501] "Means for registering users' exercise habits" refers to a function that allows users to input their own exercise habits (e.g., jogging three times a week, walking for 30 minutes every day, etc.) into the system and save that information.

[1502] "Method for generating diet meal plans, exercises, and durations" refers to a function that uses a generative AI model to create optimal diet meal plans, exercise programs, and their durations based on the user's body shape image, desired physique, and exercise habits.

[1503] "Means for editing generated menus" refers to a function that allows users to review generated diet meal plans and exercise programs and make changes or additions as needed.

[1504] "Means including an emotion engine that recognizes user emotions" refers to a function that incorporates an emotion engine into the system to estimate emotions from the user's facial expressions, tone of voice, movements during exercise, etc.

[1505] "The emotion engine's method of analyzing the user's facial expressions, tone of voice, and movements during exercise to estimate emotions" refers to the function of the emotion engine that analyzes data collected through the camera and microphone to estimate the user's emotional state.

[1506] "Means of adjusting the content and tone of voice support based on emotions" refers to a function that changes the content and tone of voice support according to the user's emotional state estimated by the emotion engine, and provides encouraging messages to the user.

[1507] "Means for automatically adjusting exercise intensity" refers to a function in which the emotional engine appropriately changes the exercise intensity based on the user's emotional state, thereby reducing the user's stress and fatigue.

[1508] Modes for carrying out the invention

[1509] This invention begins with the user taking a picture of their body shape using a digital camera or smartphone camera and registering the image in the system. Next, the user selects the image that best matches their desired body shape from several body shape images provided in the system. Furthermore, the user registers their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day) in the system. Based on this information, the system uses a generative AI model to generate an optimal diet plan, exercise program, and duration for the user. The generated plan can be edited by the user as needed.

[1510] In one embodiment of the present invention, an emotion engine that recognizes the user's emotions is incorporated into the system. This emotion engine estimates emotions from the user's facial expressions, tone of voice, and movements during exercise. For example, if the user shows a pained expression during exercise, the emotion engine captures this information and determines that the user is experiencing excessive stress. This information is fed back to the system, and the content and tone of the voice support are adjusted accordingly. Specifically, the user may be provided with encouraging messages, or the intensity of the exercise may be automatically reduced.

[1511] Hardware and software to be used

[1512] This system uses the following hardware and software.

[1513] Digital camera or smartphone camera: Used to photograph the user's body shape.

[1514] Devices (PCs, smartphones, tablets, etc.): Used by users to access the system and register images and exercise habits.

[1515] Server: Stores data sent by users and generates diet menus and exercise programs using a generation AI model.

[1516] Generative AI Model: Based on the user's body shape image, desired physique, and exercise habits, it generates an optimal diet plan and exercise program.

[1517] Emotion Engine: Analyzes the user's facial expressions, voice tone, and movements during exercise to estimate their emotions.

[1518] Specific example

[1519] Example 1: Registration of body shape images

[1520] Users take photos of their body shape with their smartphone cameras and upload the images to the system. The server receives the images and saves them to the user's profile.

[1521] Specific example 2: Choosing your desired body image

[1522] The user selects their desired body shape from body shape images within the system and sends that information to the server. The server saves the selected image as the user's target.

[1523] Example 3: Registering an exercise habit

[1524] The user enters their exercise habits, such as "I jog three times a week," into the system and sends it to the server. The server then adds this information to the user's profile.

[1525] Example 4: Creating a diet menu

[1526] The server uses an AI model to generate a "one-week diet plan" based on the user's body shape image, desired physique, and exercise habits. The generated plan is sent to the user's device, and the user can edit it as needed.

[1527] Example 5: Feedback from an emotional engine

[1528] If a user shows signs of pain during exercise, the emotion engine detects this information and sends it back to the server. The server generates an encouraging message and sends it to the user's device. Additionally, the exercise intensity is automatically reduced.

[1529] Example of a prompt

[1530] "Upload a photo of your body shape and select your desired physique. Next, input your exercise habits, and we'll generate an optimal diet plan. We'll also recognize your emotions during exercise and provide feedback."

[1531] The flow of the specific processing in Example 1 will be explained using Figure 15.

[1532] Step 1:

[1533] Users take photos of their own body shape using a digital camera or smartphone camera.

[1534] Input: User's body shape image

[1535] Output: Captured body shape image file

[1536] Specific actions: The user launches the camera app and takes a photo from the front, capturing their entire body. After taking the photo, the image file is saved.

[1537] Step 2:

[1538] The user logs into the system from their device and uploads the images they have taken.

[1539] Input: Captured body shape image file

[1540] Output: Body shape image saved on the server

[1541] Specific steps: The user accesses the system's login screen, enters their user ID and password, and logs in. After logging in, they move to the image upload screen, select the saved image file, and click the upload button.

[1542] Step 3:

[1543] The server saves the received images and associates them with the user's profile.

[1544] Input: Uploaded body shape image file

[1545] Output: Body shape image associated with the user's profile

[1546] Specific operation: The server receives the uploaded image file and saves it to the database. The saved image is linked to the user ID.

[1547] Step 4:

[1548] The user views multiple body shape images provided within the system.

[1549] Input: Body shape image data within the system

[1550] Output: List of body shape images viewed by the user

[1551] Specific operation: The user accesses a body shape image selection screen and views body shape images displayed in a slideshow format.

[1552] Step 5:

[1553] Select the image that best matches the user's desired physique.

[1554] Input: User selects body shape image

[1555] Output: Selected body shape image data

[1556] Specific action: The user selects the image that best represents their desired body shape and clicks the select button.

[1557] Step 6:

[1558] The server saves the selected image as the user's target.

[1559] Input: Selected body shape image data

[1560] Output: Target body shape image saved in the user's profile

[1561] Specific operation: The server receives the selected image data and saves it to the user's profile as a target body shape image.

[1562] Step 7:

[1563] Users input their exercise habits into the system via their device.

[1564] Input: User's exercise habits (e.g., jogs 3 times a week)

[1565] Output: Input exercise habit data

[1566] Specific actions: The user accesses the exercise habit input screen and enters their exercise habits into the text box. After entering the information, they click the save button.

[1567] Step 8:

[1568] The server receives the entered information about exercise habits and adds it to the user's profile.

[1569] Input: Entered exercise habit data

[1570] Output: Exercise habit information added to the user's profile

[1571] Specific operation: The server receives the entered exercise habit data and saves it to the database. The saved data is linked to the user ID.

[1572] Step 9:

[1573] Based on the user's body shape image, desired physique, and exercise habits, the server uses a generative AI model to generate an optimal diet plan, exercise program, and duration.

[1574] Input: User's body shape image, desired body shape, exercise habit information

[1575] Output: Generated diet meal plans and exercise programs

[1576] Specific operation: The server runs the generated AI model and creates a menu based on the input data, taking into account calorie calculations and nutritional balance.

[1577] Step 10:

[1578] The server sends the generated menu to the user's terminal.

[1579] Input: Generated diet meal plans and exercise programs

[1580] Output: Menu sent to the user's terminal

[1581] Specific operation: The server sends the generated menu to the user's terminal in PDF format or similar.

[1582] Step 11:

[1583] Review the menu received by the user and edit it as needed.

[1584] Input: Generated diet meal plans and exercise programs

[1585] Output: Edited menu

[1586] Specific actions: The user accesses the menu editing screen and makes changes to ingredients or adds exercises. After editing, they click the save button.

[1587] Step 12:

[1588] The user uses the system while exercising.

[1589] Input: User's exercise data (e.g., facial expressions, voice tone, movements during exercise)

[1590] Output: Real-time collected motion data

[1591] Specific operation: The user exercises while holding a smartphone, and data is collected through the camera and microphone.

[1592] Step 13:

[1593] The emotion engine analyzes the user's facial expressions, voice tone, and movements during exercise in real time.

[1594] Input: Real-time collected exercise data

[1595] Output: Estimated user emotional state

[1596] Specific operation: The emotion engine analyzes the collected data and estimates the user's emotional state.

[1597] Step 14:

[1598] The emotion engine estimates the user's emotions, and if it determines that the user is experiencing excessive stress, it feeds that information back to the server.

[1599] Input: Estimated user's emotional state

[1600] Output: Sentiment information fed back to the server

[1601] Specific operation: The emotion engine determines the user's emotional state and, if it detects excessive stress, sends that information to the server.

[1602] Step 15:

[1603] The server adjusts the content and tone of the voice support based on feedback and generates encouraging messages for the user.

[1604] Input: Feedback on emotional information

[1605] Output: Generated encouraging message

[1606] Specific operation: The server modifies the content and tone of the voice support based on emotional information, generating messages such as, "Keep going! You're almost there!"

[1607] Step 16:

[1608] The server automatically adjusts the exercise intensity and sends the information to the user's device.

[1609] Input: Feedback on emotional information

[1610] Output: Instructions for adjusted exercise intensity

[1611] Specific operation: The server adjusts the exercise intensity appropriately based on emotional information and sends instructions to the user's device. For example, it might instruct the user to slightly slow down their exercise pace.

[1612] (Application Example 1)

[1613] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1614] Traditional diet support systems can provide personalized menus based on a user's body shape and exercise habits, but they have the drawback of not being able to provide feedback that takes into account the user's emotional state. This has led to problems such as users experiencing excessive stress or being unable to maintain motivation.

[1615] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1616] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, means for registering the user's exercise habits, means for generating a diet menu, exercises, and duration, means for editing the generated menu, means for recognizing the user's emotions, and means for providing feedback based on the user's emotional state. This makes it possible not only to provide an optimal menu based on the user's individual body shape and exercise habits, but also to provide feedback according to the emotional state.

[1617] "Method for registering a user's body shape using an image" refers to a method for users to take a picture of their own body shape using a digital camera or smartphone camera and register that image in the system.

[1618] "A method for selecting an image that closely resembles the desired body shape using a generating AI" refers to a method for selecting the image that most closely matches the user's desired body shape from multiple body shape images provided within the system, using a generating AI.

[1619] "Means for registering users' exercise habits" refers to the means by which users register their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day, etc.) in the system.

[1620] "Methods for generating diet meal plans, exercises, and durations" refers to methods for generating optimal diet meal plans, exercise programs, and their durations for a user, using a generating AI based on the user's body shape image and exercise habits.

[1621] "Means for editing generated menus" refers to means that users can edit the generated diet meal plans and exercise programs as needed.

[1622] "Means of recognizing user emotions" refers to methods for estimating emotions from the user's facial expressions, tone of voice, and movements during exercise.

[1623] "Means of providing feedback based on the user's emotional state" refers to methods for adjusting the content and tone of voice support according to the user's emotional state, providing encouraging messages to the user, or automatically reducing the intensity of exercise.

[1624] A system for carrying out this invention includes means for registering the user's body shape with images, means for selecting images that closely resemble the desired body shape using a generating AI, means for registering the user's exercise habits, means for generating a diet menu, exercises, and duration, means for editing the generated menu, means for recognizing the user's emotions, and means for providing feedback based on the user's emotional state.

[1625] Hardware and software configuration

[1626] Hardware:

[1627] Smartphone (camera, microphone, sensors)

[1628] Server (data processing, execution of generational AI models)

[1629] software:

[1630] Image processing library (OpenCV)

[1631] Emotion recognition engine (Microsoft Azure Cognitive Services)

[1632] Generative AI model (GPT-4)

[1633] Database (MySQL)

[1634] Data processing and data calculation

[1635] User body shape image registration:

[1636] Users take photos of their body shape using their smartphone cameras and upload the images to the system. The system then uses an image processing library (OpenCV) to analyze body shape data from the images.

[1637] Select an image that closely resembles the body you want to achieve:

[1638] Using a generative AI model (GPT-4), the system selects the image that most closely matches the user's desired body shape from multiple body shape images provided within the system.

[1639] Register your exercise habits:

[1640] Users input their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day) into the system as text data.

[1641] Generating diet meal plans and exercise programs:

[1642] Enter the following prompts into the generating AI model to generate an optimal diet plan and exercise program.

[1643] Based on the user's body shape image and exercise habits, please provide the following information:

[1644] 1. Body shape image that most closely matches the user's desired physique.

[1645] 2. Optimal diet meal plan (breakfast, lunch, dinner)

[1646] 3. An optimal exercise program (e.g., jogging three times a week, walking for 30 minutes daily)

[1647] 4. Recommended period

[1648] Editing the generated menu:

[1649] Users can edit the generated diet meal plans and exercise programs as needed.

[1650] Emotional recognition and feedback:

[1651] Using an emotion recognition engine (Microsoft Azure Cognitive Services), the system analyzes the user's facial expressions and tone of voice to estimate their emotional state. If the user shows signs of pain during exercise, the system captures this information and adjusts the content and tone of the voice support. Specifically, it may provide encouraging messages or automatically reduce the intensity of the exercise.

[1652] Specific example

[1653] The user launches the app and takes a picture of their body shape with their smartphone camera. The app analyzes the image and obtains the user's body shape data. The user inputs their exercise habits (e.g., jogging 3 times a week). An emotion recognition engine analyzes the user's facial expressions and tone of voice. A generative AI model generates an optimal meal plan and exercise program based on the prompt text. The app displays the generated menu to the user, who can edit it as needed. If the user shows signs of pain during exercise, the emotion engine detects this and provides feedback (e.g., displays an encouraging message, reduces the exercise intensity).

[1654] In this way, users receive individually customized diet plans and support tailored to their emotional state.

[1655] The flow of a specific process in Application Example 1 will be explained using Figure 16.

[1656] Step 1:

[1657] The user takes a picture of their own body shape using their smartphone camera.

[1658] Input: User's body shape image

[1659] Output: Body shape image data

[1660] Specific operation: The user activates the smartphone camera and takes a full-body photo. The captured image is saved within the application.

[1661] Step 2:

[1662] The device uses an image processing library (OpenCV) to analyze the body shape image.

[1663] Input: Body shape image data

[1664] Output: Body shape analysis data

[1665] Specific operation: The device uses OpenCV to analyze the image and extract the user's body shape characteristics (e.g., height, weight, body fat percentage, etc.).

[1666] Step 3:

[1667] Users input their exercise habits as text data.

[1668] Input: Exercise habit data (e.g., jogging 3 times a week)

[1669] Output: Exercise habit text data

[1670] Specific action: The user enters their exercise habits into the application's input form and presses the submit button.

[1671] Step 4:

[1672] The server uses a generation AI model (GPT-4) to select images that closely resemble the desired body shape.

[1673] Input: Body shape analysis data, exercise habit text data

[1674] Output: Image data that closely resembles the desired physique.

[1675] Specific operation: The server inputs body shape analysis data and exercise habit text data into a generation AI model based on prompt messages, and generates an image that closely resembles the desired body shape.

[1676] Step 5:

[1677] The server uses an AI model to generate meal plans and exercise programs for weight loss.

[1678] Input: Body shape analysis data, exercise habit text data, image data that closely resembles the desired physique.

[1679] Output: Diet meal plan, exercise program, recommended duration

[1680] Specific operation: The server generates the following prompt messages and inputs them into the AI ​​model to generate an optimal meal plan and exercise program.

[1681] Based on the user's body shape image and exercise habits, please provide the following information:

[1682] 1. Body shape image that most closely matches the user's desired physique.

[1683] 2. Optimal diet meal plan (breakfast, lunch, dinner)

[1684] 3. An optimal exercise program (e.g., jogging three times a week, walking for 30 minutes daily)

[1685] 4. Recommended period

[1686] Step 6:

[1687] The user reviews the generated menu and edits it as needed.

[1688] Input: Diet meal plan, exercise program, recommended duration

[1689] Output: Edited menu

[1690] Specific actions: The user reviews the menu generated within the application and edits meal plans and exercise programs as needed.

[1691] Step 7:

[1692] The device uses an emotion recognition engine (Microsoft Azure Cognitive Services) to recognize the user's emotions.

[1693] Input: User facial image, voice tone data

[1694] Output: Emotional state data

[1695] Specific operation: The device uses its camera and microphone to capture the user's facial expressions and voice tone, which are then analyzed by an emotion recognition engine.

[1696] Step 8:

[1697] The server provides feedback based on the user's emotional state.

[1698] Input: Emotional state data

[1699] Output: Feedback message, exercise intensity adjustment

[1700] Specific operation: The server generates encouraging messages for the user based on emotional state data and automatically reduces the exercise intensity as needed.

[1701] (Example 2)

[1702] Next, we will describe Example 2 of the morphological example. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1703] Conventional exercise support systems have struggled to provide personalized exercise programs that fully consider users' exercise habits and emotional states. Furthermore, the lack of real-time voice support during exercise and the absence of emotionally-based feedback made it difficult for users to maintain their motivation to exercise consistently.

[1704] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1705] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, means for registering the user's exercise habits, means for generating a diet menu, exercises, and duration, means for editing the generated menu, means for collecting the user's exercise data, means for analyzing the collected exercise data, means for collecting the user's emotional data, means for analyzing the collected emotional data and adjusting the exercise program, and means for giving voice instructions for the exercise menu. This makes it possible to provide an exercise program based on the user's individual exercise habits and emotional state, and helps the user maintain motivation to exercise continuously.

[1706] "Means of registering a user's body shape using images" refers to devices or software that record the shape and size of a user's body as image data.

[1707] "Generative AI" refers to a system that uses artificial intelligence technology to generate data and information tailored to specific purposes.

[1708] "Means for selecting images that closely resemble the desired physique" refers to interfaces and algorithms that allow users to select images that closely match their target body image.

[1709] "Means for registering a user's exercise habits" refers to devices or software used to record a user's daily exercise patterns and frequency.

[1710] "Means for generating diet meal plans, exercises, and timelines" refers to a system that automatically creates meal plans and exercise programs tailored to the user's goals.

[1711] "Means for editing generated menus" refers to an interface that allows users to manually modify and adjust automatically generated meal plans and exercise programs.

[1712] "Means for collecting user exercise data" refers to devices or software used to record data such as the type, number of repetitions, and intensity of exercise performed by the user.

[1713] "Means for analyzing collected exercise data" refers to algorithms and systems used to analyze collected exercise data and evaluate the user's exercise performance and progress.

[1714] "Means of collecting user emotional data" refers to devices and software that record emotional states from users' facial expressions and voice.

[1715] "Means for analyzing collected emotional data and adjusting exercise programs" refers to algorithms and systems that analyze collected emotional data and optimize exercise programs according to the user's emotional state.

[1716] "Means of providing voice instructions for exercise menus" refers to devices or software that provide voice guidance to users on the next exercise menu they should perform.

[1717] This invention is a system that provides voice support to users while they exercise and further adjusts the exercise program based on the user's emotional state. Specific embodiments of this system are described below.

[1718] First, the user launches an exercise app using their smartphone or wearable device. The smartphone's camera or the wearable device's sensors are used to register the user's body shape as images. This records the user's body shape and size as image data.

[1719] Next, the user selects an image that closely resembles the body they aspire to achieve using a generative AI. The generative AI model generates images that closely match the user's desired body image, allowing the user to choose from among them.

[1720] To register users' exercise habits, applications on smartphones or wearable devices are used. This records the user's daily exercise patterns and frequency.

[1721] To generate diet meal plans, exercise routines, and timelines, the server uses data analysis tools such as Python and TensorFlow. This automatically creates meal plans and exercise programs tailored to the user's goals.

[1722] To edit the generated menus, users use a smartphone application. This allows users to manually modify and adjust the automatically generated meal plans and exercise programs.

[1723] Sensors from smartphones and wearable devices are used to collect user exercise data. For example, accelerometers and heart rate monitors are used to record data such as the type, frequency, and intensity of exercise.

[1724] To analyze the collected exercise data, the server uses Python and TensorFlow. This allows the collected exercise data to be analyzed and the user's exercise performance and progress to be evaluated.

[1725] The smartphone's camera and microphone are used to collect user emotion data. This records the user's emotional state from their facial expressions and voice.

[1726] The server uses an emotion engine to analyze collected emotional data and adjust the exercise program. This allows the collected emotional data to be analyzed, and the exercise program to be optimized according to the user's emotional state.

[1727] To provide voice instructions for exercise routines, the device uses the Google Text-to-Speech API. This allows the user to receive voice guidance on the next exercise they should perform.

[1728] Specific example:

[1729] The user launches a strength training app on their smartphone. The app gives a voice command saying, "Next, do 20 sit-ups." After the user performs the sit-ups, the device uses its accelerometer to record the number of repetitions. The server analyzes this data and determines the next exercise to do, "Next, do 15 squats," and sends this command to the device. The device reads this command aloud. If the user smiles after the exercise, the device uses its camera to record the expression and sends it to the server. The server analyzes this data and incorporates sit-ups more frequently into future exercise programs.

[1730] Example of a prompt:

[1731] "Please create a program that provides voice instructions for the next exercise a user should do during their strength training. For example, it should say, 'Next, do 20 sit-ups.' Also, analyze the user's emotional state and, if they show a joyful expression, frequently incorporate that exercise into future programs."

[1732] The flow of the specific processing in Example 2 will be explained using Figure 17.

[1733] Step 1:

[1734] The user starts exercising.

[1735] Input: The user launches an exercise app using a smartphone or wearable device.

[1736] Specific action: The user taps the app on their smartphone and selects "Strength Training".

[1737] Output: A signal to initiate movement is sent to the terminal.

[1738] Step 2:

[1739] The device provides voice instructions for the exercise routine.

[1740] Input: The signal to start exercise and the exercise menu sent from the server.

[1741] Specific action: The device uses the Google Text-to-Speech API to read aloud, "Next, do 20 sit-ups."

[1742] Output: Recognizes the exercise routine the user should perform next.

[1743] Step 3:

[1744] The user performs exercise.

[1745] Input: Voice commands from the device.

[1746] Specific action: The user performs 20 sit-ups.

[1747] Output: Performing exercise.

[1748] Step 4:

[1749] The device collects the user's exercise data.

[1750] Input: User's exercise performance.

[1751] Specific operation: The device uses an accelerometer and heart rate monitor to record the number of exercises and their intensity.

[1752] Output: Exercise data is saved to the device.

[1753] Step 5:

[1754] The server analyzes the exercise data.

[1755] Input: Exercise data transmitted from the device.

[1756] Specific operation: The server uses Python and TensorFlow to analyze exercise data and evaluate the user's exercise performance.

[1757] Output: Analysis results are generated.

[1758] Step 6:

[1759] The server determines the next exercise menu and sends it to the terminal.

[1760] Input: Analysis results of exercise data.

[1761] Specific operation: Based on the analysis results, the server determines the next exercise to perform and sends that information to the terminal. For example, it might decide, "Next, please do 15 squats."

[1762] Output: The next exercise menu will be sent to the terminal.

[1763] Step 7:

[1764] The device will provide voice instructions for the next exercise routine.

[1765] Input: The following exercise menu sent from the server.

[1766] Specific action: The device will again use the Google Text-to-Speech API to read aloud, "Now, do 15 squats."

[1767] Output: Recognizes the exercise routine the user should perform next.

[1768] Step 8:

[1769] The user finishes their exercise.

[1770] Input: User indicates their intention to end the exercise.

[1771] Specific action: The user taps the "Exit" button in the app.

[1772] Output: A signal indicating the end of the exercise is sent to the terminal.

[1773] Step 9:

[1774] The device collects user emotion data.

[1775] Input: Signal to end exercise.

[1776] Specific operation: The device uses its camera and microphone to collect emotional data from the user's facial expressions and voice.

[1777] Output: Emotional data is saved to the device.

[1778] Step 10:

[1779] The server analyzes emotional data and adjusts the exercise program accordingly.

[1780] Input: Emotional data sent from the device.

[1781] Specific operation: The server uses an emotion engine to analyze the collected emotion data and optimize the exercise program according to the user's emotional state.

[1782] Output: A modified exercise program is generated.

[1783] (Application Example 2)

[1784] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1785] Conventional exercise and diet support systems only provide menus based on the user's exercise habits and body shape, but they lack consideration for the user's emotional state when suggesting meal plans and integration with delivery services for the suggested meals. Therefore, there is a challenge in that it is difficult for users to maintain the motivation to continue exercising and eating consistently.

[1786] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1787] In this invention, the server includes means for registering the user's body shape as an image, means for selecting an image that closely resembles the desired body shape using a generation AI, means for registering the user's exercise habits, means for generating a diet meal plan, exercise routines, and duration, means for editing the generated menu, means for analyzing the user's emotional state, means for suggesting a meal plan based on the emotional state, and means for delivering the suggested meal plan in cooperation with a delivery service. This enables the suggestion of an optimal meal plan according to the user's emotional state and the rapid delivery of that menu.

[1788] "Means for registering a user's body shape using images" refers to devices or software for recording a user's body shape and dimensions as image data.

[1789] "A method for selecting images that closely resemble the desired body using generation AI" refers to a device or software that uses artificial intelligence technology to select images of the body that the user is aiming for.

[1790] "Means for registering a user's exercise habits" refers to devices or software used to record the type and frequency of a user's daily exercise.

[1791] "Means for generating diet meal plans, exercises, and durations" refers to devices or software that automatically create appropriate meal plans, exercise programs, and durations based on the user's goals and current condition.

[1792] "Means for editing generated menus" refers to devices or software that allow users to manually modify and adjust generated meal plans and exercise programs.

[1793] "Means for analyzing a user's emotional state" refers to devices or software that analyze a user's facial expressions and feedback to determine their emotional state.

[1794] "Means for suggesting meal menus based on emotional state" refers to devices or software that suggest the optimal meal menu considering the user's emotional state.

[1795] "Methods for delivering suggested meal menus in conjunction with delivery services" refers to devices or software that deliver suggested meal menus to users in conjunction with delivery services.

[1796] A system for carrying out this invention includes means for registering the user's body shape with an image, means for selecting an image that closely resembles the desired body shape using a generating AI, means for registering the user's exercise habits, means for generating a diet meal menu, exercises, and duration, means for editing the generated menu, means for analyzing the user's emotional state, means for suggesting a meal menu based on the emotional state, and means for delivering the suggested meal menu in cooperation with a delivery service.

[1797] Hardware and software to be used

[1798] hardware

[1799] smartphone

[1800] server

[1801] Delivery service terminal

[1802] software

[1803] Python

[1804] Speech recognition libraries (e.g., Google Speech-to-Text)

[1805] Sentiment analysis libraries (e.g., IBM Watson)

[1806] Generative AI models (e.g., GPT-3)

[1807] Data processing and data calculation

[1808] A method for registering a user's body shape using an image.

[1809] The user uses their smartphone camera to take a picture of their body shape and sends the image data to the server. The server stores this image data and uses it for later analysis.

[1810] A method of selecting images that closely resemble the desired body shape using generation AI.

[1811] The server uses a generative AI model to generate images of the body the user desires. The user then selects their target body from the generated images through a smartphone application.

[1812] A means of registering a user's exercise habits.

[1813] Users use a smartphone application to input the type and frequency of their daily exercise. This data is sent to and stored on a server.

[1814] Means for generating diet meal plans, exercise routines, and timeframes.

[1815] The server generates appropriate meal plans, exercise programs, and durations based on the user's body shape data, target body image, and exercise habit data. These plans are automatically created using a generation AI model.

[1816] Means for editing the generated menu

[1817] Users can manually modify and adjust the generated meal plans and exercise programs using a smartphone application. The edited data is stored on the server.

[1818] A means of analyzing a user's emotional state

[1819] The system collects user feedback after exercise or meals using the smartphone's camera and microphone. The server uses an emotion analysis library to analyze this feedback and determine the user's emotional state.

[1820] A method for suggesting meal menus based on emotional state

[1821] Based on the results of the emotion analysis, the server suggests the optimal meal menu tailored to the user's emotional state. It uses a generative AI model to automatically generate the suggested menu.

[1822] A method of delivering the suggested meal menu in cooperation with a delivery service.

[1823] The server sends the suggested meal menu to the delivery service terminal and instructs it to deliver it to the user. The delivery service then promptly delivers the suggested menu.

[1824] Examples of specific cases and prompt statements

[1825] Specific example

[1826] When a user is running, the system will notify them by voice if their current pace is not enough to reach their target time and will suggest increasing their pace.

[1827] If a user gives feedback that they "had fun" after exercising, the emotion engine will determine that they are "happy" and suggest a salad and grilled chicken.

[1828] Example of a prompt

[1829] "When a user is running, analyze their current pace and suggest increasing their pace if they are not reaching their target time."

[1830] "Analyze user feedback after exercise and suggest appropriate meal plans based on their emotional state."

[1831] The above describes the embodiments for carrying out this invention.

[1832] The flow of a specific process in Application Example 2 will be explained using Figure 18.

[1833] Step 1:

[1834] The user uses their smartphone camera to take a picture of their body shape and sends the image data to the server. The server stores this image data and uses it for later analysis. The input is the user's body shape image, and the output is the image data stored on the server.

[1835] Step 2:

[1836] The server uses a generative AI model to generate images of the body the user desires. The user selects their target body from the generated images via a smartphone application. The input is the user's body shape image and the characteristics of the target body, and the output is the image of the target body selected by the user.

[1837] Step 3:

[1838] Users input the type and frequency of their daily exercise using a smartphone application. This data is sent to and stored on a server. The input is the user's exercise habit data, and the output is the exercise habit data stored on the server.

[1839] Step 4:

[1840] The server generates appropriate meal plans, exercise programs, and durations based on the user's body shape data, target body image, and exercise habit data. These plans are automatically created using a generation AI model. The input is the user's body shape data, target body image, and exercise habit data, and the output is the generated meal plan and exercise program.

[1841] Step 5:

[1842] Users can manually modify and adjust the generated meal plans and exercise programs using a smartphone application. The edited data is saved on the server. The input is the user's edits, and the output is the edited menu saved on the server.

[1843] Step 6:

[1844] The system collects user feedback after exercise or meals using the smartphone's camera and microphone. A server uses an emotion analysis library to analyze this feedback and determine the user's emotional state. The input is the user's feedback data, and the output is the analyzed emotional state.

[1845] Step 7:

[1846] The server suggests the optimal meal menu based on the user's emotional state, using the results of the emotion analysis. A generative AI model is used to automatically generate the suggestions. The input is the analyzed emotional state, and the output is the suggested meal menu.

[1847] Step 8:

[1848] The server sends the suggested meal menu to the delivery service terminal and instructs it to deliver it to the user. The delivery service quickly delivers the suggested menu. The input is the suggested meal menu, and the output is the meal delivered to the user.

[1849] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1850] The data generation model 58 is a form of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1851] Other examples of generative AI include Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are some examples.

[1852] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1853] [Fourth Embodiment]

[1854] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1855] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1856] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1857] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1858] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1859] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1860] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1861] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1862] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1863] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1864] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1865] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1866] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[1867] "Example of form 1"

[1868] The system of this invention allows the user to take a picture of their body shape using a digital camera or smartphone camera and register the image in the system. Next, the system selects the image that best matches the user's desired body shape from a set of body shape images provided within the system. Furthermore, the user registers their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day) in the system. Based on this information, the system uses a generative AI to generate an optimal diet plan, exercise program, and duration for the user. The generated menu can be edited by the user as needed.

[1869] "Example of form 2"

[1870] The system of this invention provides voice support when a user is exercising. For example, when performing strength training, it reads out the next exercise to be done (e.g., "Next, please do 20 sit-ups"). When running, it checks the user's pace and suggests increasing or decreasing the pace as needed (e.g., "At your current pace, you may not reach your target time. Try increasing your pace a little"). This enables the user to continue exercising appropriately.

[1871] The following describes the processing flow for each example of the form.

[1872] "Example of form 1"

[1873] Step 1: The user takes a picture of their body shape using a digital camera or smartphone camera. The captured image is uploaded to the system.

[1874] Step 2: From the multiple body shape images provided within the system, the user selects the one that best matches their desired body shape. This selection is made through the system's user interface.

[1875] Step 3: The user registers their exercise habits (e.g., jogging three times a week, walking for 30 minutes every day) in the system. This registration is also done through the system's user interface.

[1876] Step 4: Based on this information, the system uses AI to generate an optimal diet plan, exercise program, and duration for the user. The generated plan can be edited by the user as needed.

[1877] "Example of form 2"

[1878] Step 1: When the user starts exercising, the system provides voice support.

[1879] Step 2: When performing strength training, the system will announce the next exercise to do (e.g., "Next, do 20 sit-ups").

[1880] Step 3: When running, the system monitors the user's pace and suggests increasing or decreasing the pace as needed (e.g., "At your current pace, you may not reach your target time. Try increasing your pace a little").

[1881] (Example 1)

[1882] Next, we will describe Embodiment 1 of Example Form 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1883] Traditional diet systems have struggled to generate personalized diet plans based on users' body types and exercise habits. Furthermore, the lack of editing capabilities for the generated plans and insufficient support during exercise made it difficult for users to consistently follow their diets.

[1884] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1885] In this invention, the server includes means for registering the user's body shape as an image, means for presenting multiple body shape images and allowing the user to select their desired body shape, means for registering the user's exercise habits, means for generating a diet meal plan, exercise program, and duration using an AI model based on the user's body shape image, desired body shape, and exercise habit information, and means for editing the generated menu. This makes it possible to generate and edit a diet plan tailored to the user's individual needs.

[1886] "Method for registering user body shape with images" refers to a function that allows users to take pictures of their own body shape using a digital camera or smartphone camera and upload those images to the system.

[1887] "A means of presenting multiple body shape images and allowing the user to select the body shape they aspire to" refers to a function that presents the user with multiple body shape images prepared within the system and allows the user to select the one that most closely matches their desired body shape.

[1888] "Means for registering users' exercise habits" refers to a function that allows users to input their own exercise habits (e.g., jogging three times a week, walking for 30 minutes every day, etc.) into the system and register that information.

[1889] "A means of generating diet meal plans, exercise programs, and their duration using a generative AI model" refers to a function that uses a generative AI model to generate optimal diet meal plans, exercise programs, and their duration based on the user's body shape image, desired physique, and exercise habits.

[1890] "Means for editing generated menus" refers to a function that allows users to review the generated diet plan and edit meal menus and exercise programs as needed.

[1891] This invention begins with a user taking a picture of their own body shape using a digital camera or smartphone camera and registering that image in the system. The user launches the camera app on their smartphone and takes a picture of their body shape. The captured image is uploaded from the terminal to the system, and the server receives the uploaded image and saves it to a database. At this time, the image quality is checked using an image processing library (e.g., OpenCV).

[1892] Next, the server presents the user with several body shape images prepared within the system. The user selects the image that most closely matches their desired body shape from the presented images. The ID of the selected body shape image is sent from the terminal to the server, and the server stores that ID in its database.

[1893] Users input their exercise habits from their device. For example, they might enter information such as "I jog three times a week" or "I walk for 30 minutes every day." The entered exercise habit data is sent from the device to the server, which stores that data in a database.

[1894] The server obtains information on the user's body shape image, desired physique, and exercise habits, and prompts a generation AI model (e.g., GPT-4) to generate a diet plan. An example of a prompt is, "Based on the user's body shape image, desired physique, and exercise habits, please generate an optimal diet meal plan, exercise program, and duration." The generation AI model generates the optimal meal plan, exercise program, and duration based on the prompt. The generated diet plan is then saved to a database by the server.

[1895] Users can view the generated diet plan on their device and edit the meal menu and exercise program as needed. For example, they can change breakfast from oatmeal to yogurt. The edited information is sent from the device to the server, which then updates the database.

[1896] In this way, the system allows users to easily create a diet plan tailored to their own goals and edit it as needed.

[1897] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1898] Ste...

Claims

[Claim 1] A means of recording the user's current body shape as image data, A means for presenting to the user, in a selectable manner, an image of the body the user desires based on the aforementioned image data, A means for registering information about the user's exercise habits, A means for generating information about the duration for which the user will continue to follow at least one of the diet meal plan and exercise program, based on information about the user's current body shape, the image, and the exercise habits of the user, means for collecting the user's exercise data, A means for analyzing the collected exercise data, including the user's pace and heart rate during running, in real time by comparing it with the generated exercise program, and generating a prompt message that, as a result of the analysis, reads out the type and number of repetitions of the next menu item to be performed, or suggests increasing or decreasing the pace to achieve the target time based on the results of checking the pace distribution during the run in real time, A means for generating voice instructions using the generated prompt sentence and the generating AI model, means for playing the generated voice instruction on the user's terminal, Includes, The means for presenting the aforementioned images to the user in a selectable manner is: A system that generates an image using a prompt message that generates a personalized target body shape image that reflects the detailed physical characteristics of the body, such as body fat percentage, muscle mass, and the shape of specific body parts, and a generation AI model, and presents the generated image to the user for selection.

Images