System

Abstract

A system is provided.SOLUTION: A system comprising: photographing means for photographing a play-video; uploading means for uploading the play-video photographed by the photographing means to a server; saving means for saving the uploaded video; analyzing means for analyzing the saved video; advice generating means for generating play improvement advice based on a result of the analysis; transmitting means for transmitting the generated advice to a terminal; and displaying means for displaying the transmitted advice.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditionally, improving sports performance and managing nutrition have required specialized knowledge and experience, making it difficult for students and amateur athletes in particular to receive direct instruction from top coaches. It has also been difficult to provide customized advice tailored to each athlete's individual characteristics. There is a need for an innovative system that can solve these issues and provide optimal advice and nutritional guidance for each individual athlete. [Means for solving the problem]

[0005] The present invention provides a system including a filming means for filming gameplay videos, an uploading means for uploading the gameplay videos filmed by the filming means to a server, a storage means for saving the uploaded videos, an analysis means for analyzing the saved videos, an advice generation means for generating gameplay improvement advice based on the analysis results, a transmission means for transmitting the generated advice to a terminal, and a display means for displaying the transmitted advice. The present invention also provides a system further including an input means for inputting nutritional management information, an information transmission means for transmitting the input nutritional management information to a server, an evaluation means for evaluating the transmitted nutritional management information, a nutritional advice generation means for generating nutritional advice based on the evaluation results, a nutritional advice transmission means for transmitting the generated nutritional advice to a terminal, and a display means for displaying the transmitted nutritional advice. The present invention also provides a system in which the gameplay improvement advice is generated by referencing multiple method databases. In this way, it is possible to provide specialized advice and nutritional guidance optimized for each individual player.

[0006] The "filming means" is a device for recording sports play as video.

[0007] The "uploading means" is a function for transmitting the video recorded by the imaging means to the server.

[0008] "Storage means" is a function for storing uploaded videos in a server.

[0009] The "analysis means" is a function for analyzing the content of the saved video and evaluating the play actions.

[0010] The "advice generation means" is a function for generating advice for improving play based on the analysis results obtained by the analysis means.

[0011] The "transmission means" is a function for transmitting the generated advice to the terminal.

[0012] The "display means" is a function for displaying the transmitted advice on the terminal.

[0013] "Input means" is a function that allows the user to input nutritional management information.

[0014] The "information transmission means" is a function for transmitting the input nutrition management information to the server.

[0015] The "evaluation means" is a function for evaluating the transmitted nutritional management information and analyzing the nutritional balance.

[0016] The "nutritional advice generating means" is a function for generating advice for nutritional management based on the evaluation results obtained by the evaluation means.

[0017] The "nutrition advice sending means" is a function for sending the generated nutrition advice to the terminal.

[0018] The "method database" is a database that includes methods used by professional athletes and famous coaches. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention is an innovative system for supporting the improvement of sports play and nutritional management. The specific operation of this system will be described below.

[0041] Overall system configuration

[0042] This system consists of a server, a terminal, and a user. The specific role of each element is explained below.

[0043] User Actions

[0044] 1. A user uses a device (e.g., a smartphone or video camera) to record their sports play.

[0045] 2. Users upload the gameplay videos they have taken using a dedicated application on their device.

[0046] 3. The user uses a dedicated application to enter nutritional management information (e.g., dietary details, weight, and exercise amount).

[0047] Device behavior

[0048] 1. The device receives the gameplay video provided by the user, converts it into the appropriate format, compresses it if necessary, and sends it to the server.

[0049] 2. The terminal receives the nutritional management information provided by the user, converts and formats it, and then sends it to the server.

[0050] 3. The terminal receives the advice (play improvement advice and nutrition advice) sent from the server and displays it through the user interface.

[0051] Server Operation

[0052] 1. The server stores the gameplay video sent from the device.

[0053] 2. The server analyzes the saved gameplay video using analytical means (AI model) and evaluates the movement.

[0054] 3. The server generates advice for improving play based on the analysis results. The quality of the generated advice is improved by referencing multiple method databases.

[0055] 4. The server sends the generated play improvement advice to the device.

[0056] 5. The server stores the nutritional management information sent from the terminal and evaluates the nutritional balance using an evaluation method (AI model).

[0057] 6. The server generates nutrition advice based on the evaluation results.

[0058] 7. The server sends the generated nutrition advice to the terminal.

[0059] Specific examples

[0060] Analyzing gameplay videos and providing advice on improvements

[0061] 1. The user uses a smartphone to film their basketball shooting practice, then uploads the video to a dedicated application.

[0062] 2. The device receives the video file, converts the file format, compresses it, and then sends it to the server.

[0063] 3. The server receives the video and stores it in a database. The stored video is then analyzed by an analytical method (AI model).

[0064] 4. Based on the analysis results, the server identifies specific areas for improvement, such as "the wrist angle when shooting is insufficient."

[0065] 5. The server compares the method with that of a professional player and generates custom advice such as "Keep your wrist at a 45-degree angle."

[0066] 6. The server sends the generated advice to the terminal, which displays the advice on the user interface.

[0067] 7. The user checks the advice and puts it into practice during their next shooting practice session.

[0068] Entering nutritional management information and providing advice

[0069] 1. The user enters the contents of their breakfast (e.g., oatmeal, banana, yogurt) in a dedicated application.

[0070] 2. The terminal receives the input information, converts and formats it, and sends it to the server.

[0071] 3. The server receives the nutrition management information and stores it in a database.

[0072] 4. The server uses an evaluation tool (AI model) to assess nutritional balance and identify specific areas for improvement, such as "you need more protein at breakfast."

[0073] 5. The server generates a custom advice such as "Add a protein shake to breakfast."

[0074] 6. The server sends the generated nutrition advice to the terminal, which displays the advice on the user interface.

[0075] 7. The user confirms the advice and applies it to their next meal.

[0076] In this way, the present invention provides users with personalized sports play improvement and nutritional management advice to help improve overall performance.

[0077] The processing flow will be explained below.

[0078] Step 1:

[0079] Users use devices such as smartphones and tablets to record video of sports play.

[0080] Step 2:

[0081] Users upload the videos they have taken from their device to the server using a dedicated application.

[0082] Step 3:

[0083] The device receives the video file and converts or compresses the file format as needed.

[0084] Step 4:

[0085] The device then sends the converted and compressed video file to the server.

[0086] Step 5:

[0087] The server saves the received video file in storage.

[0088] Step 6:

[0089] The server inputs the saved video into an analysis tool (AI model) and begins analyzing the movements within the video.

[0090] Step 7:

[0091] The server's analytics evaluates your movements and identifies specific areas for improvement (e.g., "Your wrist angle when shooting is insufficient").

[0092] Step 8:

[0093] The server references a database of methods from professional athletes and famous coaches and generates custom advice based on the analysis results.

[0094] Step 9:

[0095] The server then sends the generated advice to the device to improve your gameplay.

[0096] Step 10:

[0097] The device formats the received advice for display in a user interface.

[0098] Step 11:

[0099] The device will then display the formatted advice to the user.

[0100] Step 12:

[0101] The user can review the advice provided via the device and put it into practice the next time they practice.

[0102] Step 13:

[0103] Users use a dedicated application to enter nutritional management information (e.g., dietary content, weight, and exercise volume).

[0104] Step 14:

[0105] The terminal receives the entered nutritional management information, formats it, and sends it to the server.

[0106] Step 15:

[0107] The server receives the nutritional management information sent and stores it in a database.

[0108] Step 16:

[0109] The server uses an evaluation tool (AI model) to assess nutritional balance and extract specific nutritional areas for improvement (e.g., "lack of protein").

[0110] Step 17:

[0111] The server then references a nutrition database and generates custom nutrition advice based on the assessment results.

[0112] Step 18:

[0113] The server then sends the generated nutrition advice to the device.

[0114] Step 19:

[0115] The device then formats the received nutrition advice for display in a user interface.

[0116] Step 20:

[0117] The device will then display formatted nutrition advice to the user.

[0118] Step 21:

[0119] Users can review the nutritional advice provided via their device and incorporate it into their next meal plan.

[0120] Through these steps, the system can provide users with comprehensive and automated personalized sports play improvement and nutritional management advice.

[0121] Example 1

[0122] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0123] Conventional sports performance improvement and nutrition management systems have limitations in analyzing data and providing appropriate advice to individual users. Furthermore, analysis of gameplay videos and evaluation of nutrition management information are often done manually, resulting in insufficient automation and efficiency. This makes it difficult for users to obtain effective improvement measures in a timely manner, making it difficult to improve overall performance.

[0124] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0125] In this invention, the server includes: a filming means for filming gameplay videos; an uploading means for uploading the gameplay videos filmed by the filming means via a dedicated application on the terminal; a transmitting means for receiving the uploaded videos at the terminal, converting the file format and compressing them, and transmitting them to the server; a storing means for saving the transmitted videos on the server; an analyzing means for analyzing the saved videos using an AI model; an advice generating means for generating gameplay improvement advice based on the analysis results; a play advice sending means for sending the generated gameplay improvement advice to the terminal; a display means for displaying the transmitted gameplay improvement advice on a terminal user interface; an input means for inputting nutritional management information; a nutritional information sending means for receiving the input nutritional management information at the terminal, shaping and formatting the information, and transmitting it to the server; a nutrition saving means for saving the transmitted nutritional management information on the server; an evaluating means for evaluating the stored nutritional management information using an AI model; a nutritional advice generating means for generating nutritional advice based on the evaluation results; a nutritional advice sending means for sending the generated nutritional advice to the terminal; and a display means for displaying the transmitted nutritional advice on the terminal user interface. This will enable the analysis of individual users' sports play and nutritional management to be automated, enabling the provision of quick and effective advice on improvement.

[0126] A "play video" is a video file in which a user captures a sports play.

[0127] "Filming means" refers to devices or methods for filming sports play, specifically smartphones and video cameras.

[0128] The "uploading means" is a means for transmitting the captured gameplay video to the server, and is a function or process that is performed through a dedicated application.

[0129] "Transmission means" refers to a method for transferring data from a user terminal to a server, and includes a POST request using the HTTP protocol.

[0130] "Storage" means a system or method for retaining transmitted data, including the use of a database or storage system.

[0131] "Analysis means" refers to the processes and tools used to analyze stored data, specifically the use of AI models.

[0132] The "advice generation means" is a process for creating improvement advice for the user based on the analysis results.

[0133] The "play advice sending means" refers to a function for sending the generated improvement advice to the user terminal.

[0134] The "display means" is an interface for visually presenting information to the user on the terminal.

[0135] "Input means" refers to a method by which a user inputs nutritional management information into the system, such as a form in a dedicated application.

[0136] "Nutrition information transmission means" refers to a function for transmitting input nutrition management information to a server.

[0137] "Nutrition storage means" refers to a system or method for storing transmitted nutrition management information.

[0138] "Evaluation methods" are processes and tools for analyzing and evaluating stored nutrition management information, and use AI models.

[0139] "Nutrition advice generator" refers to a process that generates nutrition advice based on the evaluation results.

[0140] The term "nutrition advice sending means" refers to a function for sending the generated nutrition advice to the user terminal.

[0141] This invention is an innovative system for supporting sports performance improvement and nutritional management. This system is composed of a server, terminals, and users, and each element works in cooperation with each other.

[0142] User Actions

[0143] First, the user films their own sports play (e.g., a basketball shot) using a smartphone or video camera. Next, they upload the video to their device using a dedicated application. They also use the same application to enter their daily nutrition management information (e.g., dietary details, weight, and exercise volume).

[0144] Device behavior

[0145] The device receives gameplay videos uploaded by users, converts the video files to the appropriate format using tools such as FFmpeg, and compresses them by adjusting the bitrate and resolution. It then sends the videos to the server. It also receives nutritional management information entered by the user and formats the data into JSON format. It then sends the formatted nutritional information to the server. It also receives gameplay improvement advice and nutritional advice sent from the server and displays them on the user interface.

[0146] Server Operation

[0147] The server receives gameplay videos sent from the device and stores them in a database. For example, a storage system dedicated to video files (e.g., Amazon S3) can be used. The stored videos are input into an AI analysis model (e.g., OpenPose or MediaPipe) for analysis and evaluation of the movements. Based on the analysis results, advice is created to improve the user's gameplay. This advice is optimized by referencing multiple method databases. The generated advice is then sent to the device and presented to the user.

[0148] The server receives the nutritional management information sent from the device and stores it in a database. The stored nutritional information is input into an AI model (e.g., TensorFlow) for analysis and an evaluation of nutritional balance is performed. Based on the evaluation results, specific nutritional advice is generated and sent to the device for presentation to the user.

[0149] Specific examples

[0150] Analyzing gameplay videos and providing advice on improvements

[0151] For example, a user uses a smartphone to film basketball shooting practice and uploads the video to a dedicated application. The device receives the video file, converts the file format, compresses it, and sends it to a server. The server then receives the video and stores it in a database. The saved video is analyzed using an AI model (e.g., OpenPose) to identify specific areas for improvement, such as "your wrist angle when shooting is insufficient." Based on the analysis results, custom advice is generated, such as "keep your wrist at a 45-degree angle." The server then sends the generated advice to the device, which displays it on the user interface. The user then checks the advice and puts it into practice during their next shooting practice.

[0152] Entering nutritional management information and providing advice

[0153] For example, a user uses a dedicated application to input the contents of their breakfast (e.g., oatmeal, banana, yogurt). The device receives the input information, converts and formats it, and sends it to the server. The server receives the information and stores it in a database. The stored information is input into an AI model for analysis, and specific areas for improvement, such as "You need more protein for breakfast," are identified. Based on the results, custom advice, such as "Add a protein shake to breakfast," is generated, and the server sends the advice to the device. The device displays the advice on the user interface, and the user can incorporate it into their next meal.

[0154] Examples of prompt statements

[0155] An example of a prompt sentence generated by a generative AI model could be written as follows:

[0156] "What is the optimal wrist angle when shooting a basketball?"

[0157] "What foods should I add to my breakfast to improve its nutritional balance?"

[0158] In this way, the present invention provides useful advice to users in sports play and nutritional management, and helps improve overall performance.

[0159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0160] Step 1:

[0161] Users can film their own sports play using a smartphone or video camera and then upload the recorded video files using a dedicated application.

[0162] Input: Recorded video file

[0163] Output: Video file for upload

[0164] Specific operation: The user opens the dedicated application, selects the recorded video file, and presses the upload button.

[0165] Step 2:

[0166] The device receives the gameplay video file uploaded by the user, converts the video file format to MP4 format using a tool such as FFmpeg, and compresses it if necessary.

[0167] Input: User uploaded video file

[0168] Output: Converted and compressed video file

[0169] Specific operation: The device receives the video file, converts the format using FFmpeg, and compresses it by adjusting the bitrate and resolution.

[0170] Step 3:

[0171] The device sends the converted and compressed video file to the server.

[0172] Input: Converted and compressed video files

[0173] Output: Video file sent to the server

[0174] Specific behavior: Sends the file to the server using a POST request using the HTTP protocol.

[0175] Step 4:

[0176] The server receives the video file sent from the terminal and stores the video file in a database.

[0177] Input: Video file sent to the server

[0178] Output: Video files stored in a database

[0179] Specific behavior: Receives video files and stores them in a database (e.g., Amazon S3) along with associated metadata (e.g., user ID, timestamp).

[0180] Step 5:

[0181] The server analyzes the stored video files using an AI analysis model (e.g., OpenPose, MediaPipe) and evaluates the movements.

[0182] Input: Video files stored in the database

[0183] Output: Analysis results (motion evaluation data)

[0184] Specific operation: The saved video file is input into the AI ​​model, and the characteristics of the user's movements (e.g., wrist angle, body posture) are extracted.

[0185] Step 6:

[0186] The server generates advice for improving play based on the analysis results, and this advice is optimized by referencing multiple method databases.

[0187] Input: Analysis results (motion evaluation data)

[0188] Output: Advice for improving your gameplay

[0189] Specific operation: Based on the analysis results, automatically generated advice is compared with the user profile and method database to optimize the system.

[0190] Step 7:

[0191] The server transmits the generated play improvement advice to the terminal.

[0192] Input: Advice for improving your game

[0193] Output: Send advice to terminal

[0194] Specific operation: The generated advice is sent to the device using a RESTful API or WebSocket.

[0195] Step 8:

[0196] The terminal receives the play improvement advice sent from the server and displays it on the user interface.

[0197] Input: Advice sent by the server

[0198] Output: Advice displayed in the user interface

[0199] Specific behavior: The received advice is visually presented to the user in a user interface within the application.

[0200] Step 9:

[0201] The user uses a dedicated application to input nutritional management information (e.g., dietary content, weight, amount of exercise).

[0202] Input: Nutritional management information (dietary content, weight, exercise amount)

[0203] Output: Nutritional management information entered into the application

[0204] Specific operation: Enter diet and exercise data using the input form in the dedicated application.

[0205] Step 10:

[0206] The terminal receives nutritional management information entered by the user, formats the data into JSON format, and sends it to the server.

[0207] Input: Nutritional management information entered by the user

[0208] Output: Formatted data sent to the server

[0209] Specific operation: Converts input data into JSON format and sends it to the server via an HTTP POST request.

[0210] Step 11:

[0211] The server receives the nutritional management information sent from the terminal and stores it in a database.

[0212] Input: Nutritional management information sent to the server

[0213] Output: Nutritional management information stored in a database

[0214] Specific operation: Save nutritional management information in a database.

[0215] Step 12:

[0216] The server analyzes the stored nutritional management information using an AI analysis model (e.g., TensorFlow) and evaluates nutritional balance.

[0217] Input: Nutritional management information stored in the database

[0218] Output: Nutritional balance evaluation results

[0219] Specific operation: Nutritional management information is input into the AI ​​model and nutritional balance is analyzed.

[0220] Step 13:

[0221] The server generates specific nutritional advice based on the evaluation results.

[0222] Input: Nutritional balance assessment results

[0223] Output: Nutrition advice

[0224] Specific operation: Based on the analysis results, automatically generated advice is adapted to the user profile.

[0225] Step 14:

[0226] The server transmits the generated nutrition advice to the terminal.

[0227] Enter: nutrition advice

[0228] Output: Send advice to terminal

[0229] Specific operation: The generated advice is sent to the device using a RESTful API or WebSocket.

[0230] Step 15:

[0231] The terminal receives the nutrition advice sent from the server and displays it on the user interface.

[0232] Input: Advice sent by the server

[0233] Output: Advice displayed in the user interface

[0234] Specific behavior: The received advice is visually presented to the user in a user interface within the application.

[0235] (Application example 1)

[0236] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0237] Fitness gym members need professional advice to effectively improve their training methods and nutritional management. However, receiving advice from a trainer individually is time-consuming and expensive. Furthermore, proper nutritional management is difficult to achieve through self-determination, and requires specialized knowledge. Conventional methods have made it difficult to provide this advice quickly and efficiently. Therefore, there is a need for a system that can easily provide professional advice to fitness gym members and help them optimize their training results and nutritional management.

[0238] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0239] In this invention, the server includes a filming means for filming gameplay videos, an uploading means for uploading the gameplay videos filmed by the filming means to the server, a saving means for saving the uploaded videos, an analysis means for analyzing the saved videos, an advice generating means for generating gameplay improvement advice based on the analysis results, a sending means for sending the generated advice to a terminal, a display means for displaying the sent advice, an input means for a user to input nutritional management information, an information sending means for sending the nutritional management information to the server, an evaluation means for evaluating the sent nutritional management information, a nutritional advice generating means for generating nutritional advice based on the evaluation results, a nutritional advice sending means for sending the generated nutritional advice to a terminal, and a display means for displaying the sent nutritional advice, and the gameplay improvement advice and nutritional advice are generated by referring to multiple method databases. This enables fitness gym members to professionally and efficiently improve their training methods and nutritional management.

[0240] definition statement

[0241] A "play video" is a video file containing sports or fitness-related actions filmed by a user.

[0242] "Capturing means" refers to a device for capturing video using a smartphone, smart glasses, head-mounted display, etc.

[0243] The "uploading means" is a device or application that has the function of transmitting the captured video to a server via a network.

[0244] "Storage means" refers to a system that has the function of recording and storing videos and data within a server.

[0245] The "analysis means" is a system that has the function of analyzing videos and data stored on the server and evaluating training and exercise performance.

[0246] The "advice generation means" is a system that automatically creates recommendations for improvements in training and exercise and nutritional management based on the analysis results.

[0247] The "transmission means" is a system having a function for transmitting the generated advice to the user's terminal.

[0248] The "display means" is a device or application that has the function of visually presenting advice sent from the server on the user's terminal.

[0249] "Input means" refers to a device or application that allows a user to input their own nutritional management information.

[0250] The "information transmission means" is a device or application that has the function of transmitting input nutritional management information to a server.

[0251] The "evaluation means" is a system that analyzes and evaluates the nutritional management information sent to the server.

[0252] The "nutritional advice generating means" is a system that generates individual nutritional management advice based on the nutritional information analyzed by the evaluation means.

[0253] The "nutrition advice sending means" is a system having a function of sending the generated nutritional management advice to the user's terminal.

[0254] The "method database" is a collection of data that describes multiple training methods and nutritional management methods, and is a system that provides advanced advice by referring to this data.

[0255] MODE FOR CARRYING OUT THE INVENTION

[0256] System Overview

[0257] The system that realizes this application example consists of three main elements: a server, a terminal, and a user, in order to improve the user's training performance and nutritional management.

[0258] Hardware and software used

[0259] 1. Hardware and software used by the user:

[0260] Smartphone

[0261] Smart Glasses

[0262] head-mounted display

[0263] Photo capture and upload application (developed with React Native)

[0264] 2. Software and systems used by the server:

[0265] Server: Using Node.js and Express

[0266] Database: MongoDB

[0267] Video processing: FFmpeg

[0268] Analyzing AI models: TensorFlow

[0269] Specific operation of the system

[0270] 1. User Action:

[0271] Users use a smartphone, smart glasses, or head-mounted display to record their own training videos, and then use the application to prepare the videos for uploading.

[0272] Users use the application to input their own nutritional management information (e.g., dietary content, amount of exercise).

[0273] 2. Device behavior:

[0274] The device receives the training video provided by the user, converts the video file into the appropriate format, compresses the video if necessary, and sends it to the server.

[0275] The terminal receives the nutrition management information provided by the user, converts the format and formats it, and then transmits it to the server.

[0276] The terminal receives the play improvement advice and nutrition advice sent from the server and displays them through a user interface.

[0277] 3. Server Operation:

[0278] The server stores the training video sent from the terminal.

[0279] The server analyzes the saved training videos using an analytical method (a generative AI model using TensorFlow) and evaluates the movements.

[0280] The server generates advice for improving play based on the analysis results, and the quality of this advice is improved by referencing multiple method databases.

[0281] The server transmits the generated play improvement advice to the terminal.

[0282] The server stores the nutritional management information transmitted from the terminal and evaluates the nutritional balance using the evaluation means.

[0283] The server generates nutrition advice based on the evaluation results and transmits it to the terminal.

[0284] Specific processing examples

[0285] Analyzing training videos and providing improvement advice:

[0286] The user films a squat training video on their smartphone and uploads it to the server via the app. The server then analyzes the video using a TensorFlow model, generates specific advice, such as "your hip angle should be less than 90 degrees," and sends it to the device. The device then displays this advice on the user interface, allowing the user to confirm and practice it during their next training session.

[0287] Enter nutritional information and provide advice:

[0288] The user enters what they have for breakfast (e.g., oatmeal, banana, yogurt) into the application. The device formats the information and sends it to the server. The server stores the information, evaluates the nutritional balance using an AI model, and generates specific nutritional advice, such as "add a protein shake," and sends it to the device. The device displays this advice in the user interface, and the user can incorporate it into their next meal.

[0289] Prompt Sentence Examples

[0290] Below is an example of a prompt for an AI model:

[0291] Training video analysis:

[0292] Please analyze the user's squat video and rate it based on the following items.

[0293] 1. Waist angle

[0294] 2. Knee position

[0295] 3. Back Posture

[0296] Generate nutrition advice:

[0297] Based on the breakfast information entered by the user (oatmeal, banana, yogurt), assess whether there are any protein deficiencies and, if so, suggest additional foods.

[0298] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0299] Program processing steps

[0300] Processing Steps

[0301] Step 1:

[0302] Users record training videos using a smartphone, smart glasses, or a head-mounted display.

[0303] Input: Filmed training video

[0304] Output: Recorded training video file

[0305] Step 2:

[0306] Users upload their videos to the server through the app, which uses FFmpeg to convert the video files to the appropriate format and compress them if necessary.

[0307] Input: Training video file

[0308] Data processing: format conversion, video compression

[0309] Output: Converted and compressed video file

[0310] Step 3:

[0311] The server receives and stores the converted and compressed video files.

[0312] Input: Converted and compressed video files

[0313] Output: Saved video file

[0314] Step 4:

[0315] The server analyzes the stored video files using a TensorFlow model, which uses a generative AI model to generate advice for improving gameplay.

[0316] Input: Saved video file

[0317] Data Computation: Video Analysis with Generative AI Models

[0318] Output: Advice for improving your gameplay

[0319] Step 5:

[0320] The server transmits the generated play improvement advice to the terminal.

[0321] Input: Advice for improving your game

[0322] Output: Play improvement advice sent to the device

[0323] Step 6:

[0324] The terminal displays the received advice for improving play on the user interface.

[0325] Input: Submitted game improvement advice

[0326] Output: Play improvement advice displayed in the user interface

[0327] Step 7:

[0328] The user uses a dedicated application to input nutritional management information (e.g., dietary details, amount of exercise).

[0329] Input: Nutritional management information

[0330] Output: Entered nutritional management information

[0331] Step 8:

[0332] The terminal converts and formats the input nutritional management information and sends it to the server.

[0333] Input: Nutritional management information entered

[0334] Data processing: format conversion, formatting

[0335] Output: Transformed and formatted nutrition information

[0336] Step 9:

[0337] The server receives and stores the converted and formatted nutrition management information.

[0338] Input: Transformed and formatted nutrition information

[0339] Output: Saved nutritional information

[0340] Step 10:

[0341] The server evaluates the stored nutritional management information using an evaluation tool and generates nutritional advice. This evaluation also uses a generative AI model.

[0342] Input: Saved nutrition management information

[0343] Data Computation: Evaluation with Generative AI Models

[0344] Output: Nutrition advice

[0345] Step 11:

[0346] The server transmits the generated nutrition advice to the terminal.

[0347] Enter: nutrition advice

[0348] Output: Nutrition advice sent to the device

[0349] Step 12:

[0350] The terminal displays the received nutrition advice on a user interface.

[0351] Input: Submitted nutrition advice

[0352] Output: Nutrition advice displayed in a user interface

[0353] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0354] The present invention is an innovative system for supporting sports improvement and nutritional management. In particular, by combining it with an emotion engine that recognizes the user's emotions, it is possible to provide advice optimized for each individual user. The specific operation of this system is described below.

[0355] Overall system configuration

[0356] This system consists of a server, a terminal, and a user. The specific role of each element is explained below.

[0357] User Actions

[0358] 1. A user uses a device (e.g., a smartphone or video camera) to record their sports play.

[0359] 2. The user uploads the captured gameplay video from their device to the server using a dedicated application.

[0360] 3. The user uses a dedicated application to input nutritional management information (e.g., dietary details, weight, amount of exercise).

[0361] Device behavior

[0362] 1. The device receives the gameplay video provided by the user, converts it into the appropriate format, compresses it if necessary, and sends it to the server.

[0363] 2. The terminal receives the nutritional management information provided by the user, converts and formats it, and then sends it to the server.

[0364] 3. The terminal receives the advice (play improvement advice and nutrition advice) sent from the server and displays it through the user interface.

[0365] Server Operation

[0366] 1. The server stores the gameplay video sent from the device.

[0367] 2. The server inputs the saved gameplay video into the analysis means (AI model) and begins analyzing the movements in the video.

[0368] 3. The server uses the emotion engine to analyze the user's emotions from the gameplay video.

[0369] 4. The server evaluates the results of the motion analysis and emotion analysis and identifies specific areas for improvement (e.g., "The wrist angle when shooting is insufficient").

[0370] 5. The server references a database of methods from professional athletes and famous coaches and generates custom advice based on the emotion of the analysis (e.g., "Shoot with confidence").

[0371] 6. The server sends the generated play improvement advice to the device.

[0372] 7. The server stores the nutritional management information sent from the terminal and evaluates the nutritional balance using an evaluation method (AI model).

[0373] 8. The server uses an emotion engine to analyze the emotion of the user's input.

[0374] 9. The server generates nutritional advice based on the evaluation results and sentiment analysis results (e.g., "Take more vitamin C if you tend to feel depressed").

[0375] 10. The server sends the generated nutrition advice to the terminal.

[0376] Specific examples

[0377] Analyzing gameplay videos and providing advice on improvements

[0378] 1. The user uses a smartphone to film their basketball shooting practice, then uploads the video to a dedicated application.

[0379] 2. The device receives the video file, converts the file format, compresses it, and then sends it to the server.

[0380] 3. The server receives the video and stores it in a database. The stored video is then analyzed by an analytical method (AI model).

[0381] 4. Based on the analysis results, the server identifies specific areas for improvement, such as "the wrist angle when shooting is insufficient."

[0382] 5. The server uses an emotion engine to analyze the user's emotions while playing and obtain information such as "I'm nervous."

[0383] 6. The server compares the method with that of professional players and generates custom advice such as "Keep your wrist at a 45-degree angle and shoot with confidence."

[0384] 7. The server sends the generated advice to the terminal, which displays the advice on the user interface.

[0385] 8. The user checks the advice and puts it into practice during their next shooting practice session.

[0386] Entering nutritional management information and providing advice

[0387] 1. The user enters the contents of their breakfast (e.g., oatmeal, banana, yogurt) in a dedicated application.

[0388] 2. The terminal receives the input information, converts and formats it, and sends it to the server.

[0389] 3. The server receives the nutrition management information and stores it in a database.

[0390] 4. The server uses an evaluation tool (AI model) to assess nutritional balance and identify specific nutritional improvements (e.g., "You need more protein at breakfast").

[0391] 5. The server uses an emotion engine to analyze the user's emotions at the time of input and obtain information such as "I feel stressed."

[0392] 6. The server generates custom advice such as "Add a protein shake to your breakfast and eat more vitamin C to reduce stress."

[0393] 7. The server sends the generated nutrition advice to the terminal, which displays the advice on the user interface.

[0394] 8. The user reviews the advice and incorporates it into their next meal plan.

[0395] In this way, the present invention can provide users with comprehensive, automated and personalized sports play improvement and nutritional management advice, including emotion recognition.

[0396] The processing flow will be explained below.

[0397] Step 1:

[0398] Users use devices such as smartphones and tablets to record video of sports play.

[0399] Step 2:

[0400] Users upload the videos they have taken from their device to the server using a dedicated application.

[0401] Step 3:

[0402] The device receives the video file and converts or compresses the file format as needed.

[0403] Step 4:

[0404] The device then sends the converted and compressed video file to the server.

[0405] Step 5:

[0406] The server saves the received video file in storage.

[0407] Step 6:

[0408] The server inputs the saved video into an analysis tool (AI model) and begins analyzing the movements within the video.

[0409] Step 7:

[0410] The server's analytics evaluates your movements and identifies specific areas for improvement (e.g., "Your wrist angle when shooting is insufficient").

[0411] Step 8:

[0412] The server uses an emotion engine to analyze the user's emotions in real time from gameplay video (e.g., "tension" or "concentration").

[0413] Step 9:

[0414] The server generates advice to improve play based on the results of motion analysis and emotion analysis (e.g., "Keep your wrist at a 45-degree angle and shoot with confidence").

[0415] Step 10:

[0416] The server then sends the generated advice to the device to improve your gameplay.

[0417] Step 11:

[0418] The device formats the received advice for display in a user interface.

[0419] Step 12:

[0420] The device will then display the formatted advice to the user.

[0421] Step 13:

[0422] The user can review the advice provided via the device and put it into practice the next time they practice.

[0423] Step 14:

[0424] Users use a dedicated application to enter nutritional management information (e.g., dietary content, weight, and exercise volume).

[0425] Step 15:

[0426] The terminal receives the entered nutritional management information, formats it, and sends it to the server.

[0427] Step 16:

[0428] The server receives the nutritional management information sent and stores it in a database.

[0429] Step 17:

[0430] The server uses an evaluation tool (AI model) to assess nutritional balance and extract specific nutritional improvements (e.g., "You are lacking the protein you need for breakfast").

[0431] Step 18:

[0432] The server uses an emotion engine to analyze the emotions (e.g., "stress" or "fatigue") when the user enters the meal information.

[0433] Step 19:

[0434] The server generates nutritional advice based on the evaluation and sentiment analysis results (e.g., "Add a protein shake and take more vitamin C to reduce stress").

[0435] Step 20:

[0436] The server then sends the generated nutrition advice to the device.

[0437] Step 21:

[0438] The device then formats the received nutrition advice for display in a user interface.

[0439] Step 22:

[0440] The device will then display formatted nutrition advice to the user.

[0441] Step 23:

[0442] Users can review the nutritional advice provided via their device and incorporate it into their next meal plan.

[0443] Through these steps, the system can provide comprehensive and automatic personalized sports play improvement and nutritional management advice that also takes the user's emotions into account.

[0444] Example 2

[0445] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0446] In modern sports training and nutritional management, providing individually optimized advice is extremely important, and there is a particular demand for advice that takes into account the user's emotional state. However, current systems perform motion analysis of gameplay videos and evaluation of nutritional management information separately, and there are only a limited number of systems that can generate advice by integrating emotional analysis. As a result, there is a problem in that the improvement advice and nutritional advice obtained are of insufficient quality and usefulness.

[0447] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0448] In this invention, the server includes analysis means for performing motion analysis and emotion analysis using analysis means, advice generation means for generating play improvement advice based on the results of the motion analysis and emotion analysis, evaluation means for performing the motion analysis and emotion analysis, and nutrition advice generation means for generating nutrition advice based on the results of the nutrition analysis and emotion analysis, thereby making it possible to provide high-quality play improvement advice and nutrition advice that comprehensively considers the user's motion state and emotional state.

[0449] A "play video" is a video file in which a user captures a sports play.

[0450] "Photographing means" refers to a device that a user uses to photograph sports play, such as a smartphone or video camera.

[0451] "Uploading means" refers to a function or application for transmitting gameplay videos captured by the capturing means to a server.

[0452] "Storage means" refers to the function by which the server stores uploaded gameplay videos in a database or the like.

[0453] "Analysis means" refers to a function that analyzes saved gameplay videos and evaluates the user's actions and emotional state.

[0454] "Motion analysis" is the process of analyzing the user's body movements in gameplay video.

[0455] "Emotion analysis" is the process of evaluating and determining a user's emotional state based on gameplay video and user input information.

[0456] The "advice generation means" refers to a function that generates advice for improving play to be provided to the user based on the results of the motion analysis and emotion analysis.

[0457] The "transmission means" refers to a function for transmitting the generated advice to the terminal.

[0458] The "display means" refers to a function that displays the advice sent to the terminal on the user interface.

[0459] "Input means" refers to a function or application that allows the user to input nutritional management information.

[0460] "Information transmission means" refers to a function for transmitting input nutritional management information to a server.

[0461] "Evaluation means" refers to a function that analyzes the transmitted nutritional management information and evaluates the nutritional balance.

[0462] The term "nutritional advice generating means" refers to a function that generates nutritional improvement advice to be provided to a user based on the results of nutritional analysis and emotion analysis.

[0463] The term "nutritional advice sending means" refers to a function for sending the generated nutritional advice to the terminal.

[0464] A "method database" refers to a database that stores the coaching methods of professional athletes and famous coaches.

[0465] The present invention relates to a system for supporting users in improving their sports performance and managing their nutrition. The system analyzes the user's movements and emotions and provides personalized optimization advice based on the analysis. Specific embodiments of the system, including the hardware, software, and data processing methods used, are described in detail below.

[0466] Overall system configuration

[0467] This system consists of a server, a terminal, and a user. The specific role of each element is explained below.

[0468] User Actions

[0469] Users can record their sports activities using a smartphone or video camera. After recording is complete, they can use a dedicated application to upload the video to a server. Users can also enter nutritional management information (dietary content, weight, exercise volume, etc.) through the same dedicated application.

[0470] Device behavior

[0471] The device receives gameplay videos provided by the user, performs the necessary processing, and then sends them to the server. Specifically, it uses a video processing library such as FFmpeg to convert the video format and compress the size. It also receives nutritional management information provided by the user, converts its format, formats it, and sends it to the server.

[0472] Server Operation

[0473] The server receives the data sent from the terminal and performs the following processing.

[0474] 1. Video analysis

[0475] The server stores the received gameplay video. The stored video is then analyzed using a generative AI model. This analysis uses motion analysis libraries such as OpenPose and MediaPipe to analyze the user's body movements from the video frames.

[0476] 2. Emotion analysis

[0477] The server uses an emotion engine to analyze the user's emotions from the gameplay video. It uses facial recognition and voice analysis technologies to determine the user's emotional state (e.g., tension, joy, concentration).

[0478] 3. Generating advice for improving play

[0479] The server integrates the results of the motion analysis and emotion analysis, and generates personalized, custom advice by referencing a database of methods from professional athletes and famous coaches. The generated advice is then sent to the device.

[0480] 4. Evaluation of nutritional management information

[0481] The server stores the received nutritional management information and evaluates nutritional balance using a generative AI model. The evaluation results and sentiment analysis results are combined to generate appropriate custom nutrition advice, which is then sent to the device.

[0482] Specific examples

[0483] Analyzing gameplay videos and providing advice on improvements

[0484] 1. The user uses a smartphone to film their basketball shooting practice, then uploads the video to a dedicated application.

[0485] 2. The device receives the video file, converts the file format, compresses it, and then sends it to the server.

[0486] 3. The server receives the video and stores it in a database, where it is analyzed by a generative AI model.

[0487] 4. Based on the analysis results, the server identifies specific areas for improvement, such as "the wrist angle when shooting is insufficient."

[0488] 5. The server uses an emotion engine to analyze the user's emotions while playing and obtain information such as "I'm nervous."

[0489] 6. The server compares the method with that of professional players and generates custom advice such as "Keep your wrist at a 45-degree angle and shoot with confidence."

[0490] 7. The server sends the generated advice to the terminal, which displays the advice on the user interface.

[0491] 8. The user checks the advice and puts it into practice during their next shooting practice session.

[0492] Entering nutritional management information and providing advice

[0493] 1. The user enters the contents of their breakfast (e.g., oatmeal, banana, yogurt) in a dedicated application.

[0494] 2. The terminal receives the input information, converts and formats it, and sends it to the server.

[0495] 3. The server receives the nutrition management information and stores it in a database.

[0496] 4. The server uses an evaluation tool (generative AI model) to assess nutritional balance and identify specific nutritional improvements (e.g., "You need more protein at breakfast").

[0497] 5. The server uses an emotion engine to analyze the user's emotions at the time of input and obtain information such as "I feel stressed."

[0498] 6. The server generates custom advice such as "Add a protein shake to your breakfast and eat more vitamin C to reduce stress."

[0499] 7. The server sends the generated nutrition advice to the terminal, which displays the advice on the user interface.

[0500] 8. The user reviews the advice and incorporates it into their next meal plan.

[0501] Example prompts to input to the generative AI model

[0502] Example 1: "Analyze basketball shooting practice videos uploaded by users and evaluate the wrist angle during the shot."

[0503] Example 2: "Evaluate the nutritional balance of a breakfast menu of oatmeal, banana, and yogurt and generate advice on what nutrients the user needs. If the user is feeling stressed, provide advice on what to eat."

[0504] The present invention makes it possible to provide high-quality advice for improving play and nutritional advice to users, taking into consideration their behavioral and emotional states in a comprehensive manner.

[0505] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0506] Step 1:

[0507] Users can film their own sports play using a smartphone or video camera.

[0508] Input: Sports play footage

[0509] Specific operations: Launch the smartphone's camera application and start recording. Once recording is complete, save the video file.

[0510] Output: Recorded gameplay video file

[0511] Step 2:

[0512] The user launches a dedicated application, selects the gameplay video they have taken, and uploads it to the server.

[0513] Input: gameplay video file

[0514] Specific operation: Use the application's "Video Upload" function to select the play video file on the device and click the upload button.

[0515] Output: Gameplay video data sent to the server

[0516] Step 3:

[0517] The terminal receives the gameplay video sent from the user, converts the format and compresses it, and then sends it to the server.

[0518] Input: gameplay video data

[0519] What it does: It uses the FFmpeg library to convert the video format, compress it if necessary, and then sends the video data to the server over a network connection.

[0520] Output: Format-converted and compressed video data

[0521] Step 4:

[0522] The server receives the gameplay video sent from the terminal and stores it in a database.

[0523] Input: Format-converted gameplay video data

[0524] Specific operation: Connect to the database and save the video data and metadata (shooting date and time, user ID, etc.).

[0525] Output: Stored video data and metadata

[0526] Step 5:

[0527] The server analyzes the saved gameplay video using a generated AI model.

[0528] Input: Saved video data

[0529] Specific operation: Input video frames into the generative AI model and analyze the user's body movements (e.g., using the OpenPose or MediaPipe library). Save the analysis results.

[0530] Output: Motion analysis result data (e.g., position information of each body part)

[0531] Step 6:

[0532] The server uses an emotion engine to analyze the user's emotions from the gameplay video.

[0533] Input: Stored video and audio data

[0534] Specific operation: Determine the user's emotional state using facial recognition and voice analysis technology. Save the analysis results.

[0535] Output: Emotion analysis result data (e.g., tension, joy, concentration, etc.)

[0536] Step 7:

[0537] The server integrates the results of the motion analysis and emotion analysis, and generates play improvement advice by referring to a method database of professional athletes.

[0538] Input: Motion analysis result data and emotion analysis result data

[0539] Specific operation: Based on the analysis results, relevant advice is extracted from the method database and custom advice is generated.

[0540] Output: Generated play improvement advice data

[0541] Step 8:

[0542] The server transmits the generated play improvement advice to the terminal.

[0543] Input: Generated play improvement advice data

[0544] Specific operation: Advice data is sent to the terminal via a network connection.

[0545] Output: Advice data sent to the terminal

[0546] Step 9:

[0547] The terminal receives the play improvement advice sent from the server and displays it on the user interface.

[0548] Input: Play improvement advice data

[0549] Specific behavior: Display the advice text in the "Advice" section of the application.

[0550] Output: Advice displayed in the user interface

[0551] Step 10:

[0552] The user uses a dedicated application to input nutritional management information (dietary content, weight, amount of exercise, etc.).

[0553] Input: Nutritional management information

[0554] Specific operations: Enter information such as diet, weight, and exercise amount into the "Nutrition Management" form within the app and click the submit button.

[0555] Output: Input nutritional management information data

[0556] Step 11:

[0557] The terminal receives the input nutrition management information, converts and formats it, and then transmits it to the server.

[0558] Input: Nutritional management information data

[0559] Specific operation: Convert the received data into JSON format and send it to the server over the network.

[0560] Output: Formatted nutrition management information data

[0561] Step 12:

[0562] The server receives the nutritional management information sent from the terminal and stores it in a database.

[0563] Input: Formatted nutrition management information data

[0564] Specific operation: Connect to the database and save nutrition management information.

[0565] Output: Saved nutritional management information data

[0566] Step 13:

[0567] The server uses a generative AI model to assess nutritional balance and identify specific nutritional improvements.

[0568] Input: Saved nutrition management information data

[0569] Specific operation: Input nutritional management information into the generative AI model, analyze and evaluate nutritional balance, and save the evaluation results.

[0570] Output: Nutritional balance assessment result data

[0571] Step 14:

[0572] The server uses an emotion engine to analyze the user's emotions when inputting nutritional management information.

[0573] Input: Voice data and facial image data when entering nutrition management information

[0574] Specific operation: Determine the user's emotional state using facial recognition and voice analysis technology. Save the analysis results.

[0575] Output: Emotion analysis result data (e.g., stress, joy, etc.)

[0576] Step 15:

[0577] The server generates custom nutrition advice based on the nutritional assessment results and the sentiment analysis results.

[0578] Input: Nutritional balance assessment result data and emotion analysis result data

[0579] Specific operation: Integrates evaluation results and sentiment analysis results to generate appropriate nutrition advice.

[0580] Output: Generated custom nutrition advice data

[0581] Step 16:

[0582] The server transmits the generated nutrition advice to the terminal.

[0583] Input: Generated custom nutrition advice data

[0584] Specific operation: Advice data is sent to the terminal via a network connection.

[0585] Output: Advice data sent to the terminal

[0586] Step 17:

[0587] The terminal receives the nutrition advice sent from the server and displays it on the user interface.

[0588] Input: Nutrition advice data

[0589] What it does: Displays the advice in text form in the "Nutrition Advice" section of the application.

[0590] Output: Nutrition advice displayed on the user interface

[0591] Through this system, users can receive individually optimized advice on improving their sports play and nutritional management.

[0592] (Application example 2)

[0593] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0594] In recent years, automated robots have played an important role in many industries. However, these robots occasionally experience performance degradation or malfunction. To solve this problem, a system is needed that can properly monitor the robot's behavior and status and provide immediate advice for performance improvement. However, current technology does not provide a system that comprehensively analyzes the robot's behavior and emotions (state) and provides performance improvement advice. Therefore, the objective of this invention is to provide a system that comprehensively supports not only the improvement of sports performance but also the performance improvement and status monitoring of robotic work in factories.

[0595] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a filming means for filming gameplay videos; an uploading means for uploading the gameplay videos filmed by the filming means to the server; a saving means for saving the uploaded videos; an analyzing means for analyzing the saved videos; an advice generating means for generating play improvement advice based on the analysis results; a transmitting means for sending the generated advice to a terminal; a display means for displaying the sent advice; an improvement advice generating means for generating performance improvement advice for the robot based on the analysis results; an emotion analysis means by which the improvement advice generating means analyzes the state of the robot using sensor data; a transmitting means for sending the advice generated by the emotion analysis means to a terminal; and a display means for displaying the sent advice. This makes it possible to monitor the operating state of the robot in real time and immediately provide performance improvement advice based on emotion analysis.

[0596] "Photographing means" refers to a device or system for capturing videos or images.

[0597] The "uploading means" is a method or device for transmitting captured data to a server via a network.

[0598] "Storage means" refers to a method or device for temporarily or long-term storage of videos and data sent to the server.

[0599] "Analysis means" refers to a method or device for analyzing stored video or data and extracting specific information or features.

[0600] The "advice generation means" is a method or device that generates improvement instructions for the user based on the data obtained by the analysis means.

[0601] The "transmission means" is a method or device for sending information such as generated advice to the user's terminal.

[0602] "Display means" refers to a method or device for visually displaying advice or information sent to a user's terminal.

[0603] The "improvement advice generating means" is a method or device that generates instructions for improving the performance of a robot based on an analysis of the robot's movements and state.

[0604] "Emotion analysis means" refers to a method or device that analyzes the robot's sensor data and evaluates the robot's current state and performance.

[0605] "Sensor data" refers to data obtained from various sensors attached to robots and other devices, and is data that measures and records operating conditions and environmental conditions.

[0606] The present invention is a system for monitoring the operation of a factory robot and improving its performance. The system operates based on sensor data, including an image capturing means, an uploading means, a storage means, an analysis means, an advice generating means, a transmission means, a display means, an improvement advice generating means, and an emotion analysis means.

[0607] Overall system configuration

[0608] This system consists of a server, a terminal, and a user (in this case, the factory manager). The specific configuration and operation of each element are shown below.

[0609] Filming method

[0610] Cameras are used to capture the movements of robots in factories. The captured video is high resolution and contains important motion analysis data.

[0611] Upload method

[0612] The user uploads the video file of the robot's movements to the device through a dedicated application (e.g., a smartphone app). The uploaded video is converted into an appropriate format and sent to the server.

[0613] Preservation means

[0614] The server stores the received video files in a database, which allows for efficient management and storage of multiple video files.

[0615] Analysis means

[0616] The server processes the stored video using analytical means, specifically, an AI model (e.g., a motion analysis model) to analyze the robot's movement patterns and abnormalities in the video.

[0617] Advice Generation Method

[0618] Based on the data extracted by the analysis means, the advice generation means generates specific advice for improving the performance of the robot. The generated advice is optimized for the operating state of each individual robot.

[0619] Transmission method

[0620] The generated advice is transmitted from the server to the terminal, and the transmitting means enables the advice to be provided to the user in real time.

[0621] Display means

[0622] The terminal visually displays the received advice to the user. The display means includes an interface for concisely and clearly showing the content of the advice.

[0623] Improvement advice generation method

[0624] This is a means for generating more advanced performance improvement advice based on the data provided by the analysis means and the sentiment analysis means. The sentiment analysis means uses sensor data to analyze the robot's condition (e.g., temperature, load).

[0625] Hardware and software used

[0626] Hardware:

[0627] Camera: A device for filming the robot's movements.

[0628] Sensor: A device that acquires robot status data (temperature, load, etc.).

[0629] software:

[0630] Dedicated application: Upload videos, convert formats, and compress them.

[0631] Server: Stores and analyzes video and sensor data.

[0632] AI models: AI models for video analysis (e.g., Keras), AI models for sentiment analysis (e.g., Keras).

[0633] Specific examples

[0634] For example, a user can film a robot working in a factory with a camera and upload the video using a dedicated application. The video is then converted and compressed on the device and sent to the server. The server then saves the video and analyzes the robot's movements and status using analysis and emotion analysis. Based on the analysis results, the advice generation means creates specific advice for improving the robot's movements, which is then sent to the device and displayed to the user.

[0635] Examples of prompts:

[0636] "Write a program that analyzes this video file and sensor data to evaluate the robot's accuracy and health. Add logic to generate specific improvements and recommendations."

[0637] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0638] Step 1:

[0639] The user takes pictures of the robot's operation in the factory with a camera.

[0640] Input: Real-time video of robot movement.

[0641] Output: Video file of robot movement.

[0642] Specific operation: Cameras are installed in appropriate locations within the factory to capture high-resolution images of the robot's work process.

[0643] Step 2:

[0644] The user uploads the video file they have taken to the device using a dedicated application.

[0645] Input: Video file.

[0646] Output: Video file stored on your device.

[0647] Specific operation: Transfer and save video files to the device using a dedicated application.

[0648] Step 3:

[0649] The device sends the video file to the server.

[0650] Input: Video files stored on your device.

[0651] Output: Video file uploaded to the server.

[0652] Specific operation: The video file is compressed and format converted on the terminal, and then sent to the server via the network.

[0653] Step 4:

[0654] The server stores the received video file.

[0655] Input: Video files uploaded to the server.

[0656] Output: Video files stored in the database.

[0657] Specific operation: Video files are stored in a database on the server and managed appropriately.

[0658] Step 5:

[0659] The server analyzes the stored video.

[0660] Input: Video files stored in the database.

[0661] Output: Robot motion data in video.

[0662] Specific behavior: Using an AI model (e.g., a motion analysis model), the robot's movement patterns and abnormalities are analyzed for each video frame.

[0663] Step 6:

[0664] The server uses an emotion analysis means to analyze the state of the robot.

[0665] Input: Robot sensor data (temperature, load, etc.).

[0666] Output: Robot state data.

[0667] Specific behavior: Using an emotion analysis model, sensor data is analyzed to assess the robot's current behavioral state.

[0668] Step 7:

[0669] The server uses the advice generation means to create improvement advice based on the analysis results.

[0670] Input: Video analysis data and state data.

[0671] Output: Improvement advice.

[0672] Specific behavior: Based on the analysis results, specific advice is generated to optimize the robot's behavior (e.g., "reduce the part sorting speed by 20%)."

[0673] Step 8:

[0674] The server transmits the generated advice to the terminal.

[0675] Input: Improvement advice.

[0676] Output: Advice sent to the terminal.

[0677] Specific operation: The generated advice is sent to the user's terminal via the network.

[0678] Step 9:

[0679] The advice received by the terminal is displayed on a user interface.

[0680] Input: Advice sent by the server.

[0681] Output: The advice displayed to the user.

[0682] Specific operation: The advice content is displayed in the user interface in a format that is easy for the user to view.

[0683] Through these steps, the entire system is configured to effectively monitor the operation of robots in the factory and provide timely improvement advice.

[0684] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0685] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0686] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0687] [Second embodiment]

[0688] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0689] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0690] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0691] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0692] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0693] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0694] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0695] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0696] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0697] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0698] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0699] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0700] The present invention is an innovative system for supporting the improvement of sports play and nutritional management. The specific operation of this system will be described below.

[0701] Overall system configuration

[0702] This system consists of a server, a terminal, and a user. The specific role of each element is explained below.

[0703] User Actions

[0704] 1. A user uses a device (e.g., a smartphone or video camera) to record their sports play.

[0705] 2. Users upload the gameplay videos they have taken using a dedicated application on their device.

[0706] 3. The user uses a dedicated application to enter nutritional management information (e.g., dietary details, weight, and exercise amount).

[0707] Device behavior

[0708] 1. The device receives the gameplay video provided by the user, converts it into the appropriate format, compresses it if necessary, and sends it to the server.

[0709] 2. The terminal receives the nutritional management information provided by the user, converts and formats it, and then sends it to the server.

[0710] 3. The terminal receives the advice (play improvement advice and nutrition advice) sent from the server and displays it through the user interface.

[0711] Server Operation

[0712] 1. The server stores the gameplay video sent from the device.

[0713] 2. The server analyzes the saved gameplay video using analytical means (AI model) and evaluates the movement.

[0714] 3. The server generates advice for improving play based on the analysis results. The quality of the generated advice is improved by referencing multiple method databases.

[0715] 4. The server sends the generated play improvement advice to the device.

[0716] 5. The server stores the nutritional management information sent from the terminal and evaluates the nutritional balance using an evaluation method (AI model).

[0717] 6. The server generates nutrition advice based on the evaluation results.

[0718] 7. The server sends the generated nutrition advice to the terminal.

[0719] Specific examples

[0720] Analyzing gameplay videos and providing advice on improvements

[0721] 1. The user uses a smartphone to film their basketball shooting practice, then uploads the video to a dedicated application.

[0722] 2. The device receives the video file, converts the file format, compresses it, and then sends it to the server.

[0723] 3. The server receives the video and stores it in a database. The stored video is then analyzed by an analytical method (AI model).

[0724] 4. Based on the analysis results, the server identifies specific areas for improvement, such as "the wrist angle when shooting is insufficient."

[0725] 5. The server compares the method with that of a professional player and generates custom advice such as "Keep your wrist at a 45-degree angle."

[0726] 6. The server sends the generated advice to the terminal, which displays the advice on the user interface.

[0727] 7. The user checks the advice and puts it into practice during their next shooting practice session.

[0728] Entering nutritional management information and providing advice

[0729] 1. The user enters the contents of their breakfast (e.g., oatmeal, banana, yogurt) in a dedicated application.

[0730] 2. The terminal receives the input information, converts and formats it, and sends it to the server.

[0731] 3. The server receives the nutrition management information and stores it in a database.

[0732] 4. The server uses an evaluation tool (AI model) to assess nutritional balance and identify specific areas for improvement, such as "you need more protein at breakfast."

[0733] 5. The server generates a custom advice such as "Add a protein shake to breakfast."

[0734] 6. The server sends the generated nutrition advice to the terminal, which displays the advice on the user interface.

[0735] 7. The user confirms the advice and applies it to their next meal.

[0736] In this way, the present invention provides users with personalized sports play improvement and nutritional management advice to help improve overall performance.

[0737] The processing flow will be explained below.

[0738] Step 1:

[0739] Users use devices such as smartphones and tablets to record video of sports play.

[0740] Step 2:

[0741] Users upload the videos they have taken from their device to the server using a dedicated application.

[0742] Step 3:

[0743] The device receives the video file and converts or compresses the file format as needed.

[0744] Step 4:

[0745] The device then sends the converted and compressed video file to the server.

[0746] Step 5:

[0747] The server saves the received video file in storage.

[0748] Step 6:

[0749] The server inputs the saved video into an analysis tool (AI model) and begins analyzing the movements within the video.

[0750] Step 7:

[0751] The server's analytics evaluates your movements and identifies specific areas for improvement (e.g., "Your wrist angle when shooting is insufficient").

[0752] Step 8:

[0753] The server references a database of methods from professional athletes and famous coaches and generates custom advice based on the analysis results.

[0754] Step 9:

[0755] The server then sends the generated advice to the device to improve your gameplay.

[0756] Step 10:

[0757] The device formats the received advice for display in a user interface.

[0758] Step 11:

[0759] The device will then display the formatted advice to the user.

[0760] Step 12:

[0761] The user can review the advice provided via the device and put it into practice the next time they practice.

[0762] Step 13:

[0763] Users use a dedicated application to enter nutritional management information (e.g., dietary content, weight, and exercise volume).

[0764] Step 14:

[0765] The terminal receives the entered nutritional management information, formats it, and sends it to the server.

[0766] Step 15:

[0767] The server receives the nutritional management information sent and stores it in a database.

[0768] Step 16:

[0769] The server uses an evaluation tool (AI model) to assess nutritional balance and extract specific nutritional areas for improvement (e.g., "lack of protein").

[0770] Step 17:

[0771] The server then references a nutrition database and generates custom nutrition advice based on the assessment results.

[0772] Step 18:

[0773] The server then sends the generated nutrition advice to the device.

[0774] Step 19:

[0775] The device then formats the received nutrition advice for display in a user interface.

[0776] Step 20:

[0777] The device will then display formatted nutrition advice to the user.

[0778] Step 21:

[0779] Users can review the nutritional advice provided via their device and incorporate it into their next meal plan.

[0780] Through these steps, the system can provide users with comprehensive and automated personalized sports play improvement and nutritional management advice.

[0781] Example 1

[0782] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0783] Conventional sports performance improvement and nutrition management systems have limitations in analyzing data and providing appropriate advice to individual users. Furthermore, analysis of gameplay videos and evaluation of nutrition management information are often done manually, resulting in insufficient automation and efficiency. This makes it difficult for users to obtain effective improvement measures in a timely manner, making it difficult to improve overall performance.

[0784] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0785] In this invention, the server includes: a filming means for filming gameplay videos; an uploading means for uploading the gameplay videos filmed by the filming means via a dedicated application on the terminal; a transmitting means for receiving the uploaded videos at the terminal, converting the file format and compressing them, and transmitting them to the server; a storing means for saving the transmitted videos on the server; an analyzing means for analyzing the saved videos using an AI model; an advice generating means for generating gameplay improvement advice based on the analysis results; a play advice sending means for sending the generated gameplay improvement advice to the terminal; a display means for displaying the transmitted gameplay improvement advice on a terminal user interface; an input means for inputting nutritional management information; a nutritional information sending means for receiving the input nutritional management information at the terminal, shaping and formatting the information, and transmitting it to the server; a nutrition saving means for saving the transmitted nutritional management information on the server; an evaluating means for evaluating the stored nutritional management information using an AI model; a nutritional advice generating means for generating nutritional advice based on the evaluation results; a nutritional advice sending means for sending the generated nutritional advice to the terminal; and a display means for displaying the transmitted nutritional advice on the terminal user interface. This will enable the analysis of individual users' sports play and nutritional management to be automated, enabling the provision of quick and effective advice on improvement.

[0786] A "play video" is a video file in which a user captures a sports play.

[0787] "Filming means" refers to devices or methods for filming sports play, specifically smartphones and video cameras.

[0788] The "uploading means" is a means for transmitting the captured gameplay video to the server, and is a function or process that is performed through a dedicated application.

[0789] "Transmission means" refers to a method for transferring data from a user terminal to a server, and includes a POST request using the HTTP protocol.

[0790] "Storage" means a system or method for retaining transmitted data, including the use of a database or storage system.

[0791] "Analysis means" refers to the processes and tools used to analyze stored data, specifically the use of AI models.

[0792] The "advice generation means" is a process for creating improvement advice for the user based on the analysis results.

[0793] The "play advice sending means" refers to a function for sending the generated improvement advice to the user terminal.

[0794] The "display means" is an interface for visually presenting information to the user on the terminal.

[0795] "Input means" refers to a method by which a user inputs nutritional management information into the system, such as a form in a dedicated application.

[0796] "Nutrition information transmission means" refers to a function for transmitting input nutrition management information to a server.

[0797] "Nutrition storage means" refers to a system or method for storing transmitted nutrition management information.

[0798] "Evaluation methods" are processes and tools for analyzing and evaluating stored nutrition management information, and use AI models.

[0799] "Nutrition advice generator" refers to a process that generates nutrition advice based on the evaluation results.

[0800] The term "nutrition advice sending means" refers to a function for sending the generated nutrition advice to the user terminal.

[0801] This invention is an innovative system for supporting sports performance improvement and nutritional management. This system is composed of a server, terminals, and users, and each element works in cooperation with each other.

[0802] User Actions

[0803] First, the user films their own sports play (e.g., a basketball shot) using a smartphone or video camera. Next, they upload the video to their device using a dedicated application. They also use the same application to enter their daily nutrition management information (e.g., dietary details, weight, and exercise volume).

[0804] Device behavior

[0805] The device receives gameplay videos uploaded by users, converts the video files to the appropriate format using tools such as FFmpeg, and compresses them by adjusting the bitrate and resolution. It then sends the videos to the server. It also receives nutritional management information entered by the user and formats the data into JSON format. It then sends the formatted nutritional information to the server. It also receives gameplay improvement advice and nutritional advice sent from the server and displays them on the user interface.

[0806] Server Operation

[0807] The server receives gameplay videos sent from the device and stores them in a database. For example, a storage system dedicated to video files (e.g., Amazon S3) can be used. The stored videos are input into an AI analysis model (e.g., OpenPose or MediaPipe) for analysis and evaluation of the movements. Based on the analysis results, advice is created to improve the user's gameplay. This advice is optimized by referencing multiple method databases. The generated advice is then sent to the device and presented to the user.

[0808] The server receives the nutritional management information sent from the device and stores it in a database. The stored nutritional information is input into an AI model (e.g., TensorFlow) for analysis and an evaluation of nutritional balance is performed. Based on the evaluation results, specific nutritional advice is generated and sent to the device for presentation to the user.

[0809] Specific examples

[0810] Analyzing gameplay videos and providing advice on improvements

[0811] For example, a user uses a smartphone to film basketball shooting practice and uploads the video to a dedicated application. The device receives the video file, converts the file format, compresses it, and sends it to a server. The server then receives the video and stores it in a database. The saved video is analyzed using an AI model (e.g., OpenPose) to identify specific areas for improvement, such as "your wrist angle when shooting is insufficient." Based on the analysis results, custom advice is generated, such as "keep your wrist at a 45-degree angle." The server then sends the generated advice to the device, which displays it on the user interface. The user then checks the advice and puts it into practice during their next shooting practice.

[0812] Entering nutritional management information and providing advice

[0813] For example, a user uses a dedicated application to input the contents of their breakfast (e.g., oatmeal, banana, yogurt). The device receives the input information, converts and formats it, and sends it to the server. The server receives the information and stores it in a database. The stored information is input into an AI model for analysis, and specific areas for improvement, such as "You need more protein for breakfast," are identified. Based on the results, custom advice, such as "Add a protein shake to breakfast," is generated, and the server sends the advice to the device. The device displays the advice on the user interface, and the user can incorporate it into their next meal.

[0814] Examples of prompt statements

[0815] An example of a prompt sentence generated by a generative AI model could be written as follows:

[0816] "What is the optimal wrist angle when shooting a basketball?"

[0817] "What foods should I add to my breakfast to improve its nutritional balance?"

[0818] In this way, the present invention provides useful advice to users in sports play and nutritional management, and helps improve overall performance.

[0819] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0820] Step 1:

[0821] Users can film their own sports play using a smartphone or video camera and then upload the recorded video files using a dedicated application.

[0822] Input: Recorded video file

[0823] Output: Video file for upload

[0824] Specific operation: The user opens the dedicated application, selects the recorded video file, and presses the upload button.

[0825] Step 2:

[0826] The device receives the gameplay video file uploaded by the user, converts the video file format to MP4 format using a tool such as FFmpeg, and compresses it if necessary.

[0827] Input: User uploaded video file

[0828] Output: Converted and compressed video file

[0829] Specific operation: The device receives the video file, converts the format using FFmpeg, and compresses it by adjusting the bitrate and resolution.

[0830] Step 3:

[0831] The device sends the converted and compressed video file to the server.

[0832] Input: Converted and compressed video files

[0833] Output: Video file sent to the server

[0834] Specific behavior: Sends the file to the server using a POST request using the HTTP protocol.

[0835] Step 4:

[0836] The server receives the video file sent from the terminal and stores the video file in a database.

[0837] Input: Video file sent to the server

[0838] Output: Video files stored in a database

[0839] Specific behavior: Receives video files and stores them in a database (e.g., Amazon S3) along with associated metadata (e.g., user ID, timestamp).

[0840] Step 5:

[0841] The server analyzes the stored video files using an AI analysis model (e.g., OpenPose, MediaPipe) and evaluates the movements.

[0842] Input: Video files stored in the database

[0843] Output: Analysis results (motion evaluation data)

[0844] Specific operation: The saved video file is input into the AI ​​model, and the characteristics of the user's movements (e.g., wrist angle, body posture) are extracted.

[0845] Step 6:

[0846] The server generates advice for improving play based on the analysis results, and this advice is optimized by referencing multiple method databases.

[0847] Input: Analysis results (motion evaluation data)

[0848] Output: Advice for improving your gameplay

[0849] Specific operation: Based on the analysis results, automatically generated advice is compared with the user profile and method database to optimize the system.

[0850] Step 7:

[0851] The server transmits the generated play improvement advice to the terminal.

[0852] Input: Advice for improving your game

[0853] Output: Send advice to terminal

[0854] Specific operation: The generated advice is sent to the device using a RESTful API or WebSocket.

[0855] Step 8:

[0856] The terminal receives the play improvement advice sent from the server and displays it on the user interface.

[0857] Input: Advice sent by the server

[0858] Output: Advice displayed in the user interface

[0859] Specific behavior: The received advice is visually presented to the user in a user interface within the application.

[0860] Step 9:

[0861] The user uses a dedicated application to input nutritional management information (e.g., dietary content, weight, amount of exercise).

[0862] Input: Nutritional management information (dietary content, weight, exercise amount)

[0863] Output: Nutritional management information entered into the application

[0864] Specific operation: Enter diet and exercise data using the input form in the dedicated application.

[0865] Step 10:

[0866] The terminal receives nutritional management information entered by the user, formats the data into JSON format, and sends it to the server.

[0867] Input: Nutritional management information entered by the user

[0868] Output: Formatted data sent to the server

[0869] Specific operation: Converts input data into JSON format and sends it to the server via an HTTP POST request.

[0870] Step 11:

[0871] The server receives the nutritional management information sent from the terminal and stores it in a database.

[0872] Input: Nutritional management information sent to the server

[0873] Output: Nutritional management information stored in a database

[0874] Specific operation: Save nutritional management information in a database.

[0875] Step 12:

[0876] The server analyzes the stored nutritional management information using an AI analysis model (e.g., TensorFlow) and evaluates nutritional balance.

[0877] Input: Nutritional management information stored in the database

[0878] Output: Nutritional balance evaluation results

[0879] Specific operation: Nutritional management information is input into the AI ​​model and nutritional balance is analyzed.

[0880] Step 13:

[0881] The server generates specific nutritional advice based on the evaluation results.

[0882] Input: Nutritional balance assessment results

[0883] Output: Nutrition advice

[0884] Specific operation: Based on the analysis results, automatically generated advice is adapted to the user profile.

[0885] Step 14:

[0886] The server transmits the generated nutrition advice to the terminal.

[0887] Enter: nutrition advice

[0888] Output: Send advice to terminal

[0889] Specific operation: The generated advice is sent to the device using a RESTful API or WebSocket.

[0890] Step 15:

[0891] The terminal receives the nutrition advice sent from the server and displays it on the user interface.

[0892] Input: Advice sent by the server

[0893] Output: Advice displayed in the user interface

[0894] Specific behavior: The received advice is visually presented to the user in a user interface within the application.

[0895] (Application example 1)

[0896] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0897] Fitness gym members need professional advice to effectively improve their training methods and nutritional management. However, receiving advice from a trainer individually is time-consuming and expensive. Furthermore, proper nutritional management is difficult to achieve through self-determination, and requires specialized knowledge. Conventional methods have made it difficult to provide this advice quickly and efficiently. Therefore, there is a need for a system that can easily provide professional advice to fitness gym members and help them optimize their training results and nutritional management.

[0898] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0899] In this invention, the server includes a filming means for filming gameplay videos, an uploading means for uploading the gameplay videos filmed by the filming means to the server, a saving means for saving the uploaded videos, an analysis means for analyzing the saved videos, an advice generating means for generating gameplay improvement advice based on the analysis results, a sending means for sending the generated advice to a terminal, a display means for displaying the sent advice, an input means for a user to input nutritional management information, an information sending means for sending the nutritional management information to the server, an evaluation means for evaluating the sent nutritional management information, a nutritional advice generating means for generating nutritional advice based on the evaluation results, a nutritional advice sending means for sending the generated nutritional advice to a terminal, and a display means for displaying the sent nutritional advice, and the gameplay improvement advice and nutritional advice are generated by referring to multiple method databases. This enables fitness gym members to professionally and efficiently improve their training methods and nutritional management.

[0900] definition statement

[0901] A "play video" is a video file containing sports or fitness-related actions filmed by a user.

[0902] "Capturing means" refers to a device for capturing video using a smartphone, smart glasses, head-mounted display, etc.

[0903] The "uploading means" is a device or application that has the function of transmitting the captured video to a server via a network.

[0904] "Storage means" refers to a system that has the function of recording and storing videos and data within a server.

[0905] The "analysis means" is a system that has the function of analyzing videos and data stored on the server and evaluating training and exercise performance.

[0906] The "advice generation means" is a system that automatically creates recommendations for improvements in training and exercise and nutritional management based on the analysis results.

[0907] The "transmission means" is a system having a function for transmitting the generated advice to the user's terminal.

[0908] The "display means" is a device or application that has the function of visually presenting advice sent from the server on the user's terminal.

[0909] "Input means" refers to a device or application that allows a user to input their own nutritional management information.

[0910] The "information transmission means" is a device or application that has the function of transmitting input nutritional management information to a server.

[0911] The "evaluation means" is a system that analyzes and evaluates the nutritional management information sent to the server.

[0912] The "nutritional advice generating means" is a system that generates individual nutritional management advice based on the nutritional information analyzed by the evaluation means.

[0913] The "nutrition advice sending means" is a system having a function of sending the generated nutritional management advice to the user's terminal.

[0914] The "method database" is a collection of data that describes multiple training methods and nutritional management methods, and is a system that provides advanced advice by referring to this data.

[0915] MODE FOR CARRYING OUT THE INVENTION

[0916] System Overview

[0917] The system that realizes this application example consists of three main elements: a server, a terminal, and a user, in order to improve the user's training performance and nutritional management.

[0918] Hardware and software used

[0919] 1. Hardware and software used by the user:

[0920] Smartphone

[0921] Smart Glasses

[0922] head-mounted display

[0923] Photo capture and upload application (developed with React Native)

[0924] 2. Software and systems used by the server:

[0925] Server: Using Node.js and Express

[0926] Database: MongoDB

[0927] Video processing: FFmpeg

[0928] Analyzing AI models: TensorFlow

[0929] Specific operation of the system

[0930] 1. User Action:

[0931] Users use a smartphone, smart glasses, or head-mounted display to record their own training videos, and then use the application to prepare the videos for uploading.

[0932] Users use the application to input their own nutritional management information (e.g., dietary content, amount of exercise).

[0933] 2. Device behavior:

[0934] The device receives the training video provided by the user, converts the video file into the appropriate format, compresses the video if necessary, and sends it to the server.

[0935] The terminal receives the nutrition management information provided by the user, converts the format and formats it, and then transmits it to the server.

[0936] The terminal receives the play improvement advice and nutrition advice sent from the server and displays them through a user interface.

[0937] 3. Server Operation:

[0938] The server stores the training video sent from the terminal.

[0939] The server analyzes the saved training videos using an analytical method (a generative AI model using TensorFlow) and evaluates the movements.

[0940] The server generates advice for improving play based on the analysis results, and the quality of this advice is improved by referencing multiple method databases.

[0941] The server transmits the generated play improvement advice to the terminal.

[0942] The server stores the nutritional management information transmitted from the terminal and evaluates the nutritional balance using the evaluation means.

[0943] The server generates nutrition advice based on the evaluation results and transmits it to the terminal.

[0944] Specific processing examples

[0945] Analyzing training videos and providing improvement advice:

[0946] The user films a squat training video on their smartphone and uploads it to the server via the app. The server then analyzes the video using a TensorFlow model, generates specific advice, such as "your hip angle should be less than 90 degrees," and sends it to the device. The device then displays this advice on the user interface, allowing the user to confirm and practice it during their next training session.

[0947] Enter nutritional information and provide advice:

[0948] The user enters what they have for breakfast (e.g., oatmeal, banana, yogurt) into the application. The device formats the information and sends it to the server. The server stores the information, evaluates the nutritional balance using an AI model, and generates specific nutritional advice, such as "add a protein shake," and sends it to the device. The device displays this advice in the user interface, and the user can incorporate it into their next meal.

[0949] Prompt Sentence Examples

[0950] Below is an example of a prompt for an AI model:

[0951] Training video analysis:

[0952] Please analyze the user's squat video and rate it based on the following items.

[0953] 1. Waist angle

[0954] 2. Knee position

[0955] 3. Back Posture

[0956] Generate nutrition advice:

[0957] Based on the breakfast information entered by the user (oatmeal, banana, yogurt), assess whether there are any protein deficiencies and, if so, suggest additional foods.

[0958] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0959] Program processing steps

[0960] Processing Steps

[0961] Step 1:

[0962] Users record training videos using a smartphone, smart glasses, or a head-mounted display.

[0963] Input: Filmed training video

[0964] Output: Recorded training video file

[0965] Step 2:

[0966] Users upload their videos to the server through the app, which uses FFmpeg to convert the video files to the appropriate format and compress them if necessary.

[0967] Input: Training video file

[0968] Data processing: format conversion, video compression

[0969] Output: Converted and compressed video file

[0970] Step 3:

[0971] The server receives and stores the converted and compressed video files.

[0972] Input: Converted and compressed video files

[0973] Output: Saved video file

[0974] Step 4:

[0975] The server analyzes the stored video files using a TensorFlow model, which uses a generative AI model to generate advice for improving gameplay.

[0976] Input: Saved video file

[0977] Data Computation: Video Analysis with Generative AI Models

[0978] Output: Advice for improving your gameplay

[0979] Step 5:

[0980] The server transmits the generated play improvement advice to the terminal.

[0981] Input: Advice for improving your game

[0982] Output: Play improvement advice sent to the device

[0983] Step 6:

[0984] The terminal displays the received advice for improving play on the user interface.

[0985] Input: Submitted game improvement advice

[0986] Output: Play improvement advice displayed in the user interface

[0987] Step 7:

[0988] The user uses a dedicated application to input nutritional management information (e.g., dietary details, amount of exercise).

[0989] Input: Nutritional management information

[0990] Output: Entered nutritional management information

[0991] Step 8:

[0992] The terminal converts and formats the input nutritional management information and sends it to the server.

[0993] Input: Nutritional management information entered

[0994] Data processing: format conversion, formatting

[0995] Output: Transformed and formatted nutrition information

[0996] Step 9:

[0997] The server receives and stores the converted and formatted nutrition management information.

[0998] Input: Transformed and formatted nutrition information

[0999] Output: Saved nutritional information

[1000] Step 10:

[1001] The server evaluates the stored nutritional management information using an evaluation tool and generates nutritional advice. This evaluation also uses a generative AI model.

[1002] Input: Saved nutrition management information

[1003] Data Computation: Evaluation with Generative AI Models

[1004] Output: Nutrition advice

[1005] Step 11:

[1006] The server transmits the generated nutrition advice to the terminal.

[1007] Enter: nutrition advice

[1008] Output: Nutrition advice sent to the device

[1009] Step 12:

[1010] The terminal displays the received nutrition advice on a user interface.

[1011] Input: Submitted nutrition advice

[1012] Output: Nutrition advice displayed in a user interface

[1013] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1014] The present invention is an innovative system for supporting sports improvement and nutritional management. In particular, by combining it with an emotion engine that recognizes the user's emotions, it is possible to provide advice optimized for each individual user. The specific operation of this system is described below.

[1015] Overall system configuration

[1016] This system consists of a server, a terminal, and a user. The specific role of each element is explained below.

[1017] User Actions

[1018] 1. A user uses a device (e.g., a smartphone or video camera) to record their sports play.

[1019] 2. The user uploads the captured gameplay video from their device to the server using a dedicated application.

[1020] 3. The user uses a dedicated application to input nutritional management information (e.g., dietary details, weight, amount of exercise).

[1021] Device behavior

[1022] 1. The device receives the gameplay video provided by the user, converts it into the appropriate format, compresses it if necessary, and sends it to the server.

[1023] 2. The terminal receives the nutritional management information provided by the user, converts and formats it, and then sends it to the server.

[1024] 3. The terminal receives the advice (play improvement advice and nutrition advice) sent from the server and displays it through the user interface.

[1025] Server Operation

[1026] 1. The server stores the gameplay video sent from the device.

[1027] 2. The server inputs the saved gameplay video into the analysis means (AI model) and begins analyzing the movements in the video.

[1028] 3. The server uses the emotion engine to analyze the user's emotions from the gameplay video.

[1029] 4. The server evaluates the results of the motion analysis and emotion analysis and identifies specific areas for improvement (e.g., "The wrist angle when shooting is insufficient").

[1030] 5. The server references a database of methods from professional athletes and famous coaches and generates custom advice based on the emotion of the analysis (e.g., "Shoot with confidence").

[1031] 6. The server sends the generated play improvement advice to the device.

[1032] 7. The server stores the nutritional management information sent from the terminal and evaluates the nutritional balance using an evaluation method (AI model).

[1033] 8. The server uses an emotion engine to analyze the emotion of the user's input.

[1034] 9. The server generates nutritional advice based on the evaluation results and sentiment analysis results (e.g., "Take more vitamin C if you tend to feel depressed").

[1035] 10. The server sends the generated nutrition advice to the terminal.

[1036] Specific examples

[1037] Analyzing gameplay videos and providing advice on improvements

[1038] 1. The user uses a smartphone to film their basketball shooting practice, then uploads the video to a dedicated application.

[1039] 2. The device receives the video file, converts the file format, compresses it, and then sends it to the server.

[1040] 3. The server receives the video and stores it in a database. The stored video is then analyzed by an analytical method (AI model).

[1041] 4. Based on the analysis results, the server identifies specific areas for improvement, such as "the wrist angle when shooting is insufficient."

[1042] 5. The server uses an emotion engine to analyze the user's emotions while playing and obtain information such as "I'm nervous."

[1043] 6. The server compares the method with that of professional players and generates custom advice such as "Keep your wrist at a 45-degree angle and shoot with confidence."

[1044] 7. The server sends the generated advice to the terminal, which displays the advice on the user interface.

[1045] 8. The user checks the advice and puts it into practice during their next shooting practice session.

[1046] Entering nutritional management information and providing advice

[1047] 1. The user enters the contents of their breakfast (e.g., oatmeal, banana, yogurt) in a dedicated application.

[1048] 2. The terminal receives the input information, converts and formats it, and sends it to the server.

[1049] 3. The server receives the nutrition management information and stores it in a database.

[1050] 4. The server uses an evaluation tool (AI model) to assess nutritional balance and identify specific nutritional improvements (e.g., "You need more protein at breakfast").

[1051] 5. The server uses an emotion engine to analyze the user's emotions at the time of input and obtain information such as "I feel stressed."

[1052] 6. The server generates custom advice such as "Add a protein shake to your breakfast and eat more vitamin C to reduce stress."

[1053] 7. The server sends the generated nutrition advice to the terminal, which displays the advice on the user interface.

[1054] 8. The user reviews the advice and incorporates it into their next meal plan.

[1055] In this way, the present invention can provide users with comprehensive, automated and personalized sports play improvement and nutritional management advice, including emotion recognition.

[1056] The processing flow will be explained below.

[1057] Step 1:

[1058] Users use devices such as smartphones and tablets to record video of sports play.

[1059] Step 2:

[1060] Users upload the videos they have taken from their device to the server using a dedicated application.

[1061] Step 3:

[1062] The device receives the video file and converts or compresses the file format as needed.

[1063] Step 4:

[1064] The device then sends the converted and compressed video file to the server.

[1065] Step 5:

[1066] The server saves the received video file in storage.

[1067] Step 6:

[1068] The server inputs the saved video into an analysis tool (AI model) and begins analyzing the movements within the video.

[1069] Step 7:

[1070] The server's analytics evaluates your movements and identifies specific areas for improvement (e.g., "Your wrist angle when shooting is insufficient").

[1071] Step 8:

[1072] The server uses an emotion engine to analyze the user's emotions in real time from gameplay video (e.g., "tension" or "concentration").

[1073] Step 9:

[1074] The server generates advice to improve play based on the results of motion analysis and emotion analysis (e.g., "Keep your wrist at a 45-degree angle and shoot with confidence").

[1075] Step 10:

[1076] The server then sends the generated advice to the device to improve your gameplay.

[1077] Step 11:

[1078] The device formats the received advice for display in a user interface.

[1079] Step 12:

[1080] The device will then display the formatted advice to the user.

[1081] Step 13:

[1082] The user can review the advice provided via the device and put it into practice the next time they practice.

[1083] Step 14:

[1084] Users use a dedicated application to enter nutritional management information (e.g., dietary content, weight, and exercise volume).

[1085] Step 15:

[1086] The terminal receives the entered nutritional management information, formats it, and sends it to the server.

[1087] Step 16:

[1088] The server receives the nutritional management information sent and stores it in a database.

[1089] Step 17:

[1090] The server uses an evaluation tool (AI model) to assess nutritional balance and extract specific nutritional improvements (e.g., "You are lacking the protein you need for breakfast").

[1091] Step 18:

[1092] The server uses an emotion engine to analyze the emotions (e.g., "stress" or "fatigue") when the user enters the meal information.

[1093] Step 19:

[1094] The server generates nutritional advice based on the evaluation and sentiment analysis results (e.g., "Add a protein shake and take more vitamin C to reduce stress").

[1095] Step 20:

[1096] The server then sends the generated nutrition advice to the device.

[1097] Step 21:

[1098] The device then formats the received nutrition advice for display in a user interface.

[1099] Step 22:

[1100] The device will then display formatted nutrition advice to the user.

[1101] Step 23:

[1102] Users can review the nutritional advice provided via their device and incorporate it into their next meal plan.

[1103] Through these steps, the system can provide comprehensive and automatic personalized sports play improvement and nutritional management advice that also takes the user's emotions into account.

[1104] Example 2

[1105] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1106] In modern sports training and nutritional management, providing individually optimized advice is extremely important, and there is a particular demand for advice that takes into account the user's emotional state. However, current systems perform motion analysis of gameplay videos and evaluation of nutritional management information separately, and there are only a limited number of systems that can generate advice by integrating emotional analysis. As a result, there is a problem in that the improvement advice and nutritional advice obtained are of insufficient quality and usefulness.

[1107] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1108] In this invention, the server includes analysis means for performing motion analysis and emotion analysis using analysis means, advice generation means for generating play improvement advice based on the results of the motion analysis and emotion analysis, evaluation means for performing the motion analysis and emotion analysis, and nutrition advice generation means for generating nutrition advice based on the results of the nutrition analysis and emotion analysis, thereby making it possible to provide high-quality play improvement advice and nutrition advice that comprehensively considers the user's motion state and emotional state.

[1109] A "play video" is a video file in which a user captures a sports play.

[1110] "Photographing means" refers to a device that a user uses to photograph sports play, such as a smartphone or video camera.

[1111] "Uploading means" refers to a function or application for transmitting gameplay videos captured by the capturing means to a server.

[1112] "Storage means" refers to the function by which the server stores uploaded gameplay videos in a database or the like.

[1113] "Analysis means" refers to a function that analyzes saved gameplay videos and evaluates the user's actions and emotional state.

[1114] "Motion analysis" is the process of analyzing the user's body movements in gameplay video.

[1115] "Emotion analysis" is the process of evaluating and determining a user's emotional state based on gameplay video and user input information.

[1116] The "advice generation means" refers to a function that generates advice for improving play to be provided to the user based on the results of the motion analysis and emotion analysis.

[1117] The "transmission means" refers to a function for transmitting the generated advice to the terminal.

[1118] The "display means" refers to a function that displays the advice sent to the terminal on the user interface.

[1119] "Input means" refers to a function or application that allows the user to input nutritional management information.

[1120] "Information transmission means" refers to a function for transmitting input nutritional management information to a server.

[1121] "Evaluation means" refers to a function that analyzes the transmitted nutritional management information and evaluates the nutritional balance.

[1122] The term "nutritional advice generating means" refers to a function that generates nutritional improvement advice to be provided to a user based on the results of nutritional analysis and emotion analysis.

[1123] The term "nutritional advice sending means" refers to a function for sending the generated nutritional advice to the terminal.

[1124] A "method database" refers to a database that stores the coaching methods of professional athletes and famous coaches.

[1125] The present invention relates to a system for supporting users in improving their sports performance and managing their nutrition. The system analyzes the user's movements and emotions and provides personalized optimization advice based on the analysis. Specific embodiments of the system, including the hardware, software, and data processing methods used, are described in detail below.

[1126] Overall system configuration

[1127] This system consists of a server, a terminal, and a user. The specific role of each element is explained below.

[1128] User Actions

[1129] Users can record their sports activities using a smartphone or video camera. After recording is complete, they can use a dedicated application to upload the video to a server. Users can also enter nutritional management information (dietary content, weight, exercise volume, etc.) through the same dedicated application.

[1130] Device behavior

[1131] The device receives gameplay videos provided by the user, performs the necessary processing, and then sends them to the server. Specifically, it uses a video processing library such as FFmpeg to convert the video format and compress the size. It also receives nutritional management information provided by the user, converts its format, formats it, and sends it to the server.

[1132] Server Operation

[1133] The server receives the data sent from the terminal and performs the following processing.

[1134] 1. Video analysis

[1135] The server stores the received gameplay video. The stored video is then analyzed using a generative AI model. This analysis uses motion analysis libraries such as OpenPose and MediaPipe to analyze the user's body movements from the video frames.

[1136] 2. Emotion analysis

[1137] The server uses an emotion engine to analyze the user's emotions from the gameplay video. It uses facial recognition and voice analysis technologies to determine the user's emotional state (e.g., tension, joy, concentration).

[1138] 3. Generating advice for improving play

[1139] The server integrates the results of the motion analysis and emotion analysis, and generates personalized, custom advice by referencing a database of methods from professional athletes and famous coaches. The generated advice is then sent to the device.

[1140] 4. Evaluation of nutritional management information

[1141] The server stores the received nutritional management information and evaluates nutritional balance using a generative AI model. The evaluation results and sentiment analysis results are combined to generate appropriate custom nutrition advice, which is then sent to the device.

[1142] Specific examples

[1143] Analyzing gameplay videos and providing advice on improvements

[1144] 1. The user uses a smartphone to film their basketball shooting practice, then uploads the video to a dedicated application.

[1145] 2. The device receives the video file, converts the file format, compresses it, and then sends it to the server.

[1146] 3. The server receives the video and stores it in a database, where it is analyzed by a generative AI model.

[1147] 4. Based on the analysis results, the server identifies specific areas for improvement, such as "the wrist angle when shooting is insufficient."

[1148] 5. The server uses an emotion engine to analyze the user's emotions while playing and obtain information such as "I'm nervous."

[1149] 6. The server compares the method with that of professional players and generates custom advice such as "Keep your wrist at a 45-degree angle and shoot with confidence."

[1150] 7. The server sends the generated advice to the terminal, which displays the advice on the user interface.

[1151] 8. The user checks the advice and puts it into practice during their next shooting practice session.

[1152] Entering nutritional management information and providing advice

[1153] 1. The user enters the contents of their breakfast (e.g., oatmeal, banana, yogurt) in a dedicated application.

[1154] 2. The terminal receives the input information, converts and formats it, and sends it to the server.

[1155] 3. The server receives the nutrition management information and stores it in a database.

[1156] 4. The server uses an evaluation tool (generative AI model) to assess nutritional balance and identify specific nutritional improvements (e.g., "You need more protein at breakfast").

[1157] 5. The server uses an emotion engine to analyze the user's emotions at the time of input and obtain information such as "I feel stressed."

[1158] 6. The server generates custom advice such as "Add a protein shake to your breakfast and eat more vitamin C to reduce stress."

[1159] 7. The server sends the generated nutrition advice to the terminal, which displays the advice on the user interface.

[1160] 8. The user reviews the advice and incorporates it into their next meal plan.

[1161] Example prompts to input to the generative AI model

[1162] Example 1: "Analyze basketball shooting practice videos uploaded by users and evaluate the wrist angle during the shot."

[1163] Example 2: "Evaluate the nutritional balance of a breakfast menu of oatmeal, banana, and yogurt and generate advice on what nutrients the user needs. If the user is feeling stressed, provide advice on what to eat."

[1164] The present invention makes it possible to provide high-quality advice for improving play and nutritional advice to users, taking into consideration their behavioral and emotional states in a comprehensive manner.

[1165] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1166] Step 1:

[1167] Users can film their own sports play using a smartphone or video camera.

[1168] Input: Sports play footage

[1169] Specific operations: Launch the smartphone's camera application and start recording. Once recording is complete, save the video file.

[1170] Output: Recorded gameplay video file

[1171] Step 2:

[1172] The user launches a dedicated application, selects the gameplay video they have taken, and uploads it to the server.

[1173] Input: gameplay video file

[1174] Specific operation: Use the application's "Video Upload" function to select the play video file on the device and click the upload button.

[1175] Output: Gameplay video data sent to the server

[1176] Step 3:

[1177] The terminal receives the gameplay video sent from the user, converts the format and compresses it, and then sends it to the server.

[1178] Input: gameplay video data

[1179] What it does: It uses the FFmpeg library to convert the video format, compress it if necessary, and then sends the video data to the server over a network connection.

[1180] Output: Format-converted and compressed video data

[1181] Step 4:

[1182] The server receives the gameplay video sent from the terminal and stores it in a database.

[1183] Input: Format-converted gameplay video data

[1184] Specific operation: Connect to the database and save the video data and metadata (shooting date and time, user ID, etc.).

[1185] Output: Stored video data and metadata

[1186] Step 5:

[1187] The server analyzes the saved gameplay video using a generated AI model.

[1188] Input: Saved video data

[1189] Specific operation: Input video frames into the generative AI model and analyze the user's body movements (e.g., using the OpenPose or MediaPipe library). Save the analysis results.

[1190] Output: Motion analysis result data (e.g., position information of each body part)

[1191] Step 6:

[1192] The server uses an emotion engine to analyze the user's emotions from the gameplay video.

[1193] Input: Stored video and audio data

[1194] Specific operation: Determine the user's emotional state using facial recognition and voice analysis technology. Save the analysis results.

[1195] Output: Emotion analysis result data (e.g., tension, joy, concentration, etc.)

[1196] Step 7:

[1197] The server integrates the results of the motion analysis and emotion analysis, and generates play improvement advice by referring to a method database of professional athletes.

[1198] Input: Motion analysis result data and emotion analysis result data

[1199] Specific operation: Based on the analysis results, relevant advice is extracted from the method database and custom advice is generated.

[1200] Output: Generated play improvement advice data

[1201] Step 8:

[1202] The server transmits the generated play improvement advice to the terminal.

[1203] Input: Generated play improvement advice data

[1204] Specific operation: Advice data is sent to the terminal via a network connection.

[1205] Output: Advice data sent to the terminal

[1206] Step 9:

[1207] The terminal receives the play improvement advice sent from the server and displays it on the user interface.

[1208] Input: Play improvement advice data

[1209] Specific behavior: Display the advice text in the "Advice" section of the application.

[1210] Output: Advice displayed in the user interface

[1211] Step 10:

[1212] The user uses a dedicated application to input nutritional management information (dietary content, weight, amount of exercise, etc.).

[1213] Input: Nutritional management information

[1214] Specific operations: Enter information such as diet, weight, and exercise amount into the "Nutrition Management" form within the app and click the submit button.

[1215] Output: Input nutritional management information data

[1216] Step 11:

[1217] The terminal receives the input nutrition management information, converts and formats it, and then transmits it to the server.

[1218] Input: Nutritional management information data

[1219] Specific operation: Convert the received data into JSON format and send it to the server over the network.

[1220] Output: Formatted nutrition management information data

[1221] Step 12:

[1222] The server receives the nutritional management information sent from the terminal and stores it in a database.

[1223] Input: Formatted nutrition management information data

[1224] Specific operation: Connect to the database and save nutrition management information.

[1225] Output: Saved nutritional management information data

[1226] Step 13:

[1227] The server uses a generative AI model to assess nutritional balance and identify specific nutritional improvements.

[1228] Input: Saved nutrition management information data

[1229] Specific operation: Input nutritional management information into the generative AI model, analyze and evaluate nutritional balance, and save the evaluation results.

[1230] Output: Nutritional balance assessment result data

[1231] Step 14:

[1232] The server uses an emotion engine to analyze the user's emotions when inputting nutritional management information.

[1233] Input: Voice data and facial image data when entering nutrition management information

[1234] Specific operation: Determine the user's emotional state using facial recognition and voice analysis technology. Save the analysis results.

[1235] Output: Emotion analysis result data (e.g., stress, joy, etc.)

[1236] Step 15:

[1237] The server generates custom nutrition advice based on the nutritional assessment results and the sentiment analysis results.

[1238] Input: Nutritional balance assessment result data and emotion analysis result data

[1239] Specific operation: Integrates evaluation results and sentiment analysis results to generate appropriate nutrition advice.

[1240] Output: Generated custom nutrition advice data

[1241] Step 16:

[1242] The server transmits the generated nutrition advice to the terminal.

[1243] Input: Generated custom nutrition advice data

[1244] Specific operation: Advice data is sent to the terminal via a network connection.

[1245] Output: Advice data sent to the terminal

[1246] Step 17:

[1247] The terminal receives the nutrition advice sent from the server and displays it on the user interface.

[1248] Input: Nutrition advice data

[1249] What it does: Displays the advice in text form in the "Nutrition Advice" section of the application.

[1250] Output: Nutrition advice displayed on the user interface

[1251] Through this system, users can receive individually optimized advice on improving their sports play and nutritional management.

[1252] (Application example 2)

[1253] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1254] In recent years, automated robots have played an important role in many industries. However, these robots occasionally experience performance degradation or malfunction. To solve this problem, a system is needed that can properly monitor the robot's behavior and status and provide immediate advice for performance improvement. However, current technology does not provide a system that comprehensively analyzes the robot's behavior and emotions (state) and provides performance improvement advice. Therefore, the objective of this invention is to provide a system that comprehensively supports not only the improvement of sports performance but also the performance improvement and status monitoring of robotic work in factories.

[1255] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a filming means for filming gameplay videos; an uploading means for uploading the gameplay videos filmed by the filming means to the server; a saving means for saving the uploaded videos; an analyzing means for analyzing the saved videos; an advice generating means for generating play improvement advice based on the analysis results; a transmitting means for sending the generated advice to a terminal; a display means for displaying the sent advice; an improvement advice generating means for generating performance improvement advice for the robot based on the analysis results; an emotion analysis means by which the improvement advice generating means analyzes the state of the robot using sensor data; a transmitting means for sending the advice generated by the emotion analysis means to a terminal; and a display means for displaying the sent advice. This makes it possible to monitor the operating state of the robot in real time and immediately provide performance improvement advice based on emotion analysis.

[1256] "Photographing means" refers to a device or system for capturing videos or images.

[1257] The "uploading means" is a method or device for transmitting captured data to a server via a network.

[1258] "Storage means" refers to a method or device for temporarily or long-term storage of videos and data sent to the server.

[1259] "Analysis means" refers to a method or device for analyzing stored video or data and extracting specific information or features.

[1260] The "advice generation means" is a method or device that generates improvement instructions for the user based on the data obtained by the analysis means.

[1261] The "transmission means" is a method or device for sending information such as generated advice to the user's terminal.

[1262] "Display means" refers to a method or device for visually displaying advice or information sent to a user's terminal.

[1263] The "improvement advice generating means" is a method or device that generates instructions for improving the performance of a robot based on an analysis of the robot's movements and state.

[1264] "Emotion analysis means" refers to a method or device that analyzes the robot's sensor data and evaluates the robot's current state and performance.

[1265] "Sensor data" refers to data obtained from various sensors attached to robots and other devices, and is data that measures and records operating conditions and environmental conditions.

[1266] The present invention is a system for monitoring the operation of a factory robot and improving its performance. The system operates based on sensor data, including an image capturing means, an uploading means, a storage means, an analysis means, an advice generating means, a transmission means, a display means, an improvement advice generating means, and an emotion analysis means.

[1267] Overall system configuration

[1268] This system consists of a server, a terminal, and a user (in this case, the factory manager). The specific configuration and operation of each element are shown below.

[1269] Filming method

[1270] Cameras are used to capture the movements of robots in factories. The captured video is high resolution and contains important motion analysis data.

[1271] Upload method

[1272] The user uploads the video file of the robot's movements to the device through a dedicated application (e.g., a smartphone app). The uploaded video is converted into an appropriate format and sent to the server.

[1273] Preservation means

[1274] The server stores the received video files in a database, which allows for efficient management and storage of multiple video files.

[1275] Analysis means

[1276] The server processes the stored video using analytical means, specifically, an AI model (e.g., a motion analysis model) to analyze the robot's movement patterns and abnormalities in the video.

[1277] Advice Generation Method

[1278] Based on the data extracted by the analysis means, the advice generation means generates specific advice for improving the performance of the robot. The generated advice is optimized for the operating state of each individual robot.

[1279] Transmission method

[1280] The generated advice is transmitted from the server to the terminal, and the transmitting means enables the advice to be provided to the user in real time.

[1281] Display means

[1282] The terminal visually displays the received advice to the user. The display means includes an interface for concisely and clearly showing the content of the advice.

[1283] Improvement advice generation method

[1284] This is a means for generating more advanced performance improvement advice based on the data provided by the analysis means and the sentiment analysis means. The sentiment analysis means uses sensor data to analyze the robot's condition (e.g., temperature, load).

[1285] Hardware and software used

[1286] Hardware:

[1287] Camera: A device for filming the robot's movements.

[1288] Sensor: A device that acquires robot status data (temperature, load, etc.).

[1289] software:

[1290] Dedicated application: Upload videos, convert formats, and compress them.

[1291] Server: Stores and analyzes video and sensor data.

[1292] AI models: AI models for video analysis (e.g., Keras), AI models for sentiment analysis (e.g., Keras).

[1293] Specific examples

[1294] For example, a user can film a robot working in a factory with a camera and upload the video using a dedicated application. The video is then converted and compressed on the device and sent to the server. The server then saves the video and analyzes the robot's movements and status using analysis and emotion analysis. Based on the analysis results, the advice generation means creates specific advice for improving the robot's movements, which is then sent to the device and displayed to the user.

[1295] Examples of prompts:

[1296] "Write a program that analyzes this video file and sensor data to evaluate the robot's accuracy and health. Add logic to generate specific improvements and recommendations."

[1297] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1298] Step 1:

[1299] The user takes pictures of the robot's operation in the factory with a camera.

[1300] Input: Real-time video of robot movement.

[1301] Output: Video file of robot movement.

[1302] Specific operation: Cameras are installed in appropriate locations within the factory to capture high-resolution images of the robot's work process.

[1303] Step 2:

[1304] The user uploads the video file they have taken to the device using a dedicated application.

[1305] Input: Video file.

[1306] Output: Video file stored on your device.

[1307] Specific operation: Transfer and save video files to the device using a dedicated application.

[1308] Step 3:

[1309] The device sends the video file to the server.

[1310] Input: Video files stored on your device.

[1311] Output: Video file uploaded to the server.

[1312] Specific operation: The video file is compressed and format converted on the terminal, and then sent to the server via the network.

[1313] Step 4:

[1314] The server stores the received video file.

[1315] Input: Video files uploaded to the server.

[1316] Output: Video files stored in the database.

[1317] Specific operation: Video files are stored in a database on the server and managed appropriately.

[1318] Step 5:

[1319] The server analyzes the stored video.

[1320] Input: Video files stored in the database.

[1321] Output: Robot motion data in video.

[1322] Specific behavior: Using an AI model (e.g., a motion analysis model), the robot's movement patterns and abnormalities are analyzed for each video frame.

[1323] Step 6:

[1324] The server uses an emotion analysis means to analyze the state of the robot.

[1325] Input: Robot sensor data (temperature, load, etc.).

[1326] Output: Robot state data.

[1327] Specific behavior: Using an emotion analysis model, sensor data is analyzed to assess the robot's current behavioral state.

[1328] Step 7:

[1329] The server uses the advice generation means to create improvement advice based on the analysis results.

[1330] Input: Video analysis data and state data.

[1331] Output: Improvement advice.

[1332] Specific behavior: Based on the analysis results, specific advice is generated to optimize the robot's behavior (e.g., "reduce the part sorting speed by 20%)."

[1333] Step 8:

[1334] The server transmits the generated advice to the terminal.

[1335] Input: Improvement advice.

[1336] Output: Advice sent to the terminal.

[1337] Specific operation: The generated advice is sent to the user's terminal via the network.

[1338] Step 9:

[1339] The advice received by the terminal is displayed on a user interface.

[1340] Input: Advice sent by the server.

[1341] Output: The advice displayed to the user.

[1342] Specific operation: The advice content is displayed in the user interface in a format that is easy for the user to view.

[1343] Through these steps, the entire system is configured to effectively monitor the operation of robots in the factory and provide timely improvement advice.

[1344] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1345] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1346] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1347] [Third embodiment]

[1348] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1349] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1350] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1351] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1352] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1353] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1354] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1355] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1356] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1357] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1358] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1359] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1360] The present invention is an innovative system for supporting the improvement of sports play and nutritional management. The specific operation of this system will be described below.

[1361] Overall system configuration

[1362] This system consists of a server, a terminal, and a user. The specific role of each element is explained below.

[1363] User Actions

[1364] 1. A user uses a device (e.g., a smartphone or video camera) to record their sports play.

[1365] 2. Users upload the gameplay videos they have taken using a dedicated application on their device.

[1366] 3. The user uses a dedicated application to enter nutritional management information (e.g., dietary details, weight, and exercise amount).

[1367] Device behavior

[1368] 1. The device receives the gameplay video provided by the user, converts it into the appropriate format, compresses it if necessary, and sends it to the server.

[1369] 2. The terminal receives the nutritional management information provided by the user, converts and formats it, and then sends it to the server.

[1370] 3. The terminal receives the advice (play improvement advice and nutrition advice) sent from the server and displays it through the user interface.

[1371] Server Operation

[1372] 1. The server stores the gameplay video sent from the device.

[1373] 2. The server analyzes the saved gameplay video using analytical means (AI model) and evaluates the movement.

[1374] 3. The server generates advice for improving play based on the analysis results. The quality of the generated advice is improved by referencing multiple method databases.

[1375] 4. The server sends the generated play improvement advice to the device.

[1376] 5. The server stores the nutritional management information sent from the terminal and evaluates the nutritional balance using an evaluation method (AI model).

[1377] 6. The server generates nutrition advice based on the evaluation results.

[1378] 7. The server sends the generated nutrition advice to the terminal.

[1379] Specific examples

[1380] Analyzing gameplay videos and providing advice on improvements

[1381] 1. The user uses a smartphone to film their basketball shooting practice, then uploads the video to a dedicated application.

[1382] 2. The device receives the video file, converts the file format, compresses it, and then sends it to the server.

[1383] 3. The server receives the video and stores it in a database. The stored video is then analyzed by an analytical method (AI model).

[1384] 4. Based on the analysis results, the server identifies specific areas for improvement, such as "the wrist angle when shooting is insufficient."

[1385] 5. The server compares the method with that of a professional player and generates custom advice such as "Keep your wrist at a 45-degree angle."

[1386] 6. The server sends the generated advice to the terminal, which displays the advice on the user interface.

[1387] 7. The user checks the advice and puts it into practice during their next shooting practice session.

[1388] Entering nutritional management information and providing advice

[1389] 1. The user enters the contents of their breakfast (e.g., oatmeal, banana, yogurt) in a dedicated application.

[1390] 2. The terminal receives the input information, converts and formats it, and sends it to the server.

[1391] 3. The server receives the nutrition management information and stores it in a database.

[1392] 4. The server uses an evaluation tool (AI model) to assess nutritional balance and identify specific areas for improvement, such as "you need more protein at breakfast."

[1393] 5. The server generates a custom advice such as "Add a protein shake to breakfast."

[1394] 6. The server sends the generated nutrition advice to the terminal, which displays the advice on the user interface.

[1395] 7. The user confirms the advice and applies it to their next meal.

[1396] In this way, the present invention provides users with personalized sports play improvement and nutritional management advice to help improve overall performance.

[1397] The processing flow will be explained below.

[1398] Step 1:

[1399] Users use devices such as smartphones and tablets to record video of sports play.

[1400] Step 2:

[1401] Users upload the videos they have taken from their device to the server using a dedicated application.

[1402] Step 3:

[1403] The device receives the video file and converts or compresses the file format as needed.

[1404] Step 4:

[1405] The device then sends the converted and compressed video file to the server.

[1406] Step 5:

[1407] The server saves the received video file in storage.

[1408] Step 6:

[1409] The server inputs the saved video into an analysis tool (AI model) and begins analyzing the movements within the video.

[1410] Step 7:

[1411] The server's analytics evaluates your movements and identifies specific areas for improvement (e.g., "Your wrist angle when shooting is insufficient").

[1412] Step 8:

[1413] The server references a database of methods from professional athletes and famous coaches and generates custom advice based on the analysis results.

[1414] Step 9:

[1415] The server then sends the generated advice to the device to improve your gameplay.

[1416] Step 10:

[1417] The device formats the received advice for display in a user interface.

[1418] Step 11:

[1419] The device will then display the formatted advice to the user.

[1420] Step 12:

[1421] The user can review the advice provided via the device and put it into practice the next time they practice.

[1422] Step 13:

[1423] Users use a dedicated application to enter nutritional management information (e.g., dietary content, weight, and exercise volume).

[1424] Step 14:

[1425] The terminal receives the entered nutritional management information, formats it, and sends it to the server.

[1426] Step 15:

[1427] The server receives the nutritional management information sent and stores it in a database.

[1428] Step 16:

[1429] The server uses an evaluation tool (AI model) to assess nutritional balance and extract specific nutritional areas for improvement (e.g., "lack of protein").

[1430] Step 17:

[1431] The server then references a nutrition database and generates custom nutrition advice based on the assessment results.

[1432] Step 18:

[1433] The server then sends the generated nutrition advice to the device.

[1434] Step 19:

[1435] The device then formats the received nutrition advice for display in a user interface.

[1436] Step 20:

[1437] The device will then display formatted nutrition advice to the user.

[1438] Step 21:

[1439] Users can review the nutritional advice provided via their device and incorporate it into their next meal plan.

[1440] Through these steps, the system can provide users with comprehensive and automated personalized sports play improvement and nutritional management advice.

[1441] Example 1

[1442] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1443] Conventional sports performance improvement and nutrition management systems have limitations in analyzing data and providing appropriate advice to individual users. Furthermore, analysis of gameplay videos and evaluation of nutrition management information are often done manually, resulting in insufficient automation and efficiency. This makes it difficult for users to obtain effective improvement measures in a timely manner, making it difficult to improve overall performance.

[1444] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1445] In this invention, the server includes: a filming means for filming gameplay videos; an uploading means for uploading the gameplay videos filmed by the filming means via a dedicated application on the terminal; a transmitting means for receiving the uploaded videos at the terminal, converting the file format and compressing them, and transmitting them to the server; a storing means for saving the transmitted videos on the server; an analyzing means for analyzing the saved videos using an AI model; an advice generating means for generating gameplay improvement advice based on the analysis results; a play advice sending means for sending the generated gameplay improvement advice to the terminal; a display means for displaying the transmitted gameplay improvement advice on a terminal user interface; an input means for inputting nutritional management information; a nutritional information sending means for receiving the input nutritional management information at the terminal, shaping and formatting the information, and transmitting it to the server; a nutrition saving means for saving the transmitted nutritional management information on the server; an evaluating means for evaluating the stored nutritional management information using an AI model; a nutritional advice generating means for generating nutritional advice based on the evaluation results; a nutritional advice sending means for sending the generated nutritional advice to the terminal; and a display means for displaying the transmitted nutritional advice on the terminal user interface. This will enable the analysis of individual users' sports play and nutritional management to be automated, enabling the provision of quick and effective advice on improvement.

[1446] A "play video" is a video file in which a user captures a sports play.

[1447] "Filming means" refers to devices or methods for filming sports play, specifically smartphones and video cameras.

[1448] The "uploading means" is a means for transmitting the captured gameplay video to the server, and is a function or process that is performed through a dedicated application.

[1449] "Transmission means" refers to a method for transferring data from a user terminal to a server, and includes a POST request using the HTTP protocol.

[1450] "Storage" means a system or method for retaining transmitted data, including the use of a database or storage system.

[1451] "Analysis means" refers to the processes and tools used to analyze stored data, specifically the use of AI models.

[1452] The "advice generation means" is a process for creating improvement advice for the user based on the analysis results.

[1453] The "play advice sending means" refers to a function for sending the generated improvement advice to the user terminal.

[1454] The "display means" is an interface for visually presenting information to the user on the terminal.

[1455] "Input means" refers to a method by which a user inputs nutritional management information into the system, such as a form in a dedicated application.

[1456] "Nutrition information transmission means" refers to a function for transmitting input nutrition management information to a server.

[1457] "Nutrition storage means" refers to a system or method for storing transmitted nutrition management information.

[1458] "Evaluation methods" are processes and tools for analyzing and evaluating stored nutrition management information, and use AI models.

[1459] "Nutrition advice generator" refers to a process that generates nutrition advice based on the evaluation results.

[1460] The term "nutrition advice sending means" refers to a function for sending the generated nutrition advice to the user terminal.

[1461] This invention is an innovative system for supporting sports performance improvement and nutritional management. This system is composed of a server, terminals, and users, and each element works in cooperation with each other.

[1462] User Actions

[1463] First, the user films their own sports play (e.g., a basketball shot) using a smartphone or video camera. Next, they upload the video to their device using a dedicated application. They also use the same application to enter their daily nutrition management information (e.g., dietary details, weight, and exercise volume).

[1464] Device behavior

[1465] The device receives gameplay videos uploaded by users, converts the video files to the appropriate format using tools such as FFmpeg, and compresses them by adjusting the bitrate and resolution. It then sends the videos to the server. It also receives nutritional management information entered by the user and formats the data into JSON format. It then sends the formatted nutritional information to the server. It also receives gameplay improvement advice and nutritional advice sent from the server and displays them on the user interface.

[1466] Server Operation

[1467] The server receives gameplay videos sent from the device and stores them in a database. For example, a storage system dedicated to video files (e.g., Amazon S3) can be used. The stored videos are input into an AI analysis model (e.g., OpenPose or MediaPipe) for analysis and evaluation of the movements. Based on the analysis results, advice is created to improve the user's gameplay. This advice is optimized by referencing multiple method databases. The generated advice is then sent to the device and presented to the user.

[1468] The server receives the nutritional management information sent from the device and stores it in a database. The stored nutritional information is input into an AI model (e.g., TensorFlow) for analysis and an evaluation of nutritional balance is performed. Based on the evaluation results, specific nutritional advice is generated and sent to the device for presentation to the user.

[1469] Specific examples

[1470] Analyzing gameplay videos and providing advice on improvements

[1471] For example, a user uses a smartphone to film basketball shooting practice and uploads the video to a dedicated application. The device receives the video file, converts the file format, compresses it, and sends it to a server. The server then receives the video and stores it in a database. The saved video is analyzed using an AI model (e.g., OpenPose) to identify specific areas for improvement, such as "your wrist angle when shooting is insufficient." Based on the analysis results, custom advice is generated, such as "keep your wrist at a 45-degree angle." The server then sends the generated advice to the device, which displays it on the user interface. The user then checks the advice and puts it into practice during their next shooting practice.

[1472] Entering nutritional management information and providing advice

[1473] For example, a user uses a dedicated application to input the contents of their breakfast (e.g., oatmeal, banana, yogurt). The device receives the input information, converts and formats it, and sends it to the server. The server receives the information and stores it in a database. The stored information is input into an AI model for analysis, and specific areas for improvement, such as "You need more protein for breakfast," are identified. Based on the results, custom advice, such as "Add a protein shake to breakfast," is generated, and the server sends the advice to the device. The device displays the advice on the user interface, and the user can incorporate it into their next meal.

[1474] Examples of prompt statements

[1475] An example of a prompt sentence generated by a generative AI model could be written as follows:

[1476] "What is the optimal wrist angle when shooting a basketball?"

[1477] "What foods should I add to my breakfast to improve its nutritional balance?"

[1478] In this way, the present invention provides useful advice to users in sports play and nutritional management, and helps improve overall performance.

[1479] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1480] Step 1:

[1481] Users can film their own sports play using a smartphone or video camera and then upload the recorded video files using a dedicated application.

[1482] Input: Recorded video file

[1483] Output: Video file for upload

[1484] Specific operation: The user opens the dedicated application, selects the recorded video file, and presses the upload button.

[1485] Step 2:

[1486] The device receives the gameplay video file uploaded by the user, converts the video file format to MP4 format using a tool such as FFmpeg, and compresses it if necessary.

[1487] Input: User uploaded video file

[1488] Output: Converted and compressed video file

[1489] Specific operation: The device receives the video file, converts the format using FFmpeg, and compresses it by adjusting the bitrate and resolution.

[1490] Step 3:

[1491] The device sends the converted and compressed video file to the server.

[1492] Input: Converted and compressed video files

[1493] Output: Video file sent to the server

[1494] Specific behavior: Sends the file to the server using a POST request using the HTTP protocol.

[1495] Step 4:

[1496] The server receives the video file sent from the terminal and stores the video file in a database.

[1497] Input: Video file sent to the server

[1498] Output: Video files stored in a database

[1499] Specific behavior: Receives video files and stores them in a database (e.g., Amazon S3) along with associated metadata (e.g., user ID, timestamp).

[1500] Step 5:

[1501] The server analyzes the stored video files using an AI analysis model (e.g., OpenPose, MediaPipe) and evaluates the movements.

[1502] Input: Video files stored in the database

[1503] Output: Analysis results (motion evaluation data)

[1504] Specific operation: The saved video file is input into the AI ​​model, and the characteristics of the user's movements (e.g., wrist angle, body posture) are extracted.

[1505] Step 6:

[1506] The server generates advice for improving play based on the analysis results, and this advice is optimized by referencing multiple method databases.

[1507] Input: Analysis results (motion evaluation data)

[1508] Output: Advice for improving your gameplay

[1509] Specific operation: Based on the analysis results, automatically generated advice is compared with the user profile and method database to optimize the system.

[1510] Step 7:

[1511] The server transmits the generated play improvement advice to the terminal.

[1512] Input: Advice for improving your game

[1513] Output: Send advice to terminal

[1514] Specific operation: The generated advice is sent to the device using a RESTful API or WebSocket.

[1515] Step 8:

[1516] The terminal receives the play improvement advice sent from the server and displays it on the user interface.

[1517] Input: Advice sent by the server

[1518] Output: Advice displayed in the user interface

[1519] Specific behavior: The received advice is visually presented to the user in a user interface within the application.

[1520] Step 9:

[1521] The user uses a dedicated application to input nutritional management information (e.g., dietary content, weight, amount of exercise).

[1522] Input: Nutritional management information (dietary content, weight, exercise amount)

[1523] Output: Nutritional management information entered into the application

[1524] Specific operation: Enter diet and exercise data using the input form in the dedicated application.

[1525] Step 10:

[1526] The terminal receives nutritional management information entered by the user, formats the data into JSON format, and sends it to the server.

[1527] Input: Nutritional management information entered by the user

[1528] Output: Formatted data sent to the server

[1529] Specific operation: Converts input data into JSON format and sends it to the server via an HTTP POST request.

[1530] Step 11:

[1531] The server receives the nutritional management information sent from the terminal and stores it in a database.

[1532] Input: Nutritional management information sent to the server

[1533] Output: Nutritional management information stored in a database

[1534] Specific operation: Save nutritional management information in a database.

[1535] Step 12:

[1536] The server analyzes the stored nutritional management information using an AI analysis model (e.g., TensorFlow) and evaluates nutritional balance.

[1537] Input: Nutritional management information stored in the database

[1538] Output: Nutritional balance evaluation results

[1539] Specific operation: Nutritional management information is input into the AI ​​model and nutritional balance is analyzed.

[1540] Step 13:

[1541] The server generates specific nutritional advice based on the evaluation results.

[1542] Input: Nutritional balance assessment results

[1543] Output: Nutrition advice

[1544] Specific operation: Based on the analysis results, automatically generated advice is adapted to the user profile.

[1545] Step 14:

[1546] The server transmits the generated nutrition advice to the terminal.

[1547] Enter: nutrition advice

[1548] Output: Send advice to terminal

[1549] Specific operation: The generated advice is sent to the device using a RESTful API or WebSocket.

[1550] Step 15:

[1551] The terminal receives the nutrition advice sent from the server and displays it on the user interface.

[1552] Input: Advice sent by the server

[1553] Output: Advice displayed in the user interface

[1554] Specific behavior: The received advice is visually presented to the user in a user interface within the application.

[1555] (Application example 1)

[1556] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1557] Fitness gym members need professional advice to effectively improve their training methods and nutritional management. However, receiving advice from a trainer individually is time-consuming and expensive. Furthermore, proper nutritional management is difficult to achieve through self-determination, and requires specialized knowledge. Conventional methods have made it difficult to provide this advice quickly and efficiently. Therefore, there is a need for a system that can easily provide professional advice to fitness gym members and help them optimize their training results and nutritional management.

[1558] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1559] In this invention, the server includes a filming means for filming gameplay videos, an uploading means for uploading the gameplay videos filmed by the filming means to the server, a saving means for saving the uploaded videos, an analysis means for analyzing the saved videos, an advice generating means for generating gameplay improvement advice based on the analysis results, a sending means for sending the generated advice to a terminal, a display means for displaying the sent advice, an input means for a user to input nutritional management information, an information sending means for sending the nutritional management information to the server, an evaluation means for evaluating the sent nutritional management information, a nutritional advice generating means for generating nutritional advice based on the evaluation results, a nutritional advice sending means for sending the generated nutritional advice to a terminal, and a display means for displaying the sent nutritional advice, and the gameplay improvement advice and nutritional advice are generated by referring to multiple method databases. This enables fitness gym members to professionally and efficiently improve their training methods and nutritional management.

[1560] definition statement

[1561] A "play video" is a video file containing sports or fitness-related actions filmed by a user.

[1562] "Capturing means" refers to a device for capturing video using a smartphone, smart glasses, head-mounted display, etc.

[1563] The "uploading means" is a device or application that has the function of transmitting the captured video to a server via a network.

[1564] "Storage means" refers to a system that has the function of recording and storing videos and data within a server.

[1565] The "analysis means" is a system that has the function of analyzing videos and data stored on the server and evaluating training and exercise performance.

[1566] The "advice generation means" is a system that automatically creates recommendations for improvements in training and exercise and nutritional management based on the analysis results.

[1567] The "transmission means" is a system having a function for transmitting the generated advice to the user's terminal.

[1568] The "display means" is a device or application that has the function of visually presenting advice sent from the server on the user's terminal.

[1569] "Input means" refers to a device or application that allows a user to input their own nutritional management information.

[1570] The "information transmission means" is a device or application that has the function of transmitting input nutritional management information to a server.

[1571] The "evaluation means" is a system that analyzes and evaluates the nutritional management information sent to the server.

[1572] The "nutritional advice generating means" is a system that generates individual nutritional management advice based on the nutritional information analyzed by the evaluation means.

[1573] The "nutrition advice sending means" is a system having a function of sending the generated nutritional management advice to the user's terminal.

[1574] The "method database" is a collection of data that describes multiple training methods and nutritional management methods, and is a system that provides advanced advice by referring to this data.

[1575] MODE FOR CARRYING OUT THE INVENTION

[1576] System Overview

[1577] The system that realizes this application example consists of three main elements: a server, a terminal, and a user, in order to improve the user's training performance and nutritional management.

[1578] Hardware and software used

[1579] 1. Hardware and software used by the user:

[1580] Smartphone

[1581] Smart Glasses

[1582] head-mounted display

[1583] Photo capture and upload application (developed with React Native)

[1584] 2. Software and systems used by the server:

[1585] Server: Using Node.js and Express

[1586] Database: MongoDB

[1587] Video processing: FFmpeg

[1588] Analyzing AI models: TensorFlow

[1589] Specific operation of the system

[1590] 1. User Action:

[1591] Users use a smartphone, smart glasses, or head-mounted display to record their own training videos, and then use the application to prepare the videos for uploading.

[1592] Users use the application to input their own nutritional management information (e.g., dietary content, amount of exercise).

[1593] 2. Device behavior:

[1594] The device receives the training video provided by the user, converts the video file into the appropriate format, compresses the video if necessary, and sends it to the server.

[1595] The terminal receives the nutrition management information provided by the user, converts the format and formats it, and then transmits it to the server.

[1596] The terminal receives the play improvement advice and nutrition advice sent from the server and displays them through a user interface.

[1597] 3. Server Operation:

[1598] The server stores the training video sent from the terminal.

[1599] The server analyzes the saved training videos using an analytical method (a generative AI model using TensorFlow) and evaluates the movements.

[1600] The server generates advice for improving play based on the analysis results, and the quality of this advice is improved by referencing multiple method databases.

[1601] The server transmits the generated play improvement advice to the terminal.

[1602] The server stores the nutritional management information transmitted from the terminal and evaluates the nutritional balance using the evaluation means.

[1603] The server generates nutrition advice based on the evaluation results and transmits it to the terminal.

[1604] Specific processing examples

[1605] Analyzing training videos and providing improvement advice:

[1606] The user films a squat training video on their smartphone and uploads it to the server via the app. The server then analyzes the video using a TensorFlow model, generates specific advice, such as "your hip angle should be less than 90 degrees," and sends it to the device. The device then displays this advice on the user interface, allowing the user to confirm and practice it during their next training session.

[1607] Enter nutritional information and provide advice:

[1608] The user enters what they have for breakfast (e.g., oatmeal, banana, yogurt) into the application. The device formats the information and sends it to the server. The server stores the information, evaluates the nutritional balance using an AI model, and generates specific nutritional advice, such as "add a protein shake," and sends it to the device. The device displays this advice in the user interface, and the user can incorporate it into their next meal.

[1609] Prompt Sentence Examples

[1610] Below is an example of a prompt for an AI model:

[1611] Training video analysis:

[1612] Please analyze the user's squat video and rate it based on the following items.

[1613] 1. Waist angle

[1614] 2. Knee position

[1615] 3. Back Posture

[1616] Generate nutrition advice:

[1617] Based on the breakfast information entered by the user (oatmeal, banana, yogurt), assess whether there are any protein deficiencies and, if so, suggest additional foods.

[1618] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1619] Program processing steps

[1620] Processing Steps

[1621] Step 1:

[1622] Users record training videos using a smartphone, smart glasses, or a head-mounted display.

[1623] Input: Filmed training video

[1624] Output: Recorded training video file

[1625] Step 2:

[1626] Users upload their videos to the server through the app, which uses FFmpeg to convert the video files to the appropriate format and compress them if necessary.

[1627] Input: Training video file

[1628] Data processing: format conversion, video compression

[1629] Output: Converted and compressed video file

[1630] Step 3:

[1631] The server receives and stores the converted and compressed video files.

[1632] Input: Converted and compressed video files

[1633] Output: Saved video file

[1634] Step 4:

[1635] The server analyzes the stored video files using a TensorFlow model, which uses a generative AI model to generate advice for improving gameplay.

[1636] Input: Saved video file

[1637] Data Computation: Video Analysis with Generative AI Models

[1638] Output: Advice for improving your gameplay

[1639] Step 5:

[1640] The server transmits the generated play improvement advice to the terminal.

[1641] Input: Advice for improving your game

[1642] Output: Play improvement advice sent to the device

[1643] Step 6:

[1644] The terminal displays the received advice for improving play on the user interface.

[1645] Input: Submitted game improvement advice

[1646] Output: Play improvement advice displayed in the user interface

[1647] Step 7:

[1648] The user uses a dedicated application to input nutritional management information (e.g., dietary details, amount of exercise).

[1649] Input: Nutritional management information

[1650] Output: Entered nutritional management information

[1651] Step 8:

[1652] The terminal converts and formats the input nutritional management information and sends it to the server.

[1653] Input: Nutritional management information entered

[1654] Data processing: format conversion, formatting

[1655] Output: Transformed and formatted nutrition information

[1656] Step 9:

[1657] The server receives and stores the converted and formatted nutrition management information.

[1658] Input: Transformed and formatted nutrition information

[1659] Output: Saved nutritional information

[1660] Step 10:

[1661] The server evaluates the stored nutritional management information using an evaluation tool and generates nutritional advice. This evaluation also uses a generative AI model.

[1662] Input: Saved nutrition management information

[1663] Data Computation: Evaluation with Generative AI Models

[1664] Output: Nutrition advice

[1665] Step 11:

[1666] The server transmits the generated nutrition advice to the terminal.

[1667] Enter: nutrition advice

[1668] Output: Nutrition advice sent to the device

[1669] Step 12:

[1670] The terminal displays the received nutrition advice on a user interface.

[1671] Input: Submitted nutrition advice

[1672] Output: Nutrition advice displayed in a user interface

[1673] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1674] The present invention is an innovative system for supporting sports improvement and nutritional management. In particular, by combining it with an emotion engine that recognizes the user's emotions, it is possible to provide advice optimized for each individual user. The specific operation of this system is described below.

[1675] Overall system configuration

[1676] This system consists of a server, a terminal, and a user. The specific role of each element is explained below.

[1677] User Actions

[1678] 1. A user uses a device (e.g., a smartphone or video camera) to record their sports play.

[1679] 2. The user uploads the captured gameplay video from their device to the server using a dedicated application.

[1680] 3. The user uses a dedicated application to input nutritional management information (e.g., dietary details, weight, amount of exercise).

[1681] Device behavior

[1682] 1. The device receives the gameplay video provided by the user, converts it into the appropriate format, compresses it if necessary, and sends it to the server.

[1683] 2. The terminal receives the nutritional management information provided by the user, converts and formats it, and then sends it to the server.

[1684] 3. The terminal receives the advice (play improvement advice and nutrition advice) sent from the server and displays it through the user interface.

[1685] Server Operation

[1686] 1. The server stores the gameplay video sent from the device.

[1687] 2. The server inputs the saved gameplay video into the analysis means (AI model) and begins analyzing the movements in the video.

[1688] 3. The server uses the emotion engine to analyze the user's emotions from the gameplay video.

[1689] 4. The server evaluates the results of the motion analysis and emotion analysis and identifies specific areas for improvement (e.g., "The wrist angle when shooting is insufficient").

[1690] 5. The server references a database of methods from professional athletes and famous coaches and generates custom advice based on the emotion of the analysis (e.g., "Shoot with confidence").

[1691] 6. The server sends the generated play improvement advice to the device.

[1692] 7. The server stores the nutritional management information sent from the terminal and evaluates the nutritional balance using an evaluation method (AI model).

[1693] 8. The server uses an emotion engine to analyze the emotion of the user's input.

[1694] 9. The server generates nutritional advice based on the evaluation results and sentiment analysis results (e.g., "Take more vitamin C if you tend to feel depressed").

[1695] 10. The server sends the generated nutrition advice to the terminal.

[1696] Specific examples

[1697] Analyzing gameplay videos and providing advice on improvements

[1698] 1. The user uses a smartphone to film their basketball shooting practice, then uploads the video to a dedicated application.

[1699] 2. The device receives the video file, converts the file format, compresses it, and then sends it to the server.

[1700] 3. The server receives the video and stores it in a database. The stored video is then analyzed by an analytical method (AI model).

[1701] 4. Based on the analysis results, the server identifies specific areas for improvement, such as "the wrist angle when shooting is insufficient."

[1702] 5. The server uses an emotion engine to analyze the user's emotions while playing and obtain information such as "I'm nervous."

[1703] 6. The server compares the method with that of professional players and generates custom advice such as "Keep your wrist at a 45-degree angle and shoot with confidence."

[1704] 7. The server sends the generated advice to the terminal, which displays the advice on the user interface.

[1705] 8. The user checks the advice and puts it into practice during their next shooting practice session.

[1706] Entering nutritional management information and providing advice

[1707] 1. The user enters the contents of their breakfast (e.g., oatmeal, banana, yogurt) in a dedicated application.

[1708] 2. The terminal receives the input information, converts and formats it, and sends it to the server.

[1709] 3. The server receives the nutrition management information and stores it in a database.

[1710] 4. The server uses an evaluation tool (AI model) to assess nutritional balance and identify specific nutritional improvements (e.g., "You need more protein at breakfast").

[1711] 5. The server uses an emotion engine to analyze the user's emotions at the time of input and obtain information such as "I feel stressed."

[1712] 6. The server generates custom advice such as "Add a protein shake to your breakfast and eat more vitamin C to reduce stress."

[1713] 7. The server sends the generated nutrition advice to the terminal, which displays the advice on the user interface.

[1714] 8. The user reviews the advice and incorporates it into their next meal plan.

[1715] In this way, the present invention can provide users with comprehensive, automated and personalized sports play improvement and nutritional management advice, including emotion recognition.

[1716] The processing flow will be explained below.

[1717] Step 1:

[1718] Users use devices such as smartphones and tablets to record video of sports play.

[1719] Step 2:

[1720] Users upload the videos they have taken from their device to the server using a dedicated application.

[1721] Step 3:

[1722] The device receives the video file and converts or compresses the file format as needed.

[1723] Step 4:

[1724] The device then sends the converted and compressed video file to the server.

[1725] Step 5:

[1726] The server saves the received video file in storage.

[1727] Step 6:

[1728] The server inputs the saved video into an analysis tool (AI model) and begins analyzing the movements within the video.

[1729] Step 7:

[1730] The server's analytics evaluates your movements and identifies specific areas for improvement (e.g., "Your wrist angle when shooting is insufficient").

[1731] Step 8:

[1732] The server uses an emotion engine to analyze the user's emotions in real time from gameplay video (e.g., "tension" or "concentration").

[1733] Step 9:

[1734] The server generates advice to improve play based on the results of motion analysis and emotion analysis (e.g., "Keep your wrist at a 45-degree angle and shoot with confidence").

[1735] Step 10:

[1736] The server then sends the generated advice to the device to improve your gameplay.

[1737] Step 11:

[1738] The device formats the received advice for display in a user interface.

[1739] Step 12:

[1740] The device will then display the formatted advice to the user.

[1741] Step 13:

[1742] The user can review the advice provided via the device and put it into practice the next time they practice.

[1743] Step 14:

[1744] Users use a dedicated application to enter nutritional management information (e.g., dietary content, weight, and exercise volume).

[1745] Step 15:

[1746] The terminal receives the entered nutritional management information, formats it, and sends it to the server.

[1747] Step 16:

[1748] The server receives the nutritional management information sent and stores it in a database.

[1749] Step 17:

[1750] The server uses an evaluation tool (AI model) to assess nutritional balance and extract specific nutritional improvements (e.g., "You are lacking the protein you need for breakfast").

[1751] Step 18:

[1752] The server uses an emotion engine to analyze the emotions (e.g., "stress" or "fatigue") when the user enters the meal information.

[1753] Step 19:

[1754] The server generates nutritional advice based on the evaluation and sentiment analysis results (e.g., "Add a protein shake and take more vitamin C to reduce stress").

[1755] Step 20:

[1756] The server then sends the generated nutrition advice to the device.

[1757] Step 21:

[1758] The device then formats the received nutrition advice for display in a user interface.

[1759] Step 22:

[1760] The device will then display formatted nutrition advice to the user.

[1761] Step 23:

[1762] Users can review the nutritional advice provided via their device and incorporate it into their next meal plan.

[1763] Through these steps, the system can provide comprehensive and automatic personalized sports play improvement and nutritional management advice that also takes the user's emotions into account.

[1764] Example 2

[1765] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1766] In modern sports training and nutritional management, providing individually optimized advice is extremely important, and there is a particular demand for advice that takes into account the user's emotional state. However, current systems perform motion analysis of gameplay videos and evaluation of nutritional management information separately, and there are only a limited number of systems that can generate advice by integrating emotional analysis. As a result, there is a problem in that the improvement advice and nutritional advice obtained are of insufficient quality and usefulness.

[1767] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1768] In this invention, the server includes analysis means for performing motion analysis and emotion analysis using analysis means, advice generation means for generating play improvement advice based on the results of the motion analysis and emotion analysis, evaluation means for performing the motion analysis and emotion analysis, and nutrition advice generation means for generating nutrition advice based on the results of the nutrition analysis and emotion analysis, thereby making it possible to provide high-quality play improvement advice and nutrition advice that comprehensively considers the user's motion state and emotional state.

[1769] A "play video" is a video file in which a user captures a sports play.

[1770] "Photographing means" refers to a device that a user uses to photograph sports play, such as a smartphone or video camera.

[1771] "Uploading means" refers to a function or application for transmitting gameplay videos captured by the capturing means to a server.

[1772] "Storage means" refers to the function by which the server stores uploaded gameplay videos in a database or the like.

[1773] "Analysis means" refers to a function that analyzes saved gameplay videos and evaluates the user's actions and emotional state.

[1774] "Motion analysis" is the process of analyzing the user's body movements in gameplay video.

[1775] "Emotion analysis" is the process of evaluating and determining a user's emotional state based on gameplay video and user input information.

[1776] The "advice generation means" refers to a function that generates advice for improving play to be provided to the user based on the results of the motion analysis and emotion analysis.

[1777] The "transmission means" refers to a function for transmitting the generated advice to the terminal.

[1778] The "display means" refers to a function that displays the advice sent to the terminal on the user interface.

[1779] "Input means" refers to a function or application that allows the user to input nutritional management information.

[1780] "Information transmission means" refers to a function for transmitting input nutritional management information to a server.

[1781] "Evaluation means" refers to a function that analyzes the transmitted nutritional management information and evaluates the nutritional balance.

[1782] The term "nutritional advice generating means" refers to a function that generates nutritional improvement advice to be provided to a user based on the results of nutritional analysis and emotion analysis.

[1783] The term "nutritional advice sending means" refers to a function for sending the generated nutritional advice to the terminal.

[1784] A "method database" refers to a database that stores the coaching methods of professional athletes and famous coaches.

[1785] The present invention relates to a system for supporting users in improving their sports performance and managing their nutrition. The system analyzes the user's movements and emotions and provides personalized optimization advice based on the analysis. Specific embodiments of the system, including the hardware, software, and data processing methods used, are described in detail below.

[1786] Overall system configuration

[1787] This system consists of a server, a terminal, and a user. The specific role of each element is explained below.

[1788] User Actions

[1789] Users can record their sports activities using a smartphone or video camera. After recording is complete, they can use a dedicated application to upload the video to a server. Users can also enter nutritional management information (dietary content, weight, exercise volume, etc.) through the same dedicated application.

[1790] Device behavior

[1791] The device receives gameplay videos provided by the user, performs the necessary processing, and then sends them to the server. Specifically, it uses a video processing library such as FFmpeg to convert the video format and compress the size. It also receives nutritional management information provided by the user, converts its format, formats it, and sends it to the server.

[1792] Server Operation

[1793] The server receives the data sent from the terminal and performs the following processing.

[1794] 1. Video analysis

[1795] The server stores the received gameplay video. The stored video is then analyzed using a generative AI model. This analysis uses motion analysis libraries such as OpenPose and MediaPipe to analyze the user's body movements from the video frames.

[1796] 2. Emotion analysis

[1797] The server uses an emotion engine to analyze the user's emotions from the gameplay video. It uses facial recognition and voice analysis technologies to determine the user's emotional state (e.g., tension, joy, concentration).

[1798] 3. Generating advice for improving play

[1799] The server integrates the results of the motion analysis and emotion analysis, and generates personalized, custom advice by referencing a database of methods from professional athletes and famous coaches. The generated advice is then sent to the device.

[1800] 4. Evaluation of nutritional management information

[1801] The server stores the received nutritional management information and evaluates nutritional balance using a generative AI model. The evaluation results and sentiment analysis results are combined to generate appropriate custom nutrition advice, which is then sent to the device.

[1802] Specific examples

[1803] Analyzing gameplay videos and providing advice on improvements

[1804] 1. The user uses a smartphone to film their basketball shooting practice, then uploads the video to a dedicated application.

[1805] 2. The device receives the video file, converts the file format, compresses it, and then sends it to the server.

[1806] 3. The server receives the video and stores it in a database, where it is analyzed by a generative AI model.

[1807] 4. Based on the analysis results, the server identifies specific areas for improvement, such as "the wrist angle when shooting is insufficient."

[1808] 5. The server uses an emotion engine to analyze the user's emotions while playing and obtain information such as "I'm nervous."

[1809] 6. The server compares the method with that of professional players and generates custom advice such as "Keep your wrist at a 45-degree angle and shoot with confidence."

[1810] 7. The server sends the generated advice to the terminal, which displays the advice on the user interface.

[1811] 8. The user checks the advice and puts it into practice during their next shooting practice session.

[1812] Entering nutritional management information and providing advice

[1813] 1. The user enters the contents of their breakfast (e.g., oatmeal, banana, yogurt) in a dedicated application.

[1814] 2. The terminal receives the input information, converts and formats it, and sends it to the server.

[1815] 3. The server receives the nutrition management information and stores it in a database.

[1816] 4. The server uses an evaluation tool (generative AI model) to assess nutritional balance and identify specific nutritional improvements (e.g., "You need more protein at breakfast").

[1817] 5. The server uses an emotion engine to analyze the user's emotions at the time of input and obtain information such as "I feel stressed."

[1818] 6. The server generates custom advice such as "Add a protein shake to your breakfast and eat more vitamin C to reduce stress."

[1819] 7. The server sends the generated nutrition advice to the terminal, which displays the advice on the user interface.

[1820] 8. The user reviews the advice and incorporates it into their next meal plan.

[1821] Example prompts to input to the generative AI model

[1822] Example 1: "Analyze basketball shooting practice videos uploaded by users and evaluate the wrist angle during the shot."

[1823] Example 2: "Evaluate the nutritional balance of a breakfast menu of oatmeal, banana, and yogurt and generate advice on what nutrients the user needs. If the user is feeling stressed, provide advice on what to eat."

[1824] The present invention makes it possible to provide high-quality advice for improving play and nutritional advice to users, taking into consideration their behavioral and emotional states in a comprehensive manner.

[1825] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1826] Step 1:

[1827] Users can film their own sports play using a smartphone or video camera.

[1828] Input: Sports play footage

[1829] Specific operations: Launch the smartphone's camera application and start recording. Once recording is complete, save the video file.

[1830] Output: Recorded gameplay video file

[1831] Step 2:

[1832] The user launches a dedicated application, selects the gameplay video they have taken, and uploads it to the server.

[1833] Input: gameplay video file

[1834] Specific operation: Use the application's "Video Upload" function to select the play video file on the device and click the upload button.

[1835] Output: Gameplay video data sent to the server

[1836] Step 3:

[1837] The terminal receives the gameplay video sent from the user, converts the format and compresses it, and then sends it to the server.

[1838] Input: gameplay video data

[1839] What it does: It uses the FFmpeg library to convert the video format, compress it if necessary, and then sends the video data to the server over a network connection.

[1840] Output: Format-converted and compressed video data

[1841] Step 4:

[1842] The server receives the gameplay video sent from the terminal and stores it in a database.

[1843] Input: Format-converted gameplay video data

[1844] Specific operation: Connect to the database and save the video data and metadata (shooting date and time, user ID, etc.).

[1845] Output: Stored video data and metadata

[1846] Step 5:

[1847] The server analyzes the saved gameplay video using a generated AI model.

[1848] Input: Saved video data

[1849] Specific operation: Input video frames into the generative AI model and analyze the user's body movements (e.g., using the OpenPose or MediaPipe library). Save the analysis results.

[1850] Output: Motion analysis result data (e.g., position information of each body part)

[1851] Step 6:

[1852] The server uses an emotion engine to analyze the user's emotions from the gameplay video.

[1853] Input: Stored video and audio data

[1854] Specific operation: Determine the user's emotional state using facial recognition and voice analysis technology. Save the analysis results.

[1855] Output: Emotion analysis result data (e.g., tension, joy, concentration, etc.)

[1856] Step 7:

[1857] The server integrates the results of the motion analysis and emotion analysis, and generates play improvement advice by referring to a method database of professional athletes.

[1858] Input: Motion analysis result data and emotion analysis result data

[1859] Specific operation: Based on the analysis results, relevant advice is extracted from the method database and custom advice is generated.

[1860] Output: Generated play improvement advice data

[1861] Step 8:

[1862] The server transmits the generated play improvement advice to the terminal.

[1863] Input: Generated play improvement advice data

[1864] Specific operation: Advice data is sent to the terminal via a network connection.

[1865] Output: Advice data sent to the terminal

[1866] Step 9:

[1867] The terminal receives the play improvement advice sent from the server and displays it on the user interface.

[1868] Input: Play improvement advice data

[1869] Specific behavior: Display the advice text in the "Advice" section of the application.

[1870] Output: Advice displayed in the user interface

[1871] Step 10:

[1872] The user uses a dedicated application to input nutritional management information (dietary content, weight, amount of exercise, etc.).

[1873] Input: Nutritional management information

[1874] Specific operations: Enter information such as diet, weight, and exercise amount into the "Nutrition Management" form within the app and click the submit button.

[1875] Output: Input nutritional management information data

[1876] Step 11:

[1877] The terminal receives the input nutrition management information, converts and formats it, and then transmits it to the server.

[1878] Input: Nutritional management information data

[1879] Specific operation: Convert the received data into JSON format and send it to the server over the network.

[1880] Output: Formatted nutrition management information data

[1881] Step 12:

[1882] The server receives the nutritional management information sent from the terminal and stores it in a database.

[1883] Input: Formatted nutrition management information data

[1884] Specific operation: Connect to the database and save nutrition management information.

[1885] Output: Saved nutritional management information data

[1886] Step 13:

[1887] The server uses a generative AI model to assess nutritional balance and identify specific nutritional improvements.

[1888] Input: Saved nutrition management information data

[1889] Specific operation: Input nutritional management information into the generative AI model, analyze and evaluate nutritional balance, and save the evaluation results.

[1890] Output: Nutritional balance assessment result data

[1891] Step 14:

[1892] The server uses an emotion engine to analyze the user's emotions when inputting nutritional management information.

[1893] Input: Voice data and facial image data when entering nutrition management information

[1894] Specific operation: Determine the user's emotional state using facial recognition and voice analysis technology. Save the analysis results.

[1895] Output: Emotion analysis result data (e.g., stress, joy, etc.)

[1896] Step 15:

[1897] The server generates custom nutrition advice based on the nutritional assessment results and the sentiment analysis results.

[1898] Input: Nutritional balance assessment result data and emotion analysis result data

[1899] Specific operation: Integrates evaluation results and sentiment analysis results to generate appropriate nutrition advice.

[1900] Output: Generated custom nutrition advice data

[1901] Step 16:

[1902] The server transmits the generated nutrition advice to the terminal.

[1903] Input: Generated custom nutrition advice data

[1904] Specific operation: Advice data is sent to the terminal via a network connection.

[1905] Output: Advice data sent to the terminal

[1906] Step 17:

[1907] The terminal receives the nutrition advice sent from the server and displays it on the user interface.

[1908] Input: Nutrition advice data

[1909] What it does: Displays the advice in text form in the "Nutrition Advice" section of the application.

[1910] Output: Nutrition advice displayed on the user interface

[1911] Through this system, users can receive individually optimized advice on improving their sports play and nutritional management.

[1912] (Application example 2)

[1913] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1914] In recent years, automated robots have played an important role in many industries. However, these robots occasionally experience performance degradation or malfunction. To solve this problem, a system is needed that can properly monitor the robot's behavior and status and provide immediate advice for performance improvement. However, current technology does not provide a system that comprehensively analyzes the robot's behavior and emotions (state) and provides performance improvement advice. Therefore, the objective of this invention is to provide a system that comprehensively supports not only the improvement of sports performance but also the performance improvement and status monitoring of robotic work in factories.

[1915] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a filming means for filming gameplay videos; an uploading means for uploading the gameplay videos filmed by the filming means to the server; a saving means for saving the uploaded videos; an analyzing means for analyzing the saved videos; an advice generating means for generating play improvement advice based on the analysis results; a transmitting means for sending the generated advice to a terminal; a display means for displaying the sent advice; an improvement advice generating means for generating performance improvement advice for the robot based on the analysis results; an emotion analysis means by which the improvement advice generating means analyzes the state of the robot using sensor data; a transmitting means for sending the advice generated by the emotion analysis means to a terminal; and a display means for displaying the sent advice. This makes it possible to monitor the operating state of the robot in real time and immediately provide performance improvement advice based on emotion analysis.

[1916] "Photographing means" refers to a device or system for capturing videos or images.

[1917] The "uploading means" is a method or device for transmitting captured data to a server via a network.

[1918] "Storage means" refers to a method or device for temporarily or long-term storage of videos and data sent to the server.

[1919] "Analysis means" refers to a method or device for analyzing stored video or data and extracting specific information or features.

[1920] The "advice generation means" is a method or device that generates improvement instructions for the user based on the data obtained by the analysis means.

[1921] The "transmission means" is a method or device for sending information such as generated advice to the user's terminal.

[1922] "Display means" refers to a method or device for visually displaying advice or information sent to a user's terminal.

[1923] The "improvement advice generating means" is a method or device that generates instructions for improving the performance of a robot based on an analysis of the robot's movements and state.

[1924] "Emotion analysis means" refers to a method or device that analyzes the robot's sensor data and evaluates the robot's current state and performance.

[1925] "Sensor data" refers to data obtained from various sensors attached to robots and other devices, and is data that measures and records operating conditions and environmental conditions.

[1926] The present invention is a system for monitoring the operation of a factory robot and improving its performance. The system operates based on sensor data, including an image capturing means, an uploading means, a storage means, an analysis means, an advice generating means, a transmission means, a display means, an improvement advice generating means, and an emotion analysis means.

[1927] Overall system configuration

[1928] This system consists of a server, a terminal, and a user (in this case, the factory manager). The specific configuration and operation of each element are shown below.

[1929] Filming method

[1930] Cameras are used to capture the movements of robots in factories. The captured video is high resolution and contains important motion analysis data.

[1931] Upload method

[1932] The user uploads the video file of the robot's movements to the device through a dedicated application (e.g., a smartphone app). The uploaded video is converted into an appropriate format and sent to the server.

[1933] Preservation means

[1934] The server stores the received video files in a database, which allows for efficient management and storage of multiple video files.

[1935] Analysis means

[1936] The server processes the stored video using analytical means, specifically, an AI model (e.g., a motion analysis model) to analyze the robot's movement patterns and abnormalities in the video.

[1937] Advice Generation Method

[1938] Based on the data extracted by the analysis means, the advice generation means generates specific advice for improving the performance of the robot. The generated advice is optimized for the operating state of each individual robot.

[1939] Transmission method

[1940] The generated advice is transmitted from the server to the terminal, and the transmitting means enables the advice to be provided to the user in real time.

[1941] Display means

[1942] The terminal visually displays the received advice to the user. The display means includes an interface for concisely and clearly showing the content of the advice.

[1943] Improvement advice generation method

[1944] This is a means for generating more advanced performance improvement advice based on the data provided by the analysis means and the sentiment analysis means. The sentiment analysis means uses sensor data to analyze the robot's condition (e.g., temperature, load).

[1945] Hardware and software used

[1946] Hardware:

[1947] Camera: A device for filming the robot's movements.

[1948] Sensor: A device that acquires robot status data (temperature, load, etc.).

[1949] software:

[1950] Dedicated application: Upload videos, convert formats, and compress them.

[1951] Server: Stores and analyzes video and sensor data.

[1952] AI models: AI models for video analysis (e.g., Keras), AI models for sentiment analysis (e.g., Keras).

[1953] Specific examples

[1954] For example, a user can film a robot working in a factory with a camera and upload the video using a dedicated application. The video is then converted and compressed on the device and sent to the server. The server then saves the video and analyzes the robot's movements and status using analysis and emotion analysis. Based on the analysis results, the advice generation means creates specific advice for improving the robot's movements, which is then sent to the device and displayed to the user.

[1955] Examples of prompts:

[1956] "Write a program that analyzes this video file and sensor data to evaluate the robot's accuracy and health. Add logic to generate specific improvements and recommendations."

[1957] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1958] Step 1:

[1959] The user takes pictures of the robot's operation in the factory with a camera.

[1960] Input: Real-time video of robot movement.

[1961] Output: Video file of robot movement.

[1962] Specific operation: Cameras are installed in appropriate locations within the factory to capture high-resolution images of the robot's work process.

[1963] Step 2:

[1964] The user uploads the video file they have taken to the device using a dedicated application.

[1965] Input: Video file.

[1966] Output: Video file stored on your device.

[1967] Specific operation: Transfer and save video files to the device using a dedicated application.

[1968] Step 3:

[1969] The device sends the video file to the server.

[1970] Input: Video files stored on your device.

[1971] Output: Video file uploaded to the server.

[1972] Specific operation: The video file is compressed and format converted on the terminal, and then sent to the server via the network.

[1973] Step 4:

[1974] The server stores the received video file.

[1975] Input: Video files uploaded to the server.

[1976] Output: Video files stored in the database.

[1977] Specific operation: Video files are stored in a database on the server and managed appropriately.

[1978] Step 5:

[1979] The server analyzes the stored video.

[1980] Input: Video files stored in the database.

[1981] Output: Robot motion data in video.

[1982] Specific behavior: Using an AI model (e.g., a motion analysis model), the robot's movement patterns and abnormalities are analyzed for each video frame.

[1983] Step 6:

[1984] The server uses an emotion analysis means to analyze the state of the robot.

[1985] Input: Robot sensor data (temperature, load, etc.).

[1986] Output: Robot state data.

[1987] Specific behavior: Using an emotion analysis model, sensor data is analyzed to assess the robot's current behavioral state.

[1988] Step 7:

[1989] The server uses the advice generation means to create improvement advice based on the analysis results.

[1990] Input: Video analysis data and state data.

[1991] Output: Improvement advice.

[1992] Specific behavior: Based on the analysis results, specific advice is generated to optimize the robot's behavior (e.g., "reduce the part sorting speed by 20%)."

[1993] Step 8:

[1994] The server transmits the generated advice to the terminal.

[1995] Input: Improvement advice.

[1996] Output: Advice sent to the terminal.

[1997] Specific operation: The generated advice is sent to the user's terminal via the network.

[1998] Step 9:

[1999] The advice received by the terminal is displayed on a user interface.

[2000] Input: Advice sent by the server.

[2001] Output: The advice displayed to the user.

[2002] Specific operation: The advice content is displayed in the user interface in a format that is easy for the user to view.

[2003] Through these steps, the entire system is configured to effectively monitor the operation of robots in the factory and provide timely improvement advice.

[2004] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2005] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2006] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2007] [Fourth embodiment]

[2008] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2009] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2010] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2011] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2012] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2013] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2014] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2015] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2016] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2017] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2018] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2019] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2020] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2021] The present invention is an innovative system for supporting the improvement of sports play and nutritional management. The specific operation of this system will be described below.

[2022] Overall system configuration

[2023] This system consists of a server, a terminal, and a user. The specific role of each element is explained below.

[2024] User Actions

[2025] 1. A user uses a device (e.g., a smartphone or video camera) to record their sports play.

[2026] 2. Users upload the gameplay videos they have taken using a dedicated application on their device.

[2027] 3. The user uses a dedicated application to enter nutritional management information (e.g., dietary details, weight, and exercise amount).

[2028] Device behavior

[2029] 1. The device receives the gameplay video provided by the user, converts it into the appropriate format, compresses it if necessary, and sends it to the server.

[2030] 2. The terminal receives the nutritional management information provided by the user, converts and formats it, and then sends it to the server.

[2031] 3. The terminal receives the advice (play improvement advice and nutrition advice) sent from the server and displays it through the user interface.

[2032] Server Operation

[2033] 1. The server stores the gameplay video sent from the device.

[2034] 2. The server analyzes the saved gameplay video using analytical means (AI model) and evaluates the movement.

[2035] 3. The server generates advice for improving play based on the analysis results. The quality of the generated advice is improved by referencing multiple method databases.

[2036] 4. The server sends the generated play improvement advice to the device.

[2037] 5. The server stores the nutritional management information sent from the terminal and evaluates the nutritional balance using an evaluation method (AI model).

[2038] 6. The server generates nutrition advice based on the evaluation results.

[2039] 7. The server sends the generated nutrition advice to the terminal.

[2040] Specific examples

[2041] Analyzing gameplay videos and providing advice on improvements

[2042] 1. The user uses a smartphone to film their basketball shooting practice, then uploads the video to a dedicated application.

[2043] 2. The device receives the video file, converts the file format, compresses it, and then sends it to the server.

[2044] 3. The server receives the video and stores it in a database. The stored video is then analyzed by an analytical method (AI model).

[2045] 4. Based on the analysis results, the server identifies specific areas for improvement, such as "the wrist angle when shooting is insufficient."

[2046] 5. The server compares the method with that of a professional player and generates custom advice such as "Keep your wrist at a 45-degree angle."

[2047] 6. The server sends the generated advice to the terminal, which displays the advice on the user interface.

[2048] 7. The user checks the advice and puts it into practice during their next shooting practice session.

[2049] Entering nutritional management information and providing advice

[2050] 1. The user enters the contents of their breakfast (e.g., oatmeal, banana, yogurt) in a dedicated application.

[2051] 2. The terminal receives the input information, converts and formats it, and sends it to the server.

[2052] 3. The server receives the nutrition management information and stores it in a database.

[2053] 4. The server uses an evaluation tool (AI model) to assess nutritional balance and identify specific areas for improvement, such as "you need more protein at breakfast."

[2054] 5. The server generates a custom advice such as "Add a protein shake to breakfast."

[2055] 6. The server sends the generated nutrition advice to the terminal, which displays the advice on the user interface.

[2056] 7. The user confirms the advice and applies it to their next meal.

[2057] In this way, the present invention provides users with personalized sports play improvement and nutritional management advice to help improve overall performance.

[2058] The processing flow will be explained below.

[2059] Step 1:

[2060] Users use devices such as smartphones and tablets to record video of sports play.

[2061] Step 2:

[2062] Users upload the videos they have taken from their device to the server using a dedicated application.

[2063] Step 3:

[2064] The device receives the video file and converts or compresses the file format as needed.

[2065] Step 4:

[2066] The device then sends the converted and compressed video file to the server.

[2067] Step 5:

[2068] The server saves the received video file in storage.

[2069] Step 6:

[2070] The server inputs the saved video into an analysis tool (AI model) and begins analyzing the movements within the video.

[2071] Step 7:

[2072] The server's analytics evaluates your movements and identifies specific areas for improvement (e.g., "Your wrist angle when shooting is insufficient").

[2073] Step 8:

[2074] The server references a database of methods from professional athletes and famous coaches and generates custom advice based on the analysis results.

[2075] Step 9:

[2076] The server then sends the generated advice to the device to improve your gameplay.

[2077] Step 10:

[2078] The device formats the received advice for display in a user interface.

[2079] Step 11:

[2080] The device will then display the formatted advice to the user.

[2081] Step 12:

[2082] The user can review the advice provided via the device and put it into practice the next time they practice.

[2083] Step 13:

[2084] Users use a dedicated application to enter nutritional management information (e.g., dietary content, weight, and exercise volume).

[2085] Step 14:

[2086] The terminal receives the entered nutritional management information, formats it, and sends it to the server.

[2087] Step 15:

[2088] The server receives the nutritional management information sent and stores it in a database.

[2089] Step 16:

[2090] The server uses an evaluation tool (AI model) to assess nutritional balance and extract specific nutritional areas for improvement (e.g., "lack of protein").

[2091] Step 17:

[2092] The server then references a nutrition database and generates custom nutrition advice based on the assessment results.

[2093] Step 18:

[2094] The server then sends the generated nutrition advice to the device.

[2095] Step 19:

[2096] The device then formats the received nutrition advice for display in a user interface.

[2097] Step 20:

[2098] The device will then display formatted nutrition advice to the user.

[2099] Step 21:

[2100] Users can review the nutritional advice provided via their device and incorporate it into their next meal plan.

[2101] Through these steps, the system can provide users with comprehensive and automated personalized sports play improvement and nutritional management advice.

[2102] Example 1

[2103] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2104] Conventional sports performance improvement and nutrition management systems have limitations in analyzing data and providing appropriate advice to individual users. Furthermore, analysis of gameplay videos and evaluation of nutrition management information are often done manually, resulting in insufficient automation and efficiency. This makes it difficult for users to obtain effective improvement measures in a timely manner, making it difficult to improve overall performance.

[2105] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2106] In this invention, the server includes: a filming means for filming gameplay videos; an uploading means for uploading the gameplay videos filmed by the filming means via a dedicated application on the terminal; a transmitting means for receiving the uploaded videos at the terminal, converting the file format and compressing them, and transmitting them to the server; a storing means for saving the transmitted videos on the server; an analyzing means for analyzing the saved videos using an AI model; an advice generating means for generating gameplay improvement advice based on the analysis results; a play advice sending means for sending the generated gameplay improvement advice to the terminal; a display means for displaying the transmitted gameplay improvement advice on a terminal user interface; an input means for inputting nutritional management information; a nutritional information sending means for receiving the input nutritional management information at the terminal, shaping and formatting the information, and transmitting it to the server; a nutrition saving means for saving the transmitted nutritional management information on the server; an evaluating means for evaluating the stored nutritional management information using an AI model; a nutritional advice generating means for generating nutritional advice based on the evaluation results; a nutritional advice sending means for sending the generated nutritional advice to the terminal; and a display means for displaying the transmitted nutritional advice on the terminal user interface. This will enable the analysis of individual users' sports play and nutritional management to be automated, enabling the provision of quick and effective advice on improvement.

[2107] A "play video" is a video file in which a user captures a sports play.

[2108] "Filming means" refers to devices or methods for filming sports play, specifically smartphones and video cameras.

[2109] The "uploading means" is a means for transmitting the captured gameplay video to the server, and is a function or process that is performed through a dedicated application.

[2110] "Transmission means" refers to a method for transferring data from a user terminal to a server, and includes a POST request using the HTTP protocol.

[2111] "Storage" means a system or method for retaining transmitted data, including the use of a database or storage system.

[2112] "Analysis means" refers to the processes and tools used to analyze stored data, specifically the use of AI models.

[2113] The "advice generation means" is a process for creating improvement advice for the user based on the analysis results.

[2114] The "play advice sending means" refers to a function for sending the generated improvement advice to the user terminal.

[2115] The "display means" is an interface for visually presenting information to the user on the terminal.

[2116] "Input means" refers to a method by which a user inputs nutritional management information into the system, such as a form in a dedicated application.

[2117] "Nutrition information transmission means" refers to a function for transmitting input nutrition management information to a server.

[2118] "Nutrition storage means" refers to a system or method for storing transmitted nutrition management information.

[2119] "Evaluation methods" are processes and tools for analyzing and evaluating stored nutrition management information, and use AI models.

[2120] "Nutrition advice generator" refers to a process that generates nutrition advice based on the evaluation results.

[2121] The term "nutrition advice sending means" refers to a function for sending the generated nutrition advice to the user terminal.

[2122] This invention is an innovative system for supporting sports performance improvement and nutritional management. This system is composed of a server, terminals, and users, and each element works in cooperation with each other.

[2123] User Actions

[2124] First, the user films their own sports play (e.g., a basketball shot) using a smartphone or video camera. Next, they upload the video to their device using a dedicated application. They also use the same application to enter their daily nutrition management information (e.g., dietary details, weight, and exercise volume).

[2125] Device behavior

[2126] The device receives gameplay videos uploaded by users, converts the video files to the appropriate format using tools such as FFmpeg, and compresses them by adjusting the bitrate and resolution. It then sends the videos to the server. It also receives nutritional management information entered by the user and formats the data into JSON format. It then sends the formatted nutritional information to the server. It also receives gameplay improvement advice and nutritional advice sent from the server and displays them on the user interface.

[2127] Server Operation

[2128] The server receives gameplay videos sent from the device and stores them in a database. For example, a storage system dedicated to video files (e.g., Amazon S3) can be used. The stored videos are input into an AI analysis model (e.g., OpenPose or MediaPipe) for analysis and evaluation of the movements. Based on the analysis results, advice is created to improve the user's gameplay. This advice is optimized by referencing multiple method databases. The generated advice is then sent to the device and presented to the user.

[2129] The server receives the nutritional management information sent from the device and stores it in a database. The stored nutritional information is input into an AI model (e.g., TensorFlow) for analysis and an evaluation of nutritional balance is performed. Based on the evaluation results, specific nutritional advice is generated and sent to the device for presentation to the user.

[2130] Specific examples

[2131] Analyzing gameplay videos and providing advice on improvements

[2132] For example, a user uses a smartphone to film basketball shooting practice and uploads the video to a dedicated application. The device receives the video file, converts the file format, compresses it, and sends it to a server. The server then receives the video and stores it in a database. The saved video is analyzed using an AI model (e.g., OpenPose) to identify specific areas for improvement, such as "your wrist angle when shooting is insufficient." Based on the analysis results, custom advice is generated, such as "keep your wrist at a 45-degree angle." The server then sends the generated advice to the device, which displays it on the user interface. The user then checks the advice and puts it into practice during their next shooting practice.

[2133] Entering nutritional management information and providing advice

[2134] For example, a user uses a dedicated application to input the contents of their breakfast (e.g., oatmeal, banana, yogurt). The device receives the input information, converts and formats it, and sends it to the server. The server receives the information and stores it in a database. The stored information is input into an AI model for analysis, and specific areas for improvement, such as "You need more protein for breakfast," are identified. Based on the results, custom advice, such as "Add a protein shake to breakfast," is generated, and the server sends the advice to the device. The device displays the advice on the user interface, and the user can incorporate it into their next meal.

[2135] Examples of prompt statements

[2136] An example of a prompt sentence generated by a generative AI model could be written as follows:

[2137] "What is the optimal wrist angle when shooting a basketball?"

[2138] "What foods should I add to my breakfast to improve its nutritional balance?"

[2139] In this way, the present invention provides useful advice to users in sports play and nutritional management, and helps improve overall performance.

[2140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2141] Step 1:

[2142] Users can film their own sports play using a smartphone or video camera and then upload the recorded video files using a dedicated application.

[2143] Input: Recorded video file

[2144] Output: Video file for upload

[2145] Specific operation: The user opens the dedicated application, selects the recorded video file, and presses the upload button.

[2146] Step 2:

[2147] The device receives the gameplay video file uploaded by the user, converts the video file format to MP4 format using a tool such as FFmpeg, and compresses it if necessary.

[2148] Input: User uploaded video file

[2149] Output: Converted and compressed video file

[2150] Specific operation: The device receives the video file, converts the format using FFmpeg, and compresses it by adjusting the bitrate and resolution.

[2151] Step 3:

[2152] The device sends the converted and compressed video file to the server.

[2153] Input: Converted and compressed video files

[2154] Output: Video file sent to the server

[2155] Specific behavior: Sends the file to the server using a POST request using the HTTP protocol.

[2156] Step 4:

[2157] The server receives the video file sent from the terminal and stores the video file in a database.

[2158] Input: Video file sent to the server

[2159] Output: Video files stored in a database

[2160] Specific behavior: Receives video files and stores them in a database (e.g., Amazon S3) along with associated metadata (e.g., user ID, timestamp).

[2161] Step 5:

[2162] The server analyzes the stored video files using an AI analysis model (e.g., OpenPose, MediaPipe) and evaluates the movements.

[2163] Input: Video files stored in the database

[2164] Output: Analysis results (motion evaluation data)

[2165] Specific operation: The saved video file is input into the AI ​​model, and the characteristics of the user's movements (e.g., wrist angle, body posture) are extracted.

[2166] Step 6:

[2167] The server generates advice for improving play based on the analysis results, and this advice is optimized by referencing multiple method databases.

[2168] Input: Analysis results (motion evaluation data)

[2169] Output: Advice for improving your gameplay

[2170] Specific operation: Based on the analysis results, automatically generated advice is compared with the user profile and method database to optimize the system.

[2171] Step 7:

[2172] The server transmits the generated play improvement advice to the terminal.

[2173] Input: Advice for improving your game

[2174] Output: Send advice to terminal

[2175] Specific operation: The generated advice is sent to the device using a RESTful API or WebSocket.

[2176] Step 8:

[2177] The terminal receives the play improvement advice sent from the server and displays it on the user interface.

[2178] Input: Advice sent by the server

[2179] Output: Advice displayed in the user interface

[2180] Specific behavior: The received advice is visually presented to the user in a user interface within the application.

[2181] Step 9:

[2182] The user uses a dedicated application to input nutritional management information (e.g., dietary content, weight, amount of exercise).

[2183] Input: Nutritional management information (dietary content, weight, exercise amount)

[2184] Output: Nutritional management information entered into the application

[2185] Specific operation: Enter diet and exercise data using the input form in the dedicated application.

[2186] Step 10:

[2187] The terminal receives nutritional management information entered by the user, formats the data into JSON format, and sends it to the server.

[2188] Input: Nutritional management information entered by the user

[2189] Output: Formatted data sent to the server

[2190] Specific operation: Converts input data into JSON format and sends it to the server via an HTTP POST request.

[2191] Step 11:

[2192] The server receives the nutritional management information sent from the terminal and stores it in a database.

[2193] Input: Nutritional management information sent to the server

[2194] Output: Nutritional management information stored in a database

[2195] Specific operation: Save nutritional management information in a database.

[2196] Step 12:

[2197] The server analyzes the stored nutritional management information using an AI analysis model (e.g., TensorFlow) and evaluates nutritional balance.

[2198] Input: Nutritional management information stored in the database

[2199] Output: Nutritional balance evaluation results

[2200] Specific operation: Nutritional management information is input into the AI ​​model and nutritional balance is analyzed.

[2201] Step 13:

[2202] The server generates specific nutritional advice based on the evaluation results.

[2203] Input: Nutritional balance assessment results

[2204] Output: Nutrition advice

[2205] Specific operation: Based on the analysis results, automatically generated advice is adapted to the user profile.

[2206] Step 14:

[2207] The server transmits the generated nutrition advice to the terminal.

[2208] Enter: nutrition advice

[2209] Output: Send advice to terminal

[2210] Specific operation: The generated advice is sent to the device using a RESTful API or WebSocket.

[2211] Step 15:

[2212] The terminal receives the nutrition advice sent from the server and displays it on the user interface.

[2213] Input: Advice sent by the server

[2214] Output: Advice displayed in the user interface

[2215] Specific behavior: The received advice is visually presented to the user in a user interface within the application.

[2216] (Application example 1)

[2217] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2218] Fitness gym members need professional advice to effectively improve their training methods and nutritional management. However, receiving advice from a trainer individually is time-consuming and expensive. Furthermore, proper nutritional management is difficult to achieve through self-determination, and requires specialized knowledge. Conventional methods have made it difficult to provide this advice quickly and efficiently. Therefore, there is a need for a system that can easily provide professional advice to fitness gym members and help them optimize their training results and nutritional management.

[2219] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2220] In this invention, the server includes a filming means for filming gameplay videos, an uploading means for uploading the gameplay videos filmed by the filming means to the server, a saving means for saving the uploaded videos, an analysis means for analyzing the saved videos, an advice generating means for generating gameplay improvement advice based on the analysis results, a sending means for sending the generated advice to a terminal, a display means for displaying the sent advice, an input means for a user to input nutritional management information, an information sending means for sending the nutritional management information to the server, an evaluation means for evaluating the sent nutritional management information, a nutritional advice generating means for generating nutritional advice based on the evaluation results, a nutritional advice sending means for sending the generated nutritional advice to a terminal, and a display means for displaying the sent nutritional advice, and the gameplay improvement advice and nutritional advice are generated by referring to multiple method databases. This enables fitness gym members to professionally and efficiently improve their training methods and nutritional management.

[2221] definition statement

[2222] A "play video" is a video file containing sports or fitness-related actions filmed by a user.

[2223] "Capturing means" refers to a device for capturing video using a smartphone, smart glasses, head-mounted display, etc.

[2224] The "uploading means" is a device or application that has the function of transmitting the captured video to a server via a network.

[2225] "Storage means" refers to a system that has the function of recording and storing videos and data within a server.

[2226] The "analysis means" is a system that has the function of analyzing videos and data stored on the server and evaluating training and exercise performance.

[2227] The "advice generation means" is a system that automatically creates recommendations for improvements in training and exercise and nutritional management based on the analysis results.

[2228] The "transmission means" is a system having a function for transmitting the generated advice to the user's terminal.

[2229] The "display means" is a device or application that has the function of visually presenting advice sent from the server on the user's terminal.

[2230] "Input means" refers to a device or application that allows a user to input their own nutritional management information.

[2231] The "information transmission means" is a device or application that has the function of transmitting input nutritional management information to a server.

[2232] The "evaluation means" is a system that analyzes and evaluates the nutritional management information sent to the server.

[2233] The "nutritional advice generating means" is a system that generates individual nutritional management advice based on the nutritional information analyzed by the evaluation means.

[2234] The "nutrition advice sending means" is a system having a function of sending the generated nutritional management advice to the user's terminal.

[2235] The "method database" is a collection of data that describes multiple training methods and nutritional management methods, and is a system that provides advanced advice by referring to this data.

[2236] MODE FOR CARRYING OUT THE INVENTION

[2237] System Overview

[2238] The system that realizes this application example consists of three main elements: a server, a terminal, and a user, in order to improve the user's training performance and nutritional management.

[2239] Hardware and software used

[2240] 1. Hardware and software used by the user:

[2241] Smartphone

[2242] Smart Glasses

[2243] head-mounted display

[2244] Photo capture and upload application (developed with React Native)

[2245] 2. Software and systems used by the server:

[2246] Server: Using Node.js and Express

[2247] Database: MongoDB

[2248] Video processing: FFmpeg

[2249] Analyzing AI models: TensorFlow

[2250] Specific operation of the system

[2251] 1. User Action:

[2252] Users use a smartphone, smart glasses, or head-mounted display to record their own training videos, and then use the application to prepare the videos for uploading.

[2253] Users use the application to input their own nutritional management information (e.g., dietary content, amount of exercise).

[2254] 2. Device behavior:

[2255] The device receives the training video provided by the user, converts the video file into the appropriate format, compresses the video if necessary, and sends it to the server.

[2256] The terminal receives the nutrition management information provided by the user, converts the format and formats it, and then transmits it to the server.

[2257] The terminal receives the play improvement advice and nutrition advice sent from the server and displays them through a user interface.

[2258] 3. Server Operation:

[2259] The server stores the training video sent from the terminal.

[2260] The server analyzes the saved training videos using an analytical method (a generative AI model using TensorFlow) and evaluates the movements.

[2261] The server generates advice for improving play based on the analysis results, and the quality of this advice is improved by referencing multiple method databases.

[2262] The server transmits the generated play improvement advice to the terminal.

[2263] The server stores the nutritional management information transmitted from the terminal and evaluates the nutritional balance using the evaluation means.

[2264] The server generates nutrition advice based on the evaluation results and transmits it to the terminal.

[2265] Specific processing examples

[2266] Analyzing training videos and providing improvement advice:

[2267] The user films a squat training video on their smartphone and uploads it to the server via the app. The server then analyzes the video using a TensorFlow model, generates specific advice, such as "your hip angle should be less than 90 degrees," and sends it to the device. The device then displays this advice on the user interface, allowing the user to confirm and practice it during their next training session.

[2268] Enter nutritional information and provide advice:

[2269] The user enters what they have for breakfast (e.g., oatmeal, banana, yogurt) into the application. The device formats the information and sends it to the server. The server stores the information, evaluates the nutritional balance using an AI model, and generates specific nutritional advice, such as "add a protein shake," and sends it to the device. The device displays this advice in the user interface, and the user can incorporate it into their next meal.

[2270] Prompt Sentence Examples

[2271] Below is an example of a prompt for an AI model:

[2272] Training video analysis:

[2273] Please analyze the user's squat video and rate it based on the following items.

[2274] 1. Waist angle

[2275] 2. Knee position

[2276] 3. Back Posture

[2277] Generate nutrition advice:

[2278] Based on the breakfast information entered by the user (oatmeal, banana, yogurt), assess whether there are any protein deficiencies and, if so, suggest additional foods.

[2279] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2280] Program processing steps

[2281] Processing Steps

[2282] Step 1:

[2283] Users record training videos using a smartphone, smart glasses, or a head-mounted display.

[2284] Input: Filmed training video

[2285] Output: Recorded training video file

[2286] Step 2:

[2287] Users upload their videos to the server through the app, which uses FFmpeg to convert the video files to the appropriate format and compress them if necessary.

[2288] Input: Training video file

[2289] Data processing: format conversion, video compression

[2290] Output: Converted and compressed video file

[2291] Step 3:

[2292] The server receives and stores the converted and compressed video files.

[2293] Input: Converted and compressed video files

[2294] Output: Saved video file

[2295] Step 4:

[2296] The server analyzes the stored video files using a TensorFlow model, which uses a generative AI model to generate advice for improving gameplay.

[2297] Input: Saved video file

[2298] Data Computation: Video Analysis with Generative AI Models

[2299] Output: Advice for improving your gameplay

[2300] Step 5:

[2301] The server transmits the generated play improvement advice to the terminal.

[2302] Input: Advice for improving your game

[2303] Output: Play improvement advice sent to the device

[2304] Step 6:

[2305] The terminal displays the received advice for improving play on the user interface.

[2306] Input: Submitted game improvement advice

[2307] Output: Play improvement advice displayed in the user interface

[2308] Step 7:

[2309] The user uses a dedicated application to input nutritional management information (e.g., dietary details, amount of exercise).

[2310] Input: Nutritional management information

[2311] Output: Entered nutritional management information

[2312] Step 8:

[2313] The terminal converts and formats the input nutritional management information and sends it to the server.

[2314] Input: Nutritional management information entered

[2315] Data processing: format conversion, formatting

[2316] Output: Transformed and formatted nutrition information

[2317] Step 9:

[2318] The server receives and stores the converted and formatted nutrition management information.

[2319] Input: Transformed and formatted nutrition information

[2320] Output: Saved nutritional information

[2321] Step 10:

[2322] The server evaluates the stored nutritional management information using an evaluation tool and generates nutritional advice. This evaluation also uses a generative AI model.

[2323] Input: Saved nutrition management information

[2324] Data Computation: Evaluation with Generative AI Models

[2325] Output: Nutrition advice

[2326] Step 11:

[2327] The server transmits the generated nutrition advice to the terminal.

[2328] Enter: nutrition advice

[2329] Output: Nutrition advice sent to the device

[2330] Step 12:

[2331] The terminal displays the received nutrition advice on a user interface.

[2332] Input: Submitted nutrition advice

[2333] Output: Nutrition advice displayed in a user interface

[2334] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2335] The present invention is an innovative system for supporting sports improvement and nutritional management. In particular, by combining it with an emotion engine that recognizes the user's emotions, it is possible to provide advice optimized for each individual user. The specific operation of this system is described below.

[2336] Overall system configuration

[2337] This system consists of a server, a terminal, and a user. The specific role of each element is explained below.

[2338] User Actions

[2339] 1. A user uses a device (e.g., a smartphone or video camera) to record their sports play.

[2340] 2. The user uploads the captured gameplay video from their device to the server using a dedicated application.

[2341] 3. The user uses a dedicated application to input nutritional management information (e.g., dietary details, weight, amount of exercise).

[2342] Device behavior

[2343] 1. The device receives the gameplay video provided by the user, converts it into the appropriate format, compresses it if necessary, and sends it to the server.

[2344] 2. The terminal receives the nutritional management information provided by the user, converts and formats it, and then sends it to the server.

[2345] 3. The terminal receives the advice (play improvement advice and nutrition advice) sent from the server and displays it through the user interface.

[2346] Server Operation

[2347] 1. The server stores the gameplay video sent from the device.

[2348] 2. The server inputs the saved gameplay video into the analysis means (AI model) and begins analyzing the movements in the video.

[2349] 3. The server uses the emotion engine to analyze the user's emotions from the gameplay video.

[2350] 4. The server evaluates the results of the motion analysis and emotion analysis and identifies specific areas for improvement (e.g., "The wrist angle when shooting is insufficient").

[2351] 5. The server references a database of methods from professional athletes and famous coaches and generates custom advice based on the emotion of the analysis (e.g., "Shoot with confidence").

[2352] 6. The server sends the generated play improvement advice to the device.

[2353] 7. The server stores the nutritional management information sent from the terminal and evaluates the nutritional balance using an evaluation method (AI model).

[2354] 8. The server uses an emotion engine to analyze the emotion of the user's input.

[2355] 9. The server generates nutritional advice based on the evaluation results and sentiment analysis results (e.g., "Take more vitamin C if you tend to feel depressed").

[2356] 10. The server sends the generated nutrition advice to the terminal.

[2357] Specific examples

[2358] Analyzing gameplay videos and providing advice on improvements

[2359] 1. The user uses a smartphone to film their basketball shooting practice, then uploads the video to a dedicated application.

[2360] 2. The device receives the video file, converts the file format, compresses it, and then sends it to the server.

[2361] 3. The server receives the video and stores it in a database. The stored video is then analyzed by an analytical method (AI model).

[2362] 4. Based on the analysis results, the server identifies specific areas for improvement, such as "the wrist angle when shooting is insufficient."

[2363] 5. The server uses an emotion engine to analyze the user's emotions while playing and obtain information such as "I'm nervous."

[2364] 6. The server compares the method with that of professional players and generates custom advice such as "Keep your wrist at a 45-degree angle and shoot with confidence."

[2365] 7. The server sends the generated advice to the terminal, which displays the advice on the user interface.

[2366] 8. The user checks the advice and puts it into practice during their next shooting practice session.

[2367] Entering nutritional management information and providing advice

[2368] 1. The user enters the contents of their breakfast (e.g., oatmeal, banana, yogurt) in a dedicated application.

[2369] 2. The terminal receives the input information, converts and formats it, and sends it to the server.

[2370] 3. The server receives the nutrition management information and stores it in a database.

[2371] 4. The server uses an evaluation tool (AI model) to assess nutritional balance and identify specific nutritional improvements (e.g., "You need more protein at breakfast").

[2372] 5. The server uses an emotion engine to analyze the user's emotions at the time of input and obtain information such as "I feel stressed."

[2373] 6. The server generates custom advice such as "Add a protein shake to your breakfast and eat more vitamin C to reduce stress."

[2374] 7. The server sends the generated nutrition advice to the terminal, which displays the advice on the user interface.

[2375] 8. The user reviews the advice and incorporates it into their next meal plan.

[2376] In this way, the present invention can provide users with comprehensive, automated and personalized sports play improvement and nutritional management advice, including emotion recognition.

[2377] The processing flow will be explained below.

[2378] Step 1:

[2379] Users use devices such as smartphones and tablets to record video of sports play.

[2380] Step 2:

[2381] Users upload the videos they have taken from their device to the server using a dedicated application.

[2382] Step 3:

[2383] The device receives the video file and converts or compresses the file format as needed.

[2384] Step 4:

[2385] The device then sends the converted and compressed video file to the server.

[2386] Step 5:

[2387] The server saves the received video file in storage.

[2388] Step 6:

[2389] The server inputs the saved video into an analysis tool (AI model) and begins analyzing the movements within the video.

[2390] Step 7:

[2391] The server's analytics evaluates your movements and identifies specific areas for improvement (e.g., "Your wrist angle when shooting is insufficient").

[2392] Step 8:

[2393] The server uses an emotion engine to analyze the user's emotions in real time from gameplay video (e.g., "tension" or "concentration").

[2394] Step 9:

[2395] The server generates advice to improve play based on the results of motion analysis and emotion analysis (e.g., "Keep your wrist at a 45-degree angle and shoot with confidence").

[2396] Step 10:

[2397] The server then sends the generated advice to the device to improve your gameplay.

[2398] Step 11:

[2399] The device formats the received advice for display in a user interface.

[2400] Step 12:

[2401] The device will then display the formatted advice to the user.

[2402] Step 13:

[2403] The user can review the advice provided via the device and put it into practice the next time they practice.

[2404] Step 14:

[2405] Users use a dedicated application to enter nutritional management information (e.g., dietary content, weight, and exercise volume).

[2406] Step 15:

[2407] The terminal receives the entered nutritional management information, formats it, and sends it to the server.

[2408] Step 16:

[2409] The server receives the nutritional management information sent and stores it in a database.

[2410] Step 17:

[2411] The server uses an evaluation tool (AI model) to assess nutritional balance and extract specific nutritional improvements (e.g., "You are lacking the protein you need for breakfast").

[2412] Step 18:

[2413] The server uses an emotion engine to analyze the emotions (e.g., "stress" or "fatigue") when the user enters the meal information.

[2414] Step 19:

[2415] The server generates nutritional advice based on the evaluation and sentiment analysis results (e.g., "Add a protein shake and take more vitamin C to reduce stress").

[2416] Step 20:

[2417] The server then sends the generated nutrition advice to the device.

[2418] Step 21:

[2419] The device then formats the received nutrition advice for display in a user interface.

[2420] Step 22:

[2421] The device will then display formatted nutrition advice to the user.

[2422] Step 23:

[2423] Users can review the nutritional advice provided via their device and incorporate it into their next meal plan.

[2424] Through these steps, the system can provide comprehensive and automatic personalized sports play improvement and nutritional management advice that also takes the user's emotions into account.

[2425] Example 2

[2426] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2427] In modern sports training and nutritional management, providing individually optimized advice is extremely important, and there is a particular demand for advice that takes into account the user's emotional state. However, current systems perform motion analysis of gameplay videos and evaluation of nutritional management information separately, and there are only a limited number of systems that can generate advice by integrating emotional analysis. As a result, there is a problem in that the improvement advice and nutritional advice obtained are of insufficient quality and usefulness.

[2428] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2429] In this invention, the server includes analysis means for performing motion analysis and emotion analysis using analysis means, advice generation means for generating play improvement advice based on the results of the motion analysis and emotion analysis, evaluation means for performing the motion analysis and emotion analysis, and nutrition advice generation means for generating nutrition advice based on the results of the nutrition analysis and emotion analysis, thereby making it possible to provide high-quality play improvement advice and nutrition advice that comprehensively considers the user's motion state and emotional state.

[2430] A "play video" is a video file in which a user captures a sports play.

[2431] "Photographing means" refers to a device that a user uses to photograph sports play, such as a smartphone or video camera.

[2432] "Uploading means" refers to a function or application for transmitting gameplay videos captured by the capturing means to a server.

[2433] "Storage means" refers to the function by which the server stores uploaded gameplay videos in a database or the like.

[2434] "Analysis means" refers to a function that analyzes saved gameplay videos and evaluates the user's actions and emotional state.

[2435] "Motion analysis" is the process of analyzing the user's body movements in gameplay video.

[2436] "Emotion analysis" is the process of evaluating and determining a user's emotional state based on gameplay video and user input information.

[2437] The "advice generation means" refers to a function that generates advice for improving play to be provided to the user based on the results of the motion analysis and emotion analysis.

[2438] The "transmission means" refers to a function for transmitting the generated advice to the terminal.

[2439] The "display means" refers to a function that displays the advice sent to the terminal on the user interface.

[2440] "Input means" refers to a function or application that allows the user to input nutritional management information.

[2441] "Information transmission means" refers to a function for transmitting input nutritional management information to a server.

[2442] "Evaluation means" refers to a function that analyzes the transmitted nutritional management information and evaluates the nutritional balance.

[2443] The term "nutritional advice generating means" refers to a function that generates nutritional improvement advice to be provided to a user based on the results of nutritional analysis and emotion analysis.

[2444] The term "nutritional advice sending means" refers to a function for sending the generated nutritional advice to the terminal.

[2445] A "method database" refers to a database that stores the coaching methods of professional athletes and famous coaches.

[2446] The present invention relates to a system for supporting users in improving their sports performance and managing their nutrition. The system analyzes the user's movements and emotions and provides personalized optimization advice based on the analysis. Specific embodiments of the system, including the hardware, software, and data processing methods used, are described in detail below.

[2447] Overall system configuration

[2448] This system consists of a server, a terminal, and a user. The specific role of each element is explained below.

[2449] User Actions

[2450] Users can record their sports activities using a smartphone or video camera. After recording is complete, they can use a dedicated application to upload the video to a server. Users can also enter nutritional management information (dietary content, weight, exercise volume, etc.) through the same dedicated application.

[2451] Device behavior

[2452] The device receives gameplay videos provided by the user, performs the necessary processing, and then sends them to the server. Specifically, it uses a video processing library such as FFmpeg to convert the video format and compress the size. It also receives nutritional management information provided by the user, converts its format, formats it, and sends it to the server.

[2453] Server Operation

[2454] The server receives the data sent from the terminal and performs the following processing.

[2455] 1. Video analysis

[2456] The server stores the received gameplay video. The stored video is then analyzed using a generative AI model. This analysis uses motion analysis libraries such as OpenPose and MediaPipe to analyze the user's body movements from the video frames.

[2457] 2. Emotion analysis

[2458] The server uses an emotion engine to analyze the user's emotions from the gameplay video. It uses facial recognition and voice analysis technologies to determine the user's emotional state (e.g., tension, joy, concentration).

[2459] 3. Generating advice for improving play

[2460] The server integrates the results of the motion analysis and emotion analysis, and generates personalized, custom advice by referencing a database of methods from professional athletes and famous coaches. The generated advice is then sent to the device.

[2461] 4. Evaluation of nutritional management information

[2462] The server stores the received nutritional management information and evaluates nutritional balance using a generative AI model. The evaluation results and sentiment analysis results are combined to generate appropriate custom nutrition advice, which is then sent to the device.

[2463] Specific examples

[2464] Analyzing gameplay videos and providing advice on improvements

[2465] 1. The user uses a smartphone to film their basketball shooting practice, then uploads the video to a dedicated application.

[2466] 2. The device receives the video file, converts the file format, compresses it, and then sends it to the server.

[2467] 3. The server receives the video and stores it in a database, where it is analyzed by a generative AI model.

[2468] 4. Based on the analysis results, the server identifies specific areas for improvement, such as "the wrist angle when shooting is insufficient."

[2469] 5. The server uses an emotion engine to analyze the user's emotions while playing and obtain information such as "I'm nervous."

[2470] 6. The server compares the method with that of professional players and generates custom advice such as "Keep your wrist at a 45-degree angle and shoot with confidence."

[2471] 7. The server sends the generated advice to the terminal, which displays the advice on the user interface.

[2472] 8. The user checks the advice and puts it into practice during their next shooting practice session.

[2473] Entering nutritional management information and providing advice

[2474] 1. The user enters the contents of their breakfast (e.g., oatmeal, banana, yogurt...

Claims

1. A means for taking a video of the gameplay; uploading means for uploading the gameplay video captured by the capturing means to a server; A storage means for storing the uploaded video; analysis means for analyzing the stored video; an advice generation means for generating advice for improving play based on the analysis results; a transmitting means for transmitting the generated advice to a terminal; a display means for displaying the transmitted advice; A system including:

2. an input means for inputting nutritional management information; an information transmission means for transmitting the input nutrition management information to a server; evaluation means for evaluating the transmitted nutritional management information; a nutrition advice generating means for generating nutrition advice based on the evaluation results; a nutrition advice sending means for sending the generated nutrition advice to a terminal; a display means for displaying the transmitted nutrition advice; The system of claim 1 further comprising:

3. 2. The system according to claim 1, wherein the play improvement advice is generated by referring to a plurality of method databases.

Images