system

Abstract

We provide the system. [Solution] A means for users to upload images or videos of their own athletic performance, A means for analyzing uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based on that analysis, A means for generating an optimal training menu for each user based on identified technical strengths and weaknesses, Means of providing training menus to users, A means of recording the user's training progress and feedback. A means of analyzing recorded data and updating the training menu according to the user's progress. A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] A problem faced by many sports enthusiasts and athletes is that it is unclear how to efficiently improve their own skills and performance. With conventional training methods and self-analysis, it is difficult to appropriately grasp specific weaknesses and strengths, and it is difficult to construct an effective training menu. In addition, in many cases, it costs a large amount of money to receive professional guidance, and a method that allows easy access to professional guidance is required.

Means for Solving the Problems

[0005] To address these challenges, we provide a system that identifies users' technical strengths and weaknesses by having them upload images or videos of their exercise performance, which are then analyzed by a generative artificial intelligence model. Based on these identified strengths and weaknesses, the system generates and provides a personalized training menu for each user. Furthermore, it records the user's training progress and feedback, and updates the training menu according to the user's progress by analyzing the recorded data. In this way, we provide an easy way for users to receive expert guidance.

[0006] A "user" is a person who uses the system with the aim of improving their own athletic performance.

[0007] "Athletic performance" refers to all aspects of movement, form, and skills in sports and physical activities.

[0008] "Images or videos" refer to still or moving image files that record the user's exercise performance.

[0009] "Uploading" refers to the action of a user sending data (images or videos) from their device to the system.

[0010] A "generative artificial intelligence model" refers to an artificial intelligence technology that analyzes specific information from provided image or video data and performs a technical evaluation of the user.

[0011] "Analysis" is the process by which a generative artificial intelligence model processes image or video data to identify the user's technical strengths and weaknesses.

[0012] "Technical strengths and weaknesses" refer to the strengths and weaknesses of the user's athletic performance.

[0013] A "training menu" refers to a series of exercises and practice programs designed to improve performance, provided to the user based on the analysis results of a generative artificial intelligence model.

[0014] "Recording" refers to the act of users inputting their training progress and feedback, and saving that data.

[0015] "Progress" refers to the results and improvements that a user has achieved through training.

[0016] "Updating" refers to the act of changing the content of a training menu to make it up-to-date based on analytical and recorded data. [Brief explanation of the drawing]

[0017] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0019] First, the terms used in the following description will be described.

[0020] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] Modes for carrying out the invention

[0039] This invention is a system that provides users with effective training menus using captured images or videos to improve their athletic performance. This system functions through the interaction of a server, a terminal, and the user.

[0040] Image and video upload and analysis

[0041] Users select and upload images or videos of their exercise performance using a smartphone app. These images or videos must capture specific movements or forms of the user in detail.

[0042] The device has the ability to compress selected images or videos and send them to a server over the internet. The device also includes a preview function for uploaded images and videos, allowing users to check the data before sending.

[0043] The server saves images or videos received from the terminal to its storage and begins analysis using a generative artificial intelligence model. The generative AI model analyzes each frame of the image or video, evaluating the user's actions, form, timing, etc. This identifies the user's technical strengths and weaknesses.

[0044] Creating a training menu

[0045] The server generates an optimal training menu based on the analysis results, the user's past training data, and pre-set goals. This training menu is customized according to the user's characteristics and challenges, and includes specific exercises and training items.

[0046] The generated training menu is converted into a user-friendly format (e.g., PDF or in-app display) and sent to the device. The device provides an interface for the user to view the training menu and perform the training accordingly.

[0047] Training implementation and recording

[0048] The user performs exercises according to the training menu and records their progress and feedback on the device. The device temporarily saves the recorded data to local storage and uploads it to the server once an internet connection is established.

[0049] Feedback and menu updates

[0050] The server receives the training record data uploaded by the user and performs a new analysis by comparing it with previous analysis results. Based on these analysis results, it evaluates the user's progress and decides whether to update the training menu.

[0051] For example, if the analysis results confirm that "right foot kicking power has improved," the new training menu will be modified to include "improving left foot technique" as a key focus. This allows users to efficiently improve their skills while continuously receiving expert guidance.

[0052] The following describes the processing flow.

[0053] Specific steps of the program's processing

[0054] Image and video upload and analysis

[0055] Step 1:

[0056] The user launches the smartphone app and opens the image or video upload screen. The user selects an image or video of their exercise performance.

[0057] Step 2:

[0058] The device compresses the selected image or video and prepares it for upload. At this stage, the user can preview the data and, if there are no problems, begin the upload.

[0059] Step 3:

[0060] The device sends compressed images or videos to the server over the internet. Once the transmission is complete, the user receives a notification.

[0061] Step 4:

[0062] The server saves the images or videos received from the terminal to its storage and passes them to a generative artificial intelligence model to begin analysis.

[0063] Step 5:

[0064] The server uses a generative artificial intelligence model to analyze each frame of images and videos, evaluating the user's actions, form, timing, etc. Based on this evaluation, it identifies the user's technical strengths and weaknesses.

[0065] Step 6:

[0066] The server saves the analysis results to a database, which will serve as the basis for creating the next training menu.

[0067] Creating a training menu

[0068] Step 7:

[0069] The server executes an algorithm based on the analysis results, the user's past training data, and their goals to generate the optimal training menu for the user.

[0070] Step 8:

[0071] The server converts the generated training menu into a user-friendly format (e.g., PDF, in-app display).

[0072] Step 9:

[0073] The server sends the generated training menu to the terminal.

[0074] Training implementation and recording

[0075] Step 10:

[0076] The device displays the received training menu to the user and prompts them to perform the training according to the instructions.

[0077] Step 11:

[0078] Users perform training according to the training menu and record their progress and feedback on their device.

[0079] Step 12:

[0080] The device temporarily stores recorded data in local storage and synchronizes it with the server when connected to the internet.

[0081] Feedback and menu updates

[0082] Step 13:

[0083] The server receives the training record data uploaded by the user and performs a new analysis by comparing it with the results of previous analyses.

[0084] Step 14:

[0085] The server performs progress assessments and updates the training menu as needed. This assessment includes, for example, the degree of technical improvement and any newly discovered issues.

[0086] Step 15:

[0087] The server will send the updated training menu and any new feedback back to the terminal. This information will be reflected in the next training session.

[0088] Step 16:

[0089] The device notifies the user of updates and feedback, helping them prepare for the next training session.

[0090] (Example 1)

[0091] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] Modern sports training demands accurate feedback and customized training programs to effectively improve individual athletic performance. However, traditional methods, such as users showing individually filmed videos to experts for advice, were often inefficient due to their reliance on human intervention. Furthermore, there was a lack of systems to accurately record and evaluate training progress, provide appropriate feedback, and update training programs. This resulted in the problem of training effectiveness not being maximized.

[0093] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0094] In this invention, the server includes means for analyzing uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based on the analysis; means for generating an optimal training menu for each user based on the identified technical strengths and weaknesses; means for recording the user's training progress and feedback; means for analyzing the recorded data and updating the training menu according to the user's progress; means for compressing images and videos captured by the terminal and sending them to the server; means for the server to store the received images and videos; means for the server to refer to the user's past training data and generate an optimal training menu in conjunction with the analysis results; means for the server to convert the training menu into a format that is easy for the user to understand and send it to the terminal; means for the terminal to temporarily save training data in local storage and upload it to the server; and means for the server to analyze newly received training data and update the training menu. This enables the provision of highly accurate feedback based on the user's individual data and customized training menus.

[0095] "Means for uploading images or videos" refers to a function that allows users to send images or videos of their exercise performance from their device to the server.

[0096] A "generative artificial intelligence model" is an artificial intelligence algorithm used to analyze image or video data to identify a user's technical strengths and weaknesses.

[0097] "Means for identifying a user's technical strengths and weaknesses" refers to a function that uses a generative artificial intelligence model to evaluate the user's actions, form, and timing, and based on the results, clearly identifies the user's strengths and areas that need improvement.

[0098] "A means of generating an optimal training menu" refers to a function that takes into account the user's technical strengths and weaknesses and creates an individually optimized training plan.

[0099] "Means of providing training menus" refers to a function that displays or sends the generated training menus to the user in an easy-to-understand format.

[0100] "Means for recording training progress and feedback" refers to a function that allows users to input and save the training content, progress, and feedback they have received on their device.

[0101] "Means for analyzing recorded data and updating training menus" refers to a function that re-evaluates the content of the training menu based on the user's progress and changes it to the latest training menu as needed.

[0102] "Compression" refers to a function that performs encoding processing to reduce the file size of captured images and videos.

[0103] "Means for saving received images and videos" refers to the function that allows a server to store image and video data sent from a terminal in its storage.

[0104] "Means of referencing the user's past training data" refers to a function where the server retrieves and uses the user's previous training history and performance data from a database.

[0105] "Means of converting training menus into a user-friendly format" refers to a function that processes the generated training menus into a format that users can easily view, such as PDF or in-app display.

[0106] "Method for temporarily saving to local storage" refers to a function that allows training data to be temporarily saved even when the device does not have an internet connection.

[0107] "Method for uploading to the server" refers to the function of sending data temporarily stored on the device to the server once an internet connection is established.

[0108] "Means for analyzing newly received training data" refers to a function in which the server analyzes newly submitted training data from the user again using a generative artificial intelligence model to evaluate the user's progress and performance.

[0109] Modes for carrying out the invention

[0110] This invention is a system that provides users with effective training menus using captured images or videos to improve their athletic performance. This system functions through the interaction of a server, a terminal, and the user.

[0111] Image and video upload and analysis

[0112] Users launch a smartphone app and select and upload images or videos of their exercise performance. These images or videos capture specific movements or forms of the user in detail.

[0113] The device compresses selected images and videos and sends them to the server over the internet. Specifically, the device compresses the video file selected by the user after shooting (e.g., a soccer dribbling scene) using an encoding algorithm (e.g., H.264) and sends it to the server's communication endpoint. The device also has a preview function for uploaded images and videos, allowing the user to check the data before sending it.

[0114] The server saves images or videos received from the terminal to cloud storage (e.g., AWS S3) and begins analysis using a generative artificial intelligence model (e.g., YOLOv5). This analysis scans each frame of the video to evaluate the user's actions, form, and timing. The analysis results are saved as JSON data, and the user's technical strengths and weaknesses are identified.

[0115] Creating a training menu

[0116] The server generates an optimal training menu for each user based on the analysis results, the user's past training data, and pre-set goals. For example, if the server determines that the user's dribbling speed is slow, training items aimed at improving this (e.g., repetitive dribbling practice) will be added to the menu. The generated training menu is converted into PDF format or an in-app display format and sent to the device.

[0117] Training menu provided

[0118] The device displays the received training menu within the app, making it easy for the user to review. For example, a new training plan will appear in the "Training Menu" tab, and the user can then follow it to complete their workout.

[0119] Training implementation and recording

[0120] Users perform exercises according to the provided training menu and record their progress and feedback on their device. The device temporarily saves the recorded data to local storage (e.g., an SQLite database). Once an internet connection is established, the data is uploaded to the server.

[0121] Feedback and menu updates

[0122] The server analyzes the newly received training record data and compares it to previous analysis results. For example, if it confirms an improvement in the kicking power of the right foot, it uses a generative artificial intelligence model to identify areas that need further improvement (e.g., improving the technique of the left foot) and updates the training menu. The updated menu is sent back to the terminal, and the user can continue training by following the instructions.

[0123] Examples of specific cases and prompt statements

[0124] For example, if a user is practicing soccer, the following sequence of events might occur.

[0125] 1. The user films themselves practicing soccer dribbling with their smartphone.

[0126] 2. The device compresses the recorded video and sends it to the server. The user previews the video before sending it.

[0127] 3. The server analyzes the received video using an AI model to evaluate the dribbling technique.

[0128] 4. The server generates a training menu for improving dribbling skills based on the analysis results and past data.

[0129] 5. The server converts the generated menu into PDF format and sends it to the terminal. The terminal displays the menu.

[0130] 6. The user practices according to the provided menu and records their progress on the device.

[0131] 7. The server analyzes the new training records and updates the training menu as needed.

[0132] The following are examples of prompts for a generative AI model.

[0133] Analyze the user's video and output the analysis results, focusing on the following key points:

[0134] 1. Accuracy of form

[0135] 2. Smoothness of operation

[0136] 3. Timing

[0137] If you encounter any problems with any of the following points, please provide details.

[0138] This allows the system to effectively analyze the user's exercise performance and provide an optimal training program.

[0139] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0140] Understood. Now, I will explain the processing flow of this system's program in detail, divided into the following steps.

[0141] Step 1:

[0142] The user launches a smartphone app and records their athletic performance. For example, if the user is practicing soccer, they will record their dribbling and shooting movements as videos. The input is the video recorded by the smartphone's camera. The output is the athletic performance video file saved on the smartphone.

[0143] Step 2:

[0144] The device compresses the video recorded by the user and sends it to the server. Specifically, the device compresses the video file using an encoding algorithm (e.g., H.264) and sends it to the server over the internet. The input is the video file obtained in step 1, and the output is the compressed video file sent to the server.

[0145] Step 3:

[0146] The server saves the video received from the terminal to cloud storage (e.g., AWS S3). The input is the compressed video file that was sent. The output is the video file saved in cloud storage. The server also passes the received video to the AI ​​analysis process.

[0147] Step 4:

[0148] The server analyzes the video using a generative artificial intelligence model (e.g., YOLOv5). Specifically, it scans each frame of the video and evaluates the user's actions, form, timing, etc. The input is a video file stored in cloud storage, and the output is JSON data as the analysis result. This analysis identifies the user's technical strengths and weaknesses.

[0149] Step 5:

[0150] The server generates an optimal training menu for each user based on the analysis results, the user's past training data, and pre-set goals. The input is the analysis results and past training data, and the output is the generated training menu. For example, it might generate a menu such as "Repetitive practice to improve dribbling speed."

[0151] Step 6:

[0152] The server converts the generated training menu into a user-friendly format (e.g., PDF or in-app display format) and sends it to the device. The input is the generated training menu, and the output is the training menu sent to the device.

[0153] Step 7:

[0154] The device displays the received training menu within the app, allowing the user to review it. For example, a new training plan will appear in the app's "Training Menu" tab. The input is the training menu sent from the server, and the output is the screen display that the user can review.

[0155] Step 8:

[0156] The user performs exercises according to the provided training menu and records their progress and feedback on the device. The acquired data includes information about the content and achievement level of the training. The input is the training progress and feedback entered by the user, and the output is the data recorded on the device.

[0157] Step 9:

[0158] The terminal temporarily saves recorded data to local storage (e.g., an SQLite database) and uploads it to the server once an internet connection is established. The input is the training progress and feedback data recorded on the terminal, and the output is the data uploaded to the server.

[0159] Step 10:

[0160] The server analyzes the newly received training record data and compares it to previous analysis results. The input is the newly uploaded training record data, and the output is the updated analysis results. For example, if it is confirmed that the kicking power of the right foot has improved, a training item to strengthen the technique of the left foot will be added.

[0161] Step 11:

[0162] The server regenerates the optimal training menu based on the user's progress and sends it to the user's terminal. The input is the updated analysis results and user feedback data, and the output is the regenerated training menu. The process is then repeated from step 6.

[0163] Through the steps described above, this invention can effectively improve the user's athletic performance.

[0164] (Application Example 1)

[0165] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0166] Modern factory robots are required to be highly precise and efficient, but there is no guarantee that a set operation sequence is optimal, necessitating regular performance evaluations and improvements. Furthermore, the challenge lies in the fact that it requires advanced expertise and considerable time for on-site personnel to properly evaluate robot operation and identify specific areas for improvement.

[0167] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0168] In this invention, the server includes means for uploading images or videos of the user's own exercise performance or the robot's motion performance; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the technical strengths and weaknesses of the user or robot based on the analysis; and means for generating an optimal training menu or motion sequence for each user or robot based on the identified technical strengths and weaknesses. This makes it possible to objectively and automatically evaluate the robot's motion performance and provide specific methods for improvement.

[0169] A "user" is a person or person who uses the system to evaluate and improve their own exercise performance or the operational performance of the robot they manage.

[0170] "Exercise performance" is a general term for the series of movements and actions performed by a user, including specific techniques, movements, form, and timing.

[0171] A "robot" is an automated mechanical system that performs tasks automatically in a factory based on a sequence of actions.

[0172] "Images or videos" refer to visual data that captures the actions of a user or robot and is in a format that can be processed by a machine.

[0173] A "generative artificial intelligence model" is an algorithm or machine learning model that analyzes images or videos, identifies technical features and shortcomings from their content, and provides appropriate guidance and improvement suggestions.

[0174] A "strength" is a technically superior aspect of a user or robot, a specific element that already demonstrates high performance.

[0175] A "weakness" is a technical aspect that the user or robot needs improvement on, and is a specific element that is currently lacking.

[0176] A "training menu" is a coaching plan that specifically outlines the exercises and practice items that a user should perform to improve their technical skills.

[0177] An "operation sequence" refers to a set of steps or actions that a factory robot should perform.

[0178] "Progress" refers to data that shows how far user training and robot operation have progressed, and how well the plan has been achieved.

[0179] "Feedback" refers to evaluations and comments received regarding the results of training or performance, providing specific directions for the next step.

[0180] "Analysis" refers to the process of using generative artificial intelligence models to analyze image or video data in detail, revealing its technical characteristics and shortcomings.

[0181] A "server" is a central processing unit that processes data sent from users or robots and provides analysis results and training menus.

[0182] A "device" is a device used to temporarily store images or videos taken by a user and upload them to a server; this includes smartphones and tablets.

[0183] This invention is a system that uses video to provide effective improvement menus for users or factory personnel to improve the operational performance of robots. The specific implementation method of the system will be described below.

[0184] Users or factory personnel use devices such as smartphones or tablets to record video of the robot's performance. The recorded videos are saved on the device and compressed before being uploaded to the server via the internet. The device provides users with a preview function for the uploaded content, allowing them to review and edit the videos.

[0185] The server stores the uploaded video in storage and analyzes it using a generative artificial intelligence model. This analysis involves a detailed evaluation of each frame of the video, analyzing the robot's movements, accuracy, timing, and other aspects. Based on these analysis results, the server identifies the robot's technical strengths and weaknesses.

[0186] After identifying strengths and weaknesses, the server uses a generative artificial intelligence model to create a personalized improvement menu, suggesting optimal action sequences and improvement methods. This improvement menu is then sent to the device in PDF or in-app display format.

[0187] Users or factory personnel can check improvement menus via a terminal and adjust robot settings and operations as needed. They can also record the progress of the robot's operation and how performance has changed after configuration changes. The terminal temporarily stores this data in local storage and later uploads it to the server.

[0188] The server analyzes uploaded training data and motion records, and updates the training menu and motion sequences by comparing them with past analysis results. This re-analysis process makes it possible to continuously improve the accuracy and efficiency of the robot's movements.

[0189] As a concrete example, consider a procedure to improve the lifting motion of a robotic arm used in a factory. Suppose the initial analysis reveals that the timing of the lifting motion is incorrect. Based on this result, the server suggests improvements and provides a new training sequence. The user modifies the settings based on these suggestions and sends feedback back to the system. The system re-analyzes the system, finding further areas for improvement, ultimately resulting in a smoother and more efficient lifting motion.

[0190] As described above, this invention provides a system that effectively improves the operational performance of a robot by utilizing a generative artificial intelligence model and user feedback.

[0191] Example of a prompt:

[0192] "Explain how to film the movements of robots in a factory, analyze those movements, and provide suggestions for improvement using a generating AI model."

[0193] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0194] Step 1: The user films the robot's movements.

[0195] The user uses a smartphone or tablet to record the robot's performance as a video. The input includes the robot's specific action sequence. The output is the recorded video file, which is saved on the device.

[0196] Step 2: Compress and preview the recorded video on your device.

[0197] The device compresses the recorded video file and trims it if necessary. The user can also preview the video during this process and keep only the desired portion. The input is the raw, unprocessed video file, and the output is the compressed, final video file.

[0198] Step 3: Upload the video to the server.

[0199] The device uploads a compressed video file to the server. During this process, the device transmits the file using the internet. The input is the compressed video file, and the output is the video file uploaded to the server.

[0200] Step 4: The server analyzes the video.

[0201] The server saves the received video file to storage and begins video analysis using a generative artificial intelligence model. This analysis involves analyzing each frame of the video and evaluating the robot's movements, accuracy, timing, etc. The input is the uploaded video file, and the output is data indicating its technical strengths and weaknesses.

[0202] Step 5: The server generates the improvement menu.

[0203] The server uses a generative artificial intelligence model that proposes optimal action sequences and improvement methods based on the analysis results to create individual improvement menus. The input is data indicating strengths and weaknesses, and the output is improvement menus and specific action sequences.

[0204] Step 6: The server sends the improvement menu to the terminal.

[0205] The server sends the generated improvement menu to the device in PDF format or as an in-app display format. The input consists of improvement menus and action sequences, and the output is data in a format viewable on the device.

[0206] Step 7: The user changes the settings according to the improvement menu.

[0207] The user checks the improvement menu via a terminal and adjusts the robot's settings and operation as needed. The input is the improvement menu, and the output is the actual operation of the robot after the settings have been changed.

[0208] Step 8: Users record their feedback.

[0209] The user records the progress of the robot's movements and how performance has changed after configuration changes. The terminal temporarily stores this data in local storage. The input is the record of the movements after the changes, and the output is the recorded feedback data.

[0210] Step 9: The server re-analyzes the feedback.

[0211] The terminal uploads recorded feedback data to the server, which then analyzes the data. The server updates the training menu and action sequence by comparing it with past analysis results. The input is the recorded feedback data, and the output is the updated improvement menu.

[0212] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0213] Modes for carrying out the invention

[0214] This invention is a system that combines image / video analysis and an emotion engine to improve the user's athletic performance. It analyzes the user's technical strengths and weaknesses, and further considers their emotional state to provide an optimal training menu. This system is realized through the interaction of a server, a terminal, and the user.

[0215] Image and video upload and analysis

[0216] Users upload images or videos of their exercise performance using a smartphone app. This allows users to provide the system with data that enables detailed analysis of their movements.

[0217] The device compresses the image or video selected by the user and sends it to the server. The server stores the received data in storage and begins analysis using a generative artificial intelligence model. The generative AI model analyzes each frame of the video and evaluates the user's actions, form, timing, etc. As a result of the analysis, the user's technical strengths and weaknesses are identified.

[0218] Implementation of an emotion engine

[0219] The server uses an emotion engine to analyze user facial expression data in addition to image or video data. The emotion engine analyzes the user's facial expressions and voice to extract how the user feels about the training. This identifies the emotional state (e.g., motivation, fatigue, stress).

[0220] Creating a training menu

[0221] The server generates an optimal training menu based on technical analysis results and emotional state analysis results from the emotion engine. In addition to specific exercise items, adjustments are made considering the user's emotional state, and the training menu is customized individually. For example, if the server determines that the user is fatigued, a lighter training program will be suggested.

[0222] The generated training menu is converted into a user-friendly format (e.g., PDF, in-app display) and sent to the device. The device provides an interface for the user to review the training menu and perform the training accordingly.

[0223] Training implementation and recording

[0224] Users perform training according to the training menu and record their progress and feedback on their device. The device temporarily saves the recorded data to local storage and uploads it to the server when connected to the internet.

[0225] Feedback and menu updates

[0226] The server receives the training record data uploaded by the user and performs a re-analysis, comparing it to previous analysis results. This analysis includes technical progress and changes in emotional state. Based on the progress evaluation, the training menu and feedback are updated as needed.

[0227] For example, if a user's skills improve and their emotional state changes for the better, the training menu is updated to include new challenges to help them move on to the next step. The device notifies the user of updates and feedback, helping them prepare for the next training session.

[0228] Specific example

[0229] User A wants to improve their soccer dribbling skills. They record videos of themselves dribbling with their smartphone and upload them to the system. The server analyzes the videos using a generative artificial intelligence model and an emotion engine, identifying that they lack kicking power with their right foot and that they experience stress during practice.

[0230] Next, the server generates a training menu based on these results, combining "specific dribbling exercises to strengthen the kicking power of the right foot" and "relaxation exercises to reduce stress." User A practices according to this menu and records their progress on their device. The server analyzes this record, confirms that the kicking power of the right foot has improved and stress has decreased, and adjusts and provides the next menu.

[0231] The following describes the processing flow.

[0232] Specific steps of the program's processing

[0233] Image and video upload and analysis

[0234] Step 1:

[0235] The user launches the smartphone app and opens the image or video upload screen. The user selects an image or video of their exercise performance.

[0236] Step 2:

[0237] The device compresses the selected image or video and prepares it for upload. At this stage, the user can preview the data and, if there are no problems, begin the upload.

[0238] Step 3:

[0239] The device sends compressed images or videos to the server over the internet. Once the transmission is complete, the user receives a notification.

[0240] Step 4:

[0241] The server saves the images or videos received from the terminal to its storage and passes them to a generative artificial intelligence model to begin analysis.

[0242] Step 5:

[0243] The server uses a generative artificial intelligence model to analyze each frame of images and videos, evaluating the user's actions, form, timing, etc. Based on this evaluation, it identifies the user's technical strengths and weaknesses.

[0244] Step 6:

[0245] The server saves the analysis results to a database, which will serve as the basis for creating the next training menu.

[0246] Implementation of an emotion engine

[0247] Step 7:

[0248] The server activates an emotion engine that analyzes the user's facial expressions and voice, in addition to image or video data, to recognize the user's emotional state.

[0249] Step 8:

[0250] The server stores the user's emotional state (e.g., motivation, fatigue, stress) in a database based on the recognition results of the emotion engine.

[0251] Creating a training menu

[0252] Step 9:

[0253] The server integrates technical analysis results with the emotion engine analysis results to generate an optimal training menu for each individual user. This menu includes not only technical items but also adjustments based on emotional state.

[0254] Step 10:

[0255] The server converts the generated training menu into a user-friendly format (e.g., PDF, in-app display).

[0256] Step 11:

[0257] The server sends the generated training menu to the terminal.

[0258] Training implementation and recording

[0259] Step 12:

[0260] The device displays the received training menu to the user and prompts them to perform the training according to the instructions.

[0261] Step 13:

[0262] Users perform training according to the training menu and record their progress and feedback on their device.

[0263] Step 14:

[0264] The device temporarily stores recorded data in local storage and synchronizes it with the server when connected to the internet.

[0265] Feedback and menu updates

[0266] Step 15:

[0267] The server receives the training record data uploaded by the user and performs a new analysis by comparing it with the results of previous analyses.

[0268] Step 16:

[0269] The server performs progress assessments and updates the training menu as needed. This assessment includes, for example, the degree of technical improvement, newly discovered issues, and the user's emotional state.

[0270] Step 17:

[0271] The server will send the updated training menu and any new feedback back to the terminal. This information will be reflected in the next training session.

[0272] Step 18:

[0273] The device notifies the user of updates and feedback, helping them prepare for the next training session.

[0274] (Example 2)

[0275] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0276] Traditional athletic performance enhancement systems focused on technical analysis and failed to consider the user's emotional state. Therefore, they couldn't provide appropriate training programs when users were stressed or fatigued, making it difficult to achieve sustainable training results. Furthermore, the training programs provided to users were uniform, failing to adequately reflect individual technical strengths and weaknesses.

[0277] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0278] In this invention, the server includes means for uploading images or videos of the user's exercise performance; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based on the analysis; means for using an emotion engine to analyze the user's emotional state in addition to the identified technical strengths and weaknesses; means for generating an optimal training menu for each individual user based on the analyzed technical strengths and weaknesses and emotional state; means for converting the generated training menu into a user-friendly format and providing it to the user; means for recording the user's training progress and feedback on the terminal; and means for uploading the recorded data to the server and analyzing it to update the training menu according to the user's progress. This makes it possible to accurately identify the user's technical strengths and weaknesses, provide an individualized training menu based on them, and realize an appropriate training environment that takes into account the user's emotional state.

[0279] "User" refers to a person who uses the system to photograph their own exercise performance and receive analysis and training menu offerings.

[0280] "Terminal" refers to a device (e.g., smartphone, tablet, personal computer, etc.) used by the user, which has the function of compressing the photographed images and videos and transmitting them to the server.

[0281] "Server" refers to a computer system that has the functions of receiving, storing, and analyzing images and videos transmitted from the user, and generating and providing a training menu.

[0282] "Means for uploading an image or video" refers to the function used by the user to photograph their own exercise performance and transmit data into the system.

[0283] "Generative artificial intelligence model" refers to artificial intelligence technology used to analyze uploaded images and videos and identify the technical strengths and weaknesses of the user.

[0284] "Emotion engine" refers to technology used to analyze the user's expressions and movements and identify their emotional state.

[0285] <000,0902>"Means for identifying technical strengths and weaknesses" refers to the function of evaluating the user's technical capabilities from the analysis results of images and videos using a generative artificial intelligence model and identifying specific strengths and issues. [[ID=2,6]]

[0286] "Means for generating a training menu" refers to the function of creating optimal training content for each user based on the analyzed technical strengths and weaknesses and emotional state.

[0287] <00009,08>"Means for providing to the user" refers to the function of providing the generated training menu to the user in an easy-to-understand format.

[0288] "Means for recording training progress and feedback" refers to a function that allows users to record their results and impressions within the system after completing training.

[0289] "A means of updating the training menu according to the user's progress through analysis" refers to a function that analyzes recorded data and updates the training menu as needed, taking into account the user's technical progress and emotional state.

[0290] Modes for carrying out the invention

[0291] This invention is a system that combines image / video analysis and an emotion engine to improve the user's athletic performance. It analyzes the user's technical strengths and weaknesses, and further considers their emotional state to provide an optimal training menu. This system is realized through the interaction of a server, a terminal, and the user.

[0292] Image and video upload and analysis

[0293] Users upload images or videos of their exercise performance using a smartphone app. This allows users to provide the system with data that enables detailed analysis of their movements.

[0294] The terminal compresses images and videos selected by the user and sends them to the server. The server stores the received data in storage and begins analysis using a generative artificial intelligence model. The generative AI model analyzes each frame of the video and evaluates the user's actions, form, timing, etc. As a result of the analysis, the user's technical strengths and weaknesses are identified.

[0295] Implementation of an emotion engine

[0296] The server uses an emotion engine to analyze user facial expression data in addition to image or video data. The emotion engine analyzes the user's facial expressions and voice to extract how the user feels about the training. This identifies the emotional state (e.g., motivation, fatigue, stress).

[0297] Creating a training menu

[0298] The server generates an optimal training menu based on technical analysis results and emotional state analysis results from the emotion engine. In addition to specific exercise items, adjustments are made considering the user's emotional state, and the training menu is customized individually. For example, if the server determines that the user is fatigued, a lighter training program will be suggested.

[0299] The generated training menu is converted into a user-friendly format (e.g., PDF, in-app display) and sent to the device. The device provides an interface for the user to review the training menu and perform the training accordingly.

[0300] Training implementation and recording

[0301] Users perform training according to the training menu and record their progress and feedback on their device. The device temporarily saves the recorded data to local storage and uploads it to the server when connected to the internet.

[0302] Feedback and menu updates

[0303] The server receives the training record data uploaded by the user and performs a re-analysis, comparing it to previous analysis results. This analysis includes technical progress and changes in emotional state. Based on the progress evaluation, the training menu and feedback are updated as needed.

[0304] For example, when the user's skills improve and their emotional state changes positively, the training menu is updated to include new challenges for the next step. The terminal notifies the user of the updated information and feedback, and helps prepare for the next training session.

[0305] Specific example

[0306] User A wants to improve their soccer dribbling skills. They use their smartphone to record a video of their dribbling scene and upload it to the system. The server analyzes it using a generative AI model and an emotion engine, and identifies that the user's right foot kicking power is insufficient and that they feel stressed during practice.

[0307] Next, based on these results, the server generates a training menu that combines "specific dribbling training to strengthen the right foot kicking power" and "relaxation exercises to reduce stress". User A practices based on this menu and records the progress on the terminal. The server analyzes this record, confirms that the right foot kicking power has improved and the stress has decreased, and adjusts and provides the next menu.

[0308] Example of input prompt to the generative AI model

[0309] To improve the user's sports performance, analyze the video of the dribbling taken by the user with a smartphone, identify the technical strengths and weaknesses, and also analyze the user's expression in the video to identify the emotional state (e.g., motivation, fatigue, stress, etc.). Based on this data, generate an optimal training menu considering both technology and emotion.

[0310] The flow of the specific process in Example 2 will be described using FIG. 13.

[0311] Step 1:

[0312] This stage involves users taking images or videos of their exercise performance and uploading them through a smartphone app.

[0313] Input: Images or videos of athletic performance captured by the user.

[0314] Operation: The user uses a smartphone app to select a captured image or video and presses the upload button.

[0315] Output: Image or video data uploaded to the device.

[0316] Step 2:

[0317] This is the stage where the device compresses the image or video uploaded by the user and sends it to the server.

[0318] Input: Image or video data uploaded by the user on their device.

[0319] Operation: The device compresses image or video data and sends it to the server over the internet.

[0320] Output: Compressed image or video data is sent to the server.

[0321] Step 3:

[0322] This stage involves the server saving the received image or video data to storage and starting the analysis using a generative artificial intelligence model.

[0323] Input: Compressed image or video data.

[0324] Operation: The server receives data, saves it to storage, and then inputs that data into a generative artificial intelligence model to begin analysis. Each frame is analyzed, and the user's actions, form, timing, etc., are evaluated.

[0325] Output: Analysis results data showing the user's technical strengths and weaknesses.

[0326] Step 4:

[0327] This is the stage where the server uses an emotion engine to analyze the user's facial expression data from image or video data.

[0328] Input: Image or video data, and analysis result data obtained in the previous step.

[0329] Operation: The server uses an emotion engine to analyze the user's facial expressions in an image or video and extract the user's emotional state (e.g., motivation, fatigue, stress).

[0330] Output: Data indicating the user's emotional state.

[0331] Step 5:

[0332] This is the stage where the server generates an optimal training menu for each individual user based on technical analysis results and emotional state analysis results.

[0333] Input: Technical analysis results data, data indicating emotional state.

[0334] Operation: The server comprehensively analyzes this data and generates the optimal training menu. For example, if the user is fatigued, a lighter training program will be suggested.

[0335] Output: The generated training menu.

[0336] Step 6:

[0337] This stage involves the server converting the generated training menu into a user-friendly format (e.g., PDF, in-app display) and sending it to the device.

[0338] Input: Generated training menu.

[0339] Operation: The server converts the training menu into a user-friendly format and sends it to the terminal via the internet.

[0340] Output: The training menu, converted into a user-friendly format, is sent to the terminal.

[0341] Step 7:

[0342] This stage involves the user completing the training according to the training menu and recording their progress and feedback on their device.

[0343] Input: Training menu.

[0344] Operation: Users perform training according to the provided training menu, and their progress and feedback are recorded on their device via the app.

[0345] Output: Training progress and feedback data recorded on the device.

[0346] Step 8:

[0347] The terminal uploads the recorded training progress and feedback data to the server, where the server performs re-analysis.

[0348] Input: User training progress and feedback data.

[0349] Operation: The device sends recorded data to a server via the internet, and the server re-analyzes the received data. It evaluates technical progress and changes in emotional state, and updates the training menu as needed.

[0350] Output: An updated training menu and feedback are generated.

[0351] (Application Example 2)

[0352] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0353] In modern factories and production lines, robots handle many tasks, but optimizing their movements and performance remains challenging. In particular, identifying a robot's technical strengths and weaknesses and providing appropriate training programs based on that understanding is difficult. Furthermore, it's necessary to efficiently collect and analyze data on robot movements and improve them in real time. This is crucial for improving factory production efficiency.

[0354] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for uploading images or videos of the user's exercise performance; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based thereon; means for generating an optimal training menu for each user based on the identified technical strengths and weaknesses; means for providing the training menu to the user; means for recording the user's training progress and feedback; means for analyzing the recorded data and updating the training menu according to the user's progress; means for uploading images or videos to monitor the robot's movements; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the robot's technical strengths and weaknesses based thereon; and means for generating an optimal training menu for each robot based on the robot's technical strengths and weaknesses. This makes it possible to optimize the robot's movements and performance, thereby improving the production efficiency of the factory.

[0355] A "user" is an individual or organization that uses the system to improve their athletic performance.

[0356] "Exercise performance" refers to the user's physical movements and athletic abilities.

[0357] "Images or videos" refer to visual data that records the user's exercise performance.

[0358] "Uploading" refers to the act of sending images or videos that a user has taken to a system.

[0359] A "generative artificial intelligence model" is a machine learning model that analyzes uploaded images and videos to identify the user's technical strengths and weaknesses.

[0360] "Analysis" refers to the process of analyzing information from images and videos using generative artificial intelligence models.

[0361] "Technical strengths" refer to specific movements or skills in sports that a user excels at.

[0362] A "weakness" refers to an area where a user needs improvement, or a task or skill they struggle with.

[0363] A "training menu" is a personalized exercise plan generated based on the user's technical strengths and weaknesses.

[0364] "Recording" refers to the act of photographing or inputting the user's training progress and feedback.

[0365] A "robot" is an automated machine that performs tasks on production lines in factories and other facilities.

[0366] "Action" refers to the physical movements or tasks that a robot performs.

[0367] "Optimization" is the act of adjusting something to maximize the performance of a user or robot.

[0368] "Maintenance" refers to maintenance work performed to maintain the robot's operation and improve its performance.

[0369] "Factory production efficiency" refers to the efficiency of production activities in a factory, meaning an improvement in both the quality and speed of work.

[0370] A "server" is a computer system used to store and analyze uploaded data.

[0371] A "terminal" is a device used by users to input data or check training menus.

[0372] "Real-time" refers to a time frame in which data is processed immediately and results are provided instantly.

[0373] System Overview

[0374] This invention is a system designed to improve user motor performance and robot operation performance. The system is primarily realized through the interaction of a server, a terminal, and the user.

[0375] Image and video upload and analysis

[0376] Users or factory managers use smartphones or cameras to capture images and videos of exercise performance or robot movements, and upload them to the system via a terminal. The terminal compresses the selected images and videos from the user and efficiently sends them to the server. The server stores the received data in storage and begins analysis using a generative artificial intelligence model. This model analyzes each frame of the video, evaluating specific movements, forms, and timings of the user or robot, and identifies technical strengths and weaknesses.

[0377] Implementation of an emotion engine

[0378] In addition to analyzing image or video data, the server uses an emotion engine to analyze facial expression data. This emotion engine analyzes the user's facial expressions and the robot's movement parameters to extract emotional states (motivation, fatigue, stress, etc.) related to training and work.

[0379] Creating a training menu

[0380] The server generates an optimal training menu based on technical analysis results and analysis results from the emotion engine. This menu is customized to the individual user and robot. For example, if the user is determined to be fatigued, a lighter training program will be suggested. Similarly, if the robot's operation is under high load, maintenance protocols to reduce the load will be provided.

[0381] The generated training menu is sent to the terminal in a user-friendly format. The terminal provides an interface for users and factory managers to review the training menu and work through it.

[0382] Training implementation and recording

[0383] Users perform exercises according to the training menu and record their progress and feedback on a terminal. In the case of robots, factory managers monitor the robot's performance and similarly record it via a terminal. The recorded data is temporarily stored in local storage and uploaded to a server when an internet connection is available.

[0384] Feedback and menu updates

[0385] The server receives the uploaded training and movement recording data and performs a re-analysis, comparing it to previous analysis results. This analysis includes technical progress and changes in emotional state. Based on the progress evaluation, the training menu and feedback are updated as needed and notified to the user and the robot. This improves subsequent training sessions and work performance.

[0386] Examples of specific cases and prompt statements

[0387] If user A wants to improve their soccer dribbling skills, they record a video of themselves dribbling with their smartphone and upload it to the system. The server uses a generative artificial intelligence model and an emotion engine to analyze the video and identify that user A lacks kicking power with their right foot and experiences stress during practice. The server then generates a training menu that combines "specific dribbling exercises to strengthen right-foot kicking power" and "relaxation exercises to reduce stress." User A practices based on this menu and records their progress on their device.

[0388] Examples of prompt statements are as follows:

[0389] "The robot is under heavy load and is experiencing continuous malfunctions. Use the emotion engine to assess the robot's current condition and propose appropriate maintenance protocols."

[0390] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0391] Step 1:

[0392] Users capture images or videos of their exercise performance using their smartphones and upload them to the system via the device. The input from the user is the captured image or video, and the output is compressed data transmitted via the device. Specifically, users use a camera app on their smartphone, select the captured data, and press the upload button.

[0393] Step 2:

[0394] The terminal compresses image or video data provided by the user and sends it to the server. The input is video or image data uploaded by the user, and the output is a compressed data file. This compression is performed to improve the efficiency of data transfer. Typically, a compression algorithm is used.

[0395] Step 3:

[0396] The server saves the received data to storage. The input is a compressed data file, and the output is the data saved to storage. Error checking is also performed to confirm that the data was saved successfully.

[0397] Step 4:

[0398] The server analyzes incoming data using a generative artificial intelligence model. The input is stored image or video data, and the output is the user's technical strengths and weaknesses. As part of the data processing, image processing algorithms are used to analyze each frame of the video, evaluating movement, form, and timing. This identifies the strengths and weaknesses of athletic performance.

[0399] Step 5:

[0400] The server uses an emotion engine to analyze facial expression data and motion parameters. Inputs are facial expression data from videos and robot motion parameters, while outputs are emotional states (e.g., motivation, fatigue, stress). This analysis uses facial recognition and speech analysis technologies to identify the emotional state of the user or robot.

[0401] Step 6:

[0402] The server combines technical analysis results with the emotional engine's analysis results to generate an optimal training menu. The inputs are technical strengths and weaknesses, as well as emotional states, while the output is a customized training menu. Specifically, adjustments are made, such as reducing the training content based on the user's fatigue level.

[0403] Step 7:

[0404] The server sends the generated training menu to the terminal and provides it to the user. The input is the generated training menu, and the output is a display in a user-friendly format (PDF, in-app display). This allows the user to perform training based on the new menu.

[0405] Step 8:

[0406] The user performs exercises according to the provided training menu and records their progress and feedback on the device. Input is the training progress and feedback, while output is the data stored on the device. An interface is used to record actions and emotional feedback during training.

[0407] Step 9:

[0408] The terminal uploads recorded training data to the server when connected to the internet. Input is progress data and feedback stored on the terminal, while output is data transmission to the server. The transmitted data is stored in the server's storage.

[0409] Step 10:

[0410] The server re-analyzes the uploaded data and evaluates the user's technical progress and changes in emotional state. The input is the newly uploaded training data, and the output is the updated training menu and feedback. This allows the next training session to be adjusted based on the user's progress.

[0411] Examples of prompt statements are as follows:

[0412] "The robot is under heavy load and is experiencing continuous malfunctions. Use the emotion engine to assess the robot's current condition and propose appropriate maintenance protocols."

[0413] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0414] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0415] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0416] [Second Embodiment]

[0417] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0418] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0419] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0420] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0421] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0422] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0423] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0424] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0425] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0426] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0427] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0428] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0429] Modes for carrying out the invention

[0430] This invention is a system that provides users with effective training menus using captured images or videos to improve their athletic performance. This system functions through the interaction of a server, a terminal, and the user.

[0431] Image and video upload and analysis

[0432] Users select and upload images or videos of their exercise performance using a smartphone app. These images or videos must capture specific movements or forms of the user in detail.

[0433] The device has the ability to compress selected images or videos and send them to a server over the internet. The device also includes a preview function for uploaded images and videos, allowing users to check the data before sending.

[0434] The server saves images or videos received from the terminal to its storage and begins analysis using a generative artificial intelligence model. The generative AI model analyzes each frame of the image or video, evaluating the user's actions, form, timing, etc. This identifies the user's technical strengths and weaknesses.

[0435] Creating a training menu

[0436] The server generates an optimal training menu based on the analysis results, the user's past training data, and pre-set goals. This training menu is customized according to the user's characteristics and challenges, and includes specific exercises and training items.

[0437] The generated training menu is converted into a user-friendly format (e.g., PDF or in-app display) and sent to the device. The device provides an interface for the user to view the training menu and perform the training accordingly.

[0438] Training implementation and recording

[0439] The user performs exercises according to the training menu and records their progress and feedback on the device. The device temporarily saves the recorded data to local storage and uploads it to the server once an internet connection is established.

[0440] Feedback and menu updates

[0441] The server receives the training record data uploaded by the user and performs a new analysis by comparing it with previous analysis results. Based on these analysis results, it evaluates the user's progress and decides whether to update the training menu.

[0442] For example, if the analysis results confirm that "right foot kicking power has improved," the new training menu will be modified to include "improving left foot technique" as a key focus. This allows users to efficiently improve their skills while continuously receiving expert guidance.

[0443] The following describes the processing flow.

[0444] Specific steps of the program's processing

[0445] Image and video upload and analysis

[0446] Step 1:

[0447] The user launches the smartphone app and opens the image or video upload screen. The user selects an image or video of their exercise performance.

[0448] Step 2:

[0449] The device compresses the selected image or video and prepares it for upload. At this stage, the user can preview the data and, if there are no problems, begin the upload.

[0450] Step 3:

[0451] The device sends compressed images or videos to the server over the internet. Once the transmission is complete, the user receives a notification.

[0452] Step 4:

[0453] The server saves the images or videos received from the terminal to its storage and passes them to a generative artificial intelligence model to begin analysis.

[0454] Step 5:

[0455] The server uses a generative artificial intelligence model to analyze each frame of images and videos, evaluating the user's actions, form, timing, etc. Based on this evaluation, it identifies the user's technical strengths and weaknesses.

[0456] Step 6:

[0457] The server saves the analysis results to a database, which will serve as the basis for creating the next training menu.

[0458] Creating a training menu

[0459] Step 7:

[0460] The server executes an algorithm based on the analysis results, the user's past training data, and their goals to generate the optimal training menu for the user.

[0461] Step 8:

[0462] The server converts the generated training menu into a user-friendly format (e.g., PDF, in-app display).

[0463] Step 9:

[0464] The server sends the generated training menu to the terminal.

[0465] Training implementation and recording

[0466] Step 10:

[0467] The device displays the received training menu to the user and prompts them to perform the training according to the instructions.

[0468] Step 11:

[0469] Users perform training according to the training menu and record their progress and feedback on their device.

[0470] Step 12:

[0471] The device temporarily stores recorded data in local storage and synchronizes it with the server when connected to the internet.

[0472] Feedback and menu updates

[0473] Step 13:

[0474] The server receives the training record data uploaded by the user and performs a new analysis by comparing it with the results of previous analyses.

[0475] Step 14:

[0476] The server performs progress assessments and updates the training menu as needed. This assessment includes, for example, the degree of technical improvement and any newly discovered issues.

[0477] Step 15:

[0478] The server will send the updated training menu and any new feedback back to the terminal. This information will be reflected in the next training session.

[0479] Step 16:

[0480] The device notifies the user of updates and feedback, helping them prepare for the next training session.

[0481] (Example 1)

[0482] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0483] Modern sports training demands accurate feedback and customized training programs to effectively improve individual athletic performance. However, traditional methods, such as users showing individually filmed videos to experts for advice, were often inefficient due to their reliance on human intervention. Furthermore, there was a lack of systems to accurately record and evaluate training progress, provide appropriate feedback, and update training programs. This resulted in the problem of training effectiveness not being maximized.

[0484] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0485] In this invention, the server includes means for analyzing uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based on the analysis; means for generating an optimal training menu for each user based on the identified technical strengths and weaknesses; means for recording the user's training progress and feedback; means for analyzing the recorded data and updating the training menu according to the user's progress; means for compressing images and videos captured by the terminal and sending them to the server; means for the server to store the received images and videos; means for the server to refer to the user's past training data and generate an optimal training menu in conjunction with the analysis results; means for the server to convert the training menu into a format that is easy for the user to understand and send it to the terminal; means for the terminal to temporarily save training data in local storage and upload it to the server; and means for the server to analyze newly received training data and update the training menu. This enables the provision of highly accurate feedback based on the user's individual data and customized training menus.

[0486] "Means for uploading images or videos" refers to a function that allows users to send images or videos of their exercise performance from their device to the server.

[0487] A "generative artificial intelligence model" is an artificial intelligence algorithm used to analyze image or video data to identify a user's technical strengths and weaknesses.

[0488] "Means for identifying a user's technical strengths and weaknesses" refers to a function that uses a generative artificial intelligence model to evaluate the user's actions, form, and timing, and based on the results, clearly identifies the user's strengths and areas that need improvement.

[0489] "A means of generating an optimal training menu" refers to a function that takes into account the user's technical strengths and weaknesses and creates an individually optimized training plan.

[0490] "Means of providing training menus" refers to a function that displays or sends the generated training menus to the user in an easy-to-understand format.

[0491] "Means for recording training progress and feedback" refers to a function that allows users to input and save the training content, progress, and feedback they have received on their device.

[0492] "Means for analyzing recorded data and updating training menus" refers to a function that re-evaluates the content of the training menu based on the user's progress and changes it to the latest training menu as needed.

[0493] "Compression" refers to a function that performs encoding processing to reduce the file size of captured images and videos.

[0494] "Means for saving received images and videos" refers to the function that allows a server to store image and video data sent from a terminal in its storage.

[0495] "Means of referencing the user's past training data" refers to a function where the server retrieves and uses the user's previous training history and performance data from a database.

[0496] "Means of converting training menus into a user-friendly format" refers to a function that processes the generated training menus into a format that users can easily view, such as PDF or in-app display.

[0497] "Method for temporarily saving to local storage" refers to a function that allows training data to be temporarily saved even when the device does not have an internet connection.

[0498] "Method for uploading to the server" refers to the function of sending data temporarily stored on the device to the server once an internet connection is established.

[0499] "Means for analyzing newly received training data" refers to a function in which the server analyzes newly submitted training data from the user again using a generative artificial intelligence model to evaluate the user's progress and performance.

[0500] Modes for carrying out the invention

[0501] This invention is a system that provides users with effective training menus using captured images or videos to improve their athletic performance. This system functions through the interaction of a server, a terminal, and the user.

[0502] Image and video upload and analysis

[0503] Users launch a smartphone app and select and upload images or videos of their exercise performance. These images or videos capture specific movements or forms of the user in detail.

[0504] The device compresses selected images and videos and sends them to the server over the internet. Specifically, the device compresses the video file selected by the user after shooting (e.g., a soccer dribbling scene) using an encoding algorithm (e.g., H.264) and sends it to the server's communication endpoint. The device also has a preview function for uploaded images and videos, allowing the user to check the data before sending it.

[0505] The server saves images or videos received from the terminal to cloud storage (e.g., AWS S3) and begins analysis using a generative artificial intelligence model (e.g., YOLOv5). This analysis scans each frame of the video to evaluate the user's actions, form, and timing. The analysis results are saved as JSON data, and the user's technical strengths and weaknesses are identified.

[0506] Creating a training menu

[0507] The server generates an optimal training menu for each user based on the analysis results, the user's past training data, and pre-set goals. For example, if the server determines that the user's dribbling speed is slow, training items aimed at improving this (e.g., repetitive dribbling practice) will be added to the menu. The generated training menu is converted into PDF format or an in-app display format and sent to the device.

[0508] Training menu provided

[0509] The device displays the received training menu within the app, making it easy for the user to review. For example, a new training plan will appear in the "Training Menu" tab, and the user can then follow it to complete their workout.

[0510] Training implementation and recording

[0511] Users perform exercises according to the provided training menu and record their progress and feedback on their device. The device temporarily saves the recorded data to local storage (e.g., an SQLite database). Once an internet connection is established, the data is uploaded to the server.

[0512] Feedback and menu updates

[0513] The server analyzes the newly received training record data and compares it to previous analysis results. For example, if it confirms an improvement in the kicking power of the right foot, it uses a generative artificial intelligence model to identify areas that need further improvement (e.g., improving the technique of the left foot) and updates the training menu. The updated menu is sent back to the terminal, and the user can continue training by following the instructions.

[0514] Examples of specific cases and prompt statements

[0515] For example, if a user is practicing soccer, the following sequence of events might occur.

[0516] 1. The user films themselves practicing soccer dribbling with their smartphone.

[0517] 2. The device compresses the recorded video and sends it to the server. The user previews the video before sending it.

[0518] 3. The server analyzes the received video using an AI model to evaluate the dribbling technique.

[0519] 4. The server generates a training menu for improving dribbling skills based on the analysis results and past data.

[0520] 5. The server converts the generated menu into PDF format and sends it to the terminal. The terminal displays the menu.

[0521] 6. The user practices according to the provided menu and records their progress on the device.

[0522] 7. The server analyzes the new training records and updates the training menu as needed.

[0523] The following are examples of prompts for a generative AI model.

[0524] Analyze the user's video and output the analysis results, focusing on the following key points:

[0525] 1. Accuracy of form

[0526] 2. Smoothness of operation

[0527] 3. Timing

[0528] If you encounter any problems with any of the following points, please provide details.

[0529] This allows the system to effectively analyze the user's exercise performance and provide an optimal training program.

[0530] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0531] Understood. Now, I will explain the processing flow of this system's program in detail, divided into the following steps.

[0532] Step 1:

[0533] The user launches a smartphone app and records their athletic performance. For example, if the user is practicing soccer, they will record their dribbling and shooting movements as videos. The input is the video recorded by the smartphone's camera. The output is the athletic performance video file saved on the smartphone.

[0534] Step 2:

[0535] The device compresses the video recorded by the user and sends it to the server. Specifically, the device compresses the video file using an encoding algorithm (e.g., H.264) and sends it to the server over the internet. The input is the video file obtained in step 1, and the output is the compressed video file sent to the server.

[0536] Step 3:

[0537] The server saves the video received from the terminal to cloud storage (e.g., AWS S3). The input is the compressed video file that was sent. The output is the video file saved in cloud storage. The server also passes the received video to the AI ​​analysis process.

[0538] Step 4:

[0539] The server analyzes the video using a generative artificial intelligence model (e.g., YOLOv5). Specifically, it scans each frame of the video and evaluates the user's actions, form, timing, etc. The input is a video file stored in cloud storage, and the output is JSON data as the analysis result. This analysis identifies the user's technical strengths and weaknesses.

[0540] Step 5:

[0541] The server generates an optimal training menu for each user based on the analysis results, the user's past training data, and pre-set goals. The input is the analysis results and past training data, and the output is the generated training menu. For example, it might generate a menu such as "Repetitive practice to improve dribbling speed."

[0542] Step 6:

[0543] The server converts the generated training menu into a user-friendly format (e.g., PDF or in-app display format) and sends it to the device. The input is the generated training menu, and the output is the training menu sent to the device.

[0544] Step 7:

[0545] The device displays the received training menu within the app, allowing the user to review it. For example, a new training plan will appear in the app's "Training Menu" tab. The input is the training menu sent from the server, and the output is the screen display that the user can review.

[0546] Step 8:

[0547] The user performs exercises according to the provided training menu and records their progress and feedback on the device. The acquired data includes information about the content and achievement level of the training. The input is the training progress and feedback entered by the user, and the output is the data recorded on the device.

[0548] Step 9:

[0549] The terminal temporarily saves recorded data to local storage (e.g., an SQLite database) and uploads it to the server once an internet connection is established. The input is the training progress and feedback data recorded on the terminal, and the output is the data uploaded to the server.

[0550] Step 10:

[0551] The server analyzes the newly received training record data and compares it to previous analysis results. The input is the newly uploaded training record data, and the output is the updated analysis results. For example, if it is confirmed that the kicking power of the right foot has improved, a training item to strengthen the technique of the left foot will be added.

[0552] Step 11:

[0553] The server regenerates the optimal training menu based on the user's progress and sends it to the user's terminal. The input is the updated analysis results and user feedback data, and the output is the regenerated training menu. The process is then repeated from step 6.

[0554] Through the steps described above, this invention can effectively improve the user's athletic performance.

[0555] (Application Example 1)

[0556] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0557] Modern factory robots are required to be highly precise and efficient, but there is no guarantee that a set operation sequence is optimal, necessitating regular performance evaluations and improvements. Furthermore, the challenge lies in the fact that it requires advanced expertise and considerable time for on-site personnel to properly evaluate robot operation and identify specific areas for improvement.

[0558] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0559] In this invention, the server includes means for uploading images or videos of the user's own exercise performance or the robot's motion performance; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the technical strengths and weaknesses of the user or robot based on the analysis; and means for generating an optimal training menu or motion sequence for each user or robot based on the identified technical strengths and weaknesses. This makes it possible to objectively and automatically evaluate the robot's motion performance and provide specific methods for improvement.

[0560] A "user" is a person or person who uses the system to evaluate and improve their own exercise performance or the operational performance of the robot they manage.

[0561] "Exercise performance" is a general term for the series of movements and actions performed by a user, including specific techniques, movements, form, and timing.

[0562] A "robot" is an automated mechanical system that performs tasks automatically in a factory based on a sequence of actions.

[0563] "Images or videos" refer to visual data that captures the actions of a user or robot and is in a format that can be processed by a machine.

[0564] A "generative artificial intelligence model" is an algorithm or machine learning model that analyzes images or videos, identifies technical features and shortcomings from their content, and provides appropriate guidance and improvement suggestions.

[0565] A "strength" is a technically superior aspect of a user or robot, a specific element that already demonstrates high performance.

[0566] A "weakness" is a technical aspect that the user or robot needs improvement on, and is a specific element that is currently lacking.

[0567] A "training menu" is a coaching plan that specifically outlines the exercises and practice items that a user should perform to improve their technical skills.

[0568] An "operation sequence" refers to a set of steps or actions that a factory robot should perform.

[0569] "Progress" refers to data that shows how far user training and robot operation have progressed, and how well the plan has been achieved.

[0570] "Feedback" refers to evaluations and comments received regarding the results of training or performance, providing specific directions for the next step.

[0571] "Analysis" refers to the process of using generative artificial intelligence models to analyze image or video data in detail, revealing its technical characteristics and shortcomings.

[0572] A "server" is a central processing unit that processes data sent from users or robots and provides analysis results and training menus.

[0573] A "device" is a device used to temporarily store images or videos taken by a user and upload them to a server; this includes smartphones and tablets.

[0574] This invention is a system that uses video to provide effective improvement menus for users or factory personnel to improve the operational performance of robots. The specific implementation method of the system will be described below.

[0575] Users or factory personnel use devices such as smartphones or tablets to record video of the robot's performance. The recorded videos are saved on the device and compressed before being uploaded to the server via the internet. The device provides users with a preview function for the uploaded content, allowing them to review and edit the videos.

[0576] The server stores the uploaded video in storage and analyzes it using a generative artificial intelligence model. This analysis involves a detailed evaluation of each frame of the video, analyzing the robot's movements, accuracy, timing, and other aspects. Based on these analysis results, the server identifies the robot's technical strengths and weaknesses.

[0577] After identifying strengths and weaknesses, the server uses a generative artificial intelligence model to create a personalized improvement menu, suggesting optimal action sequences and improvement methods. This improvement menu is then sent to the device in PDF or in-app display format.

[0578] Users or factory personnel can check improvement menus via a terminal and adjust robot settings and operations as needed. They can also record the progress of the robot's operation and how performance has changed after configuration changes. The terminal temporarily stores this data in local storage and later uploads it to the server.

[0579] The server analyzes uploaded training data and motion records, and updates the training menu and motion sequences by comparing them with past analysis results. This re-analysis process makes it possible to continuously improve the accuracy and efficiency of the robot's movements.

[0580] As a concrete example, consider a procedure to improve the lifting motion of a robotic arm used in a factory. Suppose the initial analysis reveals that the timing of the lifting motion is incorrect. Based on this result, the server suggests improvements and provides a new training sequence. The user modifies the settings based on these suggestions and sends feedback back to the system. The system re-analyzes the system, finding further areas for improvement, ultimately resulting in a smoother and more efficient lifting motion.

[0581] As described above, this invention provides a system that effectively improves the operational performance of a robot by utilizing a generative artificial intelligence model and user feedback.

[0582] Example of a prompt:

[0583] "Explain how to film the movements of robots in a factory, analyze those movements, and provide suggestions for improvement using a generating AI model."

[0584] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0585] Step 1: The user films the robot's movements.

[0586] The user uses a smartphone or tablet to record the robot's performance as a video. The input includes the robot's specific action sequence. The output is the recorded video file, which is saved on the device.

[0587] Step 2: Compress and preview the recorded video on your device.

[0588] The device compresses the recorded video file and trims it if necessary. The user can also preview the video during this process and keep only the desired portion. The input is the raw, unprocessed video file, and the output is the compressed, final video file.

[0589] Step 3: Upload the video to the server.

[0590] The device uploads a compressed video file to the server. During this process, the device transmits the file using the internet. The input is the compressed video file, and the output is the video file uploaded to the server.

[0591] Step 4: The server analyzes the video.

[0592] The server saves the received video file to storage and begins video analysis using a generative artificial intelligence model. This analysis involves analyzing each frame of the video and evaluating the robot's movements, accuracy, timing, etc. The input is the uploaded video file, and the output is data indicating its technical strengths and weaknesses.

[0593] Step 5: The server generates the improvement menu.

[0594] The server uses a generative artificial intelligence model that proposes optimal action sequences and improvement methods based on the analysis results to create individual improvement menus. The input is data indicating strengths and weaknesses, and the output is improvement menus and specific action sequences.

[0595] Step 6: The server sends the improvement menu to the terminal.

[0596] The server sends the generated improvement menu to the device in PDF format or as an in-app display format. The input consists of improvement menus and action sequences, and the output is data in a format viewable on the device.

[0597] Step 7: The user changes the settings according to the improvement menu.

[0598] The user checks the improvement menu via a terminal and adjusts the robot's settings and operation as needed. The input is the improvement menu, and the output is the actual operation of the robot after the settings have been changed.

[0599] Step 8: Users record their feedback.

[0600] The user records the progress of the robot's movements and how performance has changed after configuration changes. The terminal temporarily stores this data in local storage. The input is the record of the movements after the changes, and the output is the recorded feedback data.

[0601] Step 9: The server re-analyzes the feedback.

[0602] The terminal uploads recorded feedback data to the server, which then analyzes the data. The server updates the training menu and action sequence by comparing it with past analysis results. The input is the recorded feedback data, and the output is the updated improvement menu.

[0603] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0604] Modes for carrying out the invention

[0605] This invention is a system that combines image / video analysis and an emotion engine to improve the user's athletic performance. It analyzes the user's technical strengths and weaknesses, and further considers their emotional state to provide an optimal training menu. This system is realized through the interaction of a server, a terminal, and the user.

[0606] Image and video upload and analysis

[0607] Users upload images or videos of their exercise performance using a smartphone app. This allows users to provide the system with data that enables detailed analysis of their movements.

[0608] The device compresses the image or video selected by the user and sends it to the server. The server stores the received data in storage and begins analysis using a generative artificial intelligence model. The generative AI model analyzes each frame of the video and evaluates the user's actions, form, timing, etc. As a result of the analysis, the user's technical strengths and weaknesses are identified.

[0609] Implementation of an emotion engine

[0610] The server uses an emotion engine to analyze user facial expression data in addition to image or video data. The emotion engine analyzes the user's facial expressions and voice to extract how the user feels about the training. This identifies the emotional state (e.g., motivation, fatigue, stress).

[0611] Creating a training menu

[0612] The server generates an optimal training menu based on technical analysis results and emotional state analysis results from the emotion engine. In addition to specific exercise items, adjustments are made considering the user's emotional state, and the training menu is customized individually. For example, if the server determines that the user is fatigued, a lighter training program will be suggested.

[0613] The generated training menu is converted into a user-friendly format (e.g., PDF, in-app display) and sent to the device. The device provides an interface for the user to review the training menu and perform the training accordingly.

[0614] Training implementation and recording

[0615] Users perform training according to the training menu and record their progress and feedback on their device. The device temporarily saves the recorded data to local storage and uploads it to the server when connected to the internet.

[0616] Feedback and menu updates

[0617] The server receives the training record data uploaded by the user and performs a re-analysis, comparing it to previous analysis results. This analysis includes technical progress and changes in emotional state. Based on the progress evaluation, the training menu and feedback are updated as needed.

[0618] For example, if a user's skills improve and their emotional state changes for the better, the training menu is updated to include new challenges to help them move on to the next step. The device notifies the user of updates and feedback, helping them prepare for the next training session.

[0619] Specific example

[0620] User A wants to improve their soccer dribbling skills. They record videos of themselves dribbling with their smartphone and upload them to the system. The server analyzes the videos using a generative artificial intelligence model and an emotion engine, identifying that they lack kicking power with their right foot and that they experience stress during practice.

[0621] Next, the server generates a training menu based on these results, combining "specific dribbling exercises to strengthen the kicking power of the right foot" and "relaxation exercises to reduce stress." User A practices according to this menu and records their progress on their device. The server analyzes this record, confirms that the kicking power of the right foot has improved and stress has decreased, and adjusts and provides the next menu.

[0622] The following describes the processing flow.

[0623] Specific steps of the program's processing

[0624] Image and video upload and analysis

[0625] Step 1:

[0626] The user launches the smartphone app and opens the image or video upload screen. The user selects an image or video of their exercise performance.

[0627] Step 2:

[0628] The device compresses the selected image or video and prepares it for upload. At this stage, the user can preview the data and, if there are no problems, begin the upload.

[0629] Step 3:

[0630] The device sends compressed images or videos to the server over the internet. Once the transmission is complete, the user receives a notification.

[0631] Step 4:

[0632] The server saves the images or videos received from the terminal to its storage and passes them to a generative artificial intelligence model to begin analysis.

[0633] Step 5:

[0634] The server uses a generative artificial intelligence model to analyze each frame of images and videos, evaluating the user's actions, form, timing, etc. Based on this evaluation, it identifies the user's technical strengths and weaknesses.

[0635] Step 6:

[0636] The server saves the analysis results to a database, which will serve as the basis for creating the next training menu.

[0637] Implementation of an emotion engine

[0638] Step 7:

[0639] The server activates an emotion engine that analyzes the user's facial expressions and voice, in addition to image or video data, to recognize the user's emotional state.

[0640] Step 8:

[0641] The server stores the user's emotional state (e.g., motivation, fatigue, stress) in a database based on the recognition results of the emotion engine.

[0642] Creating a training menu

[0643] Step 9:

[0644] The server integrates technical analysis results with the emotion engine analysis results to generate an optimal training menu for each individual user. This menu includes not only technical items but also adjustments based on emotional state.

[0645] Step 10:

[0646] The server converts the generated training menu into a user-friendly format (e.g., PDF, in-app display).

[0647] Step 11:

[0648] The server sends the generated training menu to the terminal.

[0649] Training implementation and recording

[0650] Step 12:

[0651] The device displays the received training menu to the user and prompts them to perform the training according to the instructions.

[0652] Step 13:

[0653] Users perform training according to the training menu and record their progress and feedback on their device.

[0654] Step 14:

[0655] The device temporarily stores recorded data in local storage and synchronizes it with the server when connected to the internet.

[0656] Feedback and menu updates

[0657] Step 15:

[0658] The server receives the training record data uploaded by the user and performs a new analysis by comparing it with the results of previous analyses.

[0659] Step 16:

[0660] The server performs progress assessments and updates the training menu as needed. This assessment includes, for example, the degree of technical improvement, newly discovered issues, and the user's emotional state.

[0661] Step 17:

[0662] The server will send the updated training menu and any new feedback back to the terminal. This information will be reflected in the next training session.

[0663] Step 18:

[0664] The device notifies the user of updates and feedback, helping them prepare for the next training session.

[0665] (Example 2)

[0666] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0667] Traditional athletic performance enhancement systems focused on technical analysis and failed to consider the user's emotional state. Therefore, they couldn't provide appropriate training programs when users were stressed or fatigued, making it difficult to achieve sustainable training results. Furthermore, the training programs provided to users were uniform, failing to adequately reflect individual technical strengths and weaknesses.

[0668] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0669] In this invention, the server includes means for uploading images or videos of the user's exercise performance; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based on the analysis; means for using an emotion engine to analyze the user's emotional state in addition to the identified technical strengths and weaknesses; means for generating an optimal training menu for each individual user based on the analyzed technical strengths and weaknesses and emotional state; means for converting the generated training menu into a user-friendly format and providing it to the user; means for recording the user's training progress and feedback on the terminal; and means for uploading the recorded data to the server and analyzing it to update the training menu according to the user's progress. This makes it possible to accurately identify the user's technical strengths and weaknesses, provide an individualized training menu based on them, and realize an appropriate training environment that takes into account the user's emotional state.

[0670] A "user" refers to someone who uses the system to film their own athletic performance and receive analysis and training menus.

[0671] A "terminal" refers to a device used by a user (e.g., a smartphone, tablet, or personal computer) that has the function of compressing captured images and videos and sending them to a server.

[0672] A "server" refers to a computer system that has the function of receiving, storing, and analyzing images and videos sent by users, and generating and providing training menus.

[0673] "Means of uploading images or videos" refers to the function that users use to film their own exercise performance and send the data into the system.

[0674] A "generative artificial intelligence model" refers to artificial intelligence technology used to analyze uploaded images and videos to identify the user's technical strengths and weaknesses.

[0675] An "emotion engine" refers to technology that analyzes a user's facial expressions and actions to identify their emotional state.

[0676] "Means for identifying technical strengths and weaknesses" refers to a function that uses generative artificial intelligence models to evaluate a user's technical capabilities from the results of image and video analysis, and to identify specific strengths and weaknesses.

[0677] "Means for generating training menus" refers to a function that creates optimal training content for each individual user based on analyzed technical strengths and weaknesses, as well as their emotional state.

[0678] "Means of providing to the user" refers to functions that provide the generated training menu to the user in an easy-to-understand format.

[0679] "Means for recording training progress and feedback" refers to a function that allows users to record their results and impressions within the system after completing training.

[0680] "A means of updating the training menu according to the user's progress through analysis" refers to a function that analyzes recorded data and updates the training menu as needed, taking into account the user's technical progress and emotional state.

[0681] Modes for carrying out the invention

[0682] This invention is a system that combines image / video analysis and an emotion engine to improve the user's athletic performance. It analyzes the user's technical strengths and weaknesses, and further considers their emotional state to provide an optimal training menu. This system is realized through the interaction of a server, a terminal, and the user.

[0683] Image and video upload and analysis

[0684] Users upload images or videos of their exercise performance using a smartphone app. This allows users to provide the system with data that enables detailed analysis of their movements.

[0685] The terminal compresses images and videos selected by the user and sends them to the server. The server stores the received data in storage and begins analysis using a generative artificial intelligence model. The generative AI model analyzes each frame of the video and evaluates the user's actions, form, timing, etc. As a result of the analysis, the user's technical strengths and weaknesses are identified.

[0686] Implementation of an emotion engine

[0687] The server uses an emotion engine to analyze user facial expression data in addition to image or video data. The emotion engine analyzes the user's facial expressions and voice to extract how the user feels about the training. This identifies the emotional state (e.g., motivation, fatigue, stress).

[0688] Creating a training menu

[0689] The server generates an optimal training menu based on technical analysis results and emotional state analysis results from the emotion engine. In addition to specific exercise items, adjustments are made considering the user's emotional state, and the training menu is customized individually. For example, if the server determines that the user is fatigued, a lighter training program will be suggested.

[0690] The generated training menu is converted into a user-friendly format (e.g., PDF, in-app display) and sent to the device. The device provides an interface for the user to review the training menu and perform the training accordingly.

[0691] Training implementation and recording

[0692] Users perform training according to the training menu and record their progress and feedback on their device. The device temporarily saves the recorded data to local storage and uploads it to the server when connected to the internet.

[0693] Feedback and menu updates

[0694] The server receives the training record data uploaded by the user and performs a re-analysis, comparing it to previous analysis results. This analysis includes technical progress and changes in emotional state. Based on the progress evaluation, the training menu and feedback are updated as needed.

[0695] For example, if a user's skills improve and their emotional state changes for the better, the training menu is updated to include new challenges to help them move on to the next step. The device notifies the user of updates and feedback, helping them prepare for the next training session.

[0696] Specific example

[0697] User A wants to improve their soccer dribbling skills. They record videos of themselves dribbling with their smartphone and upload them to the system. The server analyzes the videos using a generative artificial intelligence model and an emotion engine, identifying that they lack kicking power with their right foot and that they experience stress during practice.

[0698] Next, the server generates a training menu based on these results, combining "specific dribbling exercises to strengthen the kicking power of the right foot" and "relaxation exercises to reduce stress." User A practices according to this menu and records their progress on their device. The server analyzes this record, confirms that the kicking power of the right foot has improved and stress has decreased, and adjusts and provides the next menu.

[0699] Examples of input prompts for a generative AI model

[0700] To improve users' athletic performance, analyze dribbling videos they film with their smartphones to identify their technical strengths and weaknesses. Also, analyze the user's facial expressions in the videos to identify their emotional state (e.g., motivation, fatigue, stress). Based on this data, generate an optimal training program that considers both technical and emotional aspects.

[0701] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0702] Step 1:

[0703] This stage involves users taking images or videos of their exercise performance and uploading them through a smartphone app.

[0704] Input: Images or videos of athletic performance captured by the user.

[0705] Operation: The user uses a smartphone app to select a captured image or video and presses the upload button.

[0706] Output: Image or video data uploaded to the device.

[0707] Step 2:

[0708] This is the stage where the device compresses the image or video uploaded by the user and sends it to the server.

[0709] Input: Image or video data uploaded by the user on their device.

[0710] Operation: The device compresses image or video data and sends it to the server over the internet.

[0711] Output: Compressed image or video data is sent to the server.

[0712] Step 3:

[0713] This stage involves the server saving the received image or video data to storage and starting the analysis using a generative artificial intelligence model.

[0714] Input: Compressed image or video data.

[0715] Operation: The server receives data, saves it to storage, and then inputs that data into a generative artificial intelligence model to begin analysis. Each frame is analyzed, and the user's actions, form, timing, etc., are evaluated.

[0716] Output: Analysis results data showing the user's technical strengths and weaknesses.

[0717] Step 4:

[0718] This is the stage where the server uses an emotion engine to analyze the user's facial expression data from image or video data.

[0719] Input: Image or video data, and analysis result data obtained in the previous step.

[0720] Operation: The server uses an emotion engine to analyze the user's facial expressions in an image or video and extract the user's emotional state (e.g., motivation, fatigue, stress).

[0721] Output: Data indicating the user's emotional state.

[0722] Step 5:

[0723] This is the stage where the server generates an optimal training menu for each individual user based on technical analysis results and emotional state analysis results.

[0724] Input: Technical analysis results data, data indicating emotional state.

[0725] Operation: The server comprehensively analyzes this data and generates the optimal training menu. For example, if the user is fatigued, a lighter training program will be suggested.

[0726] Output: The generated training menu.

[0727] Step 6:

[0728] This stage involves the server converting the generated training menu into a user-friendly format (e.g., PDF, in-app display) and sending it to the device.

[0729] Input: Generated training menu.

[0730] Operation: The server converts the training menu into a user-friendly format and sends it to the terminal via the internet.

[0731] Output: The training menu, converted into a user-friendly format, is sent to the terminal.

[0732] Step 7:

[0733] This stage involves the user completing the training according to the training menu and recording their progress and feedback on their device.

[0734] Input: Training menu.

[0735] Operation: Users perform training according to the provided training menu, and their progress and feedback are recorded on their device via the app.

[0736] Output: Training progress and feedback data recorded on the device.

[0737] Step 8:

[0738] The terminal uploads the recorded training progress and feedback data to the server, where the server performs re-analysis.

[0739] Input: User training progress and feedback data.

[0740] Operation: The device sends recorded data to a server via the internet, and the server re-analyzes the received data. It evaluates technical progress and changes in emotional state, and updates the training menu as needed.

[0741] Output: An updated training menu and feedback are generated.

[0742] (Application Example 2)

[0743] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0744] In modern factories and production lines, robots handle many tasks, but optimizing their movements and performance remains challenging. In particular, identifying a robot's technical strengths and weaknesses and providing appropriate training programs based on that understanding is difficult. Furthermore, it's necessary to efficiently collect and analyze data on robot movements and improve them in real time. This is crucial for improving factory production efficiency.

[0745] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for uploading images or videos of the user's exercise performance; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based thereon; means for generating an optimal training menu for each user based on the identified technical strengths and weaknesses; means for providing the training menu to the user; means for recording the user's training progress and feedback; means for analyzing the recorded data and updating the training menu according to the user's progress; means for uploading images or videos to monitor the robot's movements; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the robot's technical strengths and weaknesses based thereon; and means for generating an optimal training menu for each robot based on the robot's technical strengths and weaknesses. This makes it possible to optimize the robot's movements and performance, thereby improving the production efficiency of the factory.

[0746] A "user" is an individual or organization that uses the system to improve their athletic performance.

[0747] "Exercise performance" refers to the user's physical movements and athletic abilities.

[0748] "Images or videos" refer to visual data that records the user's exercise performance.

[0749] "Uploading" refers to the act of sending images or videos that a user has taken to a system.

[0750] A "generative artificial intelligence model" is a machine learning model that analyzes uploaded images and videos to identify the user's technical strengths and weaknesses.

[0751] "Analysis" refers to the process of analyzing information from images and videos using generative artificial intelligence models.

[0752] "Technical strengths" refer to specific movements or skills in sports that a user excels at.

[0753] A "weakness" refers to an area where a user needs improvement, or a task or skill they struggle with.

[0754] A "training menu" is a personalized exercise plan generated based on the user's technical strengths and weaknesses.

[0755] "Recording" refers to the act of photographing or inputting the user's training progress and feedback.

[0756] A "robot" is an automated machine that performs tasks on production lines in factories and other facilities.

[0757] "Action" refers to the physical movements or tasks that a robot performs.

[0758] "Optimization" is the act of adjusting something to maximize the performance of a user or robot.

[0759] "Maintenance" refers to maintenance work performed to maintain the robot's operation and improve its performance.

[0760] "Factory production efficiency" refers to the efficiency of production activities in a factory, meaning an improvement in both the quality and speed of work.

[0761] A "server" is a computer system used to store and analyze uploaded data.

[0762] A "terminal" is a device used by users to input data or check training menus.

[0763] "Real-time" refers to a time frame in which data is processed immediately and results are provided instantly.

[0764] System Overview

[0765] This invention is a system designed to improve user motor performance and robot operation performance. The system is primarily realized through the interaction of a server, a terminal, and the user.

[0766] Image and video upload and analysis

[0767] Users or factory managers use smartphones or cameras to capture images and videos of exercise performance or robot movements, and upload them to the system via a terminal. The terminal compresses the selected images and videos from the user and efficiently sends them to the server. The server stores the received data in storage and begins analysis using a generative artificial intelligence model. This model analyzes each frame of the video, evaluating specific movements, forms, and timings of the user or robot, and identifies technical strengths and weaknesses.

[0768] Implementation of an emotion engine

[0769] In addition to analyzing image or video data, the server uses an emotion engine to analyze facial expression data. This emotion engine analyzes the user's facial expressions and the robot's movement parameters to extract emotional states (motivation, fatigue, stress, etc.) related to training and work.

[0770] Creating a training menu

[0771] The server generates an optimal training menu based on technical analysis results and analysis results from the emotion engine. This menu is customized to the individual user and robot. For example, if the user is determined to be fatigued, a lighter training program will be suggested. Similarly, if the robot's operation is under high load, maintenance protocols to reduce the load will be provided.

[0772] The generated training menu is sent to the terminal in a user-friendly format. The terminal provides an interface for users and factory managers to review the training menu and work through it.

[0773] Training implementation and recording

[0774] Users perform exercises according to the training menu and record their progress and feedback on a terminal. In the case of robots, factory managers monitor the robot's performance and similarly record it via a terminal. The recorded data is temporarily stored in local storage and uploaded to a server when an internet connection is available.

[0775] Feedback and menu updates

[0776] The server receives the uploaded training and movement recording data and performs a re-analysis, comparing it to previous analysis results. This analysis includes technical progress and changes in emotional state. Based on the progress evaluation, the training menu and feedback are updated as needed and notified to the user and the robot. This improves subsequent training sessions and work performance.

[0777] Examples of specific cases and prompt statements

[0778] If user A wants to improve their soccer dribbling skills, they record a video of themselves dribbling with their smartphone and upload it to the system. The server uses a generative artificial intelligence model and an emotion engine to analyze the video and identify that user A lacks kicking power with their right foot and experiences stress during practice. The server then generates a training menu that combines "specific dribbling exercises to strengthen right-foot kicking power" and "relaxation exercises to reduce stress." User A practices based on this menu and records their progress on their device.

[0779] Examples of prompt statements are as follows:

[0780] "The robot is under heavy load and is experiencing continuous malfunctions. Use the emotion engine to assess the robot's current condition and propose appropriate maintenance protocols."

[0781] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0782] Step 1:

[0783] Users capture images or videos of their exercise performance using their smartphones and upload them to the system via the device. The input from the user is the captured image or video, and the output is compressed data transmitted via the device. Specifically, users use a camera app on their smartphone, select the captured data, and press the upload button.

[0784] Step 2:

[0785] The terminal compresses image or video data provided by the user and sends it to the server. The input is video or image data uploaded by the user, and the output is a compressed data file. This compression is performed to improve the efficiency of data transfer. Typically, a compression algorithm is used.

[0786] Step 3:

[0787] The server saves the received data to storage. The input is a compressed data file, and the output is the data saved to storage. Error checking is also performed to confirm that the data was saved successfully.

[0788] Step 4:

[0789] The server analyzes incoming data using a generative artificial intelligence model. The input is stored image or video data, and the output is the user's technical strengths and weaknesses. As part of the data processing, image processing algorithms are used to analyze each frame of the video, evaluating movement, form, and timing. This identifies the strengths and weaknesses of athletic performance.

[0790] Step 5:

[0791] The server uses an emotion engine to analyze facial expression data and motion parameters. Inputs are facial expression data from videos and robot motion parameters, while outputs are emotional states (e.g., motivation, fatigue, stress). This analysis uses facial recognition and speech analysis technologies to identify the emotional state of the user or robot.

[0792] Step 6:

[0793] The server combines technical analysis results with the emotional engine's analysis results to generate an optimal training menu. The inputs are technical strengths and weaknesses, as well as emotional states, while the output is a customized training menu. Specifically, adjustments are made, such as reducing the training content based on the user's fatigue level.

[0794] Step 7:

[0795] The server sends the generated training menu to the terminal and provides it to the user. The input is the generated training menu, and the output is a display in a user-friendly format (PDF, in-app display). This allows the user to perform training based on the new menu.

[0796] Step 8:

[0797] The user performs exercises according to the provided training menu and records their progress and feedback on the device. Input is the training progress and feedback, while output is the data stored on the device. An interface is used to record actions and emotional feedback during training.

[0798] Step 9:

[0799] The terminal uploads recorded training data to the server when connected to the internet. Input is progress data and feedback stored on the terminal, while output is data transmission to the server. The transmitted data is stored in the server's storage.

[0800] Step 10:

[0801] The server re-analyzes the uploaded data and evaluates the user's technical progress and changes in emotional state. The input is the newly uploaded training data, and the output is the updated training menu and feedback. This allows the next training session to be adjusted based on the user's progress.

[0802] Examples of prompt statements are as follows:

[0803] "The robot is under heavy load and is experiencing continuous malfunctions. Use the emotion engine to assess the robot's current condition and propose appropriate maintenance protocols."

[0804] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0805] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0806] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0807] [Third Embodiment]

[0808] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0809] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0810] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0811] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0812] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0813] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0814] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0815] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0816] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0817] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0818] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0819] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0820] Modes for carrying out the invention

[0821] This invention is a system that provides users with effective training menus using captured images or videos to improve their athletic performance. This system functions through the interaction of a server, a terminal, and the user.

[0822] Image and video upload and analysis

[0823] Users select and upload images or videos of their exercise performance using a smartphone app. These images or videos must capture specific movements or forms of the user in detail.

[0824] The device has the ability to compress selected images or videos and send them to a server over the internet. The device also includes a preview function for uploaded images and videos, allowing users to check the data before sending.

[0825] The server saves images or videos received from the terminal to its storage and begins analysis using a generative artificial intelligence model. The generative AI model analyzes each frame of the image or video, evaluating the user's actions, form, timing, etc. This identifies the user's technical strengths and weaknesses.

[0826] Creating a training menu

[0827] The server generates an optimal training menu based on the analysis results, the user's past training data, and pre-set goals. This training menu is customized according to the user's characteristics and challenges, and includes specific exercises and training items.

[0828] The generated training menu is converted into a user-friendly format (e.g., PDF or in-app display) and sent to the device. The device then provides an interface for the user to view the training menu and perform the training accordingly.

[0829] Training implementation and recording

[0830] The user performs exercises according to the training menu and records their progress and feedback on the device. The device temporarily saves the recorded data to local storage and uploads it to the server once an internet connection is established.

[0831] Feedback and menu updates

[0832] The server receives the training record data uploaded by the user and performs a new analysis by comparing it with previous analysis results. Based on these analysis results, it evaluates the user's progress and decides whether to update the training menu.

[0833] For example, if the analysis results confirm that "right foot kicking power has improved," the new training menu will be modified to include "improving left foot technique" as a key focus. This allows users to continuously receive expert guidance and efficiently improve their skills.

[0834] The following describes the processing flow.

[0835] Specific steps of the program's processing

[0836] Image and video upload and analysis

[0837] Step 1:

[0838] The user launches the smartphone app and opens the image or video upload screen. The user selects an image or video of their exercise performance.

[0839] Step 2:

[0840] The device compresses the selected image or video and prepares it for upload. At this stage, the user can preview the data and, if there are no problems, begin the upload.

[0841] Step 3:

[0842] The device sends compressed images or videos to the server over the internet. Once the transmission is complete, the user receives a notification.

[0843] Step 4:

[0844] The server saves the images or videos received from the terminal to its storage and passes them to a generative artificial intelligence model to begin analysis.

[0845] Step 5:

[0846] The server uses a generative artificial intelligence model to analyze each frame of images and videos, evaluating the user's actions, form, timing, etc. Based on this evaluation, it identifies the user's technical strengths and weaknesses.

[0847] Step 6:

[0848] The server saves the analysis results to a database, which will serve as the basis for creating the next training menu.

[0849] Creating a training menu

[0850] Step 7:

[0851] The server executes an algorithm based on the analysis results, the user's past training data, and their goals to generate the optimal training menu for the user.

[0852] Step 8:

[0853] The server converts the generated training menu into a user-friendly format (e.g., PDF, in-app display).

[0854] Step 9:

[0855] The server sends the generated training menu to the terminal.

[0856] Training implementation and recording

[0857] Step 10:

[0858] The device displays the received training menu to the user and prompts them to perform the training according to the instructions.

[0859] Step 11:

[0860] Users perform training according to the training menu and record their progress and feedback on their device.

[0861] Step 12:

[0862] The device temporarily stores recorded data in local storage and synchronizes it with the server when connected to the internet.

[0863] Feedback and menu updates

[0864] Step 13:

[0865] The server receives the training record data uploaded by the user and performs a new analysis by comparing it with the results of previous analyses.

[0866] Step 14:

[0867] The server performs progress assessments and updates the training menu as needed. This assessment includes, for example, the degree of technical improvement and any newly discovered issues.

[0868] Step 15:

[0869] The server will send the updated training menu and any new feedback back to the terminal. This information will be reflected in the next training session.

[0870] Step 16:

[0871] The device notifies the user of updates and feedback, helping them prepare for the next training session.

[0872] (Example 1)

[0873] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0874] Modern sports training demands accurate feedback and customized training programs to effectively improve individual athletic performance. However, traditional methods, such as users showing individually filmed videos to experts for advice, were often inefficient due to their reliance on human intervention. Furthermore, there was a lack of systems to accurately record and evaluate training progress, provide appropriate feedback, and update training programs. This resulted in the problem of training effectiveness not being maximized.

[0875] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0876] In this invention, the server includes means for analyzing uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based on the analysis; means for generating an optimal training menu for each user based on the identified technical strengths and weaknesses; means for recording the user's training progress and feedback; means for analyzing the recorded data and updating the training menu according to the user's progress; means for compressing images and videos captured by the terminal and sending them to the server; means for the server to store the received images and videos; means for the server to refer to the user's past training data and generate an optimal training menu in conjunction with the analysis results; means for the server to convert the training menu into a format that is easy for the user to understand and send it to the terminal; means for the terminal to temporarily save training data in local storage and upload it to the server; and means for the server to analyze newly received training data and update the training menu. This enables the provision of highly accurate feedback based on the user's individual data and customized training menus.

[0877] "Means for uploading images or videos" refers to a function that allows users to send images or videos of their exercise performance from their device to the server.

[0878] A "generative artificial intelligence model" is an artificial intelligence algorithm used to analyze image or video data to identify a user's technical strengths and weaknesses.

[0879] "Means for identifying a user's technical strengths and weaknesses" refers to a function that uses a generative artificial intelligence model to evaluate the user's actions, forms, and timing, and based on the results, clearly identifies the user's strengths and areas that need improvement.

[0880] "A means of generating an optimal training menu" refers to a function that takes into account the user's technical strengths and weaknesses and creates an individually optimized training plan.

[0881] "Means of providing training menus" refers to a function that displays or sends the generated training menus to the user in an easy-to-understand format.

[0882] "Means for recording training progress and feedback" refers to a function that allows users to input and save the training content, progress, and feedback they have received on their device.

[0883] "Means for analyzing recorded data and updating training menus" refers to a function that re-evaluates the content of the training menu based on the user's progress and changes it to the latest training menu as needed.

[0884] "Compression" refers to a function that performs encoding processing to reduce the file size of captured images and videos.

[0885] "Means for saving received images and videos" refers to the function that allows a server to store image and video data sent from a terminal in its storage.

[0886] "Means of referencing the user's past training data" refers to a function where the server retrieves and uses the user's previous training history and performance data from a database.

[0887] "Means of converting training menus into a user-friendly format" refers to a function that processes the generated training menus into a format that users can easily view, such as PDF or in-app display.

[0888] "Method for temporarily saving to local storage" refers to a function that allows training data to be temporarily saved even when the device does not have an internet connection.

[0889] "Method for uploading to the server" refers to the function of sending data temporarily stored on the device to the server once an internet connection is established.

[0890] "Means for analyzing newly received training data" refers to a function in which the server analyzes newly submitted training data from the user again using a generative artificial intelligence model to evaluate the user's progress and performance.

[0891] Modes for carrying out the invention

[0892] This invention is a system that provides users with effective training menus using captured images or videos to improve their athletic performance. This system functions through the interaction of a server, a terminal, and the user.

[0893] Image and video upload and analysis

[0894] Users launch a smartphone app and select and upload images or videos of their exercise performance. These images or videos capture specific movements or forms of the user in detail.

[0895] The device compresses selected images and videos and sends them to the server over the internet. Specifically, the device compresses the video file selected by the user after shooting (e.g., a soccer dribbling scene) using an encoding algorithm (e.g., H.264) and sends it to the server's communication endpoint. The device also has a preview function for uploaded images and videos, allowing the user to check the data before sending it.

[0896] The server saves images or videos received from the terminal to cloud storage (e.g., AWS S3) and begins analysis using a generative artificial intelligence model (e.g., YOLOv5). This analysis scans each frame of the video to evaluate the user's actions, form, and timing. The analysis results are saved as JSON data, and the user's technical strengths and weaknesses are identified.

[0897] Creating a training menu

[0898] The server generates an optimal training menu for each user based on the analysis results, the user's past training data, and pre-set goals. For example, if the server determines that the user's dribbling speed is slow, training items aimed at improving this (e.g., repetitive dribbling practice) will be added to the menu. The generated training menu is converted into PDF format or an in-app display format and sent to the device.

[0899] Training menu provided

[0900] The device displays the received training menu within the app, making it easy for the user to review. For example, a new training plan will appear in the "Training Menu" tab, and the user can then follow it to complete their workout.

[0901] Training implementation and recording

[0902] Users perform exercises according to the provided training menu and record their progress and feedback on their device. The device temporarily saves the recorded data to local storage (e.g., an SQLite database). Once an internet connection is established, the data is uploaded to the server.

[0903] Feedback and menu updates

[0904] The server analyzes the newly received training record data and compares it to previous analysis results. For example, if it confirms an improvement in the kicking power of the right foot, it uses a generative artificial intelligence model to identify areas that need further improvement (e.g., improving the technique of the left foot) and updates the training menu. The updated menu is sent back to the terminal, and the user can continue training by following the instructions.

[0905] Examples of specific cases and prompt statements

[0906] For example, if a user is practicing soccer, the following sequence of events might occur.

[0907] 1. The user films themselves practicing soccer dribbling with their smartphone.

[0908] 2. The device compresses the recorded video and sends it to the server. The user previews the video before sending it.

[0909] 3. The server analyzes the received video using an AI model to evaluate the dribbling technique.

[0910] 4. The server generates a training menu for improving dribbling skills based on the analysis results and past data.

[0911] 5. The server converts the generated menu into PDF format and sends it to the terminal. The terminal displays the menu.

[0912] 6. The user practices according to the provided menu and records their progress on the device.

[0913] 7. The server analyzes the new training records and updates the training menu as needed.

[0914] The following are examples of prompts for a generative AI model.

[0915] Analyze the user's video and output the analysis results, focusing on the following key points:

[0916] 1. Accuracy of form

[0917] 2. Smoothness of operation

[0918] 3. Timing

[0919] If you encounter any problems with any of the following points, please provide details.

[0920] This allows the system to effectively analyze the user's exercise performance and provide an optimal training program.

[0921] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0922] Understood. Now, I will explain the processing flow of this system's program in detail, divided into the following steps.

[0923] Step 1:

[0924] The user launches a smartphone app and records their athletic performance. For example, if the user is practicing soccer, they will record their dribbling and shooting movements as videos. The input is the video recorded by the smartphone's camera. The output is the athletic performance video file saved on the smartphone.

[0925] Step 2:

[0926] The device compresses the video recorded by the user and sends it to the server. Specifically, the device compresses the video file using an encoding algorithm (e.g., H.264) and sends it to the server over the internet. The input is the video file obtained in step 1, and the output is the compressed video file sent to the server.

[0927] Step 3:

[0928] The server saves the video received from the terminal to cloud storage (e.g., AWS S3). The input is the compressed video file that was sent. The output is the video file saved in cloud storage. The server also passes the received video to the AI ​​analysis process.

[0929] Step 4:

[0930] The server analyzes the video using a generative artificial intelligence model (e.g., YOLOv5). Specifically, it scans each frame of the video and evaluates the user's actions, form, timing, etc. The input is a video file stored in cloud storage, and the output is JSON data as the analysis result. This analysis identifies the user's technical strengths and weaknesses.

[0931] Step 5:

[0932] The server generates an optimal training menu for each user based on the analysis results, the user's past training data, and pre-set goals. The input is the analysis results and past training data, and the output is the generated training menu. For example, it might generate a menu such as "Repetitive practice to improve dribbling speed."

[0933] Step 6:

[0934] The server converts the generated training menu into a user-friendly format (e.g., PDF or in-app display format) and sends it to the device. The input is the generated training menu, and the output is the training menu sent to the device.

[0935] Step 7:

[0936] The device displays the received training menu within the app, allowing the user to review it. For example, a new training plan will appear in the app's "Training Menu" tab. The input is the training menu sent from the server, and the output is the screen display that the user can review.

[0937] Step 8:

[0938] The user performs exercises according to the provided training menu and records their progress and feedback on the device. The acquired data includes information about the content and achievement level of the training. The input is the training progress and feedback entered by the user, and the output is the data recorded on the device.

[0939] Step 9:

[0940] The terminal temporarily saves recorded data to local storage (e.g., an SQLite database) and uploads it to the server once an internet connection is established. The input is the training progress and feedback data recorded on the terminal, and the output is the data uploaded to the server.

[0941] Step 10:

[0942] The server analyzes the newly received training record data and compares it to previous analysis results. The input is the newly uploaded training record data, and the output is the updated analysis results. For example, if it is confirmed that the kicking power of the right foot has improved, a training item to strengthen the technique of the left foot will be added.

[0943] Step 11:

[0944] The server regenerates the optimal training menu based on the user's progress and sends it to the user's terminal. The input is the updated analysis results and user feedback data, and the output is the regenerated training menu. The process is then repeated from step 6.

[0945] Through the steps described above, this invention can effectively improve the user's athletic performance.

[0946] (Application Example 1)

[0947] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0948] Robots used in modern factories are required to have high precision and efficiency, but there is no guarantee that a set operation sequence is optimal, and periodic performance evaluations and improvements are necessary. Furthermore, it is a challenge that on-site personnel need advanced expertise to properly evaluate robot operation and identify specific areas for improvement, making it a very time-consuming process.

[0949] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0950] In this invention, the server includes means for uploading images or videos of the user's own exercise performance or the robot's motion performance; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the technical strengths and weaknesses of the user or robot based on the analysis; and means for generating an optimal training menu or motion sequence for each user or robot based on the identified technical strengths and weaknesses. This makes it possible to objectively and automatically evaluate the robot's motion performance and provide specific methods for improvement.

[0951] A "user" is a person or person who uses the system to evaluate and improve their own exercise performance or the operational performance of the robot they manage.

[0952] "Exercise performance" is a general term for the series of movements and actions performed by a user, including specific techniques, movements, form, and timing.

[0953] A "robot" is an automated mechanical system that performs tasks automatically in a factory based on a sequence of actions.

[0954] "Images or videos" refer to visual data that captures the actions of a user or robot and is in a format that can be processed by a machine.

[0955] A "generative artificial intelligence model" is an algorithm or machine learning model that analyzes images or videos, identifies technical features and shortcomings from their content, and provides appropriate guidance and improvement suggestions.

[0956] A "strength" is a technically superior aspect of a user or robot, a specific element that already demonstrates high performance.

[0957] A "weakness" is a technical aspect that the user or robot needs improvement on, and is a specific element that is currently lacking.

[0958] A "training menu" is a coaching plan that specifically outlines the exercises and practice items that a user should perform to improve their technical skills.

[0959] An "operation sequence" refers to a set of steps or actions that a factory robot should perform.

[0960] "Progress" refers to data that shows how far user training and robot operation have progressed, and how well the plan has been achieved.

[0961] "Feedback" refers to evaluations and comments received regarding the results of training or performance, providing specific directions for the next step.

[0962] "Analysis" refers to the process of using generative artificial intelligence models to analyze image or video data in detail, revealing its technical characteristics and shortcomings.

[0963] A "server" is a central processing unit that processes data sent from users or robots and provides analysis results and training menus.

[0964] A "device" is a device used to temporarily store images or videos taken by a user and upload them to a server; this includes smartphones and tablets.

[0965] This invention is a system that uses video to provide effective improvement menus for users or factory personnel to improve the operational performance of robots. The specific implementation method of the system will be described below.

[0966] Users or factory personnel use devices such as smartphones or tablets to record video of the robot's performance. The recorded videos are saved on the device and compressed before being uploaded to the server via the internet. The device provides users with a preview function for the uploaded content, allowing them to review and edit the videos.

[0967] The server stores the uploaded video in storage and analyzes it using a generative artificial intelligence model. This analysis involves a detailed evaluation of each frame of the video, analyzing the robot's movements, accuracy, timing, and other aspects. Based on these analysis results, the server identifies the robot's technical strengths and weaknesses.

[0968] After identifying strengths and weaknesses, the server uses a generative artificial intelligence model to create a personalized improvement menu, suggesting optimal action sequences and improvement methods. This improvement menu is then sent to the device in PDF or in-app display format.

[0969] Users or factory personnel can check improvement menus via a terminal and adjust robot settings and operations as needed. They can also record the progress of the robot's operation and how performance has changed after configuration changes. The terminal temporarily stores this data in local storage and later uploads it to the server.

[0970] The server analyzes uploaded training data and motion records, and updates the training menu and motion sequences by comparing them with past analysis results. This re-analysis process makes it possible to continuously improve the accuracy and efficiency of the robot's movements.

[0971] As a concrete example, consider a procedure to improve the lifting motion of a robotic arm used in a factory. Suppose the initial analysis reveals that the timing of the lifting motion is incorrect. Based on this result, the server suggests improvements and provides a new training sequence. The user modifies the settings based on these suggestions and sends feedback back to the system. The system re-analyzes the system, finding further areas for improvement, ultimately resulting in a smoother and more efficient lifting motion.

[0972] As described above, this invention provides a system that effectively improves the operational performance of a robot by utilizing a generative artificial intelligence model and user feedback.

[0973] Example of a prompt:

[0974] "Explain how to film the movements of robots in a factory, analyze those movements, and provide suggestions for improvement using a generating AI model."

[0975] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0976] Step 1: The user films the robot's movements.

[0977] The user uses a smartphone or tablet to record the robot's performance as a video. The input includes the robot's specific action sequence. The output is the recorded video file, which is saved on the device.

[0978] Step 2: Compress and preview the recorded video on your device.

[0979] The device compresses the recorded video file and trims it if necessary. The user can also preview the video during this process and keep only the desired portion. The input is the raw, unprocessed video file, and the output is the compressed, final video file.

[0980] Step 3: Upload the video to the server.

[0981] The device uploads a compressed video file to the server. During this process, the device transmits the file using the internet. The input is the compressed video file, and the output is the video file uploaded to the server.

[0982] Step 4: The server analyzes the video.

[0983] The server saves the received video file to storage and begins video analysis using a generative artificial intelligence model. This analysis involves analyzing each frame of the video and evaluating the robot's movements, accuracy, timing, etc. The input is the uploaded video file, and the output is data indicating its technical strengths and weaknesses.

[0984] Step 5: The server generates the improvement menu.

[0985] The server uses a generative artificial intelligence model that proposes optimal action sequences and improvement methods based on the analysis results to create individual improvement menus. The input is data indicating strengths and weaknesses, and the output is improvement menus and specific action sequences.

[0986] Step 6: The server sends the improvement menu to the terminal.

[0987] The server sends the generated improvement menu to the device in PDF format or as an in-app display format. The input consists of improvement menus and action sequences, and the output is data in a format viewable on the device.

[0988] Step 7: The user changes the settings according to the improvement menu.

[0989] The user checks the improvement menu via a terminal and adjusts the robot's settings and operation as needed. The input is the improvement menu, and the output is the actual operation of the robot after the settings have been changed.

[0990] Step 8: Users record their feedback.

[0991] The user records the progress of the robot's movements and how performance has changed after configuration changes. The terminal temporarily stores this data in local storage. The input is the record of the movements after the changes, and the output is the recorded feedback data.

[0992] Step 9: The server re-analyzes the feedback.

[0993] The terminal uploads recorded feedback data to the server, which then analyzes the data. The server updates the training menu and action sequence by comparing it with past analysis results. The input is the recorded feedback data, and the output is the updated improvement menu.

[0994] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0995] Modes for carrying out the invention

[0996] This invention is a system that combines image / video analysis and an emotion engine to improve the user's athletic performance. It analyzes the user's technical strengths and weaknesses, and further considers their emotional state to provide an optimal training menu. This system is realized through the interaction of a server, a terminal, and the user.

[0997] Image and video upload and analysis

[0998] Users upload images or videos of their exercise performance using a smartphone app. This allows users to provide the system with data that enables detailed analysis of their movements.

[0999] The device compresses the image or video selected by the user and sends it to the server. The server stores the received data in storage and begins analysis using a generative artificial intelligence model. The generative AI model analyzes each frame of the video and evaluates the user's actions, form, timing, etc. As a result of the analysis, the user's technical strengths and weaknesses are identified.

[1000] Implementation of an emotion engine

[1001] The server uses an emotion engine to analyze user facial expression data in addition to image or video data. The emotion engine analyzes the user's facial expressions and voice to extract how the user feels about the training. This identifies the emotional state (e.g., motivation, fatigue, stress).

[1002] Creating a training menu

[1003] The server generates an optimal training menu based on technical analysis results and emotional state analysis results from the emotion engine. In addition to specific exercise items, adjustments are made considering the user's emotional state, and the training menu is customized individually. For example, if the server determines that the user is fatigued, a lighter training program will be suggested.

[1004] The generated training menu is converted into a user-friendly format (e.g., PDF, in-app display) and sent to the device. The device provides an interface for the user to review the training menu and perform the training accordingly.

[1005] Training implementation and recording

[1006] Users perform training according to the training menu and record their progress and feedback on their device. The device temporarily saves the recorded data to local storage and uploads it to the server when connected to the internet.

[1007] Feedback and menu updates

[1008] The server receives the training record data uploaded by the user and performs a re-analysis, comparing it to previous analysis results. This analysis includes technical progress and changes in emotional state. Based on the progress evaluation, the training menu and feedback are updated as needed.

[1009] For example, if a user's skills improve and their emotional state changes for the better, the training menu is updated to include new challenges to help them move on to the next step. The device notifies the user of updates and feedback, helping them prepare for the next training session.

[1010] Specific example

[1011] User A wants to improve their soccer dribbling skills. They record videos of themselves dribbling with their smartphone and upload them to the system. The server analyzes the videos using a generative artificial intelligence model and an emotion engine, identifying that they lack kicking power with their right foot and that they experience stress during practice.

[1012] Next, the server generates a training menu based on these results, combining "specific dribbling exercises to strengthen the kicking power of the right foot" and "relaxation exercises to reduce stress." User A practices according to this menu and records their progress on their device. The server analyzes this record, confirms that the kicking power of the right foot has improved and stress has decreased, and adjusts and provides the next menu.

[1013] The following describes the processing flow.

[1014] Specific steps of the program's processing

[1015] Image and video upload and analysis

[1016] Step 1:

[1017] The user launches the smartphone app and opens the image or video upload screen. The user selects an image or video of their exercise performance.

[1018] Step 2:

[1019] The device compresses the selected image or video and prepares it for upload. At this stage, the user can preview the data and, if there are no problems, begin the upload.

[1020] Step 3:

[1021] The device sends compressed images or videos to the server over the internet. Once the transmission is complete, the user receives a notification.

[1022] Step 4:

[1023] The server saves the images or videos received from the terminal to its storage and passes them to a generative artificial intelligence model to begin analysis.

[1024] Step 5:

[1025] The server uses a generative artificial intelligence model to analyze each frame of images and videos, evaluating the user's actions, form, timing, etc. Based on this evaluation, it identifies the user's technical strengths and weaknesses.

[1026] Step 6:

[1027] The server saves the analysis results to a database, which will serve as the basis for creating the next training menu.

[1028] Implementation of an emotion engine

[1029] Step 7:

[1030] The server activates an emotion engine that analyzes the user's facial expressions and voice, in addition to image or video data, to recognize the user's emotional state.

[1031] Step 8:

[1032] The server stores the user's emotional state (e.g., motivation, fatigue, stress) in a database based on the recognition results of the emotion engine.

[1033] Creating a training menu

[1034] Step 9:

[1035] The server integrates technical analysis results with the emotion engine analysis results to generate an optimal training menu for each individual user. This menu includes not only technical items but also adjustments based on emotional state.

[1036] Step 10:

[1037] The server converts the generated training menu into a user-friendly format (e.g., PDF, in-app display).

[1038] Step 11:

[1039] The server sends the generated training menu to the terminal.

[1040] Training implementation and recording

[1041] Step 12:

[1042] The device displays the received training menu to the user and prompts them to perform the training according to the instructions.

[1043] Step 13:

[1044] Users perform training according to the training menu and record their progress and feedback on their device.

[1045] Step 14:

[1046] The device temporarily stores recorded data in local storage and synchronizes it with the server when connected to the internet.

[1047] Feedback and menu updates

[1048] Step 15:

[1049] The server receives the training record data uploaded by the user and performs a new analysis by comparing it with the results of previous analyses.

[1050] Step 16:

[1051] The server performs progress assessments and updates the training menu as needed. This assessment includes, for example, the degree of technical improvement, newly discovered issues, and the user's emotional state.

[1052] Step 17:

[1053] The server will send the updated training menu and any new feedback back to the terminal. This information will be reflected in the next training session.

[1054] Step 18:

[1055] The device notifies the user of updates and feedback, helping them prepare for the next training session.

[1056] (Example 2)

[1057] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1058] Traditional athletic performance enhancement systems focused on technical analysis and failed to consider the user's emotional state. Therefore, they couldn't provide appropriate training programs when users were stressed or fatigued, making it difficult to achieve sustainable training results. Furthermore, the training programs provided to users were uniform, failing to adequately reflect individual technical strengths and weaknesses.

[1059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1060] In this invention, the server includes means for uploading images or videos of the user's exercise performance; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based on the analysis; means for using an emotion engine to analyze the user's emotional state in addition to the identified technical strengths and weaknesses; means for generating an optimal training menu for each individual user based on the analyzed technical strengths and weaknesses and emotional state; means for converting the generated training menu into a user-friendly format and providing it to the user; means for recording the user's training progress and feedback on the terminal; and means for uploading the recorded data to the server and analyzing it to update the training menu according to the user's progress. This makes it possible to accurately identify the user's technical strengths and weaknesses, provide an individualized training menu based on them, and realize an appropriate training environment that takes into account the user's emotional state.

[1061] A "user" refers to someone who uses the system to film their own athletic performance and receive analysis and training menus.

[1062] A "terminal" refers to a device used by a user (e.g., a smartphone, tablet, or personal computer) that has the function of compressing captured images and videos and sending them to a server.

[1063] A "server" refers to a computer system that has the function of receiving, storing, and analyzing images and videos sent by users, and generating and providing training menus.

[1064] "Means of uploading images or videos" refers to the function that users use to film their own exercise performance and send the data into the system.

[1065] A "generative artificial intelligence model" refers to artificial intelligence technology used to analyze uploaded images and videos to identify the user's technical strengths and weaknesses.

[1066] An "emotion engine" refers to technology that analyzes a user's facial expressions and actions to identify their emotional state.

[1067] "Means for identifying technical strengths and weaknesses" refers to a function that uses generative artificial intelligence models to evaluate a user's technical capabilities from the results of image and video analysis, and to identify specific strengths and weaknesses.

[1068] "Means for generating training menus" refers to a function that creates optimal training content for each individual user based on analyzed technical strengths and weaknesses, as well as their emotional state.

[1069] "Means of providing to the user" refers to functions that provide the generated training menu to the user in an easy-to-understand format.

[1070] "Means for recording training progress and feedback" refers to a function that allows users to record their results and impressions within the system after completing training.

[1071] "A means of updating the training menu according to the user's progress through analysis" refers to a function that analyzes recorded data and updates the training menu as needed, taking into account the user's technical progress and emotional state.

[1072] Modes for carrying out the invention

[1073] This invention is a system that combines image / video analysis and an emotion engine to improve the user's athletic performance. It analyzes the user's technical strengths and weaknesses, and further considers their emotional state to provide an optimal training menu. This system is realized through the interaction of a server, a terminal, and the user.

[1074] Image and video upload and analysis

[1075] Users upload images or videos of their exercise performance using a smartphone app. This allows users to provide the system with data that enables detailed analysis of their movements.

[1076] The terminal compresses images and videos selected by the user and sends them to the server. The server stores the received data in storage and begins analysis using a generative artificial intelligence model. The generative AI model analyzes each frame of the video and evaluates the user's actions, form, timing, etc. As a result of the analysis, the user's technical strengths and weaknesses are identified.

[1077] Implementation of an emotion engine

[1078] The server uses an emotion engine to analyze user facial expression data in addition to image or video data. The emotion engine analyzes the user's facial expressions and voice to extract how the user feels about the training. This identifies the emotional state (e.g., motivation, fatigue, stress).

[1079] Creating a training menu

[1080] The server generates an optimal training menu based on technical analysis results and emotional state analysis results from the emotion engine. In addition to specific exercise items, adjustments are made considering the user's emotional state, and the training menu is customized individually. For example, if the server determines that the user is fatigued, a lighter training program will be suggested.

[1081] The generated training menu is converted into a user-friendly format (e.g., PDF, in-app display) and sent to the device. The device provides an interface for the user to review the training menu and perform the training accordingly.

[1082] Training implementation and recording

[1083] Users perform training according to the training menu and record their progress and feedback on their device. The device temporarily saves the recorded data to local storage and uploads it to the server when connected to the internet.

[1084] Feedback and menu updates

[1085] The server receives the training record data uploaded by the user and performs a re-analysis, comparing it to previous analysis results. This analysis includes technical progress and changes in emotional state. Based on the progress evaluation, the training menu and feedback are updated as needed.

[1086] For example, if a user's skills improve and their emotional state changes for the better, the training menu is updated to include new challenges to help them move on to the next step. The device notifies the user of updates and feedback, helping them prepare for the next training session.

[1087] Specific example

[1088] User A wants to improve their soccer dribbling skills. They record videos of themselves dribbling with their smartphone and upload them to the system. The server analyzes the videos using a generative artificial intelligence model and an emotion engine, identifying that they lack kicking power with their right foot and that they experience stress during practice.

[1089] Next, the server generates a training menu based on these results, combining "specific dribbling exercises to strengthen the kicking power of the right foot" and "relaxation exercises to reduce stress." User A practices according to this menu and records their progress on their device. The server analyzes this record, confirms that the kicking power of the right foot has improved and stress has decreased, and adjusts and provides the next menu.

[1090] Examples of input prompts for a generative AI model

[1091] To improve users' athletic performance, analyze dribbling videos they film with their smartphones to identify their technical strengths and weaknesses. Also, analyze the user's facial expressions in the videos to identify their emotional state (e.g., motivation, fatigue, stress). Based on this data, generate an optimal training program that considers both technical and emotional aspects.

[1092] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1093] Step 1:

[1094] This stage involves users taking images or videos of their exercise performance and uploading them through a smartphone app.

[1095] Input: Images or videos of athletic performance captured by the user.

[1096] Operation: The user uses a smartphone app to select a captured image or video and presses the upload button.

[1097] Output: Image or video data uploaded to the device.

[1098] Step 2:

[1099] This is the stage where the device compresses the image or video uploaded by the user and sends it to the server.

[1100] Input: Image or video data uploaded by the user on their device.

[1101] Operation: The device compresses image or video data and sends it to the server over the internet.

[1102] Output: Compressed image or video data is sent to the server.

[1103] Step 3:

[1104] This stage involves the server saving the received image or video data to storage and starting the analysis using a generative artificial intelligence model.

[1105] Input: Compressed image or video data.

[1106] Operation: The server receives data, saves it to storage, and then inputs that data into a generative artificial intelligence model to begin analysis. Each frame is analyzed, and the user's actions, form, timing, etc., are evaluated.

[1107] Output: Analysis results data showing the user's technical strengths and weaknesses.

[1108] Step 4:

[1109] This is the stage where the server uses an emotion engine to analyze the user's facial expression data from image or video data.

[1110] Input: Image or video data, and analysis result data obtained in the previous step.

[1111] Operation: The server uses an emotion engine to analyze the user's facial expressions in an image or video and extract the user's emotional state (e.g., motivation, fatigue, stress).

[1112] Output: Data indicating the user's emotional state.

[1113] Step 5:

[1114] This is the stage where the server generates an optimal training menu for each individual user based on technical analysis results and emotional state analysis results.

[1115] Input: Technical analysis results data, data indicating emotional state.

[1116] Operation: The server comprehensively analyzes this data and generates the optimal training menu. For example, if the user is fatigued, a lighter training program will be suggested.

[1117] Output: The generated training menu.

[1118] Step 6:

[1119] This stage involves the server converting the generated training menu into a user-friendly format (e.g., PDF, in-app display) and sending it to the device.

[1120] Input: Generated training menu.

[1121] Operation: The server converts the training menu into a user-friendly format and sends it to the terminal via the internet.

[1122] Output: The training menu, converted into a user-friendly format, is sent to the terminal.

[1123] Step 7:

[1124] This stage involves the user completing the training according to the training menu and recording their progress and feedback on their device.

[1125] Input: Training menu.

[1126] Operation: Users perform training according to the provided training menu, and their progress and feedback are recorded on their device via the app.

[1127] Output: Training progress and feedback data recorded on the device.

[1128] Step 8:

[1129] The terminal uploads the recorded training progress and feedback data to the server, where the server performs re-analysis.

[1130] Input: User training progress and feedback data.

[1131] Operation: The device sends recorded data to a server via the internet, and the server re-analyzes the received data. It evaluates technical progress and changes in emotional state, and updates the training menu as needed.

[1132] Output: An updated training menu and feedback are generated.

[1133] (Application Example 2)

[1134] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1135] In modern factories and production lines, robots handle many tasks, but optimizing their movements and performance remains challenging. In particular, identifying a robot's technical strengths and weaknesses and providing appropriate training programs based on that understanding is difficult. Furthermore, it's necessary to efficiently collect and analyze data on robot movements and improve them in real time. This is crucial for improving factory production efficiency.

[1136] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for uploading images or videos of the user's exercise performance; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based thereon; means for generating an optimal training menu for each user based on the identified technical strengths and weaknesses; means for providing the training menu to the user; means for recording the user's training progress and feedback; means for analyzing the recorded data and updating the training menu according to the user's progress; means for uploading images or videos to monitor the robot's movements; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the robot's technical strengths and weaknesses based thereon; and means for generating an optimal training menu for each robot based on the robot's technical strengths and weaknesses. This makes it possible to optimize the robot's movements and performance, thereby improving the production efficiency of the factory.

[1137] A "user" is an individual or organization that uses the system to improve their athletic performance.

[1138] "Exercise performance" refers to the user's physical movements and athletic abilities.

[1139] "Images or videos" refer to visual data that records the user's exercise performance.

[1140] "Uploading" refers to the act of sending images or videos that a user has taken to a system.

[1141] A "generative artificial intelligence model" is a machine learning model that analyzes uploaded images and videos to identify the user's technical strengths and weaknesses.

[1142] "Analysis" refers to the process of analyzing information from images and videos using generative artificial intelligence models.

[1143] "Technical strengths" refer to specific movements or skills in sports that a user excels at.

[1144] A "weakness" refers to an area where a user needs improvement, or a task or skill they struggle with.

[1145] A "training menu" is a personalized exercise plan generated based on the user's technical strengths and weaknesses.

[1146] "Recording" refers to the act of photographing or inputting the user's training progress and feedback.

[1147] A "robot" is an automated machine that performs tasks on production lines in factories and other facilities.

[1148] "Action" refers to the physical movements or tasks that a robot performs.

[1149] "Optimization" is the act of adjusting something to maximize the performance of a user or robot.

[1150] "Maintenance" refers to maintenance work performed to maintain the robot's operation and improve its performance.

[1151] "Factory production efficiency" refers to the efficiency of production activities in a factory, meaning an improvement in both the quality and speed of work.

[1152] A "server" is a computer system used to store and analyze uploaded data.

[1153] A "terminal" is a device used by users to input data or check training menus.

[1154] "Real-time" refers to a time frame in which data is processed immediately and results are provided instantly.

[1155] System Overview

[1156] This invention is a system designed to improve user motor performance and robot operation performance. The system is primarily realized through the interaction of a server, a terminal, and the user.

[1157] Image and video upload and analysis

[1158] Users or factory managers use smartphones or cameras to capture images and videos of exercise performance or robot movements, and upload them to the system via a terminal. The terminal compresses the selected images and videos from the user and efficiently sends them to the server. The server stores the received data in storage and begins analysis using a generative artificial intelligence model. This model analyzes each frame of the video, evaluating specific movements, forms, and timings of the user or robot, and identifies technical strengths and weaknesses.

[1159] Implementation of an emotion engine

[1160] In addition to analyzing image or video data, the server uses an emotion engine to analyze facial expression data. This emotion engine analyzes the user's facial expressions and the robot's movement parameters to extract emotional states (motivation, fatigue, stress, etc.) related to training and work.

[1161] Creating a training menu

[1162] The server generates an optimal training menu based on technical analysis results and analysis results from the emotion engine. This menu is customized to the individual user and robot. For example, if the user is determined to be fatigued, a lighter training program will be suggested. Similarly, if the robot's operation is under high load, maintenance protocols to reduce the load will be provided.

[1163] The generated training menu is sent to the terminal in a user-friendly format. The terminal provides an interface for users and factory managers to review the training menu and work through it.

[1164] Training implementation and recording

[1165] Users perform exercises according to the training menu and record their progress and feedback on a terminal. In the case of robots, factory managers monitor the robot's performance and similarly record it via a terminal. The recorded data is temporarily stored in local storage and uploaded to a server when an internet connection is available.

[1166] Feedback and menu updates

[1167] The server receives the uploaded training and movement recording data and performs a re-analysis, comparing it to previous analysis results. This analysis includes technical progress and changes in emotional state. Based on the progress evaluation, the training menu and feedback are updated as needed and notified to the user and the robot. This improves subsequent training sessions and work performance.

[1168] Examples of specific cases and prompt statements

[1169] If user A wants to improve their soccer dribbling skills, they record a video of themselves dribbling with their smartphone and upload it to the system. The server uses a generative artificial intelligence model and an emotion engine to analyze the video and identify that user A lacks kicking power with their right foot and experiences stress during practice. The server then generates a training menu that combines "specific dribbling exercises to strengthen right-foot kicking power" and "relaxation exercises to reduce stress." User A practices based on this menu and records their progress on their device.

[1170] Examples of prompt statements are as follows:

[1171] "The robot is under heavy load and is experiencing continuous malfunctions. Use the emotion engine to assess the robot's current condition and propose appropriate maintenance protocols."

[1172] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1173] Step 1:

[1174] Users capture images or videos of their exercise performance using their smartphones and upload them to the system via the device. The input from the user is the captured image or video, and the output is compressed data transmitted via the device. Specifically, users use a camera app on their smartphone, select the captured data, and press the upload button.

[1175] Step 2:

[1176] The terminal compresses image or video data provided by the user and sends it to the server. The input is video or image data uploaded by the user, and the output is a compressed data file. This compression is performed to improve the efficiency of data transfer. Typically, a compression algorithm is used.

[1177] Step 3:

[1178] The server saves the received data to storage. The input is a compressed data file, and the output is the data saved to storage. Error checking is also performed to confirm that the data was saved successfully.

[1179] Step 4:

[1180] The server analyzes incoming data using a generative artificial intelligence model. The input is stored image or video data, and the output is the user's technical strengths and weaknesses. As part of the data processing, image processing algorithms are used to analyze each frame of the video, evaluating movement, form, and timing. This identifies the strengths and weaknesses of athletic performance.

[1181] Step 5:

[1182] The server uses an emotion engine to analyze facial expression data and motion parameters. Inputs are facial expression data from videos and robot motion parameters, while outputs are emotional states (e.g., motivation, fatigue, stress). This analysis uses facial recognition and speech analysis technologies to identify the emotional state of the user or robot.

[1183] Step 6:

[1184] The server combines technical analysis results with the emotional engine's analysis results to generate an optimal training menu. The inputs are technical strengths and weaknesses, as well as emotional states, while the output is a customized training menu. Specifically, adjustments are made, such as reducing the training content based on the user's fatigue level.

[1185] Step 7:

[1186] The server sends the generated training menu to the terminal and provides it to the user. The input is the generated training menu, and the output is a display in a user-friendly format (PDF, in-app display). This allows the user to perform training based on the new menu.

[1187] Step 8:

[1188] The user performs exercises according to the provided training menu and records their progress and feedback on the device. Input is the training progress and feedback, while output is the data stored on the device. An interface is used to record actions and emotional feedback during training.

[1189] Step 9:

[1190] The terminal uploads recorded training data to the server when connected to the internet. Input is progress data and feedback stored on the terminal, while output is data transmission to the server. The transmitted data is stored in the server's storage.

[1191] Step 10:

[1192] The server re-analyzes the uploaded data and evaluates the user's technical progress and changes in emotional state. The input is the newly uploaded training data, and the output is the updated training menu and feedback. This allows the next training session to be adjusted based on the user's progress.

[1193] Examples of prompt statements are as follows:

[1194] "The robot is under heavy load and is experiencing continuous malfunctions. Use the emotion engine to assess the robot's current condition and propose appropriate maintenance protocols."

[1195] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1196] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1197] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1198] [Fourth Embodiment]

[1199] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1200] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1201] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1202] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1203] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1204] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1205] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1206] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1207] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1208] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1209] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1210] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1211] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1212] Modes for carrying out the invention

[1213] This invention is a system that provides users with effective training menus using captured images or videos to improve their athletic performance. This system functions through the interaction of a server, a terminal, and the user.

[1214] Image and video upload and analysis

[1215] Users select and upload images or videos of their exercise performance using a smartphone app. These images or videos must capture specific movements or forms of the user in detail.

[1216] The device has the ability to compress selected images or videos and send them to a server over the internet. The device also includes a preview function for uploaded images and videos, allowing users to check the data before sending.

[1217] The server saves images or videos received from the terminal to its storage and begins analysis using a generative artificial intelligence model. The generative AI model analyzes each frame of the image or video, evaluating the user's actions, form, timing, etc. This identifies the user's technical strengths and weaknesses.

[1218] Creating a training menu

[1219] The server generates an optimal training menu based on the analysis results, the user's past training data, and pre-set goals. This training menu is customized according to the user's characteristics and challenges, and includes specific exercises and training items.

[1220] The generated training menu is converted into a user-friendly format (e.g., PDF or in-app display) and sent to the device. The device then provides an interface for the user to view the training menu and perform the training accordingly.

[1221] Training implementation and recording

[1222] The user performs exercises according to the training menu and records their progress and feedback on the device. The device temporarily saves the recorded data to local storage and uploads it to the server once an internet connection is established.

[1223] Feedback and menu updates

[1224] The server receives the training record data uploaded by the user and performs a new analysis by comparing it with previous analysis results. Based on these analysis results, it evaluates the user's progress and decides whether to update the training menu.

[1225] For example, if the analysis results confirm that "right foot kicking power has improved," the new training menu will be modified to include "improving left foot technique" as a key focus. This allows users to continuously receive expert guidance and efficiently improve their skills.

[1226] The following describes the processing flow.

[1227] Specific steps of the program's processing

[1228] Image and video upload and analysis

[1229] Step 1:

[1230] The user launches the smartphone app and opens the image or video upload screen. The user selects an image or video of their exercise performance.

[1231] Step 2:

[1232] The device compresses the selected image or video and prepares it for upload. At this stage, the user can preview the data and, if there are no problems, begin the upload.

[1233] Step 3:

[1234] The device sends compressed images or videos to the server over the internet. Once the transmission is complete, the user receives a notification.

[1235] Step 4:

[1236] The server saves the images or videos received from the terminal to its storage and passes them to a generative artificial intelligence model to begin analysis.

[1237] Step 5:

[1238] The server uses a generative artificial intelligence model to analyze each frame of images and videos, evaluating the user's actions, form, timing, etc. Based on this evaluation, it identifies the user's technical strengths and weaknesses.

[1239] Step 6:

[1240] The server saves the analysis results to a database, which will serve as the basis for creating the next training menu.

[1241] Creating a training menu

[1242] Step 7:

[1243] The server executes an algorithm based on the analysis results, the user's past training data, and their goals to generate the optimal training menu for the user.

[1244] Step 8:

[1245] The server converts the generated training menu into a user-friendly format (e.g., PDF, in-app display).

[1246] Step 9:

[1247] The server sends the generated training menu to the terminal.

[1248] Training implementation and recording

[1249] Step 10:

[1250] The device displays the received training menu to the user and prompts them to perform the training according to the instructions.

[1251] Step 11:

[1252] Users perform training according to the training menu and record their progress and feedback on their device.

[1253] Step 12:

[1254] The device temporarily stores recorded data in local storage and synchronizes it with the server when connected to the internet.

[1255] Feedback and menu updates

[1256] Step 13:

[1257] The server receives the training record data uploaded by the user and performs a new analysis by comparing it with the results of previous analyses.

[1258] Step 14:

[1259] The server performs progress assessments and updates the training menu as needed. This assessment includes, for example, the degree of technical improvement and any newly discovered issues.

[1260] Step 15:

[1261] The server will send the updated training menu and any new feedback back to the terminal. This information will be reflected in the next training session.

[1262] Step 16:

[1263] The device notifies the user of updates and feedback, helping them prepare for the next training session.

[1264] (Example 1)

[1265] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1266] Modern sports training demands accurate feedback and customized training programs to effectively improve individual athletic performance. However, traditional methods, such as users showing individually filmed videos to experts for advice, were often inefficient due to their reliance on human intervention. Furthermore, there was a lack of systems to accurately record and evaluate training progress, provide appropriate feedback, and update training programs. This resulted in the problem of training effectiveness not being maximized.

[1267] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1268] In this invention, the server includes means for analyzing uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based on the analysis; means for generating an optimal training menu for each user based on the identified technical strengths and weaknesses; means for recording the user's training progress and feedback; means for analyzing the recorded data and updating the training menu according to the user's progress; means for compressing images and videos captured by the terminal and sending them to the server; means for the server to store the received images and videos; means for the server to refer to the user's past training data and generate an optimal training menu in conjunction with the analysis results; means for the server to convert the training menu into a format that is easy for the user to understand and send it to the terminal; means for the terminal to temporarily save training data in local storage and upload it to the server; and means for the server to analyze newly received training data and update the training menu. This enables the provision of highly accurate feedback based on the user's individual data and customized training menus.

[1269] "Means for uploading images or videos" refers to a function that allows users to send images or videos of their exercise performance from their device to the server.

[1270] A "generative artificial intelligence model" is an artificial intelligence algorithm used to analyze image or video data to identify a user's technical strengths and weaknesses.

[1271] "Means for identifying a user's technical strengths and weaknesses" refers to a function that uses a generative artificial intelligence model to evaluate the user's actions, forms, and timing, and based on the results, clearly identifies the user's strengths and areas that need improvement.

[1272] "A means of generating an optimal training menu" refers to a function that takes into account the user's technical strengths and weaknesses and creates an individually optimized training plan.

[1273] "Means of providing training menus" refers to a function that displays or sends the generated training menus to the user in an easy-to-understand format.

[1274] "Means for recording training progress and feedback" refers to a function that allows users to input and save the training content, progress, and feedback they have received on their device.

[1275] "Means for analyzing recorded data and updating training menus" refers to a function that re-evaluates the content of the training menu based on the user's progress and changes it to the latest training menu as needed.

[1276] "Compression" refers to a function that performs encoding processing to reduce the file size of captured images and videos.

[1277] "Means for saving received images and videos" refers to the function that allows a server to store image and video data sent from a terminal in its storage.

[1278] "Means of referencing the user's past training data" refers to a function where the server retrieves and uses the user's previous training history and performance data from a database.

[1279] "Means of converting training menus into a user-friendly format" refers to a function that processes the generated training menus into a format that users can easily view, such as PDF or in-app display.

[1280] "Method for temporarily saving to local storage" refers to a function that allows training data to be temporarily saved even when the device does not have an internet connection.

[1281] "Method for uploading to the server" refers to the function of sending data temporarily stored on the device to the server once an internet connection is established.

[1282] "Means for analyzing newly received training data" refers to a function in which the server analyzes newly submitted training data from the user again using a generative artificial intelligence model to evaluate the user's progress and performance.

[1283] Modes for carrying out the invention

[1284] This invention is a system that provides users with effective training menus using captured images or videos to improve their athletic performance. This system functions through the interaction of a server, a terminal, and the user.

[1285] Image and video upload and analysis

[1286] Users launch a smartphone app and select and upload images or videos of their exercise performance. These images or videos capture specific movements or forms of the user in detail.

[1287] The device compresses selected images and videos and sends them to the server over the internet. Specifically, the device compresses the video file selected by the user after shooting (e.g., a soccer dribbling scene) using an encoding algorithm (e.g., H.264) and sends it to the server's communication endpoint. The device also has a preview function for uploaded images and videos, allowing the user to check the data before sending it.

[1288] The server saves images or videos received from the terminal to cloud storage (e.g., AWS S3) and begins analysis using a generative artificial intelligence model (e.g., YOLOv5). This analysis scans each frame of the video to evaluate the user's actions, form, and timing. The analysis results are saved as JSON data, and the user's technical strengths and weaknesses are identified.

[1289] Creating a training menu

[1290] The server generates an optimal training menu for each user based on the analysis results, the user's past training data, and pre-set goals. For example, if the server determines that the user's dribbling speed is slow, training items aimed at improving this (e.g., repetitive dribbling practice) will be added to the menu. The generated training menu is converted into PDF format or an in-app display format and sent to the device.

[1291] Training menu provided

[1292] The device displays the received training menu within the app, making it easy for the user to review. For example, a new training plan will appear in the "Training Menu" tab, and the user can then follow it to complete their workout.

[1293] Training implementation and recording

[1294] Users perform exercises according to the provided training menu and record their progress and feedback on their device. The device temporarily saves the recorded data to local storage (e.g., an SQLite database). Once an internet connection is established, the data is uploaded to the server.

[1295] Feedback and menu updates

[1296] The server analyzes the newly received training record data and compares it to previous analysis results. For example, if it confirms an improvement in the kicking power of the right foot, it uses a generative artificial intelligence model to identify areas that need further improvement (e.g., improving the technique of the left foot) and updates the training menu. The updated menu is sent back to the terminal, and the user can continue training by following the instructions.

[1297] Examples of specific cases and prompt statements

[1298] For example, if a user is practicing soccer, the following sequence of events might occur.

[1299] 1. The user films themselves practicing soccer dribbling with their smartphone.

[1300] 2. The device compresses the recorded video and sends it to the server. The user previews the video before sending it.

[1301] 3. The server analyzes the received video using an AI model to evaluate the dribbling technique.

[1302] 4. The server generates a training menu for improving dribbling skills based on the analysis results and past data.

[1303] 5. The server converts the generated menu into PDF format and sends it to the terminal. The terminal displays the menu.

[1304] 6. The user practices according to the provided menu and records their progress on the device.

[1305] 7. The server analyzes the new training records and updates the training menu as needed.

[1306] The following are examples of prompts for a generative AI model.

[1307] Analyze the user's video and output the analysis results, focusing on the following key points:

[1308] 1. Accuracy of form

[1309] 2. Smoothness of operation

[1310] 3. Timing

[1311] If you encounter any problems with any of the following points, please provide details.

[1312] This allows the system to effectively analyze the user's exercise performance and provide an optimal training program.

[1313] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1314] Understood. Now, I will explain the processing flow of this system's program in detail, divided into the following steps.

[1315] Step 1:

[1316] The user launches a smartphone app and records their athletic performance. For example, if the user is practicing soccer, they will record their dribbling and shooting movements as videos. The input is the video recorded by the smartphone's camera. The output is the athletic performance video file saved on the smartphone.

[1317] Step 2:

[1318] The device compresses the video recorded by the user and sends it to the server. Specifically, the device compresses the video file using an encoding algorithm (e.g., H.264) and sends it to the server over the internet. The input is the video file obtained in step 1, and the output is the compressed video file sent to the server.

[1319] Step 3:

[1320] The server saves the video received from the terminal to cloud storage (e.g., AWS S3). The input is the compressed video file that was sent. The output is the video file saved in cloud storage. The server also passes the received video to the AI ​​analysis process.

[1321] Step 4:

[1322] The server analyzes the video using a generative artificial intelligence model (e.g., YOLOv5). Specifically, it scans each frame of the video and evaluates the user's actions, form, timing, etc. The input is a video file stored in cloud storage, and the output is JSON data as the analysis result. This analysis identifies the user's technical strengths and weaknesses.

[1323] Step 5:

[1324] The server generates an optimal training menu for each user based on the analysis results, the user's past training data, and pre-set goals. The input is the analysis results and past training data, and the output is the generated training menu. For example, it might generate a menu such as "Repetitive practice to improve dribbling speed."

[1325] Step 6:

[1326] The server converts the generated training menu into a user-friendly format (e.g., PDF or in-app display format) and sends it to the device. The input is the generated training menu, and the output is the training menu sent to the device.

[1327] Step 7:

[1328] The device displays the received training menu within the app, allowing the user to review it. For example, a new training plan will appear in the app's "Training Menu" tab. The input is the training menu sent from the server, and the output is the screen display that the user can review.

[1329] Step 8:

[1330] The user performs exercises according to the provided training menu and records their progress and feedback on the device. The acquired data includes information about the content and achievement level of the training. The input is the training progress and feedback entered by the user, and the output is the data recorded on the device.

[1331] Step 9:

[1332] The terminal temporarily saves recorded data to local storage (e.g., an SQLite database) and uploads it to the server once an internet connection is established. The input is the training progress and feedback data recorded on the terminal, and the output is the data uploaded to the server.

[1333] Step 10:

[1334] The server analyzes the newly received training record data and compares it to previous analysis results. The input is the newly uploaded training record data, and the output is the updated analysis results. For example, if it is confirmed that the kicking power of the right foot has improved, a training item to strengthen the technique of the left foot will be added.

[1335] Step 11:

[1336] The server regenerates the optimal training menu based on the user's progress and sends it to the user's terminal. The input is the updated analysis results and user feedback data, and the output is the regenerated training menu. The process is then repeated from step 6.

[1337] Through the steps described above, this invention can effectively improve the user's athletic performance.

[1338] (Application Example 1)

[1339] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1340] Robots used in modern factories are required to have high precision and efficiency, but there is no guarantee that a set operation sequence is optimal, and periodic performance evaluations and improvements are necessary. Furthermore, it is a challenge that on-site personnel need advanced expertise to properly evaluate robot operation and identify specific areas for improvement, making it a very time-consuming process.

[1341] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1342] In this invention, the server includes means for uploading images or videos of the user's own exercise performance or the robot's motion performance; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the technical strengths and weaknesses of the user or robot based on the analysis; and means for generating an optimal training menu or motion sequence for each user or robot based on the identified technical strengths and weaknesses. This makes it possible to objectively and automatically evaluate the robot's motion performance and provide specific methods for improvement.

[1343] A "user" is a person or person who uses the system to evaluate and improve their own exercise performance or the operational performance of the robot they manage.

[1344] "Exercise performance" is a general term for the series of movements and actions performed by a user, including specific techniques, movements, form, and timing.

[1345] A "robot" is an automated mechanical system that performs tasks automatically in a factory based on a sequence of actions.

[1346] "Images or videos" refer to visual data that captures the actions of a user or robot and is in a format that can be processed by a machine.

[1347] A "generative artificial intelligence model" is an algorithm or machine learning model that analyzes images or videos, identifies technical features and shortcomings from their content, and provides appropriate guidance and improvement suggestions.

[1348] A "strength" is a technically superior aspect of a user or robot, a specific element that already demonstrates high performance.

[1349] A "weakness" is a technical aspect that the user or robot needs improvement on, and is a specific element that is currently lacking.

[1350] A "training menu" is a coaching plan that specifically outlines the exercises and practice items that a user should perform to improve their technical skills.

[1351] An "operation sequence" refers to a set of steps or actions that a factory robot should perform.

[1352] "Progress" refers to data that shows how far user training and robot operation have progressed, and how well they have been achieved against the plan.

[1353] "Feedback" refers to evaluations and comments received regarding the results of training or performance, providing specific directions for the next step.

[1354] "Analysis" refers to the process of using generative artificial intelligence models to analyze image or video data in detail, revealing its technical characteristics and shortcomings.

[1355] A "server" is a central processing unit that processes data sent from users or robots and provides analysis results and training menus.

[1356] A "device" is a device used to temporarily store images or videos taken by a user and upload them to a server; this includes smartphones and tablets.

[1357] This invention is a system that uses video to provide effective improvement menus for users or factory personnel to improve the operational performance of robots. The specific implementation method of the system will be described below.

[1358] Users or factory personnel use devices such as smartphones or tablets to record video of the robot's performance. The recorded videos are saved on the device and compressed before being uploaded to the server via the internet. The device provides users with a preview function for the uploaded content, allowing them to review and edit the videos.

[1359] The server stores the uploaded video in storage and analyzes it using a generative artificial intelligence model. This analysis involves a detailed evaluation of each frame of the video, analyzing the robot's movements, accuracy, timing, and other aspects. Based on these analysis results, the server identifies the robot's technical strengths and weaknesses.

[1360] After identifying strengths and weaknesses, the server uses a generative artificial intelligence model to create a personalized improvement menu, suggesting optimal action sequences and improvement methods. This improvement menu is then sent to the device in PDF or in-app display format.

[1361] Users or factory personnel can check improvement menus via a terminal and adjust robot settings and operations as needed. They can also record the progress of the robot's operation and how performance has changed after configuration changes. The terminal temporarily stores this data in local storage and later uploads it to the server.

[1362] The server analyzes uploaded training data and motion records, and updates the training menu and motion sequences by comparing them with past analysis results. This re-analysis process makes it possible to continuously improve the accuracy and efficiency of the robot's movements.

[1363] As a concrete example, consider a procedure to improve the lifting motion of a robotic arm used in a factory. Suppose the initial analysis reveals that the timing of the lifting motion is incorrect. Based on this result, the server suggests improvements and provides a new training sequence. The user modifies the settings based on these suggestions and sends feedback back to the system. The system re-analyzes the system, finding further areas for improvement, ultimately resulting in a smoother and more efficient lifting motion.

[1364] As described above, this invention provides a system that effectively improves the operational performance of a robot by utilizing a generative artificial intelligence model and user feedback.

[1365] Example of a prompt:

[1366] "Explain how to film the movements of robots in a factory, analyze those movements, and provide suggestions for improvement using a generating AI model."

[1367] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1368] Step 1: The user films the robot's movements.

[1369] The user uses a smartphone or tablet to record the robot's performance as a video. The input includes the robot's specific action sequence. The output is the recorded video file, which is saved on the device.

[1370] Step 2: Compress and preview the recorded video on your device.

[1371] The device compresses the recorded video file and trims it if necessary. The user can also preview the video during this process and keep only the desired portion. The input is the raw, unprocessed video file, and the output is the compressed, final video file.

[1372] Step 3: Upload the video to the server.

[1373] The device uploads a compressed video file to the server. During this process, the device transmits the file using the internet. The input is the compressed video file, and the output is the video file uploaded to the server.

[1374] Step 4: The server analyzes the video.

[1375] The server saves the received video file to storage and begins video analysis using a generative artificial intelligence model. This analysis involves analyzing each frame of the video and evaluating the robot's movements, accuracy, timing, etc. The input is the uploaded video file, and the output is data indicating its technical strengths and weaknesses.

[1376] Step 5: The server generates the improvement menu.

[1377] The server uses a generative artificial intelligence model that proposes optimal action sequences and improvement methods based on the analysis results to create individual improvement menus. The input is data indicating strengths and weaknesses, and the output is improvement menus and specific action sequences.

[1378] Step 6: The server sends the improvement menu to the terminal.

[1379] The server sends the generated improvement menu to the device in PDF format or as an in-app display format. The input consists of improvement menus and action sequences, and the output is data in a format viewable on the device.

[1380] Step 7: The user changes the settings according to the improvement menu.

[1381] The user checks the improvement menu via a terminal and adjusts the robot's settings and operation as needed. The input is the improvement menu, and the output is the actual operation of the robot after the settings have been changed.

[1382] Step 8: Users record their feedback.

[1383] The user records the progress of the robot's movements and how performance has changed after configuration modifications. The terminal temporarily stores this data in local storage. The input is the record of the modified movements, and the output is the recorded feedback data.

[1384] Step 9: The server re-analyzes the feedback.

[1385] The terminal uploads recorded feedback data to the server, which then analyzes the data. The server updates the training menu and action sequence by comparing it with past analysis results. The input is the recorded feedback data, and the output is the updated improvement menu.

[1386] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1387] Modes for carrying out the invention

[1388] This invention is a system that combines image / video analysis and an emotion engine to improve the user's athletic performance. It analyzes the user's technical strengths and weaknesses, and further considers their emotional state to provide an optimal training menu. This system is realized through the interaction of a server, a terminal, and the user.

[1389] Image and video upload and analysis

[1390] Users upload images or videos of their exercise performance using a smartphone app. This allows users to provide the system with data that enables detailed analysis of their movements.

[1391] The device compresses the image or video selected by the user and sends it to the server. The server stores the received data in storage and begins analysis using a generative artificial intelligence model. The generative AI model analyzes each frame of the video and evaluates the user's actions, form, timing, etc. As a result of the analysis, the user's technical strengths and weaknesses are identified.

[1392] Implementation of an emotion engine

[1393] The server uses an emotion engine to analyze user facial expression data in addition to image or video data. The emotion engine analyzes the user's facial expressions and voice to extract how the user feels about the training. This identifies the emotional state (e.g., motivation, fatigue, stress).

[1394] Creating a training menu

[1395] The server generates an optimal training menu based on technical analysis results and emotional state analysis results from the emotion engine. In addition to specific exercise items, adjustments are made considering the user's emotional state, and the training menu is customized individually. For example, if the server determines that the user is fatigued, a lighter training program will be suggested.

[1396] The generated training menu is converted into a user-friendly format (e.g., PDF, in-app display) and sent to the device. The device provides an interface for the user to review the training menu and perform the training accordingly.

[1397] Training implementation and recording

[1398] Users perform training according to the training menu and record their progress and feedback on their device. The device temporarily saves the recorded data to local storage and uploads it to the server when connected to the internet.

[1399] Feedback and menu updates

[1400] The server receives the training record data uploaded by the user and performs a re-analysis, comparing it to previous analysis results. This analysis includes technical progress and changes in emotional state. Based on the progress evaluation, the training menu and feedback are updated as needed.

[1401] For example, if a user's skills improve and their emotional state changes for the better, the training menu is updated to include new challenges to help them move on to the next step. The device notifies the user of updates and feedback, helping them prepare for the next training session.

[1402] Specific example

[1403] User A wants to improve their soccer dribbling skills. They record videos of themselves dribbling with their smartphone and upload them to the system. The server analyzes the videos using a generative artificial intelligence model and an emotion engine, identifying that they lack kicking power with their right foot and that they experience stress during practice.

[1404] Next, the server generates a training menu based on these results, combining "specific dribbling exercises to strengthen the kicking power of the right foot" and "relaxation exercises to reduce stress." User A practices according to this menu and records their progress on their device. The server analyzes this record, confirms that the kicking power of the right foot has improved and stress has decreased, and adjusts and provides the next menu.

[1405] The following describes the processing flow.

[1406] Specific steps of the program's processing

[1407] Image and video upload and analysis

[1408] Step 1:

[1409] The user launches the smartphone app and opens the image or video upload screen. The user selects an image or video of their exercise performance.

[1410] Step 2:

[1411] The device compresses the selected image or video and prepares it for upload. At this stage, the user can preview the data and, if there are no problems, begin the upload.

[1412] Step 3:

[1413] The device sends compressed images or videos to the server over the internet. Once the transmission is complete, the user receives a notification.

[1414] Step 4:

[1415] The server saves the images or videos received from the terminal to its storage and passes them to a generative artificial intelligence model to begin analysis.

[1416] Step 5:

[1417] The server uses a generative artificial intelligence model to analyze each frame of images and videos, evaluating the user's actions, form, timing, etc. Based on this evaluation, it identifies the user's technical strengths and weaknesses.

[1418] Step 6:

[1419] The server saves the analysis results to a database, which will serve as the basis for creating the next training menu.

[1420] Implementation of an emotion engine

[1421] Step 7:

[1422] The server activates an emotion engine that analyzes the user's facial expressions and voice, in addition to image or video data, to recognize the user's emotional state.

[1423] Step 8:

[1424] The server stores the user's emotional state (e.g., motivation, fatigue, stress) in a database based on the recognition results of the emotion engine.

[1425] Creating a training menu

[1426] Step 9:

[1427] The server integrates technical analysis results with the emotion engine analysis results to generate an optimal training menu for each individual user. This menu includes not only technical items but also adjustments based on emotional state.

[1428] Step 10:

[1429] The server converts the generated training menu into a user-friendly format (e.g., PDF, in-app display).

[1430] Step 11:

[1431] The server sends the generated training menu to the terminal.

[1432] Training implementation and recording

[1433] Step 12:

[1434] The device displays the received training menu to the user and prompts them to perform the training according to the instructions.

[1435] Step 13:

[1436] Users perform training according to the training menu and record their progress and feedback on their device.

[1437] Step 14:

[1438] The device temporarily stores recorded data in local storage and synchronizes it with the server when connected to the internet.

[1439] Feedback and menu updates

[1440] Step 15:

[1441] The server receives the training record data uploaded by the user and performs a new analysis by comparing it with the results of previous analyses.

[1442] Step 16:

[1443] The server performs progress assessments and updates the training menu as needed. This assessment includes, for example, the degree of technical improvement, newly discovered issues, and the user's emotional state.

[1444] Step 17:

[1445] The server will send the updated training menu and any new feedback back to the terminal. This information will be reflected in the next training session.

[1446] Step 18:

[1447] The device notifies the user of updates and feedback, helping them prepare for the next training session.

[1448] (Example 2)

[1449] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1450] Traditional athletic performance enhancement systems focused on technical analysis and failed to consider the user's emotional state. Therefore, they couldn't provide appropriate training programs when users were stressed or fatigued, making it difficult to achieve sustainable training results. Furthermore, the training programs provided to users were uniform, failing to adequately reflect individual technical strengths and weaknesses.

[1451] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1452] In this invention, the server includes means for uploading images or videos of the user's exercise performance; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based on the analysis; means for using an emotion engine to analyze the user's emotional state in addition to the identified technical strengths and weaknesses; means for generating an optimal training menu for each individual user based on the analyzed technical strengths and weaknesses and emotional state; means for converting the generated training menu into a user-friendly format and providing it to the user; means for recording the user's training progress and feedback on the terminal; and means for uploading the recorded data to the server and analyzing it to update the training menu according to the user's progress. This makes it possible to accurately identify the user's technical strengths and weaknesses, provide an individualized training menu based on them, and realize an appropriate training environment that takes into account the user's emotional state.

[1453] A "user" refers to someone who uses the system to film their own athletic performance and receive analysis and training menus.

[1454] A "terminal" refers to a device used by a user (e.g., a smartphone, tablet, or personal computer) that has the function of compressing captured images and videos and sending them to a server.

[1455] A "server" refers to a computer system that has the function of receiving, storing, and analyzing images and videos sent by users, and generating and providing training menus.

[1456] "Means of uploading images or videos" refers to the function that users use to film their own exercise performance and send the data into the system.

[1457] A "generative artificial intelligence model" refers to artificial intelligence technology used to analyze uploaded images and videos to identify the user's technical strengths and weaknesses.

[1458] An "emotion engine" refers to technology that analyzes a user's facial expressions and actions to identify their emotional state.

[1459] "Means for identifying technical strengths and weaknesses" refers to a function that uses generative artificial intelligence models to evaluate a user's technical capabilities from the results of image and video analysis, and to identify specific strengths and weaknesses.

[1460] "Means for generating training menus" refers to a function that creates optimal training content for each individual user based on analyzed technical strengths and weaknesses, as well as their emotional state.

[1461] "Means of providing to the user" refers to functions that provide the generated training menu to the user in an easy-to-understand format.

[1462] "Means for recording training progress and feedback" refers to a function that allows users to record their results and impressions within the system after completing training.

[1463] "A means of updating the training menu according to the user's progress through analysis" refers to a function that analyzes recorded data and updates the training menu as needed, taking into account the user's technical progress and emotional state.

[1464] Modes for carrying out the invention

[1465] This invention is a system that combines image / video analysis and an emotion engine to improve the user's athletic performance. It analyzes the user's technical strengths and weaknesses, and further considers their emotional state to provide an optimal training menu. This system is realized through the interaction of a server, a terminal, and the user.

[1466] Image and video upload and analysis

[1467] Users upload images or videos of their exercise performance using a smartphone app. This allows users to provide the system with data that enables detailed analysis of their movements.

[1468] The terminal compresses images and videos selected by the user and sends them to the server. The server stores the received data in storage and begins analysis using a generative artificial intelligence model. The generative AI model analyzes each frame of the video and evaluates the user's actions, form, timing, etc. As a result of the analysis, the user's technical strengths and weaknesses are identified.

[1469] Implementation of an emotion engine

[1470] The server uses an emotion engine to analyze user facial expression data in addition to image or video data. The emotion engine analyzes the user's facial expressions and voice to extract how the user feels about the training. This identifies the emotional state (e.g., motivation, fatigue, stress).

[1471] Creating a training menu

[1472] The server generates an optimal training menu based on technical analysis results and emotional state analysis results from the emotion engine. In addition to specific exercise items, adjustments are made considering the user's emotional state, and the training menu is customized individually. For example, if the server determines that the user is fatigued, a lighter training program will be suggested.

[1473] The generated training menu is converted into a user-friendly format (e.g., PDF, in-app display) and sent to the device. The device provides an interface for the user to review the training menu and perform the training accordingly.

[1474] Training implementation and recording

[1475] Users perform training according to the training menu and record their progress and feedback on their device. The device temporarily saves the recorded data to local storage and uploads it to the server when connected to the internet.

[1476] Feedback and menu updates

[1477] The server receives the training record data uploaded by the user and performs a re-analysis, comparing it to previous analysis results. This analysis includes technical progress and changes in emotional state. Based on the progress evaluation, the training menu and feedback are updated as needed.

[1478] For example, if a user's skills improve and their emotional state changes for the better, the training menu is updated to include new challenges to help them move on to the next step. The device notifies the user of updates and feedback, helping them prepare for the next training session.

[1479] Specific example

[1480] User A wants to improve their soccer dribbling skills. They record videos of themselves dribbling with their smartphone and upload them to the system. The server analyzes the videos using a generative artificial intelligence model and an emotion engine, identifying that they lack kicking power with their right foot and that they experience stress during practice.

[1481] Next, the server generates a training menu based on these results, combining "specific dribbling exercises to strengthen the kicking power of the right foot" and "relaxation exercises to reduce stress." User A practices according to this menu and records their progress on their device. The server analyzes this record, confirms that the kicking power of the right foot has improved and stress has decreased, and adjusts and provides the next menu.

[1482] Examples of input prompts for a generative AI model

[1483] To improve users' athletic performance, analyze dribbling videos they film with their smartphones to identify their technical strengths and weaknesses. Also, analyze the user's facial expressions in the videos to identify their emotional state (e.g., motivation, fatigue, stress). Based on this data, generate an optimal training program that considers both technical and emotional aspects.

[1484] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1485] Step 1:

[1486] This stage involves users taking images or videos of their exercise performance and uploading them through a smartphone app.

[1487] Input: Images or videos of athletic performance captured by the user.

[1488] Operation: The user uses a smartphone app to select a captured image or video and presses the upload button.

[1489] Output: Image or video data uploaded to the device.

[1490] Step 2:

[1491] This is the stage where the device compresses the image or video uploaded by the user and sends it to the server.

[1492] Input: Image or video data uploaded by the user on their device.

[1493] Operation: The device compresses image or video data and sends it to the server over the internet.

[1494] Output: Compressed image or video data is sent to the server.

[1495] Step 3:

[1496] This stage involves the server saving the received image or video data to storage and starting the analysis using a generative artificial intelligence model.

[1497] Input: Compressed image or video data.

[1498] Operation: The server receives data, saves it to storage, and then inputs that data into a generative artificial intelligence model to begin analysis. Each frame is analyzed, and the user's actions, form, timing, etc., are evaluated.

[1499] Output: Analysis results data showing the user's technical strengths and weaknesses.

[1500] Step 4:

[1501] This is the stage where the server uses an emotion engine to analyze the user's facial expression data from image or video data.

[1502] Input: Image or video data, and analysis result data obtained in the previous step.

[1503] Operation: The server uses an emotion engine to analyze the user's facial expressions in an image or video and extract the user's emotional state (e.g., motivation, fatigue, stress).

[1504] Output: Data indicating the user's emotional state.

[1505] Step 5:

[1506] This is the stage where the server generates an optimal training menu for each individual user based on technical analysis results and emotional state analysis results.

[1507] Input: Technical analysis results data, data indicating emotional state.

[1508] Operation: The server comprehensively analyzes this data and generates the optimal training menu. For example, if the user is fatigued, a lighter training program will be suggested.

[1509] Output: The generated training menu.

[1510] Step 6:

[1511] This stage involves the server converting the generated training menu into a user-friendly format (e.g., PDF, in-app display) and sending it to the device.

[1512] Input: Generated training menu.

[1513] Operation: The server converts the training menu into a user-friendly format and sends it to the terminal via the internet.

[1514] Output: The training menu, converted into a user-friendly format, is sent to the terminal.

[1515] Step 7:

[1516] This stage involves the user completing the training according to the training menu and recording their progress and feedback on their device.

[1517] Input: Training menu.

[1518] Operation: Users perform training according to the provided training menu, and their progress and feedback are recorded on their device via the app.

[1519] Output: Training progress and feedback data recorded on the device.

[1520] Step 8:

[1521] The terminal uploads the recorded training progress and feedback data to the server, where the server performs re-analysis.

[1522] Input: User training progress and feedback data.

[1523] Operation: The device sends recorded data to the server via the internet, and the server re-analyzes the received data. It evaluates technical progress and changes in emotional state, and updates the training menu as needed.

[1524] Output: An updated training menu and feedback are generated.

[1525] (Application Example 2)

[1526] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1527] In modern factories and production lines, robots handle many tasks, but optimizing their movements and performance remains challenging. In particular, identifying a robot's technical strengths and weaknesses and providing appropriate training programs based on that understanding is difficult. Furthermore, it's necessary to efficiently collect and analyze data on robot movements and improve them in real time. This is crucial for improving factory production efficiency.

[1528] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for uploading images or videos of the user's exercise performance; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based thereon; means for generating an optimal training menu for each user based on the identified technical strengths and weaknesses; means for providing the training menu to the user; means for recording the user's training progress and feedback; means for analyzing the recorded data and updating the training menu according to the user's progress; means for uploading images or videos to monitor the robot's movements; means for analyzing the uploaded images or videos using a generative artificial intelligence model and identifying the robot's technical strengths and weaknesses based thereon; and means for generating an optimal training menu for each robot based on the robot's technical strengths and weaknesses. This makes it possible to optimize the robot's movements and performance, thereby improving the production efficiency of the factory.

[1529] A "user" is an individual or organization that uses the system to improve their athletic performance.

[1530] "Exercise performance" refers to the user's physical movements and athletic abilities.

[1531] "Images or videos" refer to visual data that records the user's exercise performance.

[1532] "Uploading" refers to the act of sending images or videos that a user has taken to a system.

[1533] A "generative artificial intelligence model" is a machine learning model that analyzes uploaded images and videos to identify the user's technical strengths and weaknesses.

[1534] "Analysis" refers to the process of analyzing information from images and videos using generative artificial intelligence models.

[1535] "Technical strengths" refer to specific movements or skills in sports that a user excels at.

[1536] A "weakness" refers to an area where a user needs improvement, or a task or skill they struggle with.

[1537] A "training menu" is a personalized exercise plan generated based on the user's technical strengths and weaknesses.

[1538] "Recording" refers to the act of photographing or inputting the user's training progress and feedback.

[1539] A "robot" is an automated machine that performs tasks on production lines in factories and other facilities.

[1540] "Action" refers to the physical movements or tasks that a robot performs.

[1541] "Optimization" is the act of adjusting something to maximize the performance of a user or robot.

[1542] "Maintenance" refers to maintenance work performed to maintain the robot's operation and improve its performance.

[1543] "Factory production efficiency" refers to the efficiency of production activities in a factory, meaning an improvement in both the quality and speed of work.

[1544] A "server" is a computer system used to store and analyze uploaded data.

[1545] A "terminal" is a device used by users to input data or check training menus.

[1546] "Real-time" refers to a time frame in which data is processed immediately and results are provided instantly.

[1547] System Overview

[1548] This invention is a system designed to improve user motor performance and robot operation performance. The system is primarily realized through the interaction of a server, a terminal, and the user.

[1549] Image and video upload and analysis

[1550] Users or factory managers use smartphones or cameras to capture images and videos of exercise performance or robot movements, and upload them to the system via a terminal. The terminal compresses the selected images and videos from the user and efficiently sends them to the server. The server stores the received data in storage and begins analysis using a generative artificial intelligence model. This model analyzes each frame of the video, evaluating specific movements, forms, and timings of the user or robot, and identifies technical strengths and weaknesses.

[1551] Implementation of an emotion engine

[1552] In addition to analyzing image or video data, the server uses an emotion engine to analyze facial expression data. This emotion engine analyzes the user's facial expressions and the robot's movement parameters to extract emotional states (motivation, fatigue, stress, etc.) related to training and work.

[1553] Creating a training menu

[1554] The server generates an optimal training menu based on technical analysis results and analysis results from the emotion engine. This menu is customized to the individual user and robot. For example, if the user is determined to be fatigued, a lighter training program will be suggested. Similarly, if the robot's operation is under high load, maintenance protocols to reduce the load will be provided.

[1555] The generated training menu is sent to the terminal in a user-friendly format. The terminal provides an interface for users and factory managers to review the training menu and work through it.

[1556] Training implementation and recording

[1557] Users perform exercises according to the training menu and record their progress and feedback on a terminal. In the case of robots, factory managers monitor the robot's performance and similarly record it via a terminal. The recorded data is temporarily stored in local storage and uploaded to a server when an internet connection is available.

[1558] Feedback and menu updates

[1559] The server receives the uploaded training and movement recording data and performs a re-analysis, comparing it to previous analysis results. This analysis includes technical progress and changes in emotional state. Based on the progress evaluation, the training menu and feedback are updated as needed and notified to the user and the robot. This improves subsequent training sessions and work performance.

[1560] Examples of specific cases and prompt statements

[1561] If user A wants to improve their soccer dribbling skills, they record a video of themselves dribbling with their smartphone and upload it to the system. The server uses a generative artificial intelligence model and an emotion engine to analyze the video and identify that user A lacks kicking power with their right foot and experiences stress during practice. The server then generates a training menu that combines "specific dribbling exercises to strengthen right-foot kicking power" and "relaxation exercises to reduce stress." User A practices based on this menu and records their progress on their device.

[1562] Examples of prompt statements are as follows:

[1563] "The robot is under heavy load and is experiencing continuous malfunctions. Use the emotion engine to assess the robot's current condition and propose appropriate maintenance protocols."

[1564] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1565] Step 1:

[1566] Users capture images or videos of their exercise performance using their smartphones and upload them to the system via the device. The input from the user is the captured image or video, and the output is compressed data transmitted via the device. Specifically, users use a camera app on their smartphone, select the captured data, and press the upload button.

[1567] Step 2:

[1568] The terminal compresses image or video data provided by the user and sends it to the server. The input is video or image data uploaded by the user, and the output is a compressed data file. This compression is performed to improve the efficiency of data transfer. Typically, a compression algorithm is used.

[1569] Step 3:

[1570] The server saves the received data to storage. The input is a compressed data file, and the output is the data saved to storage. Error checking is also performed to confirm that the data was saved successfully.

[1571] Step 4:

[1572] The server analyzes incoming data using a generative artificial intelligence model. The input is stored image or video data, and the output is the user's technical strengths and weaknesses. As part of the data processing, image processing algorithms are used to analyze each frame of the video, evaluating movement, form, and timing. This identifies the strengths and weaknesses of athletic performance.

[1573] Step 5:

[1574] The server uses an emotion engine to analyze facial expression data and motion parameters. Inputs are facial expression data from videos and robot motion parameters, while outputs are emotional states (e.g., motivation, fatigue, stress). This analysis uses facial recognition and speech analysis technologies to identify the emotional state of the user or robot.

[1575] Step 6:

[1576] The server combines technical analysis results with the emotional engine's analysis results to generate an optimal training menu. The inputs are technical strengths and weaknesses, as well as emotional states, while the output is a customized training menu. Specifically, adjustments are made, such as reducing the training content based on the user's fatigue level.

[1577] Step 7:

[1578] The server sends the generated training menu to the terminal and provides it to the user. The input is the generated training menu, and the output is a display in a user-friendly format (PDF, in-app display). This allows the user to perform training based on the new menu.

[1579] Step 8:

[1580] The user performs exercises according to the provided training menu and records their progress and feedback on the device. Input is the training progress and feedback, while output is the data stored on the device. An interface is used to record actions and emotional feedback during training.

[1581] Step 9:

[1582] The terminal uploads recorded training data to the server when connected to the internet. Input is progress data and feedback stored on the terminal, while output is data transmission to the server. The transmitted data is stored in the server's storage.

[1583] Step 10:

[1584] The server re-analyzes the uploaded data and evaluates the user's technical progress and changes in emotional state. The input is the newly uploaded training data, and the output is the updated training menu and feedback. This allows the next training session to be adjusted based on the user's progress.

[1585] Examples of prompt statements are as follows:

[1586] "The robot is under heavy load and is experiencing continuous malfunctions. Use the emotion engine to assess the robot's current condition and propose appropriate maintenance protocols."

[1587] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1588] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1589] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1590] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1591] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1592] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1593] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1594] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1595] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1596] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1597] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1598] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1599] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1600] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1601] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1602] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1603] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1604] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1605] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1606] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1607] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1608] The following is further disclosed regarding the embodiments described above.

[1609] Draft claims

[1610] (Claim 1)

[1611] A means for users to upload images or videos of their own athletic performance,

[1612] A means for analyzing uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based on that analysis,

[1613] A means for generating an optimal training menu for each user based on identified technical strengths and weaknesses,

[1614] Means of providing training menus to users,

[1615] A means of recording the user's training progress and feedback.

[1616] A means of analyzing recorded data and updating the training menu according to the user's progress.

[1617] A system that includes this.

[1618] (Claim 2)

[1619] The analysis of images or videos further includes means for evaluating specific actions, forms, and timings when identifying a user's technical strengths and weaknesses.

[1620] The system according to claim 1.

[1621] (Claim 3)

[1622] This includes means for recording the user's training progress and feedback on a device and periodically uploading the data recorded by that device to a server.

[1623] The system according to claim 1.

[1624] "Example 1"

[1625] (Claim 1)

[1626] A means for users to upload images or videos of their own athletic performance,

[1627] A means for analyzing uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based on that analysis,

[1628] A means for generating an optimal training menu for each user based on identified technical strengths and weaknesses,

[1629] Means of providing training menus to users,

[1630] A means of recording the user's training progress and feedback.

[1631] A means of analyzing recorded data and updating the training menu according to the user's progress,

[1632] A means of compressing images and videos taken by the device and sending them to the server,

[1633] A means of saving images and videos received by the server,

[1634] A means by which the server references the user's past training data and generates an optimal training menu in conjunction with the analysis results,

[1635] A means by which the server converts the training menu into a format that is easy for the user to understand and sends it to the terminal,

[1636] A means for the device to temporarily save training data to local storage and upload it to the server,

[1637] A method for the server to analyze newly received training data and update the training menu.

[1638] A system that includes this.

[1639] (Claim 2)

[1640] The system according to claim 1, further comprising means for evaluating specific actions, forms, and timings when identifying a user's technical strengths and weaknesses based on the results of image or video analysis.

[1641] (Claim 3)

[1642] The system according to claim 1, comprising means for recording the user's training progress and feedback on a terminal and periodically uploading the data recorded by the terminal to a server.

[1643] "Application Example 1"

[1644] (Claim 1)

[1645] A means for users to upload images or videos of their own exercise performance or the robot's movement performance,

[1646] A means for analyzing uploaded images or videos using a generative artificial intelligence model and identifying the technical strengths and weaknesses of the user or robot based on that analysis,

[1647] Means for generating an optimal training menu or motion sequence for each user or robot based on identified technical strengths and weaknesses,

[1648] Means for providing a training menu or operation sequence to a user or factory personnel,

[1649] Means for recording the progress and feedback of user training or robot operation, and

[1650] A means for analyzing recorded data and updating the training menu or action sequence according to the user's progress or the robot's movements.

[1651] A system that includes this.

[1652] (Claim 2)

[1653] The analysis of images or videos further includes means for evaluating specific actions, forms, and timings when identifying the technical strengths and weaknesses of a user or robot.

[1654] The system according to claim 1.

[1655] (Claim 3)

[1656] This includes means for recording the user's training or robot operation progress and feedback on a terminal, and periodically uploading the data recorded by the terminal to a server.

[1657] The system according to claim 1.

[1658] "Example 2 of combining an emotion engine"

[1659] (Claim 1)

[1660] A means for users to upload images or videos of their own athletic performance,

[1661] A means for analyzing uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based on that analysis,

[1662] In addition to identified technical strengths and weaknesses, the means of using an emotion engine to analyze the user's emotional state,

[1663] A means for generating an optimal training menu for each individual user based on analyzed technical strengths and weaknesses and emotional state,

[1664] A means of converting the generated training menu into a user-friendly format and providing it to the user,

[1665] A means for recording the user's training progress and feedback on the device,

[1666] A method for updating training menus according to user progress by uploading recorded data to a server and analyzing it.

[1667] A system that includes this.

[1668] (Claim 2)

[1669] The system according to claim 1, further comprising means for evaluating specific actions, forms, and timings when identifying a user's technical strengths and weaknesses based on the results of image or video analysis.

[1670] (Claim 3)

[1671] The system according to claim 1, comprising an emotion engine that analyzes the user's facial expression data and identifies their emotional state.

[1672] "Application example 2 when combining with an emotional engine"

[1673] (Claim 1)

[1674] A means for users to upload images or videos of their own athletic performance,

[1675] A means for analyzing uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based on that analysis,

[1676] A means for generating an optimal training menu for each user based on identified technical strengths and weaknesses,

[1677] Means of providing training menus to users,

[1678] A means of recording the user's training progress and feedback.

[1679] A means of analyzing recorded data and updating the training menu according to the user's progress,

[1680] A means of uploading images or videos to monitor the robot's movements,

[1681] A means for analyzing uploaded images or videos using a generative artificial intelligence model and identifying the technical strengths and weaknesses of a robot based on that analysis,

[1682] A means of generating an optimal training menu for each robot based on its technical strengths and weaknesses.

[1683] A system that includes this.

[1684] (Claim 2)

[1685] The analysis of images or videos further includes means for evaluating specific actions, forms, and timings when identifying a user's technical strengths and weaknesses.

[1686] The system according to claim 1.

[1687] (Claim 3)

[1688] This includes means for recording the user's training progress and feedback on a device and periodically uploading the data recorded by that device to a server.

[1689] The system according to claim 1. [Explanation of Symbols]

[1690] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for users to upload images or videos of their own athletic performance, A means for analyzing uploaded images or videos using a generative artificial intelligence model and identifying the user's technical strengths and weaknesses based on that analysis, A means for generating an optimal training menu for each user based on identified technical strengths and weaknesses, Means of providing training menus to users, A means of recording the user's training progress and feedback. A means of analyzing recorded data and updating the training menu according to the user's progress. A system that includes this.

2. The analysis of images or videos further includes means for evaluating specific actions, forms, and timings when identifying a user's technical strengths and weaknesses. The system according to claim 1.

3. This includes means for recording the user's training progress and feedback on a device, and periodically uploading the data recorded by that device to a server. The system according to claim 1.

Images