System

Abstract

A system is provided.SOLUTION: A system for analyzing athletic performance of a player, comprising: A system comprising: means for receiving motion video of a player; means for analyzing the received motion video using a generative AI model and quantifying motion performance; and means for generating and providing a report of the motion performance based on the analysis.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In sports, maximizing athlete performance and preventing injuries are extremely important, but current methods make it difficult to perform detailed analysis of individual athletes' movements and manage risk. In particular, the lack of systematic analysis methods utilizing video data and motion capture data from practice and games makes it difficult to accurately identify areas for improvement in athletes' movements and injury risks. Furthermore, there are limited methods for scientifically predicting the time required for recovery and future performance after an injury. Therefore, there is a need for a system that can support athletes in improving their performance, preventing injuries, and providing efficient rehabilitation. [Means for solving the problem]

[0005] To address this issue, the present invention provides a system for analyzing an athlete's athletic performance. The system includes a means for receiving athlete movement video data and analyzing it using a generative AI model, and a means for generating and providing a movement performance report based on the analysis results. The system also includes a means for analyzing motion capture data to identify movement pattern and biomechanical issues, and a means for generating and providing a report based on the results. The system also includes a means for receiving athlete injury data, analyzing it using a generative AI model, and predicting recovery time and performance, and a means for generating and providing a recovery time and performance prediction report based on the results. This makes it possible to maximize athlete performance, reduce injury risk, and support efficient rehabilitation.

[0006] "Athlete" means an individual who participates in a sporting activity for the purposes of improving performance and / or preventing injury.

[0007] "Athletic performance" refers to the general evaluation of an athlete's physical movements and technical movements, specifically including factors such as speed, accuracy, and efficiency.

[0008] "Video data" refers to video files that record players' practice and matches, and are the subject of analysis.

[0009] A "generative AI model" is an algorithm that uses artificial intelligence to analyze video data and motion capture data.

[0010] "Analysis" refers to the process of quantifying and evaluating the player's movements based on the received data, and identifying problems and areas for performance improvement.

[0011] A "performance report" is a document or graphic generated based on the analysis results, providing information to visualize and evaluate an athlete's athletic performance.

[0012] "Motion capture data" refers to data that records in detail the specific movements of an athlete and is used for biomechanics analysis.

[0013] "Movement patterns" refer to the characteristics of a player's continuous movements and movements.

[0014] "Biomechanics" is an academic field that analyzes athletes' physical movements from a physical and mathematical perspective.

[0015] "Injury data" refers to data that includes detailed information about a player's injury, treatment progress, diagnosis, etc.

[0016] "Recovery time" refers to the amount of time required to fully recover from an injury.

[0017] "Performance prediction" is the process of predicting how well an athlete will perform once they have recovered. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This system maximizes athletes' athletic performance and prevents injuries by analyzing video and motion capture data of athletes using a generative AI model and generating detailed reports to help athletes improve their performance and prevent injuries. In the event of an injury, the system can also analyze injury data and predict recovery time and performance.

[0040] Performance Analysis System

[0041] The system first receives video data of the player's movements. The user records video during practice or a match on their device and uploads it to the system. The server receives the video data and performs a detailed analysis using a generative AI model, quantifying the speed, accuracy, and trajectory of the movement. A performance report is generated based on the analysis results and sent to the device. The user can review the report and learn how to improve their own movements.

[0042] As a concrete example, when a player uploads a video of himself practicing dribbling a basketball to the system, the system quantifies the speed, accuracy, and swiftness of the dribbling motion, and provides the user with a report with an overall evaluation.

[0043] Motion Capture Analysis System

[0044] In motion capture analysis, a player's movements are collected using a motion capture device. The movement data transferred from the device is stored on the device and uploaded to the system. The server analyzes the received data using a generative AI model to identify the player's movement patterns and biomechanical problems. A report is generated based on this and sent to the device. The user can review the report and receive guidance to improve their movement patterns.

[0045] For example, an athlete wears a motion capture device to record their running form and collect data. The system analyzes the movement patterns of their left and right feet, knee angles, and ankle movements, identifies problems, and provides a report.

[0046] Recovery Time and Performance Prediction System

[0047] When an athlete is injured, detailed injury data (diagnosis results, treatment progress, etc.) is uploaded from the device to the system. The server receives this data and analyzes it using a generative AI model. Based on the analysis results, a predicted recovery time and an optimal rehabilitation program are provided. A feedback report is sent to the device, allowing the user to use it to efficiently promote recovery.

[0048] For example, if an athlete sprains their ankle, they can input the details of the injury into the system. The system will compare the data with similar data from the past and provide a prediction of recovery time and rehabilitation program. The user can then use this information to create a rehabilitation plan and progress their recovery.

[0049] This invention will scientifically support athletes in improving their performance and preventing injuries, enabling efficient rehabilitation. In the initial stages, the system will partner with schools and other institutions to accumulate experience through trial and error, before being packaged and widely deployed. It is expected that this will enable effective support for a wide range of people, from professional athletes to students.

[0050] The processing flow will be explained below.

[0051] Performance analysis process steps

[0052] Step 1:

[0053] User: Record videos of practice and games and save them on the device.

[0054] Step 2:

[0055] Device: Open the video upload screen and select the saved video file.

[0056] Step 3:

[0057] On your device: Upload the selected video file to the server.

[0058] Step 4:

[0059] Server: Receives uploaded video data.

[0060] Step 5:

[0061] Server: Calls the generative AI model and analyzes the video data frame by frame.

[0062] Step 6:

[0063] Server: As a result of the analysis, it generates numerical data on the speed, accuracy, trajectory, etc. of the movement.

[0064] Step 7:

[0065] Server: Generates a performance report based on the analysis results.

[0066] Step 8:

[0067] Server: Sends the generated performance report to the terminal.

[0068] Step 9:

[0069] Terminal: Displays the received performance report to the user.

[0070] Processing steps for motion capture analysis

[0071] Step 1:

[0072] User: Wears a motion capture device to collect movement data during practice and games.

[0073] Step 2:

[0074] Terminal: Transfers movement data from the motion capture device to the terminal.

[0075] Step 3:

[0076] Terminal: Select the transferred operation data on the system's upload screen.

[0077] Step 4:

[0078] Terminal: Uploads the selected motion data to the server.

[0079] Step 5:

[0080] Server: Receives uploaded operational data.

[0081] Step 6:

[0082] Server: Calls the generative AI model and analyzes the behavior data.

[0083] Step 7:

[0084] Server: As a result of the analysis, problems with movement patterns and biomechanics are identified.

[0085] Step 8:

[0086] Server: Generates a motion capture report based on the analysis results.

[0087] Step 9:

[0088] Server: Sends the generated reports to the device.

[0089] Step 10:

[0090] Terminal: Displays received reports to the user.

[0091] Recovery time and performance prediction processing steps

[0092] Step 1:

[0093] User: Enter detailed data about the injury (diagnosis results, treatment progress, etc.) into the terminal.

[0094] Step 2:

[0095] Terminal: Select the entered data on the system's upload screen.

[0096] Step 3:

[0097] Terminal: Uploads the selected injury data to the server.

[0098] Step 4:

[0099] Server: Receives the uploaded data.

[0100] Step 5:

[0101] Server: Calls the generative AI model and analyzes injury data.

[0102] Step 6:

[0103] Server: The analysis results generate a recovery time and performance forecast.

[0104] Step 7:

[0105] Server: Generates a feedback report based on the analysis results.

[0106] Step 8:

[0107] Server: Sends the generated feedback report to the terminal.

[0108] Step 9:

[0109] Terminal: Displays the received feedback report to the user.

[0110] Example 1

[0111] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0112] Maximizing athletes' athletic performance, preventing injuries, and efficiently recovering from existing injuries are extremely important in the field of sports. However, conventional methods for analyzing athletes' movements, preventing injuries, and predicting recovery rely on expert knowledge and manual work, resulting in insufficient accuracy and efficiency. To solve these issues, a system is needed that can efficiently and accurately analyze athletes' movement and injury data and quickly provide scientifically based reports.

[0113] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0114] In this invention, the server includes means for acquiring motion video data of the player from an input device, means for receiving the acquired motion video data with a central processing unit, means for analyzing the received data using a generative AI model and quantifying motion performance, and means for generating a motion performance report based on the analysis results and sending it to the user device. This enables scientific and efficient analysis of the player's motion and detailed feedback for performance improvement and injury prevention.

[0115] The system also includes a means for acquiring motion capture data of the player from an input device, a means for receiving the acquired motion capture data with a central processing unit, a means for analyzing the received data using a generative AI model to identify problems with movement patterns and biomechanics, and a means for generating a report on the movement patterns and biomechanics problems based on the analysis results and sending it to the user device, thereby enabling accurate analysis of the player's movements and helping to improve movements and performance.

[0116] The system also includes means for acquiring data on the player's injury from an input device, means for receiving the acquired injury data with a central processing unit, means for analyzing the received data using a generative AI model to predict recovery time and performance, and means for generating a report on the recovery time and performance prediction based on the analysis results and sending it to the user device, thereby making it possible to streamline the injury recovery process and provide a scientific rehabilitation program.

[0117] "Motion video data" refers to digital video data that records an athlete actually exercising.

[0118] An "input device" is a device used by a user to collect data, including a smartphone, tablet, motion capture device, etc.

[0119] A "central processing unit" is a server computer or cloud platform that receives and analyzes uploaded data.

[0120] "Generative AI models" refer to algorithms and their implementations for analyzing data using machine learning and deep learning, including those using frameworks such as TensorFlow and PyTorch.

[0121] "Movement performance" is an evaluation index related to an athlete's motor functions, such as athletic ability, technical accuracy, and speed.

[0122] A "report" is a document or digital file that presents the analyzed results to the user in an easy-to-understand manner, detailing performance, problems, areas for improvement, etc.

[0123] "Motion capture data" refers to digital data recorded using a motion capture device, which represents the movements of each part of the player's body in three-dimensional coordinates.

[0124] "Movement patterns" refer to the repetition and habitual movements of an athlete, and are understood by analyzing the trajectory and timing of the movements.

[0125] "Biomechanics" is an academic field that deals with the movement and mechanical characteristics of the human body, and collectively refers to the movement of an athlete's body, the angles of their joints, and the function of their muscles during exercise.

[0126] "Injury data" refers to digital data detailing the injury sustained by a player, including information such as diagnosis, treatment progress, and pain level.

[0127] "Recovery time" refers to the period of time required for an athlete to fully recover from an injury and return to their previous performance.

[0128] "Performance prediction" refers to the analysis results that predict the likely improvement in an athlete's athletic ability and technique after recovery from an injury.

[0129] The present invention provides a system for maximizing an athlete's athletic performance, preventing injuries, and supporting efficient rehabilitation. Specific embodiments will be described below.

[0130] Performance Analysis System

[0131] First, a user uses an input device such as a smartphone or tablet to record video data of the player's movements. The recorded video data is then uploaded to a server via an application on the device. The server receives the uploaded video data and analyzes it using a generative AI model (e.g., TensorFlow or PyTorch).

[0132] Specifically, the system analyzes each frame of video data and quantifies the speed, accuracy, and trajectory of the player's movements. The server automatically generates a detailed performance report based on the analysis results and sends it to the device. The user can then review the report on their device and learn how to improve their movements.

[0133] Examples:

[0134] A user records a video of themselves practicing dribbling a basketball and uploads it to the system.

[0135] The system quantifies the speed, accuracy, and quickness of changes in direction of dribbling movements and provides an overall evaluation.

[0136] At the end, users receive a report detailing performance trends and areas for improvement.

[0137] Example prompt sentence:

[0138] "Analyze video of basketball dribbling practice, evaluate the speed, accuracy, and change of direction of the movement, and provide a detailed report of the results."

[0139] Motion Capture Analysis System

[0140] Next, in motion capture analysis, the user uses a motion capture device (e.g., Vicon) to capture the player's movements. The collected motion capture data is stored on the device and later uploaded to a server. The server analyzes the received data using a generative AI model (e.g., PyTorch).

[0141] The server analyzes and identifies problems in the player's movement patterns and biomechanics. Based on the analysis results, a detailed report is generated and sent to the device. The user can then review the report to understand where to improve the movement patterns and provide guidance.

[0142] Examples:

[0143] The user has the athlete wear a motion capture device to collect data for the purpose of analyzing their running form.

[0144] The system analyzes the movement patterns of the left and right feet, knee angles, and ankle movements to identify problems.

[0145] Users receive guidance based on a report containing detailed analysis results and areas for improvement.

[0146] Example prompt sentence:

[0147] "Analyze the motion capture data of the running form, evaluate the movement patterns of the left and right feet, knee angles, and ankle movements, and provide a detailed report of the results."

[0148] Recovery Time and Performance Prediction System

[0149] Furthermore, if a player is injured, the user inputs detailed data about the injury (diagnosis results, treatment progress, etc.) into the device. The input data is uploaded to the server, which receives it and analyzes it using a generative AI model (e.g., IBM Watson).

[0150] Through analysis, the server predicts injury recovery time and performance. Based on the analysis results, a feedback report is generated that provides a predicted recovery time and an optimal rehabilitation program, and is sent to the device. The user can then review the feedback report and create an efficient rehabilitation plan.

[0151] Examples:

[0152] A user enters injury data for a player who has a sprained ankle into the system.

[0153] The system compares similar past data to predict recovery time and provide a rehabilitation program.

[0154] Based on this, users can create a rehabilitation plan and efficiently progress with recovery.

[0155] Example prompt sentence:

[0156] "Based on detailed data on ankle sprains, predict recovery time and optimal rehabilitation program, and provide a detailed report of the results."

[0157] These embodiments make it possible to realize a system that scientifically supports athletes in improving their athletic performance, preventing injuries, and recovering efficiently.

[0158] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0159] Performance Analysis System

[0160] Step 1: Record your video

[0161] Users use input devices such as smartphones and tablets to record video data of athletes' movements.

[0162] Specific operations: Record practice and game situations and save them on the device.

[0163] Input: Player action scene

[0164] Output: Operation video data (MP4 format, etc.)

[0165] Step 2: Upload your video

[0166] The user uploads the recorded video data to the server via an application on the device.

[0167] Specific operation: Press the upload button in the application, select the video file and send it.

[0168] Input: Motion video data

[0169] Output: Video data sent to the server

[0170] Step 3: Receiving video data

[0171] The server receives the uploaded video data.

[0172] Specific behavior: Receives an HTTP request and saves the video file.

[0173] Input: Video data sent to the server

[0174] Output: Saved video file

[0175] Step 4: Analyze the video data

[0176] The server uses a generative AI model to analyze the received video data.

[0177] Specific actions: Analyze video frame by frame to quantify the speed, accuracy, and trajectory of movements.

[0178] Input: Saved video file

[0179] Output: Analysis result data (JSON format, etc.)

[0180] Step 5: Generate a performance report

[0181] The server generates a performance report based on the analysis results.

[0182] Specific operation: Automatically generate a report using analysis result data and save it in PDF or HTML format.

[0183] Input: Analysis result data

[0184] Output: Performance report

[0185] Step 6: Submit the report

[0186] The server transmits the generated performance report to the terminal.

[0187] What it does: Sends reports via email or a dedicated application.

[0188] Input: Performance Report

[0189] Output: Report sent to terminal

[0190] Step 7: Review the report

[0191] The user checks the sent report on the terminal.

[0192] What it does: Open the report you receive, review it, and learn how to improve it.

[0193] Input: Report sent to terminal

[0194] Output: Information to confirm and implement improvements.

[0195] Motion Capture Analysis System

[0196] Step 1: Collecting motion capture data

[0197] Users use motion capture devices to record the movements of players.

[0198] Specific movements: The athletes wear markers and their movements are recorded using a motion capture device.

[0199] Input: Player action scene

[0200] Output: Motion capture data (CSV, JSON, etc. formats)

[0201] Step 2: Upload your data

[0202] The user saves the collected motion capture data on the device and uploads it to the server.

[0203] Specific operation: Press the upload button in the application to send the data file to the server.

[0204] Input: Motion capture data

[0205] Output: Operational data sent to the server

[0206] Step 3: Receiving the data

[0207] The server receives the uploaded motion capture data.

[0208] Specific behavior: Receives an HTTP request and saves a data file.

[0209] Input: Operational data sent to the server

[0210] Output: Saved data file

[0211] Step 4: Analyze the data

[0212] The server uses a generative AI model to analyze the received motion capture data.

[0213] Specific Movements: Analyze data to identify movement patterns and biomechanical issues.

[0214] Input: Saved data file

[0215] Output: Analysis result data (JSON format, etc.)

[0216] Step 5: Generate reports

[0217] The server generates a report on movement patterns and biomechanical issues based on the analysis results.

[0218] Specific operation: Automatically generate a report using analysis result data and save it in PDF or HTML format.

[0219] Input: Analysis result data

[0220] Output: Movement pattern and biomechanics problem report

[0221] Step 6: Submit the report

[0222] The server sends the generated report to the terminal.

[0223] What it does: Sends reports via email or a dedicated application.

[0224] Input: Movement Pattern and Biomechanics Problem Report

[0225] Output: Report sent to terminal

[0226] Step 7: Review the report

[0227] The user checks the sent report on the terminal.

[0228] What it does: Open the report you receive, review it, and learn how to improve it.

[0229] Input: Report sent to terminal

[0230] Output: Information to confirm and implement improvements.

[0231] Recovery Time and Performance Prediction System

[0232] Step 1: Enter injury data

[0233] The user enters details about the player's injury into the terminal.

[0234] Specific operations: Enter information such as diagnosis results and treatment progress into a data entry form.

[0235] Input: Detailed injury information

[0236] Output: Input injury data

[0237] Step 2: Upload your data

[0238] The user uploads the injury data they entered to the server.

[0239] Specific operation: Press the upload button in the application to send the data.

[0240] Input: Injury data

[0241] Output: Injury data sent to the server

[0242] Step 3: Receiving the data

[0243] The server receives the uploaded injury data.

[0244] Specific behavior: Receives an HTTP request and saves a data file.

[0245] Input: Injury data sent to server

[0246] Output: Saved data file

[0247] Step 4: Analyze the data

[0248] The server uses a generative AI model to analyze the injury data it receives.

[0249] Specific actions: Compare with similar cases from the past, predict recovery time and suggest optimal rehabilitation programs.

[0250] Input: Stored injury data

[0251] Output: Analysis result data (JSON format, etc.)

[0252] Step 5: Generate a feedback report

[0253] The server generates a feedback report based on the analysis results.

[0254] Specific operation: Automatically generate a report using analysis result data and save it in PDF or HTML format.

[0255] Input: Analysis result data

[0256] Output: Feedback report

[0257] Step 6: Submit the report

[0258] The server transmits the generated feedback report to the terminal.

[0259] What it does: Sends reports via email or a dedicated application.

[0260] Input: Feedback Report

[0261] Output: Report sent to terminal

[0262] Step 7: Review the report

[0263] The user checks the sent report on the terminal and creates an efficient rehabilitation plan.

[0264] Specific actions: Open the received report, review the contents, and develop a rehabilitation plan.

[0265] Input: Report sent to terminal

[0266] Output: Efficient rehabilitation plan

[0267] (Application example 1)

[0268] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0269] The goal is to maximize the operational efficiency of machinery operating in factories and to detect signs of breakdowns and prevent them before they occur. With conventional methods, it has been difficult to analyze machine performance in real time and carry out appropriate maintenance.

[0270] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0271] In this invention, the server includes a means for receiving machine operation video data, a means for analyzing the received operation video data using a generative AI model to quantify operation efficiency, and a means for generating and providing an operation efficiency report based on the analysis results, thereby enabling scientific analysis of the machine's operation efficiency and identifying maintenance and improvement points at appropriate times.

[0272] "Machines" are automated devices and robots used in factories and other places to process, assemble, and transport objects.

[0273] "Operation video data" refers to video data that records the operation of a machine and is uploaded to a server for analysis.

[0274] A "generative AI model" is an algorithm that uses machine learning and deep learning techniques and is trained to perform data analysis.

[0275] "Operational efficiency" is an index that evaluates the energy consumption and time required when a machine performs a specified task.

[0276] The "means for receiving video data" is an interface that allows the server to receive video data from the outside.

[0277] The "means for quantifying motion efficiency" is a process of analyzing received video data and converting parameters related to motion into specific numerical values.

[0278] The "means for generating and providing a report" refers to a process for creating a report summarizing evaluation information regarding the operation of the machine based on the analyzed data and providing the report to the user.

[0279] The system for implementing this invention analyzes the operational efficiency of machines in a factory, and provides failure prevention and maintenance support. The system operates through the cooperation of three parties: a server, a terminal, and a user.

[0280] System configuration

[0281] The system includes the following hardware and software:

[0282] server

[0283] Video analysis and data processing are performed on the server side, mainly using OpenCV, Keras, and TensorFlow.

[0284] Terminal

[0285] Used to collect and upload motion video data. Applies to smartphones and head-mounted displays.

[0286] User

[0287] These are factory managers and operators who perform maintenance and improvements on machines based on reports provided by the system.

[0288] Program processing

[0289] The server first receives video data of the machine's operation sent from the device. This data is broken down into frames using OpenCV, and image preprocessing is performed. The preprocessed data is then analyzed using a generative AI model using Keras and TensorFlow. Here, parameters related to operational efficiency (speed, accuracy, movement trajectory, etc.) are quantified.

[0290] Based on the analysis results, the server generates a detailed report that includes not only an evaluation of operational efficiency but also any identified mechanical issues and suggestions for improvement, and the report is sent to the terminal for review by the user.

[0291] Users can plan and implement machine maintenance and operational improvements based on the report content. In particular, if signs of a malfunction are detected, it is necessary to carry out preventative maintenance as soon as possible.

[0292] Specific examples

[0293] Factory managers use their smartphones to record videos of robots operating in the factory and upload them to a dedicated app. The server receives the videos, analyzes operational efficiency and mechanical problems, generates a report, and sends it to the manager's device. The manager can then check the detailed report on the app and plan and implement improvements to the robot's operation and maintenance.

[0294] Prompt Sentence Examples

[0295] "Just take a video of your robots operating in your factory and upload it to the app. The system will analyze the video and provide you with a detailed report on the robots' speed, accuracy, and movement trajectory. You can then identify areas for improvement and carry out appropriate maintenance."

[0296] As described above, the system for carrying out the present invention scientifically analyzes the operation of machines in a factory and supports the prevention of breakdowns and the improvement of operation.

[0297] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0298] Step 1:

[0299] The device captures video of the machines in operation in the factory. The input is the video of the machines in operation, and the output is the captured video data. The user operates the device using a smartphone or head-mounted display.

[0300] Step 2:

[0301] The device uploads the captured video data to the server. The input is the captured video data, and the output is the video data stored on the server.

[0302] Step 3:

[0303] The server receives the video data and decomposes the video into frames using OpenCV. The input is the received video data, and the output is individual frame images for preprocessing.

[0304] Step 4:

[0305] The server performs image preprocessing (resizing, normalization) on each frame image. The input is the decomposed frame image, and the output is the preprocessed frame image.

[0306] Step 5:

[0307] The server inputs the preprocessed frame images into a generative AI model using Keras and TensorFlow to analyze parameters related to movement efficiency. The input is the preprocessed frame images, and the output is the analysis results related to movement efficiency (speed, accuracy, movement trajectory, etc.).

[0308] Step 6:

[0309] The server generates a detailed report based on the analysis results. The input is the analyzed data, and the output is a report that includes an evaluation of operational efficiency, problems, and improvement suggestions.

[0310] Step 7:

[0311] The server sends the generated report to the terminal, where the input is the generated report and the output is the report displayed on the terminal.

[0312] Step 8:

[0313] The user checks the report received on the terminal and plans and implements machine maintenance and operational improvements. The input is the report displayed on the terminal, and the output is the implementation of specific maintenance and improvement measures.

[0314] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0315] This system maximizes athletes' athletic performance and prevents injuries. It analyzes athletes' motion video data and motion capture data using a generative AI model and generates detailed reports to help improve athletes' performance and prevent injuries. By combining this with an emotion engine that recognizes athletes' emotions, it is possible to provide feedback that takes into account the user's emotional state. Furthermore, in the event of an injury, it is possible to analyze injury data and predict recovery time and performance.

[0316] Performance Analysis System

[0317] The system first receives video data of the player's movements. The user records video on their device during practice or a match and uploads it to the system. The server receives the video data and performs a detailed analysis using a generative AI model, quantifying the speed, accuracy, and trajectory of the movement. A performance report is generated based on the analysis results and sent to the device. Furthermore, an emotion engine is used to analyze the user's emotional state and reflect this in the feedback. The user can review the report and learn how to improve their own movements.

[0318] For example, when a player uploads a video of himself practicing dribbling a basketball to the system, the system quantifies the speed, accuracy, and speed of changes in direction of the dribble, and provides the user with a comprehensive evaluation report. In addition, the emotion engine evaluates how the user feels about the practice and provides advice to improve motivation.

[0319] Motion Capture Analysis System

[0320] In motion capture analysis, a player's movements are collected using a motion capture device. The movement data transferred from the device is stored on the device and uploaded to the system. The server analyzes the received data using a generative AI model to identify the player's movement patterns and biomechanical issues. A report is generated based on this and sent to the device. Furthermore, an emotion engine is used to analyze the user's emotional state and reflect it in the report. The user can review the report and receive guidance to improve their movement patterns.

[0321] For example, an athlete wears a motion capture device to record their running form and collect data. The system analyzes the movement patterns of the left and right feet, the knee angle, and the movement of the ankle, identifies problems, and provides a report. In addition, an emotion engine takes into account the user's emotional state and provides advice to improve training methods.

[0322] Recovery Time and Performance Prediction System

[0323] When an athlete is injured, detailed injury data (diagnosis results, treatment progress, etc.) is uploaded from the device to the system. The server receives this data and analyzes it using a generative AI model. Based on the analysis results, a predicted recovery time and an optimal rehabilitation program are provided. A feedback report is sent to the device, and the user can use this to promote efficient recovery. The system also takes the user's emotional state into account using an emotion engine to provide advice on maintaining motivation.

[0324] For example, if an athlete sprains their ankle, detailed injury data is entered into the system. The system compares this data with similar past data and predicts recovery time and rehabilitation programs. Based on this, the user can create a rehabilitation plan and progress with their recovery. Furthermore, an emotion engine evaluates the user's emotional state and provides advice to improve motivation during the rehabilitation period.

[0325] This invention will scientifically support athletes in improving their performance and preventing injuries, enabling efficient rehabilitation. Furthermore, by combining it with an emotion engine, it will be possible to provide comprehensive feedback that takes into account the user's emotional state, helping to maintain and improve the athlete's motivation. This is expected to provide effective support to a wide range of people, from professional athletes to students.

[0326] The processing flow will be explained below.

[0327] Performance analysis process steps

[0328] Step 1:

[0329] User: Record videos of practice and games and save them on the device.

[0330] Step 2:

[0331] Device: Open the video upload screen and select the saved video file.

[0332] Step 3:

[0333] On your device: Upload the selected video file to the server.

[0334] Step 4:

[0335] Server: Receives uploaded video data.

[0336] Step 5:

[0337] Server: Calls the generative AI model and analyzes the video data frame by frame.

[0338] Step 6:

[0339] Server: As a result of the analysis, it generates numerical data on the speed, accuracy, trajectory, etc. of the movement.

[0340] Step 7:

[0341] Server: Generates a performance report based on the analysis results.

[0342] Step 8:

[0343] Server: Calls the emotion engine and collects additional data to analyze the user's emotional state.

[0344] Step 9:

[0345] Server: The emotion engine analyzes the user's facial expressions and voice data to identify the user's emotional state.

[0346] Step 10:

[0347] Server: Generates a final performance report with adjusted feedback based on the emotional state.

[0348] Step 11:

[0349] Server: Sends the generated final performance report to the terminal.

[0350] Step 12:

[0351] Terminal: Displays the received performance report to the user.

[0352] Processing steps for motion capture analysis

[0353] Step 1:

[0354] User: Wears a motion capture device to collect movement data during practice and games.

[0355] Step 2:

[0356] Terminal: Transfers movement data from the motion capture device to the terminal.

[0357] Step 3:

[0358] Terminal: Select the transferred operation data on the system's upload screen.

[0359] Step 4:

[0360] Terminal: Uploads the selected motion data to the server.

[0361] Step 5:

[0362] Server: Receives uploaded operational data.

[0363] Step 6:

[0364] Server: Calls the generative AI model and analyzes the behavior data.

[0365] Step 7:

[0366] Server: As a result of the analysis, problems with movement patterns and biomechanics are identified.

[0367] Step 8:

[0368] Server: Generates a motion capture report based on the analysis results.

[0369] Step 9:

[0370] Server: Calls the emotion engine and collects additional data to analyze the user's emotional state.

[0371] Step 10:

[0372] Server: The emotion engine analyzes the user's facial expressions and voice data to identify the user's emotional state.

[0373] Step 11:

[0374] Server: Generates the final motion capture report with adjusted feedback based on the emotional state.

[0375] Step 12:

[0376] Server: Sends the generated final motion capture report to the device.

[0377] Step 13:

[0378] Terminal: Displays received reports to the user.

[0379] Recovery time and performance prediction processing steps

[0380] Step 1:

[0381] User: Enter detailed data about the injury (diagnosis results, treatment progress, etc.) into the terminal.

[0382] Step 2:

[0383] Terminal: Select the entered data on the system's upload screen.

[0384] Step 3:

[0385] Terminal: Uploads the selected injury data to the server.

[0386] Step 4:

[0387] Server: Receives the uploaded data.

[0388] Step 5:

[0389] Server: Calls the generative AI model and analyzes injury data.

[0390] Step 6:

[0391] Server: The analysis results generate a recovery time and performance forecast.

[0392] Step 7:

[0393] Server: Generates a feedback report based on the analysis results.

[0394] Step 8:

[0395] Server: Calls the emotion engine and collects additional data to analyze the user's emotional state.

[0396] Step 9:

[0397] Server: The emotion engine analyzes the user's facial expressions and voice data to identify the user's emotional state.

[0398] Step 10:

[0399] Server: Generates a final feedback report with adjusted feedback content based on the emotional state.

[0400] Step 11:

[0401] Server: Sends the generated final feedback report to the terminal.

[0402] Step 12:

[0403] Terminal: Displays the received feedback report to the user.

[0404] Example 2

[0405] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0406] Conventional athletic performance analysis systems are required not only to analyze athletes' movements but also to provide feedback that takes into account the athletes' emotional state. Current technology is capable of analyzing movement performance and predicting injury recovery, but it lacks the ability to appropriately evaluate athletes' motivation and emotional state and reflect them in feedback. As a result, the advice athletes receive is one-sided, limiting its effectiveness in improving overall performance and preventing injuries. To solve these problems, a system that combines athletes' movement analysis and emotional analysis is needed.

[0407] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving motion video data of a player, means for analyzing the received motion video data using a generative AI model and quantifying motion performance, means for generating and providing a report on the motion performance based on the analysis results, and means for evaluating the user's emotions using an emotion engine that analyzes the user's emotional state and reflecting the evaluation in the feedback content. This makes it possible to provide comprehensive feedback by combining the analysis results of the player's motion performance with emotion analysis.

[0408] "Motion video data" refers to video data that records the movements of players during practice or a match.

[0409] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to analyze input data and generate results.

[0410] "Movement performance" is a numerical evaluation of indicators such as the speed, accuracy, and trajectory of a player's movements.

[0411] A "Performance Report" is a document summarizing the results of analyzed operational performance, including suggestions for improvement and feedback.

[0412] An "emotion engine" is an algorithm that analyzes human emotions and is a system that evaluates emotional states based on facial expressions, voice, etc.

[0413] A "motion capture device" is a device that records the body movements of athletes in 3D, obtaining data on the movements of joints and various body parts.

[0414] Biomechanics is an academic field that deals with the movement and dynamics of the human body, and involves the scientific analysis of athletes' body movements.

[0415] "Injury Data" means detailed information about an injury sustained by a player, including diagnosis and treatment progress.

[0416] "Recovery time" is the estimated time it will take for an athlete to fully recover from an injury.

[0417] A "rehabilitation program" is an exercise and treatment plan designed to help an athlete recover from an injury and is provided to help the athlete recover efficiently.

[0418] "Feedback" refers to advice and instruction provided to players based on the results and evaluations analyzed by the system.

[0419] This invention is a system for maximizing athletes' athletic performance and preventing injuries. The system uses a generative AI model to analyze video data and motion capture data of athletes' movements and generate detailed reports to help athletes improve their performance and prevent injuries. Furthermore, by combining it with an emotion engine that recognizes athletes' emotions, it is possible to provide feedback that takes into account the user's emotional state.

[0420] Performance Analysis System

[0421] The system first receives video data of players' movements. Users film videos during practice or games using their devices (e.g., smartphones or tablets) and upload them to the system. The server receives the video data and performs detailed analysis using a generative AI model (e.g., TensorFlow or PyTorch). A high-performance server (e.g., an NVIDIA GPU or Intel processor) is used for the analysis. The speed, accuracy, trajectory, etc. of the movements in the video are quantified and a performance report is generated. Furthermore, an emotion engine is used to analyze the user's emotional state from their facial expressions and voice in the video and reflect this in the feedback content. For example, if a player uploads a video of themselves practicing dribbling a basketball to the system, the server will quantify the speed, accuracy, and quickness of the dribbling movement and provide an overall evaluation in the form of a report. The emotion engine can also be used to evaluate the user's emotional state and provide advice to improve motivation.

[0422] Example prompt sentence:

[0423] "Analyze the dribbling movements of the players in this video and quantify their speed, accuracy, and change of direction. Also, rate the players' emotional state."

[0424] Motion Capture Analysis System

[0425] In motion capture analysis, an athlete's movements are collected using a motion capture device (e.g., Vicon or OptiTrack). The movement data transferred from the device is stored on the device and uploaded to the system. The server analyzes the received data using a generative AI model to identify the athlete's movement patterns and biomechanical problems. A report is compiled into graphs and tables using a Python script and sent to the user's device. The emotion engine also analyzes the user's emotional state and reflects it in the report content. As a specific example, an athlete wears a motion capture device to record their running form and collects data. The server analyzes the movement patterns of each foot, knee angle, and ankle movement, and provides a report identifying problems. The emotion engine also takes the user's emotional state into account and provides advice for improving training methods.

[0426] Example prompt sentence:

[0427] "Based on this motion capture data, please analyze the athlete's running form and quantify the movement patterns of the left and right feet, the angle of the knees, and the movement of the ankles."

[0428] Recovery Time and Performance Prediction System

[0429] When an athlete is injured, detailed injury data (diagnosis results, treatment progress, etc.) is uploaded from the device to the system. The server receives this data and analyzes it using a generative AI model. Based on the analysis results, a predicted recovery time and an optimal rehabilitation program are provided. A feedback report detailing the analysis results is generated using a Python script and sent to the user's device. An emotion engine is used to analyze the user's emotional state and provide advice to maintain motivation during the rehabilitation period. As a specific example, if an athlete sprains their ankle, detailed injury data is entered into the system, and the server compares it with similar past data and predicts and provides a recovery time and rehabilitation program. The emotion engine also evaluates the user's emotional state and provides advice to maintain motivation.

[0430] Example prompt sentence:

[0431] "Based on this injury data, predict recovery time and the optimal rehabilitation program. Also, assess the user's emotional state."

[0432] This invention makes it possible to comprehensively manage an athlete's movements, recovery, and emotional state, providing optimal support for improving performance and preventing injuries. This is expected to provide effective support to a wide range of people, from professional athletes to students.

[0433] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0434] Performance Analysis System

[0435] Step 1:

[0436] Users use their devices to record the movements of players during practice and games. By recording using a device, it is possible to record the movements of players in real time. Specifically, videos are taken using the camera function of a smartphone or tablet.

[0437] Input: Video of player in action

[0438] Output: Recorded video file

[0439] Step 2:

[0440] Users upload the videos they have taken to the system by sending the video files to the server via a dedicated application or web portal.

[0441] Input: Recorded video file

[0442] Output: Video data stored on a cloud server

[0443] Step 3:

[0444] The server receives the video data sent by the user, and the received video data is stored in cloud storage.

[0445] Input: Video data uploaded to the cloud server

[0446] Output: Video data stored in cloud storage

[0447] Step 4:

[0448] The server inputs the received video into a generative AI model for analysis. The model is built using TensorFlow and PyTorch, and converts the speed, accuracy, and trajectory of the movements in the video into numerical data.

[0449] Input: Video data stored in cloud storage

[0450] Output: Numerical data of movement (speed, accuracy, trajectory)

[0451] Step 5:

[0452] The server generates detailed performance reports based on the analysis data, including performance metrics and areas for improvement. Using Python scripts, the results are visualized in graphs and tables and compiled in PDF format.

[0453] Input: Numerical data of the action

[0454] Output: Performance Report (PDF)

[0455] Step 6:

[0456] The server then sends the generated performance report to the user's device, and the user is notified of the report via a dedicated application or email.

[0457] Input: Performance Report

[0458] Output: Report notification delivered to the user's device

[0459] Step 7:

[0460] The server uses an emotion engine to analyze the user's emotional state in the video. Based on the analysis results, the server adds advice to improve motivation to the report.

[0461] Input: Video data

[0462] Output: Sentiment analysis results, feedback advice

[0463] Motion Capture Analysis System

[0464] Step 1:

[0465] Users collect movement data using motion capture devices, such as Vicon or OptiTrack, which record the movements of the athlete's joints and body as 3D coordinate data.

[0466] Input: Player movement

[0467] Output: Motion capture data (3D coordinate data)

[0468] Step 2:

[0469] The user saves the collected operation data on the device and uploads it to the system, where it is sent to the cloud server via a PC.

[0470] Input: Motion capture data

[0471] Output: Operation data stored on the cloud server

[0472] Step 3:

[0473] The server receives the data sent from the motion capture device and stores it in cloud storage.

[0474] Input: Motion data uploaded to the cloud server

[0475] Output: Data stored in cloud storage

[0476] Step 4:

[0477] The server analyzes the movement data using generative AI models, leveraging ML models such as Scikit-learn to identify movement patterns and biomechanical issues.

[0478] Input: Motion data stored in cloud storage

[0479] Power output: movement patterns and biomechanics issues

[0480] Step 5:

[0481] The server generates a report based on the analysis results. Using a Python script, the analysis results are visualized and compiled into a PDF report.

[0482] Input: Movement patterns and biomechanical issues

[0483] Output: Movement pattern and biomechanics report (PDF)

[0484] Step 6:

[0485] The server sends the generated report to the user's device, and the user is notified of the report via a dedicated application or email.

[0486] Input: Movement patterns and biomechanics reports

[0487] Output: Report notification delivered to the user's device

[0488] Step 7:

[0489] The server uses an emotion engine to analyze the user's emotional state and provides advice on how to improve the training method based on the analysis results.

[0490] Input: Operation data

[0491] Output: Sentiment analysis results, advice on improving training

[0492] Recovery Time and Performance Prediction System

[0493] Step 1:

[0494] The user inputs detailed injury data (diagnosis results, treatment progress, etc.) into the terminal, enters the data using electronic medical record software, and then exports and uploads it to the system.

[0495] Input: Injury details

[0496] Output: Injury data stored on the device

[0497] Step 2:

[0498] The user uploads the entered data to the system, which then sends the data to the cloud server.

[0499] Input: Injury data stored on the device

[0500] Output: Injury data stored on a cloud server

[0501] Step 3:

[0502] The server receives the uploaded injury data and stores it in a database (e.g., PostgreSQL).

[0503] Input: Injury data uploaded to a cloud server

[0504] Output: Injury data stored in a database

[0505] Step 4:

[0506] The server analyzes injury data using a generative AI model, comparing it with past data to predict recovery time and optimal rehabilitation programs.

[0507] Input: Injury data stored in a database

[0508] Output: Prediction of recovery time and rehabilitation program

[0509] Step 5:

[0510] The server generates a feedback report based on the analysis results. A Python script is used to create a PDF report detailing the recovery process and rehabilitation protocol.

[0511] Input: Prediction of recovery time and rehabilitation program

[0512] Output: Feedback report (PDF)

[0513] Step 6:

[0514] The server sends the generated report to the user's device, and the user is notified of the report via a dedicated application or email.

[0515] Input: Feedback Report

[0516] Output: Report notification delivered to the user's device

[0517] Step 7:

[0518] The server uses an emotion engine to analyze the user's emotional state and adds advice to help maintain motivation during the rehabilitation period.

[0519] Input: Injury data

[0520] Output: Emotion analysis results, advice for maintaining motivation

[0521] This allows the system of the present invention to comprehensively manage an athlete's movements, recovery, and emotional state, providing optimal support for improving performance and preventing injuries.

[0522] (Application example 2)

[0523] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0524] Robots used in factories require regular maintenance to maintain their precision and speed, but current methods make it difficult to efficiently optimize performance and perform preventative maintenance. Furthermore, the emotional state of robot operators tends to be ignored, which can have a negative impact on work efficiency and safety. Furthermore, when abnormal behavior occurs, quickly and accurately identifying the cause and providing an appropriate repair program is also a challenge.

[0525] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0526] In this invention, the server includes: [means for receiving robot operation video data;] [means for analyzing the received operation video data using a generative AI model and quantifying operation performance;] [means for generating and providing an operation performance report based on the analysis results;] [means for reflecting the operator's emotional state in the feedback content using emotion analysis means; and [means for, if the robot performs an abnormal operation, analyzing the abnormal operation data, identifying the cause, and generating a repair program.] This enables optimization of the robot's operation performance, preventive maintenance management, and comprehensive feedback that takes into account the operator's emotional state.

[0527] "Robot operation video data" refers to video information of the operation of a robot used in a factory recorded by a video recording device such as a video camera.

[0528] A "generative AI model" is an artificial intelligence model used to analyze robot motion data and other input data to calculate specific performance metrics.

[0529] "Motion performance" refers to a comprehensive evaluation of various indicators related to work efficiency and quality, such as the accuracy, speed, and trajectory of the movements performed by the robot.

[0530] A "report" is a document or data summarizing the analysis results obtained based on a generative AI model, including quantification of behavioral performance, indications of behavioral patterns, and areas for improvement.

[0531] "Emotion analysis means" refers to technology or devices that analyze the emotional state of operators, allowing for the assessment of their stress levels, motivation, etc.

[0532] "Feedback" refers to suggestions for improvement and advice provided based on the analysis results, allowing the robot and operator to take appropriate measures.

[0533] "Abnormal behavior data" refers to data that records the robot's unexpected or abnormal behavior.

[0534] A "repair program" is a repair procedure or process designed based on the cause of the abnormal behavior, which will restore the robot's normal operation.

[0535] Biomechanics is an academic field that analyzes the mechanical elements of robot movement. It is often applied to the analysis of human movement, but here we will also apply it to the analysis of robot movement.

[0536] To realize this invention, it is first necessary to collect video data and motion capture data of each robot's movements and upload them to a server. Users film the robot's movements in the factory with a video camera or other video recording device, and also record detailed movement patterns using a motion capture device.

[0537] The server analyzes the received motion video data using a dedicated AI model library (for example, "some_AI_model_library"). This analysis quantifies the speed, accuracy, trajectory, etc. of the motion and evaluates detailed motion performance. If abnormal motion occurs, the abnormal motion data is also analyzed to identify the cause and generate an optimal repair program. The results of this analysis are compiled into a report and sent to the user's device.

[0538] Additionally, an emotion analysis tool (e.g., "some_emotion_engine") is used to evaluate the emotional state of the operator. This allows the operator's stress level and motivation to be assessed and reflected in the feedback. The server-generated report also includes advice and areas for improvement based on the operator's emotional state, providing comprehensive feedback.

[0539] As a concrete example, video and motion capture data of a factory robot assembling parts is collected and uploaded to a server. Based on this data, the server quantifies the robot's movement accuracy and speed and generates a detailed report. This report is provided to the operator and includes advice on improving performance and preventative maintenance. If any abnormal behavior occurs, the cause is identified and a repair program is quickly implemented. The operator's emotional state is also taken into consideration, and advice on maintaining motivation is also included.

[0540] An example of a prompt for a generative AI model is, "Analyze the video and motion capture data of the factory robot's movements, quantify the accuracy, speed, and trajectory of the movements, and generate a report. Also, evaluate the operator's emotional state (such as stress level) and include it in the report."

[0541] This system not only enables efficient and effective performance optimization and maintenance management of factory robots, but also provides comprehensive support that takes into account the emotional state of the operator.

[0542] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0543] Step 1:

[0544] The user films the factory robot's movements with a video camera and records detailed movement patterns with a motion capture device.

[0545] Input: Robot motion video data, motion capture data

[0546] Output: Motion video data and motion capture data saved on the device

[0547] Specific movements: The user positions the camera and motion capture devices appropriately to film and record the robot's movements.

[0548] Step 2:

[0549] The video data and motion capture data that the user has filmed and recorded are uploaded to the server.

[0550] Input: Motion video data stored on the device, motion capture data

[0551] Output: Motion video data and motion capture data transferred to the server

[0552] Specific operation: The user transfers data to the server using a dedicated upload tool.

[0553] Step 3:

[0554] The video data received by the server is analyzed using a generative AI model.

[0555] Input: Motion video data stored on the server

[0556] Output: Analysis results such as robot movement speed, accuracy, and trajectory

[0557] Specific operation: The server inputs video data into the AI ​​model, calculates the movement speed and accuracy for each frame, and tracks and quantifies the trajectory.

[0558] Step 4:

[0559] The server analyzes the received motion capture data using a generative AI model to identify movement patterns and biomechanical issues.

[0560] Input: Motion capture data stored on the server

[0561] Output: Movement patterns and biomechanics issues

[0562] Specific movements: The server inputs motion capture data into an AI model, which analyzes the movement of each joint and identifies movement patterns and biomechanical issues.

[0563] Step 5:

[0564] The server generates and provides a report of operational performance based on the analysis results.

[0565] Input: movement speed, accuracy, trajectory, movement patterns, biomechanical issues

[0566] Output: Operational performance report

[0567] What it does: The server consolidates the analysis results and generates a visually comprehensible report, including areas for improvement and recommended actions.

[0568] Step 6:

[0569] The server uses an emotion analysis means to evaluate the emotional state of the operator and provides feedback content.

[0570] Input: Operator emotion data

[0571] Output: Improvements and advice reflecting the operator's emotional state

[0572] Specific operation: The server analyzes the emotional data using the emotion analysis means to assess stress levels and motivation, and includes this in the report.

[0573] Step 7:

[0574] If the robot behaves abnormally, the server analyzes the abnormal behavior data, identifies the cause, and generates a repair program.

[0575] Input: Abnormal behavior data

[0576] Output: Cause identification results, repair program

[0577] Specific operation: The server inputs abnormal operation data into the AI ​​model, uses prompt sentences to identify the cause, and generates repair procedures.

[0578] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0579] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0580] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0581] [Second embodiment]

[0582] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0583] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0584] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0585] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0586] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0587] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0588] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0589] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0590] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0591] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0592] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0593] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0594] This system maximizes athletes' athletic performance and prevents injuries by analyzing video and motion capture data of athletes using a generative AI model and generating detailed reports to help athletes improve their performance and prevent injuries. In the event of an injury, the system can also analyze injury data and predict recovery time and performance.

[0595] Performance Analysis System

[0596] The system first receives video data of the player's movements. The user records video during practice or a match on their device and uploads it to the system. The server receives the video data and performs a detailed analysis using a generative AI model, quantifying the speed, accuracy, and trajectory of the movement. A performance report is generated based on the analysis results and sent to the device. The user can review the report and learn how to improve their own movements.

[0597] As a concrete example, when a player uploads a video of himself practicing dribbling a basketball to the system, the system quantifies the speed, accuracy, and swiftness of the dribbling motion, and provides the user with a report with an overall evaluation.

[0598] Motion Capture Analysis System

[0599] In motion capture analysis, a player's movements are collected using a motion capture device. The movement data transferred from the device is stored on the device and uploaded to the system. The server analyzes the received data using a generative AI model to identify the player's movement patterns and biomechanical problems. A report is generated based on this and sent to the device. The user can review the report and receive guidance to improve their movement patterns.

[0600] For example, an athlete wears a motion capture device to record their running form and collect data. The system analyzes the movement patterns of their left and right feet, knee angles, and ankle movements, identifies problems, and provides a report.

[0601] Recovery Time and Performance Prediction System

[0602] When an athlete is injured, detailed injury data (diagnosis results, treatment progress, etc.) is uploaded from the device to the system. The server receives this data and analyzes it using a generative AI model. Based on the analysis results, a predicted recovery time and an optimal rehabilitation program are provided. A feedback report is sent to the device, allowing the user to use it to efficiently promote recovery.

[0603] For example, if an athlete sprains their ankle, they can input the details of the injury into the system. The system will compare the data with similar data from the past and provide a prediction of recovery time and rehabilitation program. The user can then use this information to create a rehabilitation plan and progress their recovery.

[0604] This invention will scientifically support athletes in improving their performance and preventing injuries, enabling efficient rehabilitation. In the initial stages, the system will partner with schools and other institutions to accumulate experience through trial and error, before being packaged and widely deployed. It is expected that this will enable effective support for a wide range of people, from professional athletes to students.

[0605] The processing flow will be explained below.

[0606] Performance analysis process steps

[0607] Step 1:

[0608] User: Record videos of practice and games and save them on the device.

[0609] Step 2:

[0610] Device: Open the video upload screen and select the saved video file.

[0611] Step 3:

[0612] On your device: Upload the selected video file to the server.

[0613] Step 4:

[0614] Server: Receives uploaded video data.

[0615] Step 5:

[0616] Server: Calls the generative AI model and analyzes the video data frame by frame.

[0617] Step 6:

[0618] Server: As a result of the analysis, it generates numerical data on the speed, accuracy, trajectory, etc. of the movement.

[0619] Step 7:

[0620] Server: Generates a performance report based on the analysis results.

[0621] Step 8:

[0622] Server: Sends the generated performance report to the terminal.

[0623] Step 9:

[0624] Terminal: Displays the received performance report to the user.

[0625] Processing steps for motion capture analysis

[0626] Step 1:

[0627] User: Wears a motion capture device to collect movement data during practice and games.

[0628] Step 2:

[0629] Terminal: Transfers movement data from the motion capture device to the terminal.

[0630] Step 3:

[0631] Terminal: Select the transferred operation data on the system's upload screen.

[0632] Step 4:

[0633] Terminal: Uploads the selected motion data to the server.

[0634] Step 5:

[0635] Server: Receives uploaded operational data.

[0636] Step 6:

[0637] Server: Calls the generative AI model and analyzes the behavior data.

[0638] Step 7:

[0639] Server: As a result of the analysis, problems with movement patterns and biomechanics are identified.

[0640] Step 8:

[0641] Server: Generates a motion capture report based on the analysis results.

[0642] Step 9:

[0643] Server: Sends the generated reports to the device.

[0644] Step 10:

[0645] Terminal: Displays received reports to the user.

[0646] Recovery time and performance prediction processing steps

[0647] Step 1:

[0648] User: Enter detailed data about the injury (diagnosis results, treatment progress, etc.) into the terminal.

[0649] Step 2:

[0650] Terminal: Select the entered data on the system's upload screen.

[0651] Step 3:

[0652] Terminal: Uploads the selected injury data to the server.

[0653] Step 4:

[0654] Server: Receives the uploaded data.

[0655] Step 5:

[0656] Server: Calls the generative AI model and analyzes injury data.

[0657] Step 6:

[0658] Server: The analysis results generate a recovery time and performance forecast.

[0659] Step 7:

[0660] Server: Generates a feedback report based on the analysis results.

[0661] Step 8:

[0662] Server: Sends the generated feedback report to the terminal.

[0663] Step 9:

[0664] Terminal: Displays the received feedback report to the user.

[0665] Example 1

[0666] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0667] Maximizing athletes' athletic performance, preventing injuries, and efficiently recovering from existing injuries are extremely important in the field of sports. However, conventional methods for analyzing athletes' movements, preventing injuries, and predicting recovery rely on expert knowledge and manual work, resulting in insufficient accuracy and efficiency. To solve these issues, a system is needed that can efficiently and accurately analyze athletes' movement and injury data and quickly provide scientifically based reports.

[0668] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0669] In this invention, the server includes means for acquiring motion video data of the player from an input device, means for receiving the acquired motion video data with a central processing unit, means for analyzing the received data using a generative AI model and quantifying motion performance, and means for generating a motion performance report based on the analysis results and sending it to the user device. This enables scientific and efficient analysis of the player's motion and detailed feedback for performance improvement and injury prevention.

[0670] The system also includes a means for acquiring motion capture data of the player from an input device, a means for receiving the acquired motion capture data with a central processing unit, a means for analyzing the received data using a generative AI model to identify problems with movement patterns and biomechanics, and a means for generating a report on the movement patterns and biomechanics problems based on the analysis results and sending it to the user device, thereby enabling accurate analysis of the player's movements and helping to improve movements and performance.

[0671] The system also includes means for acquiring data on the player's injury from an input device, means for receiving the acquired injury data with a central processing unit, means for analyzing the received data using a generative AI model to predict recovery time and performance, and means for generating a report on the recovery time and performance prediction based on the analysis results and sending it to the user device, thereby making it possible to streamline the injury recovery process and provide a scientific rehabilitation program.

[0672] "Motion video data" refers to digital video data that records an athlete actually exercising.

[0673] An "input device" is a device used by a user to collect data, including a smartphone, tablet, motion capture device, etc.

[0674] A "central processing unit" is a server computer or cloud platform that receives and analyzes uploaded data.

[0675] "Generative AI models" refer to algorithms and their implementations for analyzing data using machine learning and deep learning, including those using frameworks such as TensorFlow and PyTorch.

[0676] "Movement performance" is an evaluation index related to an athlete's motor functions, such as athletic ability, technical accuracy, and speed.

[0677] A "report" is a document or digital file that presents the analyzed results to the user in an easy-to-understand manner, detailing performance, problems, areas for improvement, etc.

[0678] "Motion capture data" refers to digital data recorded using a motion capture device, which represents the movements of each part of the player's body in three-dimensional coordinates.

[0679] "Movement patterns" refer to the repetition and habitual movements of an athlete, and are understood by analyzing the trajectory and timing of the movements.

[0680] "Biomechanics" is an academic field that deals with the movement and mechanical characteristics of the human body, and collectively refers to the movement of an athlete's body, the angles of their joints, and the function of their muscles during exercise.

[0681] "Injury data" refers to digital data detailing the injury sustained by a player, including information such as diagnosis, treatment progress, and pain level.

[0682] "Recovery time" refers to the period of time required for an athlete to fully recover from an injury and return to their previous performance.

[0683] "Performance prediction" refers to the analysis results that predict the likely improvement in an athlete's athletic ability and technique after recovery from an injury.

[0684] The present invention provides a system for maximizing an athlete's athletic performance, preventing injuries, and supporting efficient rehabilitation. Specific embodiments will be described below.

[0685] Performance Analysis System

[0686] First, a user uses an input device such as a smartphone or tablet to record video data of the player's movements. The recorded video data is then uploaded to a server via an application on the device. The server receives the uploaded video data and analyzes it using a generative AI model (e.g., TensorFlow or PyTorch).

[0687] Specifically, the system analyzes each frame of video data and quantifies the speed, accuracy, and trajectory of the player's movements. The server automatically generates a detailed performance report based on the analysis results and sends it to the device. The user can then review the report on their device and learn how to improve their movements.

[0688] Examples:

[0689] A user records a video of themselves practicing dribbling a basketball and uploads it to the system.

[0690] The system quantifies the speed, accuracy, and quickness of changes in direction of dribbling movements and provides an overall evaluation.

[0691] At the end, users receive a report detailing performance trends and areas for improvement.

[0692] Example prompt sentence:

[0693] "Analyze video of basketball dribbling practice, evaluate the speed, accuracy, and change of direction of the movement, and provide a detailed report of the results."

[0694] Motion Capture Analysis System

[0695] Next, in motion capture analysis, the user uses a motion capture device (e.g., Vicon) to capture the player's movements. The collected motion capture data is stored on the device and later uploaded to a server. The server analyzes the received data using a generative AI model (e.g., PyTorch).

[0696] The server analyzes and identifies problems in the player's movement patterns and biomechanics. Based on the analysis results, a detailed report is generated and sent to the device. The user can then review the report to understand where to improve the movement patterns and provide guidance.

[0697] Examples:

[0698] The user has the athlete wear a motion capture device to collect data for the purpose of analyzing their running form.

[0699] The system analyzes the movement patterns of the left and right feet, knee angles, and ankle movements to identify problems.

[0700] Users receive guidance based on a report containing detailed analysis results and areas for improvement.

[0701] Example prompt sentence:

[0702] "Analyze the motion capture data of the running form, evaluate the movement patterns of the left and right feet, knee angles, and ankle movements, and provide a detailed report of the results."

[0703] Recovery Time and Performance Prediction System

[0704] Furthermore, if a player is injured, the user inputs detailed data about the injury (diagnosis results, treatment progress, etc.) into the device. The input data is uploaded to the server, which receives it and analyzes it using a generative AI model (e.g., IBM Watson).

[0705] Through analysis, the server predicts injury recovery time and performance. Based on the analysis results, a feedback report is generated that provides a predicted recovery time and an optimal rehabilitation program, and is sent to the device. The user can then review the feedback report and create an efficient rehabilitation plan.

[0706] Examples:

[0707] A user enters injury data for a player who has a sprained ankle into the system.

[0708] The system compares similar past data to predict recovery time and provide a rehabilitation program.

[0709] Based on this, users can create a rehabilitation plan and efficiently progress with recovery.

[0710] Example prompt sentence:

[0711] "Based on detailed data on ankle sprains, predict recovery time and optimal rehabilitation program, and provide a detailed report of the results."

[0712] These embodiments make it possible to realize a system that scientifically supports athletes in improving their athletic performance, preventing injuries, and recovering efficiently.

[0713] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0714] Performance Analysis System

[0715] Step 1: Record your video

[0716] Users use input devices such as smartphones and tablets to record video data of athletes' movements.

[0717] Specific operations: Record practice and game situations and save them on the device.

[0718] Input: Player action scene

[0719] Output: Operation video data (MP4 format, etc.)

[0720] Step 2: Upload your video

[0721] The user uploads the recorded video data to the server via an application on the device.

[0722] Specific operation: Press the upload button in the application, select the video file and send it.

[0723] Input: Motion video data

[0724] Output: Video data sent to the server

[0725] Step 3: Receiving video data

[0726] The server receives the uploaded video data.

[0727] Specific behavior: Receives an HTTP request and saves the video file.

[0728] Input: Video data sent to the server

[0729] Output: Saved video file

[0730] Step 4: Analyze the video data

[0731] The server uses a generative AI model to analyze the received video data.

[0732] Specific actions: Analyze video frame by frame to quantify the speed, accuracy, and trajectory of movements.

[0733] Input: Saved video file

[0734] Output: Analysis result data (JSON format, etc.)

[0735] Step 5: Generate a performance report

[0736] The server generates a performance report based on the analysis results.

[0737] Specific operation: Automatically generate a report using analysis result data and save it in PDF or HTML format.

[0738] Input: Analysis result data

[0739] Output: Performance report

[0740] Step 6: Submit the report

[0741] The server transmits the generated performance report to the terminal.

[0742] What it does: Sends reports via email or a dedicated application.

[0743] Input: Performance Report

[0744] Output: Report sent to terminal

[0745] Step 7: Review the report

[0746] The user checks the sent report on the terminal.

[0747] What it does: Open the report you receive, review it, and learn how to improve it.

[0748] Input: Report sent to terminal

[0749] Output: Information to confirm and implement improvements.

[0750] Motion Capture Analysis System

[0751] Step 1: Collecting motion capture data

[0752] Users use motion capture devices to record the movements of players.

[0753] Specific movements: The athletes wear markers and their movements are recorded using a motion capture device.

[0754] Input: Player action scene

[0755] Output: Motion capture data (CSV, JSON, etc. formats)

[0756] Step 2: Upload your data

[0757] The user saves the collected motion capture data on the device and uploads it to the server.

[0758] Specific operation: Press the upload button in the application to send the data file to the server.

[0759] Input: Motion capture data

[0760] Output: Operational data sent to the server

[0761] Step 3: Receiving the data

[0762] The server receives the uploaded motion capture data.

[0763] Specific behavior: Receives an HTTP request and saves a data file.

[0764] Input: Operational data sent to the server

[0765] Output: Saved data file

[0766] Step 4: Analyze the data

[0767] The server uses a generative AI model to analyze the received motion capture data.

[0768] Specific Movements: Analyze data to identify movement patterns and biomechanical issues.

[0769] Input: Saved data file

[0770] Output: Analysis result data (JSON format, etc.)

[0771] Step 5: Generate reports

[0772] The server generates a report on movement patterns and biomechanical issues based on the analysis results.

[0773] Specific operation: Automatically generate a report using analysis result data and save it in PDF or HTML format.

[0774] Input: Analysis result data

[0775] Output: Movement pattern and biomechanics problem report

[0776] Step 6: Submit the report

[0777] The server sends the generated report to the terminal.

[0778] What it does: Sends reports via email or a dedicated application.

[0779] Input: Movement Pattern and Biomechanics Problem Report

[0780] Output: Report sent to terminal

[0781] Step 7: Review the report

[0782] The user checks the sent report on the terminal.

[0783] What it does: Open the report you receive, review it, and learn how to improve it.

[0784] Input: Report sent to terminal

[0785] Output: Information to confirm and implement improvements.

[0786] Recovery Time and Performance Prediction System

[0787] Step 1: Enter injury data

[0788] The user enters details about the player's injury into the terminal.

[0789] Specific operations: Enter information such as diagnosis results and treatment progress into a data entry form.

[0790] Input: Detailed injury information

[0791] Output: Input injury data

[0792] Step 2: Upload your data

[0793] The user uploads the injury data they entered to the server.

[0794] Specific operation: Press the upload button in the application to send the data.

[0795] Input: Injury data

[0796] Output: Injury data sent to the server

[0797] Step 3: Receiving the data

[0798] The server receives the uploaded injury data.

[0799] Specific behavior: Receives an HTTP request and saves a data file.

[0800] Input: Injury data sent to server

[0801] Output: Saved data file

[0802] Step 4: Analyze the data

[0803] The server uses a generative AI model to analyze the injury data it receives.

[0804] Specific actions: Compare with similar cases from the past, predict recovery time and suggest optimal rehabilitation programs.

[0805] Input: Stored injury data

[0806] Output: Analysis result data (JSON format, etc.)

[0807] Step 5: Generate a feedback report

[0808] The server generates a feedback report based on the analysis results.

[0809] Specific operation: Automatically generate a report using analysis result data and save it in PDF or HTML format.

[0810] Input: Analysis result data

[0811] Output: Feedback report

[0812] Step 6: Submit the report

[0813] The server transmits the generated feedback report to the terminal.

[0814] What it does: Sends reports via email or a dedicated application.

[0815] Input: Feedback Report

[0816] Output: Report sent to terminal

[0817] Step 7: Review the report

[0818] The user checks the sent report on the terminal and creates an efficient rehabilitation plan.

[0819] Specific actions: Open the received report, review the contents, and develop a rehabilitation plan.

[0820] Input: Report sent to terminal

[0821] Output: Efficient rehabilitation plan

[0822] (Application example 1)

[0823] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0824] The goal is to maximize the operational efficiency of machinery operating in factories and to detect signs of breakdowns and prevent them before they occur. With conventional methods, it has been difficult to analyze machine performance in real time and carry out appropriate maintenance.

[0825] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0826] In this invention, the server includes a means for receiving machine operation video data, a means for analyzing the received operation video data using a generative AI model to quantify operation efficiency, and a means for generating and providing an operation efficiency report based on the analysis results, thereby enabling scientific analysis of the machine's operation efficiency and identifying maintenance and improvement points at appropriate times.

[0827] "Machines" are automated devices and robots used in factories and other places to process, assemble, and transport objects.

[0828] "Operation video data" refers to video data that records the operation of a machine and is uploaded to a server for analysis.

[0829] A "generative AI model" is an algorithm that uses machine learning and deep learning techniques and is trained to perform data analysis.

[0830] "Operational efficiency" is an index that evaluates the energy consumption and time required when a machine performs a specified task.

[0831] The "means for receiving video data" is an interface that allows the server to receive video data from the outside.

[0832] The "means for quantifying motion efficiency" is a process of analyzing received video data and converting parameters related to motion into specific numerical values.

[0833] The "means for generating and providing a report" refers to a process for creating a report summarizing evaluation information regarding the operation of the machine based on the analyzed data and providing the report to the user.

[0834] The system for implementing this invention analyzes the operational efficiency of machines in a factory, and provides failure prevention and maintenance support. The system operates through the cooperation of three parties: a server, a terminal, and a user.

[0835] System configuration

[0836] The system includes the following hardware and software:

[0837] server

[0838] Video analysis and data processing are performed on the server side, mainly using OpenCV, Keras, and TensorFlow.

[0839] Terminal

[0840] Used to collect and upload motion video data. Applies to smartphones and head-mounted displays.

[0841] User

[0842] These are factory managers and operators who perform maintenance and improvements on machines based on reports provided by the system.

[0843] Program processing

[0844] The server first receives video data of the machine's operation sent from the device. This data is broken down into frames using OpenCV, and image preprocessing is performed. The preprocessed data is then analyzed using a generative AI model using Keras and TensorFlow. Here, parameters related to operational efficiency (speed, accuracy, movement trajectory, etc.) are quantified.

[0845] Based on the analysis results, the server generates a detailed report that includes not only an evaluation of operational efficiency but also any identified mechanical issues and suggestions for improvement, and the report is sent to the terminal for review by the user.

[0846] Users can plan and implement machine maintenance and operational improvements based on the report content. In particular, if signs of a malfunction are detected, it is necessary to carry out preventative maintenance as soon as possible.

[0847] Specific examples

[0848] Factory managers use their smartphones to record videos of robots operating in the factory and upload them to a dedicated app. The server receives the videos, analyzes operational efficiency and mechanical problems, generates a report, and sends it to the manager's device. The manager can then check the detailed report on the app and plan and implement improvements to the robot's operation and maintenance.

[0849] Prompt Sentence Examples

[0850] "Just take a video of your robots operating in your factory and upload it to the app. The system will analyze the video and provide you with a detailed report on the robots' speed, accuracy, and movement trajectory. You can then identify areas for improvement and carry out appropriate maintenance."

[0851] As described above, the system for carrying out the present invention scientifically analyzes the operation of machines in a factory and supports the prevention of breakdowns and the improvement of operation.

[0852] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0853] Step 1:

[0854] The device captures video of the machines in operation in the factory. The input is the video of the machines in operation, and the output is the captured video data. The user operates the device using a smartphone or head-mounted display.

[0855] Step 2:

[0856] The device uploads the captured video data to the server. The input is the captured video data, and the output is the video data stored on the server.

[0857] Step 3:

[0858] The server receives the video data and decomposes the video into frames using OpenCV. The input is the received video data, and the output is individual frame images for preprocessing.

[0859] Step 4:

[0860] The server performs image preprocessing (resizing, normalization) on each frame image. The input is the decomposed frame image, and the output is the preprocessed frame image.

[0861] Step 5:

[0862] The server inputs the preprocessed frame images into a generative AI model using Keras and TensorFlow to analyze parameters related to movement efficiency. The input is the preprocessed frame images, and the output is the analysis results related to movement efficiency (speed, accuracy, movement trajectory, etc.).

[0863] Step 6:

[0864] The server generates a detailed report based on the analysis results. The input is the analyzed data, and the output is a report that includes an evaluation of operational efficiency, problems, and improvement suggestions.

[0865] Step 7:

[0866] The server sends the generated report to the terminal, where the input is the generated report and the output is the report displayed on the terminal.

[0867] Step 8:

[0868] The user checks the report received on the terminal and plans and implements machine maintenance and operational improvements. The input is the report displayed on the terminal, and the output is the implementation of specific maintenance and improvement measures.

[0869] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0870] This system maximizes athletes' athletic performance and prevents injuries. It analyzes athletes' motion video data and motion capture data using a generative AI model and generates detailed reports to help improve athletes' performance and prevent injuries. By combining this with an emotion engine that recognizes athletes' emotions, it is possible to provide feedback that takes into account the user's emotional state. Furthermore, in the event of an injury, it is possible to analyze injury data and predict recovery time and performance.

[0871] Performance Analysis System

[0872] The system first receives video data of the player's movements. The user records video on their device during practice or a match and uploads it to the system. The server receives the video data and performs a detailed analysis using a generative AI model, quantifying the speed, accuracy, and trajectory of the movement. A performance report is generated based on the analysis results and sent to the device. Furthermore, an emotion engine is used to analyze the user's emotional state and reflect this in the feedback. The user can review the report and learn how to improve their own movements.

[0873] For example, when a player uploads a video of himself practicing dribbling a basketball to the system, the system quantifies the speed, accuracy, and speed of changes in direction of the dribble, and provides the user with a comprehensive evaluation report. In addition, the emotion engine evaluates how the user feels about the practice and provides advice to improve motivation.

[0874] Motion Capture Analysis System

[0875] In motion capture analysis, a player's movements are collected using a motion capture device. The movement data transferred from the device is stored on the device and uploaded to the system. The server analyzes the received data using a generative AI model to identify the player's movement patterns and biomechanical issues. A report is generated based on this and sent to the device. Furthermore, an emotion engine is used to analyze the user's emotional state and reflect it in the report. The user can review the report and receive guidance to improve their movement patterns.

[0876] For example, an athlete wears a motion capture device to record their running form and collect data. The system analyzes the movement patterns of the left and right feet, the knee angle, and the movement of the ankle, identifies problems, and provides a report. In addition, an emotion engine takes into account the user's emotional state and provides advice to improve training methods.

[0877] Recovery Time and Performance Prediction System

[0878] When an athlete is injured, detailed injury data (diagnosis results, treatment progress, etc.) is uploaded from the device to the system. The server receives this data and analyzes it using a generative AI model. Based on the analysis results, a predicted recovery time and an optimal rehabilitation program are provided. A feedback report is sent to the device, and the user can use this to promote efficient recovery. The system also takes the user's emotional state into account using an emotion engine to provide advice on maintaining motivation.

[0879] For example, if an athlete sprains their ankle, detailed injury data is entered into the system. The system compares this data with similar past data and predicts recovery time and rehabilitation programs. Based on this, the user can create a rehabilitation plan and progress with their recovery. Furthermore, an emotion engine evaluates the user's emotional state and provides advice to improve motivation during the rehabilitation period.

[0880] This invention will scientifically support athletes in improving their performance and preventing injuries, enabling efficient rehabilitation. Furthermore, by combining it with an emotion engine, it will be possible to provide comprehensive feedback that takes into account the user's emotional state, helping to maintain and improve the athlete's motivation. This is expected to provide effective support to a wide range of people, from professional athletes to students.

[0881] The processing flow will be explained below.

[0882] Performance analysis process steps

[0883] Step 1:

[0884] User: Record videos of practice and games and save them on the device.

[0885] Step 2:

[0886] Device: Open the video upload screen and select the saved video file.

[0887] Step 3:

[0888] On your device: Upload the selected video file to the server.

[0889] Step 4:

[0890] Server: Receives uploaded video data.

[0891] Step 5:

[0892] Server: Calls the generative AI model and analyzes the video data frame by frame.

[0893] Step 6:

[0894] Server: As a result of the analysis, it generates numerical data on the speed, accuracy, trajectory, etc. of the movement.

[0895] Step 7:

[0896] Server: Generates a performance report based on the analysis results.

[0897] Step 8:

[0898] Server: Calls the emotion engine and collects additional data to analyze the user's emotional state.

[0899] Step 9:

[0900] Server: The emotion engine analyzes the user's facial expressions and voice data to identify the user's emotional state.

[0901] Step 10:

[0902] Server: Generates a final performance report with adjusted feedback based on the emotional state.

[0903] Step 11:

[0904] Server: Sends the generated final performance report to the terminal.

[0905] Step 12:

[0906] Terminal: Displays the received performance report to the user.

[0907] Processing steps for motion capture analysis

[0908] Step 1:

[0909] User: Wears a motion capture device to collect movement data during practice and games.

[0910] Step 2:

[0911] Terminal: Transfers movement data from the motion capture device to the terminal.

[0912] Step 3:

[0913] Terminal: Select the transferred operation data on the system's upload screen.

[0914] Step 4:

[0915] Terminal: Uploads the selected motion data to the server.

[0916] Step 5:

[0917] Server: Receives uploaded operational data.

[0918] Step 6:

[0919] Server: Calls the generative AI model and analyzes the behavior data.

[0920] Step 7:

[0921] Server: As a result of the analysis, problems with movement patterns and biomechanics are identified.

[0922] Step 8:

[0923] Server: Generates a motion capture report based on the analysis results.

[0924] Step 9:

[0925] Server: Calls the emotion engine and collects additional data to analyze the user's emotional state.

[0926] Step 10:

[0927] Server: The emotion engine analyzes the user's facial expressions and voice data to identify the user's emotional state.

[0928] Step 11:

[0929] Server: Generates the final motion capture report with adjusted feedback based on the emotional state.

[0930] Step 12:

[0931] Server: Sends the generated final motion capture report to the device.

[0932] Step 13:

[0933] Terminal: Displays received reports to the user.

[0934] Recovery time and performance prediction processing steps

[0935] Step 1:

[0936] User: Enter detailed data about the injury (diagnosis results, treatment progress, etc.) into the terminal.

[0937] Step 2:

[0938] Terminal: Select the entered data on the system's upload screen.

[0939] Step 3:

[0940] Terminal: Uploads the selected injury data to the server.

[0941] Step 4:

[0942] Server: Receives the uploaded data.

[0943] Step 5:

[0944] Server: Calls the generative AI model and analyzes injury data.

[0945] Step 6:

[0946] Server: The analysis results generate a recovery time and performance forecast.

[0947] Step 7:

[0948] Server: Generates a feedback report based on the analysis results.

[0949] Step 8:

[0950] Server: Calls the emotion engine and collects additional data to analyze the user's emotional state.

[0951] Step 9:

[0952] Server: The emotion engine analyzes the user's facial expressions and voice data to identify the user's emotional state.

[0953] Step 10:

[0954] Server: Generates a final feedback report with adjusted feedback content based on the emotional state.

[0955] Step 11:

[0956] Server: Sends the generated final feedback report to the terminal.

[0957] Step 12:

[0958] Terminal: Displays the received feedback report to the user.

[0959] Example 2

[0960] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0961] Conventional athletic performance analysis systems are required not only to analyze athletes' movements but also to provide feedback that takes into account the athletes' emotional state. Current technology is capable of analyzing movement performance and predicting injury recovery, but it lacks the ability to appropriately evaluate athletes' motivation and emotional state and reflect them in feedback. As a result, the advice athletes receive is one-sided, limiting its effectiveness in improving overall performance and preventing injuries. To solve these problems, a system that combines athletes' movement analysis and emotional analysis is needed.

[0962] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving motion video data of a player, means for analyzing the received motion video data using a generative AI model and quantifying motion performance, means for generating and providing a report on the motion performance based on the analysis results, and means for evaluating the user's emotions using an emotion engine that analyzes the user's emotional state and reflecting the evaluation in the feedback content. This makes it possible to provide comprehensive feedback by combining the analysis results of the player's motion performance with emotion analysis.

[0963] "Motion video data" refers to video data that records the movements of players during practice or a match.

[0964] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to analyze input data and generate results.

[0965] "Movement performance" is a numerical evaluation of indicators such as the speed, accuracy, and trajectory of a player's movements.

[0966] A "Performance Report" is a document summarizing the results of analyzed operational performance, including suggestions for improvement and feedback.

[0967] An "emotion engine" is an algorithm that analyzes human emotions and is a system that evaluates emotional states based on facial expressions, voice, etc.

[0968] A "motion capture device" is a device that records the body movements of athletes in 3D, obtaining data on the movements of joints and various body parts.

[0969] Biomechanics is an academic field that deals with the movement and dynamics of the human body, and involves the scientific analysis of athletes' body movements.

[0970] "Injury Data" means detailed information about an injury sustained by a player, including diagnosis and treatment progress.

[0971] "Recovery time" is the estimated time it will take for an athlete to fully recover from an injury.

[0972] A "rehabilitation program" is an exercise and treatment plan designed to help an athlete recover from an injury and is provided to help the athlete recover efficiently.

[0973] "Feedback" refers to advice and instruction provided to players based on the results and evaluations analyzed by the system.

[0974] This invention is a system for maximizing athletes' athletic performance and preventing injuries. The system uses a generative AI model to analyze video data and motion capture data of athletes' movements and generate detailed reports to help athletes improve their performance and prevent injuries. Furthermore, by combining it with an emotion engine that recognizes athletes' emotions, it is possible to provide feedback that takes into account the user's emotional state.

[0975] Performance Analysis System

[0976] The system first receives video data of players' movements. Users film videos during practice or games using their devices (e.g., smartphones or tablets) and upload them to the system. The server receives the video data and performs detailed analysis using a generative AI model (e.g., TensorFlow or PyTorch). A high-performance server (e.g., an NVIDIA GPU or Intel processor) is used for the analysis. The speed, accuracy, trajectory, etc. of the movements in the video are quantified and a performance report is generated. Furthermore, an emotion engine is used to analyze the user's emotional state from their facial expressions and voice in the video and reflect this in the feedback content. For example, if a player uploads a video of themselves practicing dribbling a basketball to the system, the server will quantify the speed, accuracy, and quickness of the dribbling movement and provide an overall evaluation in the form of a report. The emotion engine can also be used to evaluate the user's emotional state and provide advice to improve motivation.

[0977] Example prompt sentence:

[0978] "Analyze the dribbling movements of the players in this video and quantify their speed, accuracy, and change of direction. Also, rate the players' emotional state."

[0979] Motion Capture Analysis System

[0980] In motion capture analysis, an athlete's movements are collected using a motion capture device (e.g., Vicon or OptiTrack). The movement data transferred from the device is stored on the device and uploaded to the system. The server analyzes the received data using a generative AI model to identify the athlete's movement patterns and biomechanical problems. A report is compiled into graphs and tables using a Python script and sent to the user's device. The emotion engine also analyzes the user's emotional state and reflects it in the report content. As a specific example, an athlete wears a motion capture device to record their running form and collects data. The server analyzes the movement patterns of each foot, knee angle, and ankle movement, and provides a report identifying problems. The emotion engine also takes the user's emotional state into account and provides advice for improving training methods.

[0981] Example prompt sentence:

[0982] "Based on this motion capture data, please analyze the athlete's running form and quantify the movement patterns of the left and right feet, the angle of the knees, and the movement of the ankles."

[0983] Recovery Time and Performance Prediction System

[0984] When an athlete is injured, detailed injury data (diagnosis results, treatment progress, etc.) is uploaded from the device to the system. The server receives this data and analyzes it using a generative AI model. Based on the analysis results, a predicted recovery time and an optimal rehabilitation program are provided. A feedback report detailing the analysis results is generated using a Python script and sent to the user's device. An emotion engine is used to analyze the user's emotional state and provide advice to maintain motivation during the rehabilitation period. As a specific example, if an athlete sprains their ankle, detailed injury data is entered into the system, and the server compares it with similar past data and predicts and provides a recovery time and rehabilitation program. The emotion engine also evaluates the user's emotional state and provides advice to maintain motivation.

[0985] Example prompt sentence:

[0986] "Based on this injury data, predict recovery time and the optimal rehabilitation program. Also, assess the user's emotional state."

[0987] This invention makes it possible to comprehensively manage an athlete's movements, recovery, and emotional state, providing optimal support for improving performance and preventing injuries. This is expected to provide effective support to a wide range of people, from professional athletes to students.

[0988] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0989] Performance Analysis System

[0990] Step 1:

[0991] Users use their devices to record the movements of players during practice and games. By recording using a device, it is possible to record the movements of players in real time. Specifically, videos are taken using the camera function of a smartphone or tablet.

[0992] Input: Video of player in action

[0993] Output: Recorded video file

[0994] Step 2:

[0995] Users upload the videos they have taken to the system by sending the video files to the server via a dedicated application or web portal.

[0996] Input: Recorded video file

[0997] Output: Video data stored on a cloud server

[0998] Step 3:

[0999] The server receives the video data sent by the user, and the received video data is stored in cloud storage.

[1000] Input: Video data uploaded to the cloud server

[1001] Output: Video data stored in cloud storage

[1002] Step 4:

[1003] The server inputs the received video into a generative AI model for analysis. The model is built using TensorFlow and PyTorch, and converts the speed, accuracy, and trajectory of the movements in the video into numerical data.

[1004] Input: Video data stored in cloud storage

[1005] Output: Numerical data of movement (speed, accuracy, trajectory)

[1006] Step 5:

[1007] The server generates detailed performance reports based on the analysis data, including performance metrics and areas for improvement. Using Python scripts, the results are visualized in graphs and tables and compiled in PDF format.

[1008] Input: Numerical data of the action

[1009] Output: Performance Report (PDF)

[1010] Step 6:

[1011] The server then sends the generated performance report to the user's device, and the user is notified of the report via a dedicated application or email.

[1012] Input: Performance Report

[1013] Output: Report notification delivered to the user's device

[1014] Step 7:

[1015] The server uses an emotion engine to analyze the user's emotional state in the video. Based on the analysis results, the server adds advice to improve motivation to the report.

[1016] Input: Video data

[1017] Output: Sentiment analysis results, feedback advice

[1018] Motion Capture Analysis System

[1019] Step 1:

[1020] Users collect movement data using motion capture devices, such as Vicon or OptiTrack, which record the movements of the athlete's joints and body as 3D coordinate data.

[1021] Input: Player movement

[1022] Output: Motion capture data (3D coordinate data)

[1023] Step 2:

[1024] The user saves the collected operation data on the device and uploads it to the system, where it is sent to the cloud server via a PC.

[1025] Input: Motion capture data

[1026] Output: Operation data stored on the cloud server

[1027] Step 3:

[1028] The server receives the data sent from the motion capture device and stores it in cloud storage.

[1029] Input: Motion data uploaded to the cloud server

[1030] Output: Data stored in cloud storage

[1031] Step 4:

[1032] The server analyzes the movement data using generative AI models, leveraging ML models such as Scikit-learn to identify movement patterns and biomechanical issues.

[1033] Input: Motion data stored in cloud storage

[1034] Power output: movement patterns and biomechanics issues

[1035] Step 5:

[1036] The server generates a report based on the analysis results. Using a Python script, the analysis results are visualized and compiled into a PDF report.

[1037] Input: Movement patterns and biomechanical issues

[1038] Output: Movement pattern and biomechanics report (PDF)

[1039] Step 6:

[1040] The server sends the generated report to the user's device, and the user is notified of the report via a dedicated application or email.

[1041] Input: Movement patterns and biomechanics reports

[1042] Output: Report notification delivered to the user's device

[1043] Step 7:

[1044] The server uses an emotion engine to analyze the user's emotional state and provides advice on how to improve the training method based on the analysis results.

[1045] Input: Operation data

[1046] Output: Sentiment analysis results, advice on improving training

[1047] Recovery Time and Performance Prediction System

[1048] Step 1:

[1049] The user inputs detailed injury data (diagnosis results, treatment progress, etc.) into the terminal, enters the data using electronic medical record software, and then exports and uploads it to the system.

[1050] Input: Injury details

[1051] Output: Injury data stored on the device

[1052] Step 2:

[1053] The user uploads the entered data to the system, which then sends the data to the cloud server.

[1054] Input: Injury data stored on the device

[1055] Output: Injury data stored on a cloud server

[1056] Step 3:

[1057] The server receives the uploaded injury data and stores it in a database (e.g., PostgreSQL).

[1058] Input: Injury data uploaded to a cloud server

[1059] Output: Injury data stored in a database

[1060] Step 4:

[1061] The server analyzes injury data using a generative AI model, comparing it with past data to predict recovery time and optimal rehabilitation programs.

[1062] Input: Injury data stored in a database

[1063] Output: Prediction of recovery time and rehabilitation program

[1064] Step 5:

[1065] The server generates a feedback report based on the analysis results. A Python script is used to create a PDF report detailing the recovery process and rehabilitation protocol.

[1066] Input: Prediction of recovery time and rehabilitation program

[1067] Output: Feedback report (PDF)

[1068] Step 6:

[1069] The server sends the generated report to the user's device, and the user is notified of the report via a dedicated application or email.

[1070] Input: Feedback Report

[1071] Output: Report notification delivered to the user's device

[1072] Step 7:

[1073] The server uses an emotion engine to analyze the user's emotional state and adds advice to help maintain motivation during the rehabilitation period.

[1074] Input: Injury data

[1075] Output: Emotion analysis results, advice for maintaining motivation

[1076] This allows the system of the present invention to comprehensively manage an athlete's movements, recovery, and emotional state, providing optimal support for improving performance and preventing injuries.

[1077] (Application example 2)

[1078] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1079] Robots used in factories require regular maintenance to maintain their precision and speed, but current methods make it difficult to efficiently optimize performance and perform preventative maintenance. Furthermore, the emotional state of robot operators tends to be ignored, which can have a negative impact on work efficiency and safety. Furthermore, when abnormal behavior occurs, quickly and accurately identifying the cause and providing an appropriate repair program is also a challenge.

[1080] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1081] In this invention, the server includes: [means for receiving robot operation video data;] [means for analyzing the received operation video data using a generative AI model and quantifying operation performance;] [means for generating and providing an operation performance report based on the analysis results;] [means for reflecting the operator's emotional state in the feedback content using emotion analysis means; and [means for, if the robot performs an abnormal operation, analyzing the abnormal operation data, identifying the cause, and generating a repair program.] This enables optimization of the robot's operation performance, preventive maintenance management, and comprehensive feedback that takes into account the operator's emotional state.

[1082] "Robot operation video data" refers to video information of the operation of a robot used in a factory recorded by a video recording device such as a video camera.

[1083] A "generative AI model" is an artificial intelligence model used to analyze robot motion data and other input data to calculate specific performance metrics.

[1084] "Motion performance" refers to a comprehensive evaluation of various indicators related to work efficiency and quality, such as the accuracy, speed, and trajectory of the movements performed by the robot.

[1085] A "report" is a document or data summarizing the analysis results obtained based on a generative AI model, including quantification of behavioral performance, indications of behavioral patterns, and areas for improvement.

[1086] "Emotion analysis means" refers to technology or devices that analyze the emotional state of operators, allowing for the assessment of their stress levels, motivation, etc.

[1087] "Feedback" refers to suggestions for improvement and advice provided based on the analysis results, allowing the robot and operator to take appropriate measures.

[1088] "Abnormal behavior data" refers to data that records the robot's unexpected or abnormal behavior.

[1089] A "repair program" is a repair procedure or process designed based on the cause of the abnormal behavior, which will restore the robot's normal operation.

[1090] Biomechanics is an academic field that analyzes the mechanical elements of robot movement. It is often applied to the analysis of human movement, but here we will also apply it to the analysis of robot movement.

[1091] To realize this invention, it is first necessary to collect video data and motion capture data of each robot's movements and upload them to a server. Users film the robot's movements in the factory with a video camera or other video recording device, and also record detailed movement patterns using a motion capture device.

[1092] The server analyzes the received motion video data using a dedicated AI model library (for example, "some_AI_model_library"). This analysis quantifies the speed, accuracy, trajectory, etc. of the motion and evaluates detailed motion performance. If abnormal motion occurs, the abnormal motion data is also analyzed to identify the cause and generate an optimal repair program. The results of this analysis are compiled into a report and sent to the user's device.

[1093] Additionally, an emotion analysis tool (e.g., "some_emotion_engine") is used to evaluate the emotional state of the operator. This allows the operator's stress level and motivation to be assessed and reflected in the feedback. The server-generated report also includes advice and areas for improvement based on the operator's emotional state, providing comprehensive feedback.

[1094] As a concrete example, video and motion capture data of a factory robot assembling parts is collected and uploaded to a server. Based on this data, the server quantifies the robot's movement accuracy and speed and generates a detailed report. This report is provided to the operator and includes advice on improving performance and preventative maintenance. If any abnormal behavior occurs, the cause is identified and a repair program is quickly implemented. The operator's emotional state is also taken into consideration, and advice on maintaining motivation is also included.

[1095] An example of a prompt for a generative AI model is, "Analyze the video and motion capture data of the factory robot's movements, quantify the accuracy, speed, and trajectory of the movements, and generate a report. Also, evaluate the operator's emotional state (such as stress level) and include it in the report."

[1096] This system not only enables efficient and effective performance optimization and maintenance management of factory robots, but also provides comprehensive support that takes into account the emotional state of the operator.

[1097] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1098] Step 1:

[1099] The user films the factory robot's movements with a video camera and records detailed movement patterns with a motion capture device.

[1100] Input: Robot motion video data, motion capture data

[1101] Output: Motion video data and motion capture data saved on the device

[1102] Specific movements: The user positions the camera and motion capture devices appropriately to film and record the robot's movements.

[1103] Step 2:

[1104] The video data and motion capture data that the user has filmed and recorded are uploaded to the server.

[1105] Input: Motion video data stored on the device, motion capture data

[1106] Output: Motion video data and motion capture data transferred to the server

[1107] Specific operation: The user transfers data to the server using a dedicated upload tool.

[1108] Step 3:

[1109] The video data received by the server is analyzed using a generative AI model.

[1110] Input: Motion video data stored on the server

[1111] Output: Analysis results such as robot movement speed, accuracy, and trajectory

[1112] Specific operation: The server inputs video data into the AI ​​model, calculates the movement speed and accuracy for each frame, and tracks and quantifies the trajectory.

[1113] Step 4:

[1114] The server analyzes the received motion capture data using a generative AI model to identify movement patterns and biomechanical issues.

[1115] Input: Motion capture data stored on the server

[1116] Output: Movement patterns and biomechanics issues

[1117] Specific movements: The server inputs motion capture data into an AI model, which analyzes the movement of each joint and identifies movement patterns and biomechanical issues.

[1118] Step 5:

[1119] The server generates and provides a report of operational performance based on the analysis results.

[1120] Input: movement speed, accuracy, trajectory, movement patterns, biomechanical issues

[1121] Output: Operational performance report

[1122] What it does: The server consolidates the analysis results and generates a visually comprehensible report, including areas for improvement and recommended actions.

[1123] Step 6:

[1124] The server uses an emotion analysis means to evaluate the emotional state of the operator and provides feedback content.

[1125] Input: Operator emotion data

[1126] Output: Improvements and advice reflecting the operator's emotional state

[1127] Specific operation: The server analyzes the emotional data using the emotion analysis means to assess stress levels and motivation, and includes this in the report.

[1128] Step 7:

[1129] If the robot behaves abnormally, the server analyzes the abnormal behavior data, identifies the cause, and generates a repair program.

[1130] Input: Abnormal behavior data

[1131] Output: Cause identification results, repair program

[1132] Specific operation: The server inputs abnormal operation data into the AI ​​model, uses prompt sentences to identify the cause, and generates repair procedures.

[1133] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1134] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1135] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1136] [Third embodiment]

[1137] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1138] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1139] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1140] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1141] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1142] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1143] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1144] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1145] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1146] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1147] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1148] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1149] This system maximizes athletes' athletic performance and prevents injuries by analyzing video and motion capture data of athletes using a generative AI model and generating detailed reports to help athletes improve their performance and prevent injuries. In the event of an injury, the system can also analyze injury data and predict recovery time and performance.

[1150] Performance Analysis System

[1151] The system first receives video data of the player's movements. The user records video during practice or a match on their device and uploads it to the system. The server receives the video data and performs a detailed analysis using a generative AI model, quantifying the speed, accuracy, and trajectory of the movement. A performance report is generated based on the analysis results and sent to the device. The user can review the report and learn how to improve their own movements.

[1152] As a concrete example, when a player uploads a video of himself practicing dribbling a basketball to the system, the system quantifies the speed, accuracy, and swiftness of the dribbling motion, and provides the user with a report with an overall evaluation.

[1153] Motion Capture Analysis System

[1154] In motion capture analysis, a player's movements are collected using a motion capture device. The movement data transferred from the device is stored on the device and uploaded to the system. The server analyzes the received data using a generative AI model to identify the player's movement patterns and biomechanical problems. A report is generated based on this and sent to the device. The user can review the report and receive guidance to improve their movement patterns.

[1155] For example, an athlete wears a motion capture device to record their running form and collect data. The system analyzes the movement patterns of their left and right feet, knee angles, and ankle movements, identifies problems, and provides a report.

[1156] Recovery Time and Performance Prediction System

[1157] When an athlete is injured, detailed injury data (diagnosis results, treatment progress, etc.) is uploaded from the device to the system. The server receives this data and analyzes it using a generative AI model. Based on the analysis results, a predicted recovery time and an optimal rehabilitation program are provided. A feedback report is sent to the device, allowing the user to use it to efficiently promote recovery.

[1158] For example, if an athlete sprains their ankle, they can input the details of the injury into the system. The system will compare the data with similar data from the past and provide a prediction of recovery time and rehabilitation program. The user can then use this information to create a rehabilitation plan and progress their recovery.

[1159] This invention will scientifically support athletes in improving their performance and preventing injuries, enabling efficient rehabilitation. In the initial stages, the system will partner with schools and other institutions to accumulate experience through trial and error, before being packaged and widely deployed. It is expected that this will enable effective support for a wide range of people, from professional athletes to students.

[1160] The processing flow will be explained below.

[1161] Performance analysis process steps

[1162] Step 1:

[1163] User: Record videos of practice and games and save them on the device.

[1164] Step 2:

[1165] Device: Open the video upload screen and select the saved video file.

[1166] Step 3:

[1167] On your device: Upload the selected video file to the server.

[1168] Step 4:

[1169] Server: Receives uploaded video data.

[1170] Step 5:

[1171] Server: Calls the generative AI model and analyzes the video data frame by frame.

[1172] Step 6:

[1173] Server: As a result of the analysis, it generates numerical data on the speed, accuracy, trajectory, etc. of the movement.

[1174] Step 7:

[1175] Server: Generates a performance report based on the analysis results.

[1176] Step 8:

[1177] Server: Sends the generated performance report to the terminal.

[1178] Step 9:

[1179] Terminal: Displays the received performance report to the user.

[1180] Processing steps for motion capture analysis

[1181] Step 1:

[1182] User: Wears a motion capture device to collect movement data during practice and games.

[1183] Step 2:

[1184] Terminal: Transfers movement data from the motion capture device to the terminal.

[1185] Step 3:

[1186] Terminal: Select the transferred operation data on the system's upload screen.

[1187] Step 4:

[1188] Terminal: Uploads the selected motion data to the server.

[1189] Step 5:

[1190] Server: Receives uploaded operational data.

[1191] Step 6:

[1192] Server: Calls the generative AI model and analyzes the behavior data.

[1193] Step 7:

[1194] Server: As a result of the analysis, problems with movement patterns and biomechanics are identified.

[1195] Step 8:

[1196] Server: Generates a motion capture report based on the analysis results.

[1197] Step 9:

[1198] Server: Sends the generated reports to the device.

[1199] Step 10:

[1200] Terminal: Displays received reports to the user.

[1201] Recovery time and performance prediction processing steps

[1202] Step 1:

[1203] User: Enter detailed data about the injury (diagnosis results, treatment progress, etc.) into the terminal.

[1204] Step 2:

[1205] Terminal: Select the entered data on the system's upload screen.

[1206] Step 3:

[1207] Terminal: Uploads the selected injury data to the server.

[1208] Step 4:

[1209] Server: Receives the uploaded data.

[1210] Step 5:

[1211] Server: Calls the generative AI model and analyzes injury data.

[1212] Step 6:

[1213] Server: The analysis results generate a recovery time and performance forecast.

[1214] Step 7:

[1215] Server: Generates a feedback report based on the analysis results.

[1216] Step 8:

[1217] Server: Sends the generated feedback report to the terminal.

[1218] Step 9:

[1219] Terminal: Displays the received feedback report to the user.

[1220] Example 1

[1221] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1222] Maximizing athletes' athletic performance, preventing injuries, and efficiently recovering from existing injuries are extremely important in the field of sports. However, conventional methods for analyzing athletes' movements, preventing injuries, and predicting recovery rely on expert knowledge and manual work, resulting in insufficient accuracy and efficiency. To solve these issues, a system is needed that can efficiently and accurately analyze athletes' movement and injury data and quickly provide scientifically based reports.

[1223] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1224] In this invention, the server includes means for acquiring motion video data of the player from an input device, means for receiving the acquired motion video data with a central processing unit, means for analyzing the received data using a generative AI model and quantifying motion performance, and means for generating a motion performance report based on the analysis results and sending it to the user device. This enables scientific and efficient analysis of the player's motion and detailed feedback for performance improvement and injury prevention.

[1225] The system also includes a means for acquiring motion capture data of the player from an input device, a means for receiving the acquired motion capture data with a central processing unit, a means for analyzing the received data using a generative AI model to identify problems with movement patterns and biomechanics, and a means for generating a report on the movement patterns and biomechanics problems based on the analysis results and sending it to the user device, thereby enabling accurate analysis of the player's movements and helping to improve movements and performance.

[1226] The system also includes means for acquiring data on the player's injury from an input device, means for receiving the acquired injury data with a central processing unit, means for analyzing the received data using a generative AI model to predict recovery time and performance, and means for generating a report on the recovery time and performance prediction based on the analysis results and sending it to the user device, thereby making it possible to streamline the injury recovery process and provide a scientific rehabilitation program.

[1227] "Motion video data" refers to digital video data that records an athlete actually exercising.

[1228] An "input device" is a device used by a user to collect data, including a smartphone, tablet, motion capture device, etc.

[1229] A "central processing unit" is a server computer or cloud platform that receives and analyzes uploaded data.

[1230] "Generative AI models" refer to algorithms and their implementations for analyzing data using machine learning and deep learning, including those using frameworks such as TensorFlow and PyTorch.

[1231] "Movement performance" is an evaluation index related to an athlete's motor functions, such as athletic ability, technical accuracy, and speed.

[1232] A "report" is a document or digital file that presents the analyzed results to the user in an easy-to-understand manner, detailing performance, problems, areas for improvement, etc.

[1233] "Motion capture data" refers to digital data recorded using a motion capture device, which represents the movements of each part of the player's body in three-dimensional coordinates.

[1234] "Movement patterns" refer to the repetition and habitual movements of an athlete, and are understood by analyzing the trajectory and timing of the movements.

[1235] "Biomechanics" is an academic field that deals with the movement and mechanical characteristics of the human body, and collectively refers to the movement of an athlete's body, the angles of their joints, and the function of their muscles during exercise.

[1236] "Injury data" refers to digital data detailing the injury sustained by a player, including information such as diagnosis, treatment progress, and pain level.

[1237] "Recovery time" refers to the period of time required for an athlete to fully recover from an injury and return to their previous performance.

[1238] "Performance prediction" refers to the analysis results that predict the likely improvement in an athlete's athletic ability and technique after recovery from an injury.

[1239] The present invention provides a system for maximizing an athlete's athletic performance, preventing injuries, and supporting efficient rehabilitation. Specific embodiments will be described below.

[1240] Performance Analysis System

[1241] First, a user uses an input device such as a smartphone or tablet to record video data of the player's movements. The recorded video data is then uploaded to a server via an application on the device. The server receives the uploaded video data and analyzes it using a generative AI model (e.g., TensorFlow or PyTorch).

[1242] Specifically, the system analyzes each frame of video data and quantifies the speed, accuracy, and trajectory of the player's movements. The server automatically generates a detailed performance report based on the analysis results and sends it to the device. The user can then review the report on their device and learn how to improve their movements.

[1243] Examples:

[1244] A user records a video of themselves practicing dribbling a basketball and uploads it to the system.

[1245] The system quantifies the speed, accuracy, and quickness of changes in direction of dribbling movements and provides an overall evaluation.

[1246] At the end, users receive a report detailing performance trends and areas for improvement.

[1247] Example prompt sentence:

[1248] "Analyze video of basketball dribbling practice, evaluate the speed, accuracy, and change of direction of the movement, and provide a detailed report of the results."

[1249] Motion Capture Analysis System

[1250] Next, in motion capture analysis, the user uses a motion capture device (e.g., Vicon) to capture the player's movements. The collected motion capture data is stored on the device and later uploaded to a server. The server analyzes the received data using a generative AI model (e.g., PyTorch).

[1251] The server analyzes and identifies problems in the player's movement patterns and biomechanics. Based on the analysis results, a detailed report is generated and sent to the device. The user can then review the report to understand where to improve the movement patterns and provide guidance.

[1252] Examples:

[1253] The user has the athlete wear a motion capture device to collect data for the purpose of analyzing their running form.

[1254] The system analyzes the movement patterns of the left and right feet, knee angles, and ankle movements to identify problems.

[1255] Users receive guidance based on a report containing detailed analysis results and areas for improvement.

[1256] Example prompt sentence:

[1257] "Analyze the motion capture data of the running form, evaluate the movement patterns of the left and right feet, knee angles, and ankle movements, and provide a detailed report of the results."

[1258] Recovery Time and Performance Prediction System

[1259] Furthermore, if a player is injured, the user inputs detailed data about the injury (diagnosis results, treatment progress, etc.) into the device. The input data is uploaded to the server, which receives it and analyzes it using a generative AI model (e.g., IBM Watson).

[1260] Through analysis, the server predicts injury recovery time and performance. Based on the analysis results, a feedback report is generated that provides a predicted recovery time and an optimal rehabilitation program, and is sent to the device. The user can then review the feedback report and create an efficient rehabilitation plan.

[1261] Examples:

[1262] A user enters injury data for a player who has a sprained ankle into the system.

[1263] The system compares similar past data to predict recovery time and provide a rehabilitation program.

[1264] Based on this, users can create a rehabilitation plan and efficiently progress with recovery.

[1265] Example prompt sentence:

[1266] "Based on detailed data on ankle sprains, predict recovery time and optimal rehabilitation program, and provide a detailed report of the results."

[1267] These embodiments make it possible to realize a system that scientifically supports athletes in improving their athletic performance, preventing injuries, and recovering efficiently.

[1268] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1269] Performance Analysis System

[1270] Step 1: Record your video

[1271] Users use input devices such as smartphones and tablets to record video data of athletes' movements.

[1272] Specific operations: Record practice and game situations and save them on the device.

[1273] Input: Player action scene

[1274] Output: Operation video data (MP4 format, etc.)

[1275] Step 2: Upload your video

[1276] The user uploads the recorded video data to the server via an application on the device.

[1277] Specific operation: Press the upload button in the application, select the video file and send it.

[1278] Input: Motion video data

[1279] Output: Video data sent to the server

[1280] Step 3: Receiving video data

[1281] The server receives the uploaded video data.

[1282] Specific behavior: Receives an HTTP request and saves the video file.

[1283] Input: Video data sent to the server

[1284] Output: Saved video file

[1285] Step 4: Analyze the video data

[1286] The server uses a generative AI model to analyze the received video data.

[1287] Specific actions: Analyze video frame by frame to quantify the speed, accuracy, and trajectory of movements.

[1288] Input: Saved video file

[1289] Output: Analysis result data (JSON format, etc.)

[1290] Step 5: Generate a performance report

[1291] The server generates a performance report based on the analysis results.

[1292] Specific operation: Automatically generate a report using analysis result data and save it in PDF or HTML format.

[1293] Input: Analysis result data

[1294] Output: Performance report

[1295] Step 6: Submit the report

[1296] The server transmits the generated performance report to the terminal.

[1297] What it does: Sends reports via email or a dedicated application.

[1298] Input: Performance Report

[1299] Output: Report sent to terminal

[1300] Step 7: Review the report

[1301] The user checks the sent report on the terminal.

[1302] What it does: Open the report you receive, review it, and learn how to improve it.

[1303] Input: Report sent to terminal

[1304] Output: Information to confirm and implement improvements.

[1305] Motion Capture Analysis System

[1306] Step 1: Collecting motion capture data

[1307] Users use motion capture devices to record the movements of players.

[1308] Specific movements: The athletes wear markers and their movements are recorded using a motion capture device.

[1309] Input: Player action scene

[1310] Output: Motion capture data (CSV, JSON, etc. formats)

[1311] Step 2: Upload your data

[1312] The user saves the collected motion capture data on the device and uploads it to the server.

[1313] Specific operation: Press the upload button in the application to send the data file to the server.

[1314] Input: Motion capture data

[1315] Output: Operational data sent to the server

[1316] Step 3: Receiving the data

[1317] The server receives the uploaded motion capture data.

[1318] Specific behavior: Receives an HTTP request and saves a data file.

[1319] Input: Operational data sent to the server

[1320] Output: Saved data file

[1321] Step 4: Analyze the data

[1322] The server uses a generative AI model to analyze the received motion capture data.

[1323] Specific Movements: Analyze data to identify movement patterns and biomechanical issues.

[1324] Input: Saved data file

[1325] Output: Analysis result data (JSON format, etc.)

[1326] Step 5: Generate reports

[1327] The server generates a report on movement patterns and biomechanical issues based on the analysis results.

[1328] Specific operation: Automatically generate a report using analysis result data and save it in PDF or HTML format.

[1329] Input: Analysis result data

[1330] Output: Movement pattern and biomechanics problem report

[1331] Step 6: Submit the report

[1332] The server sends the generated report to the terminal.

[1333] What it does: Sends reports via email or a dedicated application.

[1334] Input: Movement Pattern and Biomechanics Problem Report

[1335] Output: Report sent to terminal

[1336] Step 7: Review the report

[1337] The user checks the sent report on the terminal.

[1338] What it does: Open the report you receive, review it, and learn how to improve it.

[1339] Input: Report sent to terminal

[1340] Output: Information to confirm and implement improvements.

[1341] Recovery Time and Performance Prediction System

[1342] Step 1: Enter injury data

[1343] The user enters details about the player's injury into the terminal.

[1344] Specific operations: Enter information such as diagnosis results and treatment progress into a data entry form.

[1345] Input: Detailed injury information

[1346] Output: Input injury data

[1347] Step 2: Upload your data

[1348] The user uploads the injury data they entered to the server.

[1349] Specific operation: Press the upload button in the application to send the data.

[1350] Input: Injury data

[1351] Output: Injury data sent to the server

[1352] Step 3: Receiving the data

[1353] The server receives the uploaded injury data.

[1354] Specific behavior: Receives an HTTP request and saves a data file.

[1355] Input: Injury data sent to server

[1356] Output: Saved data file

[1357] Step 4: Analyze the data

[1358] The server uses a generative AI model to analyze the injury data it receives.

[1359] Specific actions: Compare with similar cases from the past, predict recovery time and suggest optimal rehabilitation programs.

[1360] Input: Stored injury data

[1361] Output: Analysis result data (JSON format, etc.)

[1362] Step 5: Generate a feedback report

[1363] The server generates a feedback report based on the analysis results.

[1364] Specific operation: Automatically generate a report using analysis result data and save it in PDF or HTML format.

[1365] Input: Analysis result data

[1366] Output: Feedback report

[1367] Step 6: Submit the report

[1368] The server transmits the generated feedback report to the terminal.

[1369] What it does: Sends reports via email or a dedicated application.

[1370] Input: Feedback Report

[1371] Output: Report sent to terminal

[1372] Step 7: Review the report

[1373] The user checks the sent report on the terminal and creates an efficient rehabilitation plan.

[1374] Specific actions: Open the received report, review the contents, and develop a rehabilitation plan.

[1375] Input: Report sent to terminal

[1376] Output: Efficient rehabilitation plan

[1377] (Application example 1)

[1378] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1379] The goal is to maximize the operational efficiency of machinery operating in factories and to detect signs of breakdowns and prevent them before they occur. With conventional methods, it has been difficult to analyze machine performance in real time and carry out appropriate maintenance.

[1380] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1381] In this invention, the server includes a means for receiving machine operation video data, a means for analyzing the received operation video data using a generative AI model to quantify operation efficiency, and a means for generating and providing an operation efficiency report based on the analysis results, thereby enabling scientific analysis of the machine's operation efficiency and identifying maintenance and improvement points at appropriate times.

[1382] "Machines" are automated devices and robots used in factories and other places to process, assemble, and transport objects.

[1383] "Operation video data" refers to video data that records the operation of a machine and is uploaded to a server for analysis.

[1384] A "generative AI model" is an algorithm that uses machine learning and deep learning techniques and is trained to perform data analysis.

[1385] "Operational efficiency" is an index that evaluates the energy consumption and time required when a machine performs a specified task.

[1386] The "means for receiving video data" is an interface that allows the server to receive video data from the outside.

[1387] The "means for quantifying motion efficiency" is a process of analyzing received video data and converting parameters related to motion into specific numerical values.

[1388] The "means for generating and providing a report" refers to a process for creating a report summarizing evaluation information regarding the operation of the machine based on the analyzed data and providing the report to the user.

[1389] The system for implementing this invention analyzes the operational efficiency of machines in a factory, and provides failure prevention and maintenance support. The system operates through the cooperation of three parties: a server, a terminal, and a user.

[1390] System configuration

[1391] The system includes the following hardware and software:

[1392] server

[1393] Video analysis and data processing are performed on the server side, mainly using OpenCV, Keras, and TensorFlow.

[1394] Terminal

[1395] Used to collect and upload motion video data. Applies to smartphones and head-mounted displays.

[1396] User

[1397] These are factory managers and operators who perform maintenance and improvements on machines based on reports provided by the system.

[1398] Program processing

[1399] The server first receives video data of the machine's operation sent from the device. This data is broken down into frames using OpenCV, and image preprocessing is performed. The preprocessed data is then analyzed using a generative AI model using Keras and TensorFlow. Here, parameters related to operational efficiency (speed, accuracy, movement trajectory, etc.) are quantified.

[1400] Based on the analysis results, the server generates a detailed report that includes not only an evaluation of operational efficiency but also any identified mechanical issues and suggestions for improvement, and the report is sent to the terminal for review by the user.

[1401] Users can plan and implement machine maintenance and operational improvements based on the report content. In particular, if signs of a malfunction are detected, it is necessary to carry out preventative maintenance as soon as possible.

[1402] Specific examples

[1403] Factory managers use their smartphones to record videos of robots operating in the factory and upload them to a dedicated app. The server receives the videos, analyzes operational efficiency and mechanical problems, generates a report, and sends it to the manager's device. The manager can then check the detailed report on the app and plan and implement improvements to the robot's operation and maintenance.

[1404] Prompt Sentence Examples

[1405] "Just take a video of your robots operating in your factory and upload it to the app. The system will analyze the video and provide you with a detailed report on the robots' speed, accuracy, and movement trajectory. You can then identify areas for improvement and carry out appropriate maintenance."

[1406] As described above, the system for carrying out the present invention scientifically analyzes the operation of machines in a factory and supports the prevention of breakdowns and the improvement of operation.

[1407] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1408] Step 1:

[1409] The device captures video of the machines in operation in the factory. The input is the video of the machines in operation, and the output is the captured video data. The user operates the device using a smartphone or head-mounted display.

[1410] Step 2:

[1411] The device uploads the captured video data to the server. The input is the captured video data, and the output is the video data stored on the server.

[1412] Step 3:

[1413] The server receives the video data and decomposes the video into frames using OpenCV. The input is the received video data, and the output is individual frame images for preprocessing.

[1414] Step 4:

[1415] The server performs image preprocessing (resizing, normalization) on each frame image. The input is the decomposed frame image, and the output is the preprocessed frame image.

[1416] Step 5:

[1417] The server inputs the preprocessed frame images into a generative AI model using Keras and TensorFlow to analyze parameters related to movement efficiency. The input is the preprocessed frame images, and the output is the analysis results related to movement efficiency (speed, accuracy, movement trajectory, etc.).

[1418] Step 6:

[1419] The server generates a detailed report based on the analysis results. The input is the analyzed data, and the output is a report that includes an evaluation of operational efficiency, problems, and improvement suggestions.

[1420] Step 7:

[1421] The server sends the generated report to the terminal, where the input is the generated report and the output is the report displayed on the terminal.

[1422] Step 8:

[1423] The user checks the report received on the terminal and plans and implements machine maintenance and operational improvements. The input is the report displayed on the terminal, and the output is the implementation of specific maintenance and improvement measures.

[1424] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1425] This system maximizes athletes' athletic performance and prevents injuries. It analyzes athletes' motion video data and motion capture data using a generative AI model and generates detailed reports to help improve athletes' performance and prevent injuries. By combining this with an emotion engine that recognizes athletes' emotions, it is possible to provide feedback that takes into account the user's emotional state. Furthermore, in the event of an injury, it is possible to analyze injury data and predict recovery time and performance.

[1426] Performance Analysis System

[1427] The system first receives video data of the player's movements. The user records video on their device during practice or a match and uploads it to the system. The server receives the video data and performs a detailed analysis using a generative AI model, quantifying the speed, accuracy, and trajectory of the movement. A performance report is generated based on the analysis results and sent to the device. Furthermore, an emotion engine is used to analyze the user's emotional state and reflect this in the feedback. The user can review the report and learn how to improve their own movements.

[1428] For example, when a player uploads a video of himself practicing dribbling a basketball to the system, the system quantifies the speed, accuracy, and speed of changes in direction of the dribble, and provides the user with a comprehensive evaluation report. In addition, the emotion engine evaluates how the user feels about the practice and provides advice to improve motivation.

[1429] Motion Capture Analysis System

[1430] In motion capture analysis, a player's movements are collected using a motion capture device. The movement data transferred from the device is stored on the device and uploaded to the system. The server analyzes the received data using a generative AI model to identify the player's movement patterns and biomechanical issues. A report is generated based on this and sent to the device. Furthermore, an emotion engine is used to analyze the user's emotional state and reflect it in the report. The user can review the report and receive guidance to improve their movement patterns.

[1431] For example, an athlete wears a motion capture device to record their running form and collect data. The system analyzes the movement patterns of the left and right feet, the knee angle, and the movement of the ankle, identifies problems, and provides a report. In addition, an emotion engine takes into account the user's emotional state and provides advice to improve training methods.

[1432] Recovery Time and Performance Prediction System

[1433] When an athlete is injured, detailed injury data (diagnosis results, treatment progress, etc.) is uploaded from the device to the system. The server receives this data and analyzes it using a generative AI model. Based on the analysis results, a predicted recovery time and an optimal rehabilitation program are provided. A feedback report is sent to the device, and the user can use this to promote efficient recovery. The system also takes the user's emotional state into account using an emotion engine to provide advice on maintaining motivation.

[1434] For example, if an athlete sprains their ankle, detailed injury data is entered into the system. The system compares this data with similar past data and predicts recovery time and rehabilitation programs. Based on this, the user can create a rehabilitation plan and progress with their recovery. Furthermore, an emotion engine evaluates the user's emotional state and provides advice to improve motivation during the rehabilitation period.

[1435] This invention will scientifically support athletes in improving their performance and preventing injuries, enabling efficient rehabilitation. Furthermore, by combining it with an emotion engine, it will be possible to provide comprehensive feedback that takes into account the user's emotional state, helping to maintain and improve the athlete's motivation. This is expected to provide effective support to a wide range of people, from professional athletes to students.

[1436] The processing flow will be explained below.

[1437] Performance analysis process steps

[1438] Step 1:

[1439] User: Record videos of practice and games and save them on the device.

[1440] Step 2:

[1441] Device: Open the video upload screen and select the saved video file.

[1442] Step 3:

[1443] On your device: Upload the selected video file to the server.

[1444] Step 4:

[1445] Server: Receives uploaded video data.

[1446] Step 5:

[1447] Server: Calls the generative AI model and analyzes the video data frame by frame.

[1448] Step 6:

[1449] Server: As a result of the analysis, it generates numerical data on the speed, accuracy, trajectory, etc. of the movement.

[1450] Step 7:

[1451] Server: Generates a performance report based on the analysis results.

[1452] Step 8:

[1453] Server: Calls the emotion engine and collects additional data to analyze the user's emotional state.

[1454] Step 9:

[1455] Server: The emotion engine analyzes the user's facial expressions and voice data to identify the user's emotional state.

[1456] Step 10:

[1457] Server: Generates a final performance report with adjusted feedback based on the emotional state.

[1458] Step 11:

[1459] Server: Sends the generated final performance report to the terminal.

[1460] Step 12:

[1461] Terminal: Displays the received performance report to the user.

[1462] Processing steps for motion capture analysis

[1463] Step 1:

[1464] User: Wears a motion capture device to collect movement data during practice and games.

[1465] Step 2:

[1466] Terminal: Transfers movement data from the motion capture device to the terminal.

[1467] Step 3:

[1468] Terminal: Select the transferred operation data on the system's upload screen.

[1469] Step 4:

[1470] Terminal: Uploads the selected motion data to the server.

[1471] Step 5:

[1472] Server: Receives uploaded operational data.

[1473] Step 6:

[1474] Server: Calls the generative AI model and analyzes the behavior data.

[1475] Step 7:

[1476] Server: As a result of the analysis, problems with movement patterns and biomechanics are identified.

[1477] Step 8:

[1478] Server: Generates a motion capture report based on the analysis results.

[1479] Step 9:

[1480] Server: Calls the emotion engine and collects additional data to analyze the user's emotional state.

[1481] Step 10:

[1482] Server: The emotion engine analyzes the user's facial expressions and voice data to identify the user's emotional state.

[1483] Step 11:

[1484] Server: Generates the final motion capture report with adjusted feedback based on the emotional state.

[1485] Step 12:

[1486] Server: Sends the generated final motion capture report to the device.

[1487] Step 13:

[1488] Terminal: Displays received reports to the user.

[1489] Recovery time and performance prediction processing steps

[1490] Step 1:

[1491] User: Enter detailed data about the injury (diagnosis results, treatment progress, etc.) into the terminal.

[1492] Step 2:

[1493] Terminal: Select the entered data on the system's upload screen.

[1494] Step 3:

[1495] Terminal: Uploads the selected injury data to the server.

[1496] Step 4:

[1497] Server: Receives the uploaded data.

[1498] Step 5:

[1499] Server: Calls the generative AI model and analyzes injury data.

[1500] Step 6:

[1501] Server: The analysis results generate a recovery time and performance forecast.

[1502] Step 7:

[1503] Server: Generates a feedback report based on the analysis results.

[1504] Step 8:

[1505] Server: Calls the emotion engine and collects additional data to analyze the user's emotional state.

[1506] Step 9:

[1507] Server: The emotion engine analyzes the user's facial expressions and voice data to identify the user's emotional state.

[1508] Step 10:

[1509] Server: Generates a final feedback report with adjusted feedback content based on the emotional state.

[1510] Step 11:

[1511] Server: Sends the generated final feedback report to the terminal.

[1512] Step 12:

[1513] Terminal: Displays the received feedback report to the user.

[1514] Example 2

[1515] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1516] Conventional athletic performance analysis systems are required not only to analyze athletes' movements but also to provide feedback that takes into account the athletes' emotional state. Current technology is capable of analyzing movement performance and predicting injury recovery, but it lacks the ability to appropriately evaluate athletes' motivation and emotional state and reflect them in feedback. As a result, the advice athletes receive is one-sided, limiting its effectiveness in improving overall performance and preventing injuries. To solve these problems, a system that combines athletes' movement analysis and emotional analysis is needed.

[1517] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving motion video data of a player, means for analyzing the received motion video data using a generative AI model and quantifying motion performance, means for generating and providing a report on the motion performance based on the analysis results, and means for evaluating the user's emotions using an emotion engine that analyzes the user's emotional state and reflecting the evaluation in the feedback content. This makes it possible to provide comprehensive feedback by combining the analysis results of the player's motion performance with emotion analysis.

[1518] "Motion video data" refers to video data that records the movements of players during practice or a match.

[1519] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to analyze input data and generate results.

[1520] "Movement performance" is a numerical evaluation of indicators such as the speed, accuracy, and trajectory of a player's movements.

[1521] A "Performance Report" is a document summarizing the results of analyzed operational performance, including suggestions for improvement and feedback.

[1522] An "emotion engine" is an algorithm that analyzes human emotions and is a system that evaluates emotional states based on facial expressions, voice, etc.

[1523] A "motion capture device" is a device that records the body movements of athletes in 3D, obtaining data on the movements of joints and various body parts.

[1524] Biomechanics is an academic field that deals with the movement and dynamics of the human body, and involves the scientific analysis of athletes' body movements.

[1525] "Injury Data" means detailed information about an injury sustained by a player, including diagnosis and treatment progress.

[1526] "Recovery time" is the estimated time it will take for an athlete to fully recover from an injury.

[1527] A "rehabilitation program" is an exercise and treatment plan designed to help an athlete recover from an injury and is provided to help the athlete recover efficiently.

[1528] "Feedback" refers to advice and instruction provided to players based on the results and evaluations analyzed by the system.

[1529] This invention is a system for maximizing athletes' athletic performance and preventing injuries. The system uses a generative AI model to analyze video data and motion capture data of athletes' movements and generate detailed reports to help athletes improve their performance and prevent injuries. Furthermore, by combining it with an emotion engine that recognizes athletes' emotions, it is possible to provide feedback that takes into account the user's emotional state.

[1530] Performance Analysis System

[1531] The system first receives video data of players' movements. Users film videos during practice or games using their devices (e.g., smartphones or tablets) and upload them to the system. The server receives the video data and performs detailed analysis using a generative AI model (e.g., TensorFlow or PyTorch). A high-performance server (e.g., an NVIDIA GPU or Intel processor) is used for the analysis. The speed, accuracy, trajectory, etc. of the movements in the video are quantified and a performance report is generated. Furthermore, an emotion engine is used to analyze the user's emotional state from their facial expressions and voice in the video and reflect this in the feedback content. For example, if a player uploads a video of themselves practicing dribbling a basketball to the system, the server will quantify the speed, accuracy, and quickness of the dribbling movement and provide an overall evaluation in the form of a report. The emotion engine can also be used to evaluate the user's emotional state and provide advice to improve motivation.

[1532] Example prompt sentence:

[1533] "Analyze the dribbling movements of the players in this video and quantify their speed, accuracy, and change of direction. Also, rate the players' emotional state."

[1534] Motion Capture Analysis System

[1535] In motion capture analysis, an athlete's movements are collected using a motion capture device (e.g., Vicon or OptiTrack). The movement data transferred from the device is stored on the device and uploaded to the system. The server analyzes the received data using a generative AI model to identify the athlete's movement patterns and biomechanical problems. A report is compiled into graphs and tables using a Python script and sent to the user's device. The emotion engine also analyzes the user's emotional state and reflects it in the report content. As a specific example, an athlete wears a motion capture device to record their running form and collects data. The server analyzes the movement patterns of each foot, knee angle, and ankle movement, and provides a report identifying problems. The emotion engine also takes the user's emotional state into account and provides advice for improving training methods.

[1536] Example prompt sentence:

[1537] "Based on this motion capture data, please analyze the athlete's running form and quantify the movement patterns of the left and right feet, the angle of the knees, and the movement of the ankles."

[1538] Recovery Time and Performance Prediction System

[1539] When an athlete is injured, detailed injury data (diagnosis results, treatment progress, etc.) is uploaded from the device to the system. The server receives this data and analyzes it using a generative AI model. Based on the analysis results, a predicted recovery time and an optimal rehabilitation program are provided. A feedback report detailing the analysis results is generated using a Python script and sent to the user's device. An emotion engine is used to analyze the user's emotional state and provide advice to maintain motivation during the rehabilitation period. As a specific example, if an athlete sprains their ankle, detailed injury data is entered into the system, and the server compares it with similar past data and predicts and provides a recovery time and rehabilitation program. The emotion engine also evaluates the user's emotional state and provides advice to maintain motivation.

[1540] Example prompt sentence:

[1541] "Based on this injury data, predict recovery time and the optimal rehabilitation program. Also, assess the user's emotional state."

[1542] This invention makes it possible to comprehensively manage an athlete's movements, recovery, and emotional state, providing optimal support for improving performance and preventing injuries. This is expected to provide effective support to a wide range of people, from professional athletes to students.

[1543] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1544] Performance Analysis System

[1545] Step 1:

[1546] Users use their devices to record the movements of players during practice and games. By recording using a device, it is possible to record the movements of players in real time. Specifically, videos are taken using the camera function of a smartphone or tablet.

[1547] Input: Video of player in action

[1548] Output: Recorded video file

[1549] Step 2:

[1550] Users upload the videos they have taken to the system by sending the video files to the server via a dedicated application or web portal.

[1551] Input: Recorded video file

[1552] Output: Video data stored on a cloud server

[1553] Step 3:

[1554] The server receives the video data sent by the user, and the received video data is stored in cloud storage.

[1555] Input: Video data uploaded to the cloud server

[1556] Output: Video data stored in cloud storage

[1557] Step 4:

[1558] The server inputs the received video into a generative AI model for analysis. The model is built using TensorFlow and PyTorch, and converts the speed, accuracy, and trajectory of the movements in the video into numerical data.

[1559] Input: Video data stored in cloud storage

[1560] Output: Numerical data of movement (speed, accuracy, trajectory)

[1561] Step 5:

[1562] The server generates detailed performance reports based on the analysis data, including performance metrics and areas for improvement. Using Python scripts, the results are visualized in graphs and tables and compiled in PDF format.

[1563] Input: Numerical data of the action

[1564] Output: Performance Report (PDF)

[1565] Step 6:

[1566] The server then sends the generated performance report to the user's device, and the user is notified of the report via a dedicated application or email.

[1567] Input: Performance Report

[1568] Output: Report notification delivered to the user's device

[1569] Step 7:

[1570] The server uses an emotion engine to analyze the user's emotional state in the video. Based on the analysis results, the server adds advice to improve motivation to the report.

[1571] Input: Video data

[1572] Output: Sentiment analysis results, feedback advice

[1573] Motion Capture Analysis System

[1574] Step 1:

[1575] Users collect movement data using motion capture devices, such as Vicon or OptiTrack, which record the movements of the athlete's joints and body as 3D coordinate data.

[1576] Input: Player movement

[1577] Output: Motion capture data (3D coordinate data)

[1578] Step 2:

[1579] The user saves the collected operation data on the device and uploads it to the system, where it is sent to the cloud server via a PC.

[1580] Input: Motion capture data

[1581] Output: Operation data stored on the cloud server

[1582] Step 3:

[1583] The server receives the data sent from the motion capture device and stores it in cloud storage.

[1584] Input: Motion data uploaded to the cloud server

[1585] Output: Data stored in cloud storage

[1586] Step 4:

[1587] The server analyzes the movement data using generative AI models, leveraging ML models such as Scikit-learn to identify movement patterns and biomechanical issues.

[1588] Input: Motion data stored in cloud storage

[1589] Power output: movement patterns and biomechanics issues

[1590] Step 5:

[1591] The server generates a report based on the analysis results. Using a Python script, the analysis results are visualized and compiled into a PDF report.

[1592] Input: Movement patterns and biomechanical issues

[1593] Output: Movement pattern and biomechanics report (PDF)

[1594] Step 6:

[1595] The server sends the generated report to the user's device, and the user is notified of the report via a dedicated application or email.

[1596] Input: Movement patterns and biomechanics reports

[1597] Output: Report notification delivered to the user's device

[1598] Step 7:

[1599] The server uses an emotion engine to analyze the user's emotional state and provides advice on how to improve the training method based on the analysis results.

[1600] Input: Operation data

[1601] Output: Sentiment analysis results, advice on improving training

[1602] Recovery Time and Performance Prediction System

[1603] Step 1:

[1604] The user inputs detailed injury data (diagnosis results, treatment progress, etc.) into the terminal, enters the data using electronic medical record software, and then exports and uploads it to the system.

[1605] Input: Injury details

[1606] Output: Injury data stored on the device

[1607] Step 2:

[1608] The user uploads the entered data to the system, which then sends the data to the cloud server.

[1609] Input: Injury data stored on the device

[1610] Output: Injury data stored on a cloud server

[1611] Step 3:

[1612] The server receives the uploaded injury data and stores it in a database (e.g., PostgreSQL).

[1613] Input: Injury data uploaded to a cloud server

[1614] Output: Injury data stored in a database

[1615] Step 4:

[1616] The server analyzes injury data using a generative AI model, comparing it with past data to predict recovery time and optimal rehabilitation programs.

[1617] Input: Injury data stored in a database

[1618] Output: Prediction of recovery time and rehabilitation program

[1619] Step 5:

[1620] The server generates a feedback report based on the analysis results. A Python script is used to create a PDF report detailing the recovery process and rehabilitation protocol.

[1621] Input: Prediction of recovery time and rehabilitation program

[1622] Output: Feedback report (PDF)

[1623] Step 6:

[1624] The server sends the generated report to the user's device, and the user is notified of the report via a dedicated application or email.

[1625] Input: Feedback Report

[1626] Output: Report notification delivered to the user's device

[1627] Step 7:

[1628] The server uses an emotion engine to analyze the user's emotional state and adds advice to help maintain motivation during the rehabilitation period.

[1629] Input: Injury data

[1630] Output: Emotion analysis results, advice for maintaining motivation

[1631] This allows the system of the present invention to comprehensively manage an athlete's movements, recovery, and emotional state, providing optimal support for improving performance and preventing injuries.

[1632] (Application example 2)

[1633] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1634] Robots used in factories require regular maintenance to maintain their precision and speed, but current methods make it difficult to efficiently optimize performance and perform preventative maintenance. Furthermore, the emotional state of robot operators tends to be ignored, which can have a negative impact on work efficiency and safety. Furthermore, when abnormal behavior occurs, quickly and accurately identifying the cause and providing an appropriate repair program is also a challenge.

[1635] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1636] In this invention, the server includes: [means for receiving robot operation video data;] [means for analyzing the received operation video data using a generative AI model and quantifying operation performance;] [means for generating and providing an operation performance report based on the analysis results;] [means for reflecting the operator's emotional state in the feedback content using emotion analysis means; and [means for, if the robot performs an abnormal operation, analyzing the abnormal operation data, identifying the cause, and generating a repair program.] This enables optimization of the robot's operation performance, preventive maintenance management, and comprehensive feedback that takes into account the operator's emotional state.

[1637] "Robot operation video data" refers to video information of the operation of a robot used in a factory recorded by a video recording device such as a video camera.

[1638] A "generative AI model" is an artificial intelligence model used to analyze robot motion data and other input data to calculate specific performance metrics.

[1639] "Motion performance" refers to a comprehensive evaluation of various indicators related to work efficiency and quality, such as the accuracy, speed, and trajectory of the movements performed by the robot.

[1640] A "report" is a document or data summarizing the analysis results obtained based on a generative AI model, including quantification of behavioral performance, indications of behavioral patterns, and areas for improvement.

[1641] "Emotion analysis means" refers to technology or devices that analyze the emotional state of operators, allowing for the assessment of their stress levels, motivation, etc.

[1642] "Feedback" refers to suggestions for improvement and advice provided based on the analysis results, allowing the robot and operator to take appropriate measures.

[1643] "Abnormal behavior data" refers to data that records the robot's unexpected or abnormal behavior.

[1644] A "repair program" is a repair procedure or process designed based on the cause of the abnormal behavior, which will restore the robot's normal operation.

[1645] Biomechanics is an academic field that analyzes the mechanical elements of robot movement. It is often applied to the analysis of human movement, but here we will also apply it to the analysis of robot movement.

[1646] To realize this invention, it is first necessary to collect video data and motion capture data of each robot's movements and upload them to a server. Users film the robot's movements in the factory with a video camera or other video recording device, and also record detailed movement patterns using a motion capture device.

[1647] The server analyzes the received motion video data using a dedicated AI model library (for example, "some_AI_model_library"). This analysis quantifies the speed, accuracy, trajectory, etc. of the motion and evaluates detailed motion performance. If abnormal motion occurs, the abnormal motion data is also analyzed to identify the cause and generate an optimal repair program. The results of this analysis are compiled into a report and sent to the user's device.

[1648] Additionally, an emotion analysis tool (e.g., "some_emotion_engine") is used to evaluate the emotional state of the operator. This allows the operator's stress level and motivation to be assessed and reflected in the feedback. The server-generated report also includes advice and areas for improvement based on the operator's emotional state, providing comprehensive feedback.

[1649] As a concrete example, video and motion capture data of a factory robot assembling parts is collected and uploaded to a server. Based on this data, the server quantifies the robot's movement accuracy and speed and generates a detailed report. This report is provided to the operator and includes advice on improving performance and preventative maintenance. If any abnormal behavior occurs, the cause is identified and a repair program is quickly implemented. The operator's emotional state is also taken into consideration, and advice on maintaining motivation is also included.

[1650] An example of a prompt for a generative AI model is, "Analyze the video and motion capture data of the factory robot's movements, quantify the accuracy, speed, and trajectory of the movements, and generate a report. Also, evaluate the operator's emotional state (such as stress level) and include it in the report."

[1651] This system not only enables efficient and effective performance optimization and maintenance management of factory robots, but also provides comprehensive support that takes into account the emotional state of the operator.

[1652] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1653] Step 1:

[1654] The user films the factory robot's movements with a video camera and records detailed movement patterns with a motion capture device.

[1655] Input: Robot motion video data, motion capture data

[1656] Output: Motion video data and motion capture data saved on the device

[1657] Specific movements: The user positions the camera and motion capture devices appropriately to film and record the robot's movements.

[1658] Step 2:

[1659] The video data and motion capture data that the user has filmed and recorded are uploaded to the server.

[1660] Input: Motion video data stored on the device, motion capture data

[1661] Output: Motion video data and motion capture data transferred to the server

[1662] Specific operation: The user transfers data to the server using a dedicated upload tool.

[1663] Step 3:

[1664] The video data received by the server is analyzed using a generative AI model.

[1665] Input: Motion video data stored on the server

[1666] Output: Analysis results such as robot movement speed, accuracy, and trajectory

[1667] Specific operation: The server inputs video data into the AI ​​model, calculates the movement speed and accuracy for each frame, and tracks and quantifies the trajectory.

[1668] Step 4:

[1669] The server analyzes the received motion capture data using a generative AI model to identify movement patterns and biomechanical issues.

[1670] Input: Motion capture data stored on the server

[1671] Output: Movement patterns and biomechanics issues

[1672] Specific movements: The server inputs motion capture data into an AI model, which analyzes the movement of each joint and identifies movement patterns and biomechanical issues.

[1673] Step 5:

[1674] The server generates and provides a report of operational performance based on the analysis results.

[1675] Input: movement speed, accuracy, trajectory, movement patterns, biomechanical issues

[1676] Output: Operational performance report

[1677] What it does: The server consolidates the analysis results and generates a visually comprehensible report, including areas for improvement and recommended actions.

[1678] Step 6:

[1679] The server uses an emotion analysis means to evaluate the emotional state of the operator and provides feedback content.

[1680] Input: Operator emotion data

[1681] Output: Improvements and advice reflecting the operator's emotional state

[1682] Specific operation: The server analyzes the emotional data using the emotion analysis means to assess stress levels and motivation, and includes this in the report.

[1683] Step 7:

[1684] If the robot behaves abnormally, the server analyzes the abnormal behavior data, identifies the cause, and generates a repair program.

[1685] Input: Abnormal behavior data

[1686] Output: Cause identification results, repair program

[1687] Specific operation: The server inputs abnormal operation data into the AI ​​model, uses prompt sentences to identify the cause, and generates repair procedures.

[1688] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1689] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1690] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1691] [Fourth embodiment]

[1692] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1693] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1694] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1695] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1696] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1697] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1698] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1699] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1700] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1701] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1702] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1703] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1704] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1705] This system maximizes athletes' athletic performance and prevents injuries by analyzing video and motion capture data of athletes using a generative AI model and generating detailed reports to help athletes improve their performance and prevent injuries. In the event of an injury, the system can also analyze injury data and predict recovery time and performance.

[1706] Performance Analysis System

[1707] The system first receives video data of the player's movements. The user records video during practice or a match on their device and uploads it to the system. The server receives the video data and performs a detailed analysis using a generative AI model, quantifying the speed, accuracy, and trajectory of the movement. A performance report is generated based on the analysis results and sent to the device. The user can review the report and learn how to improve their own movements.

[1708] As a concrete example, when a player uploads a video of himself practicing dribbling a basketball to the system, the system quantifies the speed, accuracy, and swiftness of the dribbling motion, and provides the user with a report with an overall evaluation.

[1709] Motion Capture Analysis System

[1710] In motion capture analysis, a player's movements are collected using a motion capture device. The movement data transferred from the device is stored on the device and uploaded to the system. The server analyzes the received data using a generative AI model to identify the player's movement patterns and biomechanical problems. A report is generated based on this and sent to the device. The user can review the report and receive guidance to improve their movement patterns.

[1711] For example, an athlete wears a motion capture device to record their running form and collect data. The system analyzes the movement patterns of their left and right feet, knee angles, and ankle movements, identifies problems, and provides a report.

[1712] Recovery Time and Performance Prediction System

[1713] When an athlete is injured, detailed injury data (diagnosis results, treatment progress, etc.) is uploaded from the device to the system. The server receives this data and analyzes it using a generative AI model. Based on the analysis results, a predicted recovery time and an optimal rehabilitation program are provided. A feedback report is sent to the device, allowing the user to use it to efficiently promote recovery.

[1714] For example, if an athlete sprains their ankle, they can input the details of the injury into the system. The system will compare the data with similar data from the past and provide a prediction of recovery time and rehabilitation program. The user can then use this information to create a rehabilitation plan and progress their recovery.

[1715] This invention will scientifically support athletes in improving their performance and preventing injuries, enabling efficient rehabilitation. In the initial stages, the system will partner with schools and other institutions to accumulate experience through trial and error, before being packaged and widely deployed. It is expected that this will enable effective support for a wide range of people, from professional athletes to students.

[1716] The processing flow will be explained below.

[1717] Performance analysis process steps

[1718] Step 1:

[1719] User: Record videos of practice and games and save them on the device.

[1720] Step 2:

[1721] Device: Open the video upload screen and select the saved video file.

[1722] Step 3:

[1723] On your device: Upload the selected video file to the server.

[1724] Step 4:

[1725] Server: Receives uploaded video data.

[1726] Step 5:

[1727] Server: Calls the generative AI model and analyzes the video data frame by frame.

[1728] Step 6:

[1729] Server: As a result of the analysis, it generates numerical data on the speed, accuracy, trajectory, etc. of the movement.

[1730] Step 7:

[1731] Server: Generates a performance report based on the analysis results.

[1732] Step 8:

[1733] Server: Sends the generated performance report to the terminal.

[1734] Step 9:

[1735] Terminal: Displays the received performance report to the user.

[1736] Processing steps for motion capture analysis

[1737] Step 1:

[1738] User: Wears a motion capture device to collect movement data during practice and games.

[1739] Step 2:

[1740] Terminal: Transfers movement data from the motion capture device to the terminal.

[1741] Step 3:

[1742] Terminal: Select the transferred operation data on the system's upload screen.

[1743] Step 4:

[1744] Terminal: Uploads the selected motion data to the server.

[1745] Step 5:

[1746] Server: Receives uploaded operational data.

[1747] Step 6:

[1748] Server: Calls the generative AI model and analyzes the behavior data.

[1749] Step 7:

[1750] Server: As a result of the analysis, problems with movement patterns and biomechanics are identified.

[1751] Step 8:

[1752] Server: Generates a motion capture report based on the analysis results.

[1753] Step 9:

[1754] Server: Sends the generated reports to the device.

[1755] Step 10:

[1756] Terminal: Displays received reports to the user.

[1757] Recovery time and performance prediction processing steps

[1758] Step 1:

[1759] User: Enter detailed data about the injury (diagnosis results, treatment progress, etc.) into the terminal.

[1760] Step 2:

[1761] Terminal: Select the entered data on the system's upload screen.

[1762] Step 3:

[1763] Terminal: Uploads the selected injury data to the server.

[1764] Step 4:

[1765] Server: Receives the uploaded data.

[1766] Step 5:

[1767] Server: Calls the generative AI model and analyzes injury data.

[1768] Step 6:

[1769] Server: The analysis results generate a recovery time and performance forecast.

[1770] Step 7:

[1771] Server: Generates a feedback report based on the analysis results.

[1772] Step 8:

[1773] Server: Sends the generated feedback report to the terminal.

[1774] Step 9:

[1775] Terminal: Displays the received feedback report to the user.

[1776] Example 1

[1777] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1778] Maximizing athletes' athletic performance, preventing injuries, and efficiently recovering from existing injuries are extremely important in the field of sports. However, conventional methods for analyzing athletes' movements, preventing injuries, and predicting recovery rely on expert knowledge and manual work, resulting in insufficient accuracy and efficiency. To solve these issues, a system is needed that can efficiently and accurately analyze athletes' movement and injury data and quickly provide scientifically based reports.

[1779] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1780] In this invention, the server includes means for acquiring motion video data of the player from an input device, means for receiving the acquired motion video data with a central processing unit, means for analyzing the received data using a generative AI model and quantifying motion performance, and means for generating a motion performance report based on the analysis results and sending it to the user device. This enables scientific and efficient analysis of the player's motion and detailed feedback for performance improvement and injury prevention.

[1781] The system also includes a means for acquiring motion capture data of the player from an input device, a means for receiving the acquired motion capture data with a central processing unit, a means for analyzing the received data using a generative AI model to identify problems with movement patterns and biomechanics, and a means for generating a report on the movement patterns and biomechanics problems based on the analysis results and sending it to the user device, thereby enabling accurate analysis of the player's movements and helping to improve movements and performance.

[1782] The system also includes means for acquiring data on the player's injury from an input device, means for receiving the acquired injury data with a central processing unit, means for analyzing the received data using a generative AI model to predict recovery time and performance, and means for generating a report on the recovery time and performance prediction based on the analysis results and sending it to the user device, thereby making it possible to streamline the injury recovery process and provide a scientific rehabilitation program.

[1783] "Motion video data" refers to digital video data that records an athlete actually exercising.

[1784] An "input device" is a device used by a user to collect data, including a smartphone, tablet, motion capture device, etc.

[1785] A "central processing unit" is a server computer or cloud platform that receives and analyzes uploaded data.

[1786] "Generative AI models" refer to algorithms and their implementations for analyzing data using machine learning and deep learning, including those using frameworks such as TensorFlow and PyTorch.

[1787] "Movement performance" is an evaluation index related to an athlete's motor functions, such as athletic ability, technical accuracy, and speed.

[1788] A "report" is a document or digital file that presents the analyzed results to the user in an easy-to-understand manner, detailing performance, problems, areas for improvement, etc.

[1789] "Motion capture data" refers to digital data recorded using a motion capture device, which represents the movements of each part of the player's body in three-dimensional coordinates.

[1790] "Movement patterns" refer to the repetition and habitual movements of an athlete, and are understood by analyzing the trajectory and timing of the movements.

[1791] "Biomechanics" is an academic field that deals with the movement and mechanical characteristics of the human body, and collectively refers to the movement of an athlete's body, the angles of their joints, and the function of their muscles during exercise.

[1792] "Injury data" refers to digital data detailing the injury sustained by a player, including information such as diagnosis, treatment progress, and pain level.

[1793] "Recovery time" refers to the period of time required for an athlete to fully recover from an injury and return to their previous performance.

[1794] "Performance prediction" refers to the analysis results that predict the likely improvement in an athlete's athletic ability and technique after recovery from an injury.

[1795] The present invention provides a system for maximizing an athlete's athletic performance, preventing injuries, and supporting efficient rehabilitation. Specific embodiments will be described below.

[1796] Performance Analysis System

[1797] First, a user uses an input device such as a smartphone or tablet to record video data of the player's movements. The recorded video data is then uploaded to a server via an application on the device. The server receives the uploaded video data and analyzes it using a generative AI model (e.g., TensorFlow or PyTorch).

[1798] Specifically, the system analyzes each frame of video data and quantifies the speed, accuracy, and trajectory of the player's movements. The server automatically generates a detailed performance report based on the analysis results and sends it to the device. The user can then review the report on their device and learn how to improve their movements.

[1799] Examples:

[1800] A user records a video of themselves practicing dribbling a basketball and uploads it to the system.

[1801] The system quantifies the speed, accuracy, and quickness of changes in direction of dribbling movements and provides an overall evaluation.

[1802] At the end, users receive a report detailing performance trends and areas for improvement.

[1803] Example prompt sentence:

[1804] "Analyze video of basketball dribbling practice, evaluate the speed, accuracy, and change of direction of the movement, and provide a detailed report of the results."

[1805] Motion Capture Analysis System

[1806] Next, in motion capture analysis, the user uses a motion capture device (e.g., Vicon) to capture the player's movements. The collected motion capture data is stored on the device and later uploaded to a server. The server analyzes the received data using a generative AI model (e.g., PyTorch).

[1807] The server analyzes and identifies problems in the player's movement patterns and biomechanics. Based on the analysis results, a detailed report is generated and sent to the device. The user can then review the report to understand where to improve the movement patterns and provide guidance.

[1808] Examples:

[1809] The user has the athlete wear a motion capture device to collect data for the purpose of analyzing their running form.

[1810] The system analyzes the movement patterns of the left and right feet, knee angles, and ankle movements to identify problems.

[1811] Users receive guidance based on a report containing detailed analysis results and areas for improvement.

[1812] Example prompt sentence:

[1813] "Analyze the motion capture data of the running form, evaluate the movement patterns of the left and right feet, knee angles, and ankle movements, and provide a detailed report of the results."

[1814] Recovery Time and Performance Prediction System

[1815] Furthermore, if a player is injured, the user inputs detailed data about the injury (diagnosis results, treatment progress, etc.) into the device. The input data is uploaded to the server, which receives it and analyzes it using a generative AI model (e.g., IBM Watson).

[1816] Through analysis, the server predicts injury recovery time and performance. Based on the analysis results, a feedback report is generated that provides a predicted recovery time and an optimal rehabilitation program, and is sent to the device. The user can then review the feedback report and create an efficient rehabilitation plan.

[1817] Examples:

[1818] A user enters injury data for a player who has a sprained ankle into the system.

[1819] The system compares similar past data to predict recovery time and provide a rehabilitation program.

[1820] Based on this, users can create a rehabilitation plan and efficiently progress with recovery.

[1821] Example prompt sentence:

[1822] "Based on detailed data on ankle sprains, predict recovery time and optimal rehabilitation program, and provide a detailed report of the results."

[1823] These embodiments make it possible to realize a system that scientifically supports athletes in improving their athletic performance, preventing injuries, and recovering efficiently.

[1824] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1825] Performance Analysis System

[1826] Step 1: Record your video

[1827] Users use input devices such as smartphones and tablets to record video data of athletes' movements.

[1828] Specific operations: Record practice and game situations and save them on the device.

[1829] Input: Player action scene

[1830] Output: Operation video data (MP4 format, etc.)

[1831] Step 2: Upload your video

[1832] The user uploads the recorded video data to the server via an application on the device.

[1833] Specific operation: Press the upload button in the application, select the video file and send it.

[1834] Input: Motion video data

[1835] Output: Video data sent to the server

[1836] Step 3: Receiving video data

[1837] The server receives the uploaded video data.

[1838] Specific behavior: Receives an HTTP request and saves the video file.

[1839] Input: Video data sent to the server

[1840] Output: Saved video file

[1841] Step 4: Analyze the video data

[1842] The server uses a generative AI model to analyze the received video data.

[1843] Specific actions: Analyze video frame by frame to quantify the speed, accuracy, and trajectory of movements.

[1844] Input: Saved video file

[1845] Output: Analysis result data (JSON format, etc.)

[1846] Step 5: Generate a performance report

[1847] The server generates a performance report based on the analysis results.

[1848] Specific operation: Automatically generate a report using analysis result data and save it in PDF or HTML format.

[1849] Input: Analysis result data

[1850] Output: Performance report

[1851] Step 6: Submit the report

[1852] The server transmits the generated performance report to the terminal.

[1853] What it does: Sends reports via email or a dedicated application.

[1854] Input: Performance Report

[1855] Output: Report sent to terminal

[1856] Step 7: Review the report

[1857] The user checks the sent report on the terminal.

[1858] What it does: Open the report you receive, review it, and learn how to improve it.

[1859] Input: Report sent to terminal

[1860] Output: Information to confirm and implement improvements.

[1861] Motion Capture Analysis System

[1862] Step 1: Collecting motion capture data

[1863] Users use motion capture devices to record the movements of players.

[1864] Specific movements: The athletes wear markers and their movements are recorded using a motion capture device.

[1865] Input: Player action scene

[1866] Output: Motion capture data (CSV, JSON, etc. formats)

[1867] Step 2: Upload your data

[1868] The user saves the collected motion capture data on the device and uploads it to the server.

[1869] Specific operation: Press the upload button in the application to send the data file to the server.

[1870] Input: Motion capture data

[1871] Output: Operational data sent to the server

[1872] Step 3: Receiving the data

[1873] The server receives the uploaded motion capture data.

[1874] Specific behavior: Receives an HTTP request and saves a data file.

[1875] Input: Operational data sent to the server

[1876] Output: Saved data file

[1877] Step 4: Analyze the data

[1878] The server uses a generative AI model to analyze the received motion capture data.

[1879] Specific Movements: Analyze data to identify movement patterns and biomechanical issues.

[1880] Input: Saved data file

[1881] Output: Analysis result data (JSON format, etc.)

[1882] Step 5: Generate reports

[1883] The server generates a report on movement patterns and biomechanical issues based on the analysis results.

[1884] Specific operation: Automatically generate a report using analysis result data and save it in PDF or HTML format.

[1885] Input: Analysis result data

[1886] Output: Movement pattern and biomechanics problem report

[1887] Step 6: Submit the report

[1888] The server sends the generated report to the terminal.

[1889] What it does: Sends reports via email or a dedicated application.

[1890] Input: Movement Pattern and Biomechanics Problem Report

[1891] Output: Report sent to terminal

[1892] Step 7: Review the report

[1893] The user checks the sent report on the terminal.

[1894] What it does: Open the report you receive, review it, and learn how to improve it.

[1895] Input: Report sent to terminal

[1896] Output: Information to confirm and implement improvements.

[1897] Recovery Time and Performance Prediction System

[1898] Step 1: Enter injury data

[1899] The user enters details about the player's injury into the terminal.

[1900] Specific operations: Enter information such as diagnosis results and treatment progress into a data entry form.

[1901] Input: Detailed injury information

[1902] Output: Input injury data

[1903] Step 2: Upload your data

[1904] The user uploads the injury data they entered to the server.

[1905] Specific operation: Press the upload button in the application to send the data.

[1906] Input: Injury data

[1907] Output: Injury data sent to the server

[1908] Step 3: Receiving the data

[1909] The server receives the uploaded injury data.

[1910] Specific behavior: Receives an HTTP request and saves a data file.

[1911] Input: Injury data sent to server

[1912] Output: Saved data file

[1913] Step 4: Analyze the data

[1914] The server uses a generative AI model to analyze the injury data it receives.

[1915] Specific actions: Compare with similar cases from the past, predict recovery time and suggest optimal rehabilitation programs.

[1916] Input: Stored injury data

[1917] Output: Analysis result data (JSON format, etc.)

[1918] Step 5: Generate a feedback report

[1919] The server generates a feedback report based on the analysis results.

[1920] Specific operation: Automatically generate a report using analysis result data and save it in PDF or HTML format.

[1921] Input: Analysis result data

[1922] Output: Feedback report

[1923] Step 6: Submit the report

[1924] The server transmits the generated feedback report to the terminal.

[1925] What it does: Sends reports via email or a dedicated application.

[1926] Input: Feedback Report

[1927] Output: Report sent to terminal

[1928] Step 7: Review the report

[1929] The user checks the sent report on the terminal and creates an efficient rehabilitation plan.

[1930] Specific actions: Open the received report, review the contents, and develop a rehabilitation plan.

[1931] Input: Report sent to terminal

[1932] Output: Efficient rehabilitation plan

[1933] (Application example 1)

[1934] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1935] The goal is to maximize the operational efficiency of machinery operating in factories and to detect signs of breakdowns and prevent them before they occur. With conventional methods, it has been difficult to analyze machine performance in real time and carry out appropriate maintenance.

[1936] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1937] In this invention, the server includes a means for receiving machine operation video data, a means for analyzing the received operation video data using a generative AI model to quantify operation efficiency, and a means for generating and providing an operation efficiency report based on the analysis results, thereby enabling scientific analysis of the machine's operation efficiency and identifying maintenance and improvement points at appropriate times.

[1938] "Machines" are automated devices and robots used in factories and other places to process, assemble, and transport objects.

[1939] "Operation video data" refers to video data that records the operation of a machine and is uploaded to a server for analysis.

[1940] A "generative AI model" is an algorithm that uses machine learning and deep learning techniques and is trained to perform data analysis.

[1941] "Operational efficiency" is an index that evaluates the energy consumption and time required when a machine performs a specified task.

[1942] The "means for receiving video data" is an interface that allows the server to receive video data from the outside.

[1943] The "means for quantifying motion efficiency" is a process of analyzing received video data and converting parameters related to motion into specific numerical values.

[1944] The "means for generating and providing a report" refers to a process for creating a report summarizing evaluation information regarding the operation of the machine based on the analyzed data and providing the report to the user.

[1945] The system for implementing this invention analyzes the operational efficiency of machines in a factory, and provides failure prevention and maintenance support. The system operates through the cooperation of three parties: a server, a terminal, and a user.

[1946] System configuration

[1947] The system includes the following hardware and software:

[1948] server

[1949] Video analysis and data processing are performed on the server side, mainly using OpenCV, Keras, and TensorFlow.

[1950] Terminal

[1951] Used to collect and upload motion video data. Applies to smartphones and head-mounted displays.

[1952] User

[1953] These are factory managers and operators who perform maintenance and improvements on machines based on reports provided by the system.

[1954] Program processing

[1955] The server first receives video data of the machine's operation sent from the device. This data is broken down into frames using OpenCV, and image preprocessing is performed. The preprocessed data is then analyzed using a generative AI model using Keras and TensorFlow. Here, parameters related to operational efficiency (speed, accuracy, movement trajectory, etc.) are quantified.

[1956] Based on the analysis results, the server generates a detailed report that includes not only an evaluation of operational efficiency but also any identified mechanical issues and suggestions for improvement, and the report is sent to the terminal for review by the user.

[1957] Users can plan and implement machine maintenance and operational improvements based on the report content. In particular, if signs of a malfunction are detected, it is necessary to carry out preventative maintenance as soon as possible.

[1958] Specific examples

[1959] Factory managers use their smartphones to record videos of robots operating in the factory and upload them to a dedicated app. The server receives the videos, analyzes operational efficiency and mechanical problems, generates a report, and sends it to the manager's device. The manager can then check the detailed report on the app and plan and implement improvements to the robot's operation and maintenance.

[1960] Prompt Sentence Examples

[1961] "Just take a video of your robots operating in your factory and upload it to the app. The system will analyze the video and provide you with a detailed report on the robots' speed, accuracy, and movement trajectory. You can then identify areas for improvement and carry out appropriate maintenance."

[1962] As described above, the system for carrying out the present invention scientifically analyzes the operation of machines in a factory and supports the prevention of breakdowns and the improvement of operation.

[1963] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1964] Step 1:

[1965] The device captures video of the machines in operation in the factory. The input is the video of the machines in operation, and the output is the captured video data. The user operates the device using a smartphone or head-mounted display.

[1966] Step 2:

[1967] The device uploads the captured video data to the server. The input is the captured video data, and the output is the video data stored on the server.

[1968] Step 3:

[1969] The server receives the video data and decomposes the video into frames using OpenCV. The input is the received video data, and the output is individual frame images for preprocessing.

[1970] Step 4:

[1971] The server performs image preprocessing (resizing, normalization) on each frame image. The input is the decomposed frame image, and the output is the preprocessed frame image.

[1972] Step 5:

[1973] The server inputs the preprocessed frame images into a generative AI model using Keras and TensorFlow to analyze parameters related to movement efficiency. The input is the preprocessed frame images, and the output is the analysis results related to movement efficiency (speed, accuracy, movement trajectory, etc.).

[1974] Step 6:

[1975] The server generates a detailed report based on the analysis results. The input is the analyzed data, and the output is a report that includes an evaluation of operational efficiency, problems, and improvement suggestions.

[1976] Step 7:

[1977] The server sends the generated report to the terminal, where the input is the generated report and the output is the report displayed on the terminal.

[1978] Step 8:

[1979] The user checks the report received on the terminal and plans and implements machine maintenance and operational improvements. The input is the report displayed on the terminal, and the output is the implementation of specific maintenance and improvement measures.

[1980] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1981] This system maximizes athletes' athletic performance and prevents injuries. It analyzes athletes' motion video data and motion capture data using a generative AI model and generates detailed reports to help improve athletes' performance and prevent injuries. By combining this with an emotion engine that recognizes athletes' emotions, it is possible to provide feedback that takes into account the user's emotional state. Furthermore, in the event of an injury, it is possible to analyze injury data and predict recovery time and performance.

[1982] Performance Analysis System

[1983] The system first receives video data of the player's movements. The user records video on their device during practice or a match and uploads it to the system. The server receives the video data and performs a detailed analysis using a generative AI model, quantifying the speed, accuracy, and trajectory of the movement. A performance report is generated based on the analysis results and sent to the device. Furthermore, an emotion engine is used to analyze the user's emotional state and reflect this in the feedback. The user can review the report and learn how to improve their own movements.

[1984] For example, when a player uploads a video of himself practicing dribbling a basketball to the system, the system quantifies the speed, accuracy, and speed of changes in direction of the dribble, and provides the user with a comprehensive evaluation report. In addition, the emotion engine evaluates how the user feels about the practice and provides advice to improve motivation.

[1985] Motion Capture Analysis System

[1986] In motion capture analysis, a player's movements are collected using a motion capture device. The movement data transferred from the device is stored on the device and uploaded to the system. The server analyzes the received data using a generative AI model to identify the player's movement patterns and biomechanical issues. A report is generated based on this and sent to the device. Furthermore, an emotion engine is used to analyze the user's emotional state and reflect it in the report. The user can review the report and receive guidance to improve their movement patterns.

[1987] For example, an athlete wears a motion capture device to record their running form and collect data. The system analyzes the movement patterns of the left and right feet, the knee angle, and the movement of the ankle, identifies problems, and provides a report. In addition, an emotion engine takes into account the user's emotional state and provides advice to improve training methods.

[1988] Recovery Time and Performance Prediction System

[1989] When an athlete is injured, detailed injury data (diagnosis results, treatment progress, etc.) is uploaded from the device to the system. The server receives this data and analyzes it using a generative AI model. Based on the analysis results, a predicted recovery time and an optimal rehabilitation program are provided. A feedback report is sent to the device, and the user can use this to promote efficient recovery. The system also takes the user's emotional state into account using an emotion engine to provide advice on maintaining motivation.

[1990] For example, if an athlete sprains their ankle, detailed injury data is entered into the system. The system compares this data with similar past data and predicts recovery time and rehabilitation programs. Based on this, the user can create a rehabilitation plan and progress with their recovery. Furthermore, an emotion engine evaluates the user's emotional state and provides advice to improve motivation during the rehabilitation period.

[1991] This invention will scientifically support athletes in improving their performance and preventing injuries, enabling efficient rehabilitation. Furthermore, by combining it with an emotion engine, it will be possible to provide comprehensive feedback that takes into account the user's emotional state, helping to maintain and improve the athlete's motivation. This is expected to provide effective support to a wide range of people, from professional athletes to students.

[1992] The processing flow will be explained below.

[1993] Performance analysis process steps

[1994] Step 1:

[1995] User: Record videos of practice and games and save them on the device.

[1996] Step 2:

[1997] Device: Open the video upload screen and select the saved video file.

[1998] Step 3:

[1999] On your device: Upload the selected video file to the server.

[2000] Step 4:

[2001] Server: Receives uploaded video data.

[2002] Step 5:

[2003] Server: Calls the generative AI model and analyzes the video data frame by frame.

[2004] Step 6:

[2005] Server: As a result of the analysis, it generates numerical data on the speed, accuracy, trajectory, etc. of the movement.

[2006] Step 7:

[2007] Server: Generates a performance report based on the analysis results.

[2008] Step 8:

[2009] Server: Calls the emotion engine and collects additional data to analyze the user's emotional state.

[2010] Step 9:

[2011] Server: The emotion engine analyzes the user's facial expressions and voice data to identify the user's emotional state.

[2012] Step 10:

[2013] Server: Generates a final performance report with adjusted feedback based on the emotional state.

[2014] Step 11:

[2015] Server: Sends the generated final performance report to the terminal.

[2016] Step 12:

[2017] Terminal: Displays the received performance report to the user.

[2018] Processing steps for motion capture analysis

[2019] Step 1:

[2020] User: Wears a motion capture device to collect movement data during practice and games.

[2021] Step 2:

[2022] Terminal: Transfers movement data from the motion capture device to the terminal.

[2023] Step 3:

[2024] Terminal: Select the transferred operation data on the system's upload screen.

[2025] Step 4:

[2026] Terminal: Uploads the selected motion data to the server.

[2027] Step 5:

[2028] Server: Receives uploaded operational data.

[2029] Step 6:

[2030] Server: Calls the generative AI model and analyzes the behavior data.

[2031] Step 7:

[2032] Server: As a result of the analysis, problems with movement patterns and biomechanics are identified.

[2033] Step 8:

[2034] Server: Generates a motion capture report based on the analysis results.

[2035] Step 9:

[2036] Server: Calls the emotion engine and collects additional data to analyze the user's emotional state.

[2037] Step 10:

[2038] Server: The emotion engine analyzes the user's facial expressions and voice data to identify the user's emotional state.

[2039] Step 11:

[2040] Server: Generates the final motion capture report with adjusted feedback based on the emotional state.

[2041] Step 12:

[2042] Server: Sends the generated final motion capture report to the device.

[2043] Step 13:

[2044] Terminal: Displays received reports to the user.

[2045] Recovery time and performance prediction processing steps

[2046] Step 1:

[2047] User: Enter detailed data about the injury (diagnosis results, treatment progress, etc.) into the terminal.

[2048] Step 2:

[2049] Terminal: Select the entered data on the system's upload screen.

[2050] Step 3:

[2051] Terminal: Uploads the selected injury data to the server.

[2052] Step 4:

[2053] Server: Receives the uploaded data.

[2054] Step 5:

[2055] Server: Calls the generative AI model and analyzes injury data.

[2056] Step 6:

[2057] Server: The analysis results generate a recovery time and performance forecast.

[2058] Step 7:

[2059] Server: Generates a feedback report based on the analysis results.

[2060] Step 8:

[2061] Server: Calls the emotion engine and collects additional data to analyze the user's emotional state.

[2062] Step 9:

[2063] Server: The emotion engine analyzes the user's facial expressions and voice data to identify the user's emotional state.

[2064] Step 10:

[2065] Server: Generates a final feedback report with adjusted feedback content based on the emotional state.

[2066] Step 11:

[2067] Server: Sends the generated final feedback report to the terminal.

[2068] Step 12:

[2069] Terminal: Displays the received feedback report to the user.

[2070] Example 2

[2071] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2072] Conventional athletic performance analysis systems are required not only to analyze athletes' movements but also to provide feedback that takes into account the athletes' emotional state. Current technology is capable of analyzing movement performance and predicting injury recovery, but it lacks the ability to appropriately evaluate athletes' motivation and emotional state and reflect them in feedback. As a result, the advice athletes receive is one-sided, limiting its effectiveness in improving overall performance and preventing injuries. To solve these problems, a system that combines athletes' movement analysis and emotional analysis is needed.

[2073] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving motion video data of a player, means for analyzing the received motion video data using a generative AI model and quantifying motion performance, means for generating and providing a report on the motion performance based on the analysis results, and means for evaluating the user's emotions using an emotion engine that analyzes the user's emotional state and reflecting the evaluation in the feedback content. This makes it possible to provide comprehensive feedback by combining the analysis results of the player's motion performance with emotion analysis.

[2074] "Motion video data" refers to video data that records the movements of players during practice or a match.

[2075] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to analyze input data and generate results.

[2076] "Movement performance" is a numerical evaluation of indicators such as the speed, accuracy, and trajectory of a player's movements.

[2077] A "Performance Report" is a document summarizing the results of analyzed operational performance, including suggestions for improvement and feedback.

[2078] An "emotion engine" is an algorithm that analyzes human emotions and is a system that evaluates emotional states based on facial expressions, voice, etc.

[2079] A "motion capture device" is a device that records the body movements of athletes in 3D, obtaining data on the movements of joints and various body parts.

[2080] Biomechanics is an academic field that deals with the movement and dynamics of the human body, and involves the scientific analysis of athletes' body movements.

[2081] "Injury Data" means detailed information about an injury sustained by a player, including diagnosis and treatment progress.

[2082] "Recovery time" is the estimated time it will take for an athlete to fully recover from an injury.

[2083] A "rehabilitation program" is an exercise and treatment plan designed to help an athlete recover from an injury and is provided to help the athlete recover efficiently.

[2084] "Feedback" refers to advice and instruction provided to players based on the results and evaluations analyzed by the system.

[2085] This invention is a system for maximizing athletes' athletic performance and preventing injuries. The system uses a generative AI model to analyze video data and motion capture data of athletes' movements and generate detailed reports to help athletes improve their performance and prevent injuries. Furthermore, by combining it with an emotion engine that recognizes athletes' emotions, it is possible to provide feedback that takes into account the user's emotional state.

[2086] Performance Analysis System

[2087] The system first receives video data of players' movements. Users film videos during practice or games using their devices (e.g., smartphones or tablets) and upload them to the system. The server receives the video data and performs detailed analysis using a generative AI model (e.g., TensorFlow or PyTorch). A high-performance server (e.g., an NVIDIA GPU or Intel processor) is used for the analysis. The speed, accuracy, trajectory, etc. of the movements in the video are quantified and a performance report is generated. Furthermore, an emotion engine is used to analyze the user's emotional state from their facial expressions and voice in the video and reflect this in the feedback content. For example, if a player uploads a video of themselves practicing dribbling a basketball to the system, the server will quantify the speed, accuracy, and quickness of the dribbling movement and provide an overall evaluation in the form of a report. The emotion engine can also be used to evaluate the user's emotional state and provide advice to improve motivation.

[2088] Example prompt sentence:

[2089] "Analyze the dribbling movements of the players in this video and quantify their speed, accuracy, and change of direction. Also, rate the players' emotional state."

[2090] Motion Capture Analysis System

[2091] In motion capture analysis, an athlete's movements are collected using a motion capture device (e.g., Vicon or OptiTrack). The movement data transferred from the device is stored on the device and uploaded to the system. The server analyzes the received data using a generative AI model to identify the athlete's movement patterns and biomechanical problems. A report is compiled into graphs and tables using a Python script and sent to the user's device. The emotion engine also analyzes the user's emotional state and reflects it in the report content. As a specific example, an athlete wears a motion capture device to record their running form and collects data. The server analyzes the movement patterns of each foot, knee angle, and ankle movement, and provides a report identifying problems. The emotion engine also takes the user's emotional state into account and provides advice for improving training methods.

[2092] Example prompt sentence:

[2093] "Based on this motion capture data, please analyze the athlete's running form and quantify the movement patterns of the left and right feet, the angle of the knees, and the movement of the ankles."

[2094] Recovery Time and Performance Prediction System

[2095] When an athlete is injured, detailed injury data (diagnosis results, treatment progress, etc.) is uploaded from the device to the system. The server receives this data and analyzes it using a generative AI model. Based on the analysis results, a predicted recovery time and an optimal rehabilitation program are provided. A feedback report detailing the analysis results is generated using a Python script and sent to the user's device. An emotion engine is used to analyze the user's emotional state and provide advice to maintain motivation during the rehabilitation period. As a specific example, if an athlete sprains their ankle, detailed injury data is entered into the system, and the server compares it with similar past data and predicts and provides a recovery time and rehabilitation program. The emotion engine also evaluates the user's emotional state and provides advice to maintain motivation.

[2096] Example prompt sentence:

[2097] "Based on this injury data, predict recovery time and the optimal rehabilitation program. Also, assess the user's emotional state."

[2098] This invention makes it possible to comprehensively manage an athlete's movements, recovery, and emotional state, providing optimal support for improving performance and preventing injuries. This is expected to provide effective support to a wide range of people, from professional athletes to students.

[2099] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2100] Performance Analysis System

[2101] Step 1:

[2102] Users use their devices to record the movements of players during practice and games. By recording using a device, it is possible to record the movements of players in real time. Specifically, videos are taken using the camera function of a smartphone or tablet.

[2103] Input: Video of player in action

[2104] Output: Recorded video file

[2105] Step 2:

[2106] Users upload the videos they have taken to the system by sending the video files to the server via a dedicated application or web portal.

[2107] Input: Recorded video file

[2108] Output: Video data stored on a cloud server

[2109] Step 3:

[2110] The server receives the video data sent by the user, and the received video data is stored in cloud storage.

[2111] Input: Video data uploaded to the cloud server

[2112] Output: Video data stored in cloud storage

[2113] Step 4:

[2114] The server inputs the received video into a generative AI model for analysis. The model is built using TensorFlow and PyTorch, and converts the speed, accuracy, and trajectory of the movements in the video into numerical data.

[2115] Input: Video data stored in cloud storage

[2116] Output: Numerical data of movement (speed, accuracy, trajectory)

[2117] Step 5:

[2118] The server generates detailed performance reports based on the analysis data, including performance metrics and areas for improvement. Using Python scripts, the results are visualized in graphs and tables and compiled in PDF format.

[2119] Input: Numerical data of the action

[2120] Output: Performance Report (PDF)

[2121] Step 6:

[2122] The server then sends the generated performance report to the user's device, and the user is notified of the report via a dedicated application or email.

[2123] Input: Performance Report

[2124] Output: Report notification delivered to the user's device

[2125] Step 7:

[2126] The server uses an emotion engine to analyze the user's emotional state in the video. Based on the analysis results, the server adds advice to improve motivation to the report.

[2127] Input: Video data

[2128] Output: Sentiment analysis results, feedback advice

[2129] Motion Capture Analysis System

[2130] Step 1:

[2131] Users collect movement data using motion capture devices, such as Vicon or OptiTrack, which record the movements of the athlete's joints and body as 3D coordinate data.

[2132] Input: Player movement

[2133] Output: Motion capture data (3D coordinate data)

[2134] Step 2:

[2135] The user saves the collected operation data on the device and uploads it to the system, where it is sent to the cloud server via a PC.

[2136] Input: Motion capture data

[2137] Output: Operation data stored on the cloud server

[2138] Step 3:

[2139] The server receives the data sent from the motion capture device and stores it in cloud storage.

[2140] Input: Motion data uploaded to the cloud server

[2141] Output: Data stored in cloud storage

[2142] Step 4:

[2143] The server analyzes the movement data using generative AI models, leveraging ML models such as Scikit-learn to identify movement patterns and biomechanical issues.

[2144] Input: Motion data stored in cloud storage

[2145] Power output: movement patterns and biomechanics issues

[2146] Step 5:

[2147] The server generates a report based on the analysis results. Using a Python script, the analysis results are visualized and compiled into a PDF report.

[2148] Input: Movement patterns and biomechanical issues

[2149] Output: Movement pattern and biomechanics report (PDF)

[2150] Step 6:

[2151] The server sends the generated report to the user's device, and the user is notified of the report via a dedicated application or email.

[2152] Input: Movement patterns and biomechanics reports

[2153] Output: Report notification delivered to the user's device

[2154] Step 7:

[2155] The server uses an emotion engine to analyze the user's emotional state and provides advice on how to improve the training method based on the analysis results.

[2156] Input: Operation data

[2157] Output: Sentiment analysis results, advice on improving training

[2158] Recovery Time and Performance Prediction System

[2159] Step 1:

[2160] The user inputs detailed injury data (diagnosis results, treatment progress, etc.) into the terminal, enters the data using electronic medical record software, and then exports and uploads it to the system.

[2161] Input: Injury details

[2162] Output: Injury data stored on the device

[2163] Step 2:

[2164] The user uploads the entered data to the system, which then sends the data to the cloud server.

[2165] Input: Injury data stored on the device

[2166] Output: Injury data stored on a cloud server

[2167] Step 3:

[2168] The server receives the uploaded injury data and stores it in a database (e.g., PostgreSQL).

[2169] Input: Injury data uploaded to a cloud server

[2170] Output: Injury data stored in a database

[2171] Step 4:

[2172] The server analyzes injury data using a generative AI model, comparing it with past data to predict recovery time and optimal rehabilitation programs.

[2173] Input: Injury data stored in a database

[2174] Output: Prediction of recovery time and rehabilitation program

[2175] Step 5:

[2176] The server generates a feedback report based on the analysis results. A Python script is used to create a PDF report detailing the recovery process and rehabilitation protocol.

[2177] Input: Prediction of recovery time and rehabilitation program

[2178] Output: Feedback report (PDF)

[2179] Step 6:

[2180] The server sends the generated report to the user's device, and the user is notified of the report via a dedicated application or email.

[2181] Input: Feedback Report

[2182] Output: Report notification delivered to the user's device

[2183] Step 7:

[2184] The server uses an emotion engine to analyze the user's emotional state and adds advice to help maintain motivation during the rehabilitation period.

[2185] Input: Injury data

[2186] Output: Emotion analysis results, advice for maintaining motivation

[2187] This allows the system of the present invention to comprehensively manage an athlete's movements, recovery, and emotional state, providing optimal support for improving performance and preventing injuries.

[2188] (Application example 2)

[2189] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2190] Robots used in factories require regular maintenance to maintain their precision and speed, but current methods make it difficult to efficiently optimize performance and perform preventative maintenance. Furthermore, the emotional state of robot operators tends to be ignored, which can have a negative impact on work efficiency and safety. Furthermore, when abnormal behavior occurs, quickly and accurately identifying the cause and providing an appropriate repair program is also a challenge.

[2191] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2192] In this invention, the server includes: [means for receiving robot operation video data;] [means for analyzing the received operation video data using a generative AI model and quantifying operation performance;] [means for generating and providing an operation performance report based on the analysis results;] [means for reflecting the operator's emotional state in the feedback content using emotion analysis means; and [means for, if the robot performs an abnormal operation, analyzing the abnormal operation data, identifying the cause, and generating a repair program.] This enables optimization of the robot's operation performance, preventive maintenance management, and comprehensive feedback that takes into account the operator's emotional state.

[2193] "Robot operation video data" refers to video information of the operation of a robot used in a factory recorded by a video recording device such as a video camera.

[2194] A "generative AI model" is an artificial intelligence model used to analyze robot motion data and other input data to calculate specific performance metrics.

[2195] "Motion performance" refers to a comprehensive evaluation of various indicators related to work efficiency and quality, such as the accuracy, speed, and trajectory of the movements performed by the robot.

[2196] A "report" is a document or data summarizing the analysis results obtained based on a generative AI model, including quantification of behavioral performance, indications of behavioral patterns, and areas for improvement.

[2197] "Emotion analysis means" refers to technology or devices that analyze the emotional state of operators, allowing for the assessment of their stress levels, motivation, etc.

[2198] "Feedback" refers to suggestions for improvement and advice provided based on the analysis results, allowing the robot and operator to take appropriate measures.

[2199] "Abnormal behavior data" refers to data that records the robot's unexpected or abnormal behavior.

[2200] A "repair program" is a repair procedure or process designed based on the cause of the abnormal behavior, which will restore the robot's normal operation.

[2201] Biomechanics is an academic field that analyzes the mechanical elements of robot movement. It is often applied to the analysis of human movement, but here we will also apply it to the analysis of robot movement.

[2202] To realize this invention, it is first necessary to collect video data and motion capture data of each robot's movements and upload them to a server. Users film the robot's movements in the factory with a video camera or other video recording device, and also record detailed movement patterns using a motion capture device.

[2203] The server analyzes the received motion video data using a dedicated AI model library (for example, "some_AI_model_library"). This analysis quantifies the speed, accuracy, trajectory, etc. of the motion and evaluates detailed motion performance. If abnormal motion occurs, the abnormal motion data is also analyzed to identify the cause and generate an optimal repair program. The results of this analysis are compiled into a report and sent to the user's device.

[2204] Additionally, an emotion analysis tool (e.g., "some_emotion_engine") is used to evaluate the emotional state of the operator. This allows the operator's stress level and motivation to be assessed and reflected in the feedback. The server-generated report also includes advice and areas for improvement based on the operator's emotional state, providing comprehensive feedback.

[2205] As a concrete example, video and motion capture data of a factory robot assembling parts is collected and uploaded to a server. Based on this data, the server quantifies the robot's movement accuracy and speed and generates a detailed report. This report is provided to the operator and includes advice on improving performance and preventative maintenance. If any abnormal behavior occurs, the cause is identified and a repair program is quickly implemented. The operator's emotional state is also taken into consideration, and advice on maintaining motivation is also included.

[2206] An example of a prompt for a generative AI model is, "Analyze the video and motion capture data of the factory robot's movements, quantify the accuracy, speed, and trajectory of the movements, and generate a report. Also, evaluate the operator's emotional state (such as stress level) and include it in the report."

[2207] This system not only enables efficient and effective performance optimization and maintenance management of factory robots, but also provides comprehensive support that takes into account the emotional state of the operator.

[2208] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2209] Step 1:

[2210] The user films the factory robot's movements with a video camera and records detailed movement patterns with a motion capture device.

[2211] Input: Robot motion video data, motion capture data

[2212] Output: Motion video data and motion capture data saved on the device

[2213] Specific movements: The user positions the camera and motion capture devices appropriately to film and record the robot's movements.

[2214] Step 2:

[2215] The video data and motion capture data that the user has filmed and recorded are uploaded to the server.

[2216] Input: Motion video data stored on the device, motion capture data

[2217] Output: Motion video data and motion capture data transferred to the server

[2218] Specific operation: The user transfers data to the server using a dedicated upload tool.

[2219] Step 3:

[2220] The video data received by the server is analyzed using a generative AI model.

[2221] Input: Motion video data stored on the server

[2222] Output: Analysis results such as robot movement speed, accuracy, and trajectory

[2223] Specific operation: The server inputs video data into the AI ​​model, calculates the movement speed and accuracy for each frame, and tracks and quantifies the trajectory.

[2224] Step 4:

[2225] The server analyzes the received motion capture data using a generative AI model to identify movement patterns and biomechanical issues.

[2226] Input: Motion capture data stored on the server

[2227] Output: Movement patterns and biomechanics issues

[2228] Specific movements: The server inputs motion capture data into an AI model, which analyzes the movement of each joint and identifies movement patterns and biomechanical issues.

[2229] Step 5:

[2230] The server generates and provides a report of operational performance based on the analysis results.

[2231] Input: movement speed, accuracy, trajectory, movement patterns, biomechanical issues

[2232] Output: Operational performance report

[2233] What it does: The server consolidates the analysis results and generates a visually comprehensible report, including areas for improvement and recommended actions.

[2234] Step 6:

[2235] The server uses an emotion analysis means to evaluate the emotional state of the operator and provides feedback content.

[2236] Input: Operator emotion data

[2237] Output: Improvements and advice reflecting the operator's emotional state

[2238] Specific operation: The server analyzes the emotional data using the emotion analysis means to assess stress levels and motivation, and includes this in the report.

[2239] Step 7:

[2240] If the robot behaves abnormally, the server analyzes the abnormal behavior data, identifies the cause, and generates a repair program.

[2241] Input: Abnormal behavior data

[2242] Output: Cause identification results, repair program

[2243] Specific operation: The server inputs abnormal operation data into the AI ​​model, uses prompt sentences to identify the cause, and generates repair procedures.

[2244] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2245] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2246] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2247] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2248] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2249] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2250] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2251] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2252] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2253] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2254] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2255] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2256] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2257] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2258] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2259] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2260] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2261] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2262] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2263] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2264] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2265] The following is further disclosed regarding the above embodiment.

[2266] (Claim 1)

[2267] A system for analyzing an athlete's athletic performance, comprising:

[2268] [Means for receiving player motion video data;

[2269] [Means for analyzing the received motion video data using a generative AI model and quantifying motion performance;

[2270] [means for generating and providing operational performance reports based on the analysis results;

[2271] A system including:

[2272] (Claim 2)

[2273] A system for analyzing motion capture data of an athlete, the system comprising:

[2274] [Means for receiving motion capture data of the player;

[2275] [Means of analyzing received motion capture data using generative AI models to identify movement patterns and biomechanical issues; and

[2276] [Means for generating and providing reports on movement patterns and biomechanical issues based on the analysis results; and

[2277] 10. The system of claim 1, comprising:

[2278] (Claim 3)

[2279] A system for predicting recovery from an injury of a player, comprising the following means:

[2280] [Means for receiving data regarding player injuries; and

[2281] [Means for analyzing received injury data using a generative AI model to predict recovery time and performance; and

[2282] [means for generating and providing reports on recovery times and performance predictions based on the analysis results; and

[2283] 10. The system of claim 1, comprising:

[2284] "Example 1"

[2285] (Claim 1)

[2286] [Means for acquiring player motion video data from an input device;

[2287] [Means for receiving the acquired motion video data in a central processing unit;

[2288] [Means for analyzing the received motion video data using a generative AI model and quantifying motion performance;

[2289] [means for generating and transmitting to the user device a report of operational performance based on the analysis results;

[2290] A system including:

[2291] (Claim 2)

[2292] [Means for acquiring motion capture data of the player from an input device;

[2293] [Means for receiving the acquired motion capture data in a central processing unit;

[2294] [Means for analyzing received motion capture data using a generative AI model to identify movement patterns and biomechanical issues; and

[2295] [means for generating and transmitting to the user device a report on movement patterns and biomechanical issues based on the analysis results;

[2296] 10. The system of claim 1, comprising:

[2297] (Claim 3)

[2298] [means for obtaining data relating to player injuries from an input device;

[2299] [means for receiving the acquired injury data at a central processing unit;

[2300] [Means for analyzing received injury data using a generative AI model to predict recovery time and performance; and

[2301] [means for generating and transmitting to the user device a report regarding recovery time and performance prediction based on the analysis results;

[2302] 10. The system of claim 1, comprising:

[2303] "Application Example 1"

[2304] (Claim 1)

[2305] [Means for receiving machine operation video dat...

Claims

1. A system for analyzing an athlete's athletic performance, comprising: A means for receiving player motion video data; A means for analyzing the received motion video data using a generative AI model and quantifying motion performance; means for generating and providing operational performance reports based on the analysis results; A system including:

2. A system for analyzing motion capture data of an athlete, the system comprising: a means for receiving motion capture data of the player; A means of analyzing received motion capture data using a generative AI model to identify movement patterns and biomechanical issues; and A means for generating and providing a report on movement patterns and biomechanical issues based on the analysis results; The system of claim 1 , comprising:

3. A system for predicting recovery from an injury of a player, comprising the following means: a means for receiving player injury data; a means for analyzing the received injury data using a generative AI model to generate recovery time and performance predictions; means for generating and providing reports regarding recovery times and performance predictions based on the analysis results; The system of claim 1 , comprising:

Images