System

Abstract

To provide a system for providing appropriate guidance by analyzing the athletic ability of a child.SOLUTION: The system includes a camera for analyzing the athletic ability of the child, means for acquiring an exercise video of the child, means for transmitting the acquired video to a server, means for analyzing the video data and evaluating the athletic ability of the child in the server, means for generating an exercise tips video or animation for the child based on the evaluation result, means for transmitting the generated video or animation to a terminal, means for presenting the transmitted video or animation to the child, means for generating a support method for a teacher based on the evaluation result of the athletic ability in the server, means for transmitting the generated support method to the terminal, means for displaying the transmitted support method to the teacher, and means for the teacher to refer to the evaluation result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Basic exercises such as jumping rope, horizontal bar, and vaulting box are more difficult to teach than they look, and the wide variation in children's athletic abilities places a heavy burden on teachers. In particular, it is difficult for teachers to provide appropriate instruction to each child, and there is an issue of insufficient support to provide opportunities for further development for children who are good at them and to help children who are not good at them overcome their sense of weakness. [Means for solving the problem]

[0005] The present invention is a system for analyzing a child's motor skills and providing appropriate guidance. Specifically, the system includes the following means:

[0006] 1. A means for capturing images of a child's athletic performance using a camera for analyzing the child's athletic ability.

[0007] 2. Means for sending captured images to the server

[0008] 3. A means of analyzing video data on a server and evaluating a child's motor skills

[0009] 4. A method for generating exercise tips videos and animations for children based on the evaluation results

[0010] 5. A means to send the generated video or animation to the device

[0011] 6. How to show the submitted videos and animations to your child

[0012] 7. A means for generating support methods for teachers based on the results of motor ability evaluation on the server

[0013] 8. Means for sending the generated support method to the device

[0014] 9. A way for teachers to view the support methods submitted

[0015] 10. How teachers can view evaluation results

[0016] Furthermore, the system includes a means for generating training approaches for children who are good at sports to acquire advanced techniques, and for providing video content for children who are not good at sports to relearn from the basics.The system also includes a means for using an encrypted communication protocol for data transmission to ensure the security of the transmitted video data, thereby providing a system that realizes effective and safe exercise instruction.

[0017] The "camera" is a photographing device for capturing images of a child's exercise.

[0018] A "terminal" is a device operated by a user (teacher or student) that acquires and displays images and communicates with a server.

[0019] The "server" is a central processing unit that analyzes video data, evaluates children's motor skills, and generates videos of tips and support methods for teachers.

[0020] "Video" refers to video data that records a child's movements captured by a camera.

[0021] "Athletic ability" refers to a child's level of technical skill and physical strength when performing athletic activities.

[0022] "Evaluation" is the process in which the server analyzes the video data and judges the child's athletic ability based on numerical values ​​and items.

[0023] "Tips videos" and "animations" are visual content created to clearly explain exercise tips and areas for improvement to children.

[0024] "Support methods" refer to specific teaching guidelines and training methods that teachers can use as a reference when providing physical education instruction to children.

[0025] "Data transmission" is the process of sending and communicating data from a terminal to a server or from a server to a terminal.

[0026] An "encrypted communications protocol" is a technical means for ensuring the security of transmitted data, and is a technology for preventing third parties from intercepting the transmission and reception of data.

[0027] "Relearning content" is educational videos and animations designed to help struggling children relearn the basics. [Brief explanation of the drawings]

[0028] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0029] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0030] First, the terms used in the following description will be explained.

[0031] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0032] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0033] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0034] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0035] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0036] [First embodiment]

[0037] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0038] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0039] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0040] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0041] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0042] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0043] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0044] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0045] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0046] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0047] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0048] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0049] 1. System Overview

[0050] The system of the present invention analyzes the motor skills of children and provides appropriate guidance. The main components of the system are a terminal, a server, and a user (teacher or student). An embodiment of the system will be described in detail below.

[0051] 2. Program processing explanation

[0052] Video acquisition and transmission

[0053] User: The user launches a dedicated app installed on a device (e.g., a tablet or smartphone). A teacher uses a camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[0054] Terminal: The acquired video is stored locally and prepared for the next process. The video data is sent to the server using a secure communication protocol (e.g., HTTPS).

[0055] Video analysis

[0056] Server: Prepares the received video data for input into an AI model (e.g., computer vision algorithm) for analysis. The AI ​​model analyzes the video and evaluates the child's athletic ability (jump height, rhythm, speed, etc.). The evaluation results are stored in a database.

[0057] Teaching the tricks

[0058] Server: Based on the analysis results, the server generates videos and animations that address each child's specific weaknesses and areas for improvement. The generated content includes audio guides and text to make it easy for children to understand.

[0059] Server: Sends the generated video or animation to the device.

[0060] On the device: The received videos and animations are played in the app and presented to the child. The video playback status is recorded and reflected in the next lesson plan.

[0061] Support suggestions for teachers

[0062] Server: Generates support methods for teachers based on the results of each child's motor ability assessment. For example, it creates a teaching policy such as, "This child is not good at rhythm. Rhythm training would be effective."

[0063] Server: Sends the generated support method to the device.

[0064] On the device: The teacher checks the suggestions received through the app, displays the suggestions in an easy-to-understand format for the teacher, and provides detailed explanations.

[0065] Leveling up and assessing your child

[0066] Server: Generates training methods for advanced techniques (e.g. double jumps) for children who are good at the skill, and provides video content for children who are not good at the skill to relearn the basics.

[0067] Device: For children who need to refresh their knowledge, provide video content that explains the basic steps.

[0068] User: Teachers can view the assessment results in the app and use the assessment of each child's motor skills to assign grades and plan the next lesson.

[0069] 3. Specific Examples

[0070] For example, if a child is having trouble finding the right rhythm when jumping rope, you can use this system to teach them the following:

[0071] Device: The teacher records the jump rope video with a camera and sends it to the server.

[0072] Server: The AI ​​model analyzes the footage and determines that there is a problem with the rhythm.

[0073] Server: Generates specific animations and audio guides to improve rhythm and sends them to the device.

[0074] Device: Children watch the animation and learn the correct rhythm for skipping rope.

[0075] Teachers can refer to the support methods provided by the devices and use rhythm training aids and music to effectively train children.

[0076] In this way, by using the system of the present invention, it is possible to provide instruction that is tailored to each child's individual athletic ability, thereby realizing effective and efficient exercise instruction.

[0077] The processing flow will be explained below.

[0078] Step 1:

[0079] User: A teacher or child launches a dedicated app installed on a device (e.g., a tablet or smartphone). The teacher uses the camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[0080] Step 2:

[0081] Terminal: Stores the recorded video locally, converts the video data into an appropriate format, and compresses the video data for transmission.

[0082] Step 3:

[0083] Terminal: The compressed video data is sent to the server, using a secure communication protocol (e.g., HTTPS) to ensure data security.

[0084] Step 4:

[0085] Server: Prepares the received video data for input into a pre-trained AI model for analysis. Imports the video data and begins image processing.

[0086] Step 5:

[0087] Server: Using an AI model, the server tracks the child's movements in the video and calculates evaluation items related to athletic ability (such as jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[0088] Step 6:

[0089] Server: Identifies each child's specific weaknesses and areas for improvement, and generates videos and animations based on those weaknesses and areas for improvement. The generated content is then processed to add audio guides and text that are easy for children to understand.

[0090] Step 7:

[0091] Server: Sends the generated video or animation to the device, performing checks to ensure the integrity and security of the data in transit.

[0092] Step 8:

[0093] Device: The received video or animation is stored locally and played within the app, with display adjustments and layout settings made, and presented to the child.

[0094] Step 9:

[0095] Server: Generates support methods (e.g., specific rhythm training methods) for teachers based on the results of each child's motor ability assessment. The generated support methods are documented and stored in a database.

[0096] Step 10:

[0097] Server: Sends the generated support method to the device. Checks the integrity of the data when sending and transfers it in the correct format.

[0098] Step 11:

[0099] Device: The teacher receives suggestions via the app and displays them on the user interface. The suggestions are formatted so that they are easy to understand.

[0100] Step 12:

[0101] Server: Generates training approaches to help skilled children master advanced techniques (e.g., double jumps), and prepares to provide video content to help weaker children relearn the basics.

[0102] Step 13:

[0103] Device: For children who need to relearn the basics, the generated video content for basic learning can be played and they can learn step by step.

[0104] Step 14:

[0105] User: Teachers can view the assessment results in the app and use the assessment of each child's motor skills to assign grades and plan the next lesson.

[0106] In this way, the system performs a series of processes to effectively evaluate and improve children's motor skills.

[0107] Example 1

[0108] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] Currently, it is difficult to properly evaluate children's athletic ability and provide effective instruction based on that evaluation. Conventional athletic ability assessments struggle to grasp the specific characteristics and weaknesses of individual children in detail, making it impossible to provide effective training methods. Furthermore, there is a lack of means to generate specific instructional content based on the evaluation results, making it difficult for instructors to provide appropriate instruction tailored to each child's abilities. Furthermore, ensuring security when transmitting video data is also a major issue.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0111] In this invention, the server includes a camera for analyzing a child's athletic ability, a means for capturing video of the child's athletic activity, a means for transmitting the captured video to a data center, a means for analyzing the video data at the data center and evaluating the child's athletic ability, a means for generating videos and moving images of exercise tips for the child based on the evaluation results, a means for transmitting the generated videos and moving images to a display device, a means for presenting the transmitted videos and moving images to the child, a means for generating support methods for teachers based on the athletic ability evaluation results at the data center, a means for transmitting the generated support methods to a display device, a means for displaying the transmitted support methods to the teacher, a means for the teacher to refer to the evaluation results, a means for utilizing a generative AI model based on the child's athletic ability to provide specific training methods and learning plans, and a means for inputting prompts related to the target exercise into the generative AI model to generate appropriate instruction content. This enables a detailed evaluation of each child's athletic ability and the provision of effective instruction based on the evaluation results in a timely and appropriate manner. Security can also be ensured during the transmission of video data.

[0112] "Photographing device" refers to a device for recording footage of a child's exercise, and includes cameras, video cameras, and the like.

[0113] A "data center" is a centralized management facility for receiving, storing, and analyzing acquired video data.

[0114] "Video data" refers to video files of a child's exercise captured by a camera.

[0115] "Analysis" is the process of evaluating a child's motor skills based on the acquired video data, using computer vision algorithms, etc.

[0116] "Evaluation results" refer to the numerical values ​​and indicators of a child's physical abilities obtained through analysis.

[0117] "Exercise Tips Videos" are videos that provide specific advice and training methods to help children improve their athletic abilities.

[0118] "Moving Images" refers to content that includes animations and visual guides presented to children.

[0119] A "display device" is a device used to present received videos and moving images to children and teachers, and includes tablets, smartphones, and the like.

[0120] "Support methods" refer to teaching strategies and advice provided to teachers to help them provide effective instruction to children.

[0121] A "generative AI model" is an artificial intelligence model that generates specific training methods and learning plans based on a child's athletic ability.

[0122] A "prompt" is a document that describes the instructions or requirements that are input to a generative AI model.

[0123] The present invention is a system for analyzing children's athletic abilities and providing optimal instruction for each child. This system comprises a camera, a data center, a display device, and users (teachers and children).

[0124] The user is a teacher who uses a camera (e.g., a camera built into a tablet or smartphone) to record the target child's exercise (e.g., skipping rope, horizontal bar, vaulting horse). The recorded video data is stored locally on the device (tablet or smartphone) and then sent to a data center using a secure communication protocol (e.g., HTTPS).

[0125] The data center then inputs the received video data into an AI model using computer vision algorithms. The AI ​​model analyzes the video and evaluates the child's athletic ability (e.g., jump height, rhythm, speed, etc.). The evaluation results are stored in a database.

[0126] Based on the assessment results, the data center generates instructional content tailored to each child's specific weaknesses and areas for improvement. Using a generative AI model, it generates videos and animations of exercise tips tailored to each child. This content includes audio guides and text to make it easy for children to understand.

[0127] The generated content is sent to a display device (such as a tablet or smartphone). The content is played on the display device and presented to the child. The playback status is recorded and reflected in future learning plans. Based on the evaluation results, the data center also generates support methods for teachers and sends them to the device. Teachers can check the support methods sent to them on the display device and appropriately convey the content to the children.

[0128] This system provides training methods for advanced athletic skills (e.g., double jumps) to children who are good at it, and video content to encourage children who are not good at it to relearn the basics.In addition, to ensure the security of the video data being transmitted, an encrypted data communication protocol is used.

[0129] For example, if a child has trouble finding the right rhythm when skipping rope, you can use this system to teach them the following steps:

[0130] 1. A teacher records the jump rope video with a camera and sends it to the data center.

[0131] 2. The data center analyzes the video and determines that there is a problem with the rhythm.

[0132] 3. Generate specific animations and audio guides to improve rhythm and send them to the display device.

[0133] 4. Children watch the animation and learn the correct rhythm of skipping rope.

[0134] An example prompt is:

[0135] "Analyze a video of a child skipping rope and assess whether there is a problem with his rhythm. If he is not getting the rhythm right, provide specific areas for improvement and suggest training methods based on those. Your suggestions should include animations and audio guides that are easy for children to understand."

[0136] By using this system in this way, highly efficient instruction tailored to each child's individual motor abilities can be achieved, improving educational effectiveness.

[0137] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0138] Step 1:

[0139] User: A teacher starts a dedicated app on their device to record a child's exercise. They set the exercise to be recorded (e.g., jumping rope) and press the record button to record the child's exercise.

[0140] Input: Device camera, teacher operation

[0141] Output: Recorded exercise video data

[0142] Specific behavior:

[0143] The teacher taps the "Start Recording" button on the app.

[0144] The device's camera will activate and capture video of your workout.

[0145] During video capture, the device temporarily stores the data in a buffer memory.

[0146] After recording is complete, the data is saved to the internal storage.

[0147] Step 2:

[0148] Terminal: Sends stored video data to the data center.

[0149] Input: Recorded exercise video data

[0150] Output: Video data sent to the data center

[0151] Specific behavior:

[0152] The device acquires the video data and uploads it to a data center using a secure communication protocol (HTTPS).

[0153] The data center receives and stores the uploaded data.

[0154] Step 3:

[0155] Server: Prepares for analyzing the received video data. Inputs the data into the AI ​​model and begins analysis.

[0156] Input: Video data stored in the data center

[0157] Output: Evaluation results of athletic ability

[0158] Specific behavior:

[0159] The server inputs the video data into an AI model for analysis.

[0160] The AI ​​model analyzes the video frame by frame and extracts movement parameters (jump height, rhythm, speed, etc.).

[0161] Athletic ability is evaluated based on the extracted data, and the results are stored in a database.

[0162] Step 4:

[0163] Server: Based on the analysis results, it generates videos and moving images of exercise tips for children.

[0164] Input: Athletic ability evaluation results

[0165] Output: Generated exercise tips videos and moving images

[0166] Specific behavior:

[0167] Based on the evaluation results, the server sends a prompt to the generative AI model.

[0168] The generative AI model generates videos and animations with appropriate tips and improvements.

[0169] Audio guides and text are added to the generated videos and moving images.

[0170] Step 5:

[0171] Server: Sends the generated video and moving images to the device.

[0172] Input: Generated exercise tips videos and moving images

[0173] Output: Videos and images sent to the device

[0174] Specific behavior:

[0175] The server retrieves the generated content and transmits it to the terminal.

[0176] The device receives the transmitted data and stores it in local storage.

[0177] Step 6:

[0178] Device: Plays the received videos and images and shows them to the child.

[0179] Input: Received video or moving images

[0180] Output: Played video and moving images

[0181] Specific behavior:

[0182] A teacher or child taps the "Start Playing" button in the app.

[0183] The device plays videos and moving images and shows them to the child.

[0184] Record playback status and reflect it in future learning plans.

[0185] Step 7:

[0186] Server: Generates support methods for teachers and sends them to the devices.

[0187] Input: Athletic ability evaluation results

[0188] Output: Generated support instructions for teachers

[0189] Specific behavior:

[0190] The server generates support policies for teachers based on the evaluation results.

[0191] The generated support policy is sent to the terminal.

[0192] The device displays the support method received and notifies the teacher.

[0193] Step 8:

[0194] User: Teachers refer to the assessment results and provide appropriate guidance to children.

[0195] Input: Evaluation results, generated support methods

[0196] Output: Teacher's teaching content

[0197] Specific behavior:

[0198] The teacher checks the app's evaluation results screen.

[0199] Based on the information obtained, specific guidance is given to the child.

[0200] If necessary, have your child review the training videos and animations provided by the app.

[0201] This makes it possible to evaluate each child's motor skills in detail and provide effective instruction based on that evaluation.

[0202] (Application example 1)

[0203] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0204] It is difficult to provide a system that evaluates the efficiency of workers' actions in a factory and provides appropriate training methods. It is also difficult to generate content for individualized instruction and training that takes into account the strengths and weaknesses of each worker. This creates the challenge of making it difficult to achieve the expected improvement in the production efficiency of the entire factory and the skills of workers.

[0205] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0206] In this invention, the server includes means for acquiring video of the movements of factory workers, means for transmitting the acquired video to a device on the network, means for analyzing the video data in the device on the network and evaluating the movement efficiency of the factory workers, means for generating instructional videos and animations for improving the movements of the factory workers based on the evaluation results, means for transmitting the generated videos and animations to a terminal, means for presenting the transmitted videos and animations to the factory workers, means for generating support methods for a manager based on the evaluation results of the movement efficiency in the device on the network, means for transmitting the generated support methods to the terminal, means for displaying the transmitted support methods to the manager, and means for the manager to refer to the evaluation results. This makes it possible to efficiently analyze the movements of workers in a factory and provide specific instruction methods tailored to each worker.

[0207] "Factory workers" refer to workers engaged in production activities within a factory.

[0208] "Motion video" refers to video data that records the movements of factory workers while they are working.

[0209] "Capture equipment" refers to cameras or other image capture devices used to capture footage of factory workers in action.

[0210] "Devices on a network" refers to computing resources such as servers connected via a network such as the Internet or an intranet.

[0211] "Video data" refers to digital data that records the actions of factory workers.

[0212] "Operational efficiency" is an index that evaluates how efficiently factory workers are performing their work.

[0213] "Instructional videos" refer to video content that visually teaches factory workers how to improve their performance.

[0214] "Animation" refers to visual media consisting of a sequence of image frames that visually represent behavioral improvement techniques.

[0215] "Terminal" refers to a device such as a smartphone or tablet used to play instructional videos and animations.

[0216] "Supervisor" refers to the person responsible for overseeing the activities of factory workers and providing guidance on how to support them.

[0217] "Support methods" refer to specific guidance and improvement measures provided to administrators based on evaluations of operational efficiency.

[0218] 1. System Overview

[0219] The system of the present invention analyzes the operational efficiency of factory workers and provides appropriate guidance methods. The main components of the system are terminals, a server, and users (managers or workers).

[0220] 2. Program processing explanation

[0221] Video acquisition and transmission

[0222] User: The user launches a dedicated app installed on a device (e.g., a tablet or smartphone). The administrator uses a camera to record the actions of factory workers. The device stores the captured video locally and prepares it for further processing. The video data is then sent to a device (server) on the network using a secure communication protocol (e.g., HTTPS).

[0223] Video analysis

[0224] Server: Prepares the received video data for input into an AI model (e.g., computer vision algorithm) for analysis. The AI ​​model analyzes the video and evaluates the efficiency of the factory workers' movements (such as accuracy, speed, and whether there are any unnecessary movements). The evaluation results are stored in a database.

[0225] Teaching the tricks

[0226] Server: Based on the analysis results, the server generates videos and animations that show each factory worker's specific weaknesses and areas for improvement. The generated content includes audio guides and text to make it easy for factory workers to understand. The generated videos and animations are sent to the device.

[0227] Device: The received videos and animations are played back within the app and presented to factory workers. The video playback status is recorded and reflected in the next training plan.

[0228] Support suggestions for administrators

[0229] Server: Generates support methods for managers based on the evaluation results of each factory worker's operational efficiency. For example, it creates a training policy such as, "This worker is slow. Speed ​​training would be effective." The generated support methods are sent to the terminal.

[0230] On the device: The administrator can review the suggestions received through the app, display the suggestions clearly to the administrator, and provide detailed explanations.

[0231] Worker level-up and evaluation

[0232] Server: Generates training methods for advanced skills for skilled factory workers, and provides video content for less skilled factory workers to refresh their knowledge from the basics.

[0233] Devices: For factory workers who need a refresher, video content is presented that explains the basic steps.

[0234] User: The manager checks the evaluation results in the app and assigns an evaluation and plans the next training based on the evaluation of each factory worker's operational efficiency.

[0235] 3. Specific Examples

[0236] For example, if a worker makes a lot of unnecessary movements when carrying equipment, the system can provide the following guidance:

[0237] Terminal: The administrator uses a camera to record footage of the workers' actions and sends it to the server.

[0238] Server: The AI ​​model analyzes the video and determines that there are many unnecessary movements. Based on the analysis results, it generates specific animations and audio guidance to improve the movement and sends them to the device.

[0239] Terminal: Workers watch the animation and learn efficient movements.

[0240] Managers can effectively train workers by referring to the support methods provided by the terminals and using aids and guidelines to promote efficient operations.

[0241] Prompt Sentence Examples

[0242] Please analyze the following video and provide specific guidance on how to improve work efficiency. This video shows Worker A carrying equipment.

[0243] In this way, by using the system of the present invention, it is possible to provide guidance that is tailored to the individual operational efficiency of factory workers, thereby realizing effective and efficient operational guidance.

[0244] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0245] Step 1:

[0246] User: The user launches a dedicated app installed on a device (tablet or smartphone). The administrator uses a camera to record the actions of the factory workers. The input is the video of the factory workers' actions.

[0247] Step 2:

[0248] Terminal: Stores the acquired video locally. Prepares for the next process and sends the video data to the server using a secure communication protocol (e.g., HTTPS). The input is the locally stored video data, and the output is the data to be sent to the server.

[0249] Step 3:

[0250] Server: Prepares the received video data for input into the AI ​​model in order to analyze it. Specifically, it converts the data format and performs preprocessing. The input is the video data sent to the server, and the output is data in a format suitable for the AI ​​model.

[0251] Step 4:

[0252] Server: The AI ​​model analyzes the video and evaluates the efficiency of factory workers' movements. Specifically, it calculates the accuracy, speed, and presence or absence of unnecessary movements. The input is preprocessed video data, and the output is the evaluation result of movement efficiency.

[0253] Step 5:

[0254] Server: The evaluation results are stored in a database. This database is later used to generate instructional videos. The input is the evaluation results of movement efficiency, and the output is the data stored in the database.

[0255] Step 6:

[0256] Server: Based on the analysis results, it generates videos and animations that show each factory worker's specific weaknesses and areas for improvement. The generated content includes audio guides and text to make it easy for factory workers to understand. The input is the evaluation results stored in the database, and the output is the generated instructional videos and animations.

[0257] Step 7:

[0258] Server: Sends the generated instructional videos and animations to the terminal. The input is the generated videos and animations, and the output is the data sent to the terminal.

[0259] Step 8:

[0260] Device: The received videos and animations are played within the app and presented to factory workers. The video playback status is recorded and reflected in the next training plan. The input is the instructional videos and animations sent from the server, and the output is the video playback and a record of the playback status.

[0261] Step 9:

[0262] Server: Generates support methods for managers based on the evaluation results of each factory worker's operational efficiency. Specifically, it creates guidance guidelines such as, "This worker is slow. Speed ​​training would be effective." The input is the evaluation results stored in the database, and the output is the generated support methods.

[0263] Step 10:

[0264] Server: Sends the generated support method to the terminal. The input is the generated support method, and the output is the data sent to the terminal.

[0265] Step 11:

[0266] Terminal: The administrator checks the suggestions received through the app. The suggestions are displayed in an easy-to-understand manner for the administrator, and detailed explanations are provided. The input is the support method sent from the server, and the output is the display of the suggestions.

[0267] Step 12:

[0268] Server: Generates training methods for advanced skills for skilled factory workers, and provides video content for weaker factory workers to refresh their knowledge from the basics. The input is the evaluation results stored in the database, and the output is the training methods and video content.

[0269] Step 13:

[0270] Terminal: For factory workers who need to refresh their knowledge from the basics, video content explaining basic steps is presented. The input is the training method and video content sent from the server, and the output is the video presentation.

[0271] Step 14:

[0272] User: The manager checks the evaluation results on the app and creates an evaluation and next training plan based on the evaluation of each factory worker's operational efficiency. The input is the evaluation results and training methods displayed on the device, and the output is the evaluation and training plan.

[0273] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0274] 1. System Overview

[0275] The system of the present invention analyzes children's athletic abilities and provides appropriate instruction, and furthermore, by combining it with an emotion engine, recognizes the user's emotions and makes the instruction more effective. The main components of the system are a terminal, a server, a user (teacher or student), and an emotion engine. An embodiment of this system is described in detail below.

[0276] 2. Program processing explanation

[0277] Video acquisition and transmission

[0278] User: A teacher or child launches a dedicated app installed on a device (e.g., a tablet or smartphone). The teacher uses the camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[0279] Terminal: The acquired video is stored locally, and the video data is converted into an appropriate format. It also compresses the video data to make it suitable for transmission.

[0280] Terminal: The compressed video data is sent to the server, using a secure communication protocol (e.g., HTTPS) to ensure data security.

[0281] Video analysis

[0282] Server: Prepares the received video data for input into a pre-trained AI model for analysis. Imports the video data and begins image processing.

[0283] Server: Using an AI model, the server tracks the child's movements in the video and calculates evaluation items related to athletic ability (such as jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[0284] Emotion Recognition and Analysis

[0285] Device: During recording, emotional data is collected in real time from the child's facial expressions and voice, including facial expression analysis and voice tone analysis.

[0286] Emotion Engine: Analyzes collected emotion data to determine the child's current emotional state (e.g., excitement, confusion, joy, etc.).

[0287] Device: The determined emotion data is sent to the server, again using a secure communication protocol.

[0288] Tips and emotional feedback

[0289] Server: Based on the analysis results and emotional data, it generates videos and animations with tips tailored to each child's specific weaknesses and areas for improvement. For example, if a child is confused, it will create a video with gentler training methods to resolve the issue.

[0290] Server: Sends the generated video or animation to the device.

[0291] Device: The received videos and animations are played in the app and presented to the child. The video playback status and effectiveness are recorded and reflected in the next lesson plan.

[0292] Support suggestions for teachers

[0293] Server: Based on the results of each child's motor skill assessment and emotional data, the server generates support methods for teachers (e.g., specific approaches for children who are struggling). For example, for a child who is struggling with rhythm, the server suggests a method using music for rhythm training.

[0294] Server: Sends the generated support method to the device.

[0295] Device: The teacher receives suggestions through the app and displays them on the user interface for review.

[0296] Leveling up and assessing your child

[0297] Server: Generates training approaches to help skilled children master advanced techniques (e.g., double jumps), and prepares to provide video content to help weaker children relearn the basics.

[0298] Device: For children who need to relearn the basics, the generated video content for basic learning can be played and they can learn step by step.

[0299] User: Teachers can view the assessment results in the app and use the assessment of each child's motor skills to assign grades and plan the next lesson.

[0300] 3. Specific Examples

[0301] For example, if a child is having trouble finding the right rhythm when jumping rope, you can use this system to teach them the following:

[0302] Device: The teacher records the jump rope with a camera, and at the same time, the emotion engine analyzes the child's facial expressions and tone of voice to determine whether they are confused.

[0303] Server: The AI ​​model analyzes the video and determines that there is a problem with the rhythm. It also receives emotional data indicating confusion.

[0304] Server: Generates specific animations to improve rhythm and gentle, encouraging audio guidance to resolve confusion, and sends them to the device.

[0305] Device: The animation and guide help children learn the correct skipping rhythm and reduce confusion.

[0306] Teachers can refer to the support methods provided by the devices and use rhythm training aids and music to effectively train children.

[0307] In this way, by using the system of the present invention, it is possible to provide instruction that is tailored to each child's individual motor abilities and emotional state, thereby achieving effective and efficient exercise instruction.

[0308] The processing flow will be explained below.

[0309] Step 1:

[0310] User: A teacher or child launches a dedicated app installed on a device (e.g., a tablet or smartphone). The teacher uses the camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[0311] Step 2:

[0312] Terminal: The acquired video is stored locally, and the video data is converted into an appropriate format. It also compresses the video data to make it suitable for transmission.

[0313] Step 3:

[0314] Terminal: The compressed video data is sent to the server, using a secure communication protocol (e.g., HTTPS) to ensure data security.

[0315] Step 4:

[0316] Server: Prepares the received video data for input into a pre-trained AI model for analysis. Imports the video data and begins image processing.

[0317] Step 5:

[0318] Server: Using an AI model, the server tracks the child's movements in the video and calculates evaluation items related to athletic ability (such as jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[0319] Step 6:

[0320] Device: During recording, emotional data is collected in real time from the child's facial expressions and voice, including facial expression analysis and voice tone analysis.

[0321] Step 7:

[0322] Emotion Engine: Analyzes collected emotion data to determine the child's current emotional state (e.g., excitement, confusion, joy, etc.).

[0323] Step 8:

[0324] Device: The determined emotion data is sent to the server, again using a secure communication protocol.

[0325] Step 9:

[0326] Server: Based on the analysis results and emotional data, it generates videos and animations with tips tailored to each child's specific weaknesses and areas for improvement. For example, if a child is confused, it will create a video with gentler training methods to resolve the issue.

[0327] Step 10:

[0328] Server: Sends the generated video or animation to the device, performing checks to ensure the integrity and security of the data in transit.

[0329] Step 11:

[0330] Device: The received video or animation is stored locally and played within the app, with display adjustments and layout settings made, and presented to the child.

[0331] Step 12:

[0332] Server: Based on the results of each child's motor skill assessment and emotional data, the server generates support methods for teachers (e.g., specific approaches for children who are struggling). For example, for a child who is struggling with rhythm, the server suggests a method using music for rhythm training.

[0333] Step 13:

[0334] Server: Sends the generated support method to the device. Checks the integrity of the data when sending and transfers it in the correct format.

[0335] Step 14:

[0336] Device: The teacher receives suggestions via the app and displays them on the user interface. The suggestions are formatted so that they are easy to understand.

[0337] Step 15:

[0338] Server: Generates training approaches to help skilled children master advanced techniques (e.g., double jumps), and prepares to provide video content to help weaker children relearn the basics.

[0339] Step 16:

[0340] Device: For children who need to relearn the basics, the generated video content for basic learning can be played and they can learn step by step.

[0341] Step 17:

[0342] User: Teachers can view the assessment results in the app and grade students and plan the next lesson based on their assessment of each child's motor skills and emotional data.

[0343] In this way, the system performs a series of processes to effectively evaluate and improve children's motor skills, and by combining this with emotional data, it is possible to provide optimal teaching methods for each individual.

[0344] Example 2

[0345] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0346] Conventional motor skill assessment systems rely primarily on visual information to analyze motor skills, and do not take into account the child's emotional state when providing instruction, which has the drawback of not being effective enough. Furthermore, there is a lack of individualized instruction for strong and weak areas, making it difficult to provide follow-up that responds to a child's growth.

[0347] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0348] In this invention, the server includes a means for analyzing video information and evaluating a child's athletic ability, a means for analyzing the child's emotional state using an emotion recognition engine and transmitting the analysis results to a data processing device, and a means for generating exercise instruction videos and visual effects for children based on the evaluation results, thereby enabling effective instruction that takes into account not only the child's athletic ability but also their emotional state.

[0349] "Video capture devices" refer to devices used to capture footage of children's athletic skills, and examples include cameras and smartphones.

[0350] The term "communication device" refers to a device for transmitting acquired video data to a data processing device, and specific examples include a wireless communication module and an internet connection device.

[0351] The "data processing device" refers to a device that analyzes received video information and evaluates a child's motor skills, and specific examples include a server and a computer.

[0352] "Video information" refers to video data that captures a child's athletic skills, specifically information that is saved and transferred in video file format.

[0353] "Motor ability" refers to the ability to assess multiple indicators related to a child's motor skills (e.g., jump height, rhythm, speed, etc.).

[0354] "Evaluation results" refers to the results of the evaluation of athletic ability analyzed by the data processing device, and includes numerical values ​​and text data stored in the database.

[0355] "Exercise method video" refers to a video created for children based on the evaluation results to help them master exercise techniques.

[0356] "Visual effects" refers to animations and graphic elements added to exercise videos, techniques used to present information in a way that is easy for children to understand.

[0357] "Display device" refers to a device for presenting the generated exercise method video and visual effects to a child, and examples include a tablet or smartphone.

[0358] "Educator" refers to a person responsible for teaching children athletic skills, including school teachers and instructors.

[0359] An "emotion recognition engine" refers to algorithms and software that analyze a child's emotional state from their facial expressions, voice, etc.

[0360] "Emotional state" refers to the child's current emotional state (e.g., excited, confused, happy, etc.), and includes information sent to the data processing device as an analysis result.

[0361] "Support measures" refer to specific methods and approaches that educators use to improve children's motor skills, and are created based on the evaluation results.

[0362] "Encrypted communications protocol" refers to the communications method used to ensure the security of transmitted video information, and examples include HTTPS and SSL.

[0363] The system of the present invention analyzes children's athletic abilities and provides appropriate instruction, and furthermore, by combining it with an emotion engine, recognizes the user's emotions and makes the instruction more effective. The main components of the system are a terminal, a server, a user (teacher or student), and an emotion engine. An embodiment of this system is described in detail below.

[0364] The system is equipped with the following hardware and software. Tablets and smartphones are used as terminals, with a dedicated application installed. The server uses a high-performance computer as the data processing device, and a deep learning model is implemented as the emotion recognition engine. OpenPose and FFmpeg are used for video analysis, and Praat is used for audio analysis. HTTPS is used as the communication protocol to ensure data security.

[0365] Video acquisition and transmission

[0366] The user, either a teacher or a child, launches a dedicated app installed on the device and prepares to record. Using the device's built-in camera, the teacher records the child's exercise (for example, skipping rope, horizontal bar, or vaulting box). When recording is complete, the device saves the captured video locally and converts the video data into an appropriate format (for example, MP4). It also performs a compression process to make it suitable for transmission. The compressed video data is sent to the server using the HTTPS protocol.

[0367] Video analysis

[0368] The server prepares the received video data for input into the AI ​​model. FFmpeg is used to break down the video data into frames and analyze each frame. The OpenPose library is used to track the child's movements in the video and calculate evaluation items related to athletic ability (jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[0369] Emotion Recognition and Analysis

[0370] While recording, the device collects emotional data in real time based on the child's facial expressions and voice. It uses OpenCV facial recognition technology and Praat voice tone analysis. The collected emotional data is analyzed by an emotion recognition engine to determine the child's current emotional state (excitement, confusion, joy, etc.). The determined emotional data is then sent back to the server using the HTTPS protocol.

[0371] Tips and emotional feedback

[0372] Based on the analysis results and emotional data, the server generates videos and animations that address each child's specific weaknesses and areas for improvement. Tools such as Adobe After Effects are used here, too. For example, a video containing gentler training methods can be generated for a child who is struggling. These videos and animations are then sent from the server to the device. The device then plays the received videos and animations within the app and presents them to the child. The device also has a function to monitor the child's reactions in real time during playback, and this is reflected in the next lesson plan.

[0373] Support suggestions for teachers

[0374] The server generates support methods for teachers based on each child's motor ability assessment results and emotional data. For children who are struggling with rhythm, it suggests methods such as using rhythmic training music. The generated support methods are sent to the device, which displays the suggestions received through the app on a user interface so that teachers can check them.

[0375] Leveling up and assessing your child

[0376] The server generates training approaches for children who excel at a skill to master advanced techniques (e.g., double jumps), and prepares video content for children who are not so good at a skill to relearn from the basics. For example, a step-by-step training program can be designed using Adobe After Effects. The device plays the generated video content for basic learning for children who need to relearn from the basics, providing a function for them to learn step by step.

[0377] Examples of prompt statements

[0378] "I would like to analyze the video and emotional data of a child skipping rope, and suggest an appropriate training method for a child who is having trouble. Please generate a video and audio guide for this purpose."

[0379] In this way, by using the system of the present invention, it is possible to provide instruction that is tailored to each child's individual motor abilities and emotional state, thereby achieving effective and efficient exercise instruction.

[0380] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0381] Step 1:

[0382] The user launches a dedicated app installed on their device (tablet or smartphone) and prepares for recording. This includes the login procedure and camera startup. The input is the user ID and password, and the output is the home screen of the app.

[0383] Step 2:

[0384] The user uses the device's camera to record their child's exercise (e.g., skipping rope, horizontal bar, vaulting box). During recording, the camera angle and distance are adjusted to capture the entire exercise. The input is the camera image, and the output is the recorded data.

[0385] Step 3:

[0386] The device stores the captured video locally and the video data is converted to an appropriate format (e.g., MP4). The input is the recorded data, and the output is the converted MP4 format video data.

[0387] Step 4:

[0388] The terminal then compresses the converted video data using the H.264 codec to make it suitable for transmission. The input is video data in MP4 format, and the output is compressed video data.

[0389] Step 5:

[0390] The terminal sends the compressed video data to the server using the HTTPS protocol, with the compressed video data as input and a confirmation of successful data transfer to the server as output.

[0391] Step 6:

[0392] The server prepares the received video data for input to the AI ​​model. To do this, it uses FFmpeg to break down the video data into frames. The input is compressed video data, and the output is image data for each frame.

[0393] Step 7:

[0394] The server uses the OpenPose library to track the child's movements frame by frame and calculates evaluation items related to athletic ability (jump height, rhythm, speed, etc.) Image data for each frame is input, and numerical data based on the evaluation items is output.

[0395] Step 8:

[0396] The server saves the calculated evaluation results in a database such as MySQL or PostgreSQL. The input is numerical data based on the evaluation items, and the output is recorded in the database.

[0397] Step 9:

[0398] The device collects emotional data in real time based on the child's facial expressions and voice during recording. This includes facial recognition technology (e.g., OpenCV) and tone analysis (e.g., Praat). The input is captured facial images and voice data, and the output is analyzed emotional data.

[0399] Step 10:

[0400] The emotion recognition engine analyzes the collected emotional data using a deep learning model to determine the child's current emotional state (excitement, confusion, joy, etc.) The input is the analyzed emotional data, and the output is a judgment of the child's emotional state.

[0401] Step 11:

[0402] The device transmits the determined emotion data to the server using the HTTPS protocol. The input is the result of the determination of the emotional state, and the output is confirmation of successful data transfer to the server.

[0403] Step 12:

[0404] Based on the analysis results and emotional data, the server generates videos and animations that provide tips for each child based on their specific weaknesses and areas for improvement. Tools such as Adobe After Effects are used for this. The inputs are the evaluation results and emotional data, and the output is the generated videos and animations.

[0405] Step 13:

[0406] The server sends the generated video or animation to the display device, with the generated video as input and confirmation of successful data transfer to the display device as output.

[0407] Step 14:

[0408] The device then plays the received video or animation within the app and presents it to the child. The input is the received video or animation, and the output is the video playback screen. The app also has a function to monitor the child's reaction in real time during playback.

[0409] Step 15:

[0410] The server generates support methods for educators based on each child's motor ability assessment results and emotional data. For example, for a child who is struggling with rhythm, it suggests a method using rhythm training music. The inputs are the assessment results and emotional data, and the output is the generated support method.

[0411] Step 16:

[0412] The server sends the generated support method to the display device, with the generated support method as input and confirmation of successful data transfer to the display device as output.

[0413] Step 17:

[0414] The device displays the suggestions received through the app on a user interface so that the teacher can check them. The input is the received support method, and the output is a screen showing the suggestions.

[0415] Step 18:

[0416] Teachers, who are users, check the assessment results on the app and use the assessment to grade each child and plan the next lesson. The input is the assessment results and support methods, and the output is a grade report and lesson plan.

[0417] Through the above steps, the system can provide effective and efficient exercise instruction tailored to each child's individual motor abilities and emotional state.

[0418] (Application example 2)

[0419] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0420] Currently, specific instruction is required to enhance children's motor skills, and appropriate feedback and training methods are needed. However, individual instruction tailored to each child's motor skills and emotional state is time-consuming and difficult to provide efficiently. Providing appropriate instruction and feedback to improve the operational efficiency and safety of robotic workers in factories is also difficult. In particular, systems that analyze the emotions and state of robots while they are operating and adjust their operations based on this information are still in the development stage, and there are many challenges before they can be applied in real-world situations.

[0421] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: video acquisition means equipped with a camera for analyzing the motor skills of children and the movements of robot workers; means for transmitting the acquired video to the server; means for analyzing and evaluating the video data in the server; means for transmitting generated training content to the terminal; means for presenting the transmitted training content; means for generating support methods for teachers or work instructors based on the motor skill evaluation results and emotional data in the server; means for transmitting the generated support methods to the terminal; means for displaying the transmitted support methods; means for analyzing emotional states using an emotion engine; means for providing feedback based on the evaluation results and emotional states; and means for improving work efficiency using a motor video analysis system. This enables detailed instruction and feedback to improve the movement efficiency of individual children and robot workers.

[0422] The "camera" is an image capturing device that captures the movements and work actions of the child and robot worker in real time.

[0423] The "server" is a central processing unit that analyzes the collected video data and emotional data and generates evaluation results regarding athletic ability and work efficiency.

[0424] "Image capture means" refers to a method or device for capturing images of a subject's movements or actions using a camera and acquiring the image data.

[0425] The "video transmission means" is a method or device for transmitting captured video data to a server via a network.

[0426] "Video analysis means" refers to a method or device by which the server analyzes video data and evaluates athletic ability and work efficiency.

[0427] The "training content generation means" is a method or device for generating motion programs for exercise training and work improvement for children and robot workers based on the evaluation results.

[0428] The "content transmission means" is a method or device for transmitting the generated training content and feedback to the terminal.

[0429] The "content presentation means" is a method or device for displaying the transmitted training content to children and robot workers.

[0430] The "emotion engine" is a system for analyzing a subject's emotional state from their facial expressions, movements, voice data, etc.

[0431] An "emotion analysis means" is a method or device for analyzing collected emotion data and determining a subject's current emotional state.

[0432] The "feedback providing means" is a method or device for providing appropriate feedback to a subject based on the analysis results and emotional state.

[0433] The "motion improvement means" is a method or device for the server to improve work efficiency based on the evaluation results using the motion video analysis system.

[0434] 1. System Configuration

[0435] This invention is a system that analyzes children's motor skills and the work movements of factory robots, and provides appropriate guidance and feedback. The main components of the system are a camera, a server, a terminal, a user (teacher or work instructor), and an emotion engine.

[0436] 2. Program processing explanation

[0437] Video acquisition and transmission

[0438] Camera: Records the movements and working actions of children and robot workers in real time. The camera has high resolution and clearly captures their movements.

[0439] Device: Stores the captured video locally, converts it into an appropriate format, compresses the video data for transmission, and sends it to the server using a secure communication protocol (HTTPS).

[0440] Video analysis

[0441] Server: Uses a pre-trained AI model to analyze the received video data. Imports the video data and starts image processing. Uses TensorFlow or PyTorch as the AI ​​model framework.

[0442] Server: The AI ​​model tracks the movements of the child or robot worker in the video and calculates evaluation items (speed, accuracy, safety, etc.) related to motor skills and work efficiency. This includes the speed and accuracy of exercise and work movements, as well as fatigue level. The calculated evaluation results are stored in a database.

[0443] Emotion Recognition and Analysis

[0444] Device: During recording, emotional data is collected in real time from the facial expressions and voices of the child and robotic worker, including facial expression analysis and voice tone analysis.

[0445] Emotion engine: Analyzes collected emotion data and determines the subject's current emotional state (e.g., excited, confused, tired, etc.). The emotion engine can use Microsoft Azure Cognitive Services.

[0446] Terminal: The determined emotion data is sent to the server, and a secure communication protocol is used to ensure the security of the data.

[0447] Generating and Providing Feedback

[0448] Server: Based on the analysis results and emotional data, it generates feedback tailored to each subject's specific weaknesses and areas for improvement. For example, if a robot is overworked, it generates feedback recommending appropriate rest or adjustments to its behavior.

[0449] Server: Sends the generated feedback to the device and displays it on the device. The server also generates and sends support methods to teachers and instructors.

[0450] Terminal: The received feedback and support methods are displayed on the user interface and presented to the target.

[0451] 3. Specific Examples

[0452] For example, if a robotic arm in a factory is moving slower than usual, its built-in camera will capture its movements and send them to a server. The server will then use an AI model and emotion engine to recognize that the robot is overworked. It will then generate feedback recommending appropriate breaks or minor adjustments to its movements and send it to the robot. For children, a teacher will record a video of the robot skipping rope with a camera, while the emotion engine will analyze the child's facial expressions and tone of voice to determine whether the child is confused. The server will then analyze the video, determine that there is a problem with the rhythm, and receive the emotional data indicating confusion. Based on this, it will generate specific animations to improve the rhythm and gentle, encouraging audio guidance to resolve the confusion, which will then be sent to the device. The child will then learn the correct skipping rhythm and become less confused.

[0453] Example of a generative AI model prompt:

[0454] To ensure that factory robots are working properly, how can you analyze video footage captured by built-in cameras, recognize the robot's performance evaluation and emotional state, and provide optimal feedback?

[0455] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0456] Step 1:

[0457] Footage collection

[0458] User: A teacher or instructor uses a camera to record the child's exercise and the robot's task movements.

[0459] Input: Child's movement or robot's task movement.

[0460] Processing: The camera captures the action and acquires the video data.

[0461] Output: Captured high-resolution video data.

[0462] Step 2:

[0463] Video storage and conversion

[0464] Device: Stores the captured video in local storage and converts it to the appropriate format.

[0465] Input: Acquired video data.

[0466] Processing: Video data is compressed and converted into a format suitable for transmission.

[0467] Output: Compressed video data.

[0468] Step 3:

[0469] Video data transmission

[0470] Terminal: Compressed video data is sent to the server using a secure protocol (HTTPS).

[0471] Input: Compressed video data.

[0472] Processing: Data transmission via secure communication protocols.

[0473] Output: Video data sent to the server.

[0474] Step 4:

[0475] Preparing video data for analysis

[0476] Server: Prepares to analyze the received video data.

[0477] Input: Video data sent to the server.

[0478] Processing: Preparing the video data in the appropriate format to input into the AI ​​model.

[0479] Output: Video data that can be input into an AI model.

[0480] Step 5:

[0481] Video analysis using AI models

[0482] Server: Uses AI models to analyze movements in the video (e.g., jump height, rhythm, and work efficiency).

[0483] Input: Video data.

[0484] Processing: Apply video analysis algorithms to evaluate subject's movements.

[0485] Output: Evaluation results of motor skills and work efficiency.

[0486] Step 6:

[0487] Emotion data collection and analysis

[0488] Device: Collects emotional data in real time based on the facial expressions and voices of children and robots.

[0489] Input: facial and voice data.

[0490] Processing: Facial expression analysis and voice tone analysis are performed to obtain emotion data.

[0491] Output: The obtained emotion data.

[0492] Step 7:

[0493] Emotion analysis using an emotion engine

[0494] Emotion engine: Analyzes the acquired emotion data and determines the current emotional state.

[0495] Input: Collected emotion data.

[0496] Processing: Determining emotional state.

[0497] Output: Determined emotional state data.

[0498] Step 8:

[0499] Sending emotional data

[0500] Device: Sends the determined emotion data to the server.

[0501] Input: Determined emotional state data.

[0502] Processing: Data transmission via secure communication protocol (HTTPS).

[0503] Output: Emotional state data sent to the server.

[0504] Step 9:

[0505] Generate feedback

[0506] Server: Based on the analysis results and emotional data, it generates feedback tailored to each subject's specific weaknesses and areas for improvement.

[0507] Input: Evaluation results and emotional state data.

[0508] Processing: Feedback generation and customization.

[0509] Output: The generated feedback data.

[0510] Step 10:

[0511] Providing feedback

[0512] Terminal: Display the received feedback on the user interface.

[0513] Input: Feedback data.

[0514] Action: View feedback.

[0515] Output: Feedback presented to the child and robot worker.

[0516] Through the above steps, it is possible to realize a system that properly analyzes the motion data and emotional state of children and robot workers, and provides detailed guidance and feedback.

[0517] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0518] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0519] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0520] [Second embodiment]

[0521] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0522] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0523] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0524] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0525] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0526] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0527] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0528] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0529] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0530] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0531] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0532] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0533] 1. System Overview

[0534] The system of the present invention analyzes the motor skills of children and provides appropriate guidance. The main components of the system are a terminal, a server, and a user (teacher or student). An embodiment of the system will be described in detail below.

[0535] 2. Program processing explanation

[0536] Video acquisition and transmission

[0537] User: The user launches a dedicated app installed on a device (e.g., a tablet or smartphone). A teacher uses a camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[0538] Terminal: The acquired video is stored locally and prepared for the next process. The video data is sent to the server using a secure communication protocol (e.g., HTTPS).

[0539] Video analysis

[0540] Server: Prepares the received video data for input into an AI model (e.g., computer vision algorithm) for analysis. The AI ​​model analyzes the video and evaluates the child's athletic ability (jump height, rhythm, speed, etc.). The evaluation results are stored in a database.

[0541] Teaching the tricks

[0542] Server: Based on the analysis results, the server generates videos and animations that address each child's specific weaknesses and areas for improvement. The generated content includes audio guides and text to make it easy for children to understand.

[0543] Server: Sends the generated video or animation to the device.

[0544] On the device: The received videos and animations are played in the app and presented to the child. The video playback status is recorded and reflected in the next lesson plan.

[0545] Support suggestions for teachers

[0546] Server: Generates support methods for teachers based on the results of each child's motor ability assessment. For example, it creates a teaching policy such as, "This child is not good at rhythm. Rhythm training would be effective."

[0547] Server: Sends the generated support method to the device.

[0548] On the device: The teacher checks the suggestions received through the app, displays the suggestions in an easy-to-understand format for the teacher, and provides detailed explanations.

[0549] Leveling up and assessing your child

[0550] Server: Generates training methods for advanced techniques (e.g. double jumps) for children who are good at the skill, and provides video content for children who are not good at the skill to relearn the basics.

[0551] Device: For children who need to refresh their knowledge, provide video content that explains the basic steps.

[0552] User: Teachers can view the assessment results in the app and use the assessment of each child's motor skills to assign grades and plan the next lesson.

[0553] 3. Specific Examples

[0554] For example, if a child is having trouble finding the right rhythm when jumping rope, you can use this system to teach them the following:

[0555] Device: The teacher records the jump rope video with a camera and sends it to the server.

[0556] Server: The AI ​​model analyzes the footage and determines that there is a problem with the rhythm.

[0557] Server: Generates specific animations and audio guides to improve rhythm and sends them to the device.

[0558] Device: Children watch the animation and learn the correct rhythm for skipping rope.

[0559] Teachers can refer to the support methods provided by the devices and use rhythm training aids and music to effectively train children.

[0560] In this way, by using the system of the present invention, it is possible to provide instruction that is tailored to each child's individual athletic ability, thereby realizing effective and efficient exercise instruction.

[0561] The processing flow will be explained below.

[0562] Step 1:

[0563] User: A teacher or child launches a dedicated app installed on a device (e.g., a tablet or smartphone). The teacher uses the camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[0564] Step 2:

[0565] Terminal: Stores the recorded video locally, converts the video data into an appropriate format, and compresses the video data for transmission.

[0566] Step 3:

[0567] Terminal: The compressed video data is sent to the server, using a secure communication protocol (e.g., HTTPS) to ensure data security.

[0568] Step 4:

[0569] Server: Prepares the received video data for input into a pre-trained AI model for analysis. Imports the video data and begins image processing.

[0570] Step 5:

[0571] Server: Using an AI model, the server tracks the child's movements in the video and calculates evaluation items related to athletic ability (such as jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[0572] Step 6:

[0573] Server: Identifies each child's specific weaknesses and areas for improvement, and generates videos and animations based on those weaknesses and areas for improvement. The generated content is then processed to add audio guides and text that are easy for children to understand.

[0574] Step 7:

[0575] Server: Sends the generated video or animation to the device, performing checks to ensure the integrity and security of the data in transit.

[0576] Step 8:

[0577] Device: The received video or animation is stored locally and played within the app, with display adjustments and layout settings made, and presented to the child.

[0578] Step 9:

[0579] Server: Generates support methods (e.g., specific rhythm training methods) for teachers based on the results of each child's motor ability assessment. The generated support methods are documented and stored in a database.

[0580] Step 10:

[0581] Server: Sends the generated support method to the device. Checks the integrity of the data when sending and transfers it in the correct format.

[0582] Step 11:

[0583] Device: The teacher receives suggestions via the app and displays them on the user interface. The suggestions are formatted so that they are easy to understand.

[0584] Step 12:

[0585] Server: Generates training approaches to help skilled children master advanced techniques (e.g., double jumps), and prepares to provide video content to help weaker children relearn the basics.

[0586] Step 13:

[0587] Device: For children who need to relearn the basics, the generated video content for basic learning can be played and they can learn step by step.

[0588] Step 14:

[0589] User: Teachers can view the assessment results in the app and use the assessment of each child's motor skills to assign grades and plan the next lesson.

[0590] In this way, the system performs a series of processes to effectively evaluate and improve children's motor skills.

[0591] Example 1

[0592] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0593] Currently, it is difficult to properly evaluate children's athletic ability and provide effective instruction based on that evaluation. Conventional athletic ability assessments struggle to grasp the specific characteristics and weaknesses of individual children in detail, making it impossible to provide effective training methods. Furthermore, there is a lack of means to generate specific instructional content based on the evaluation results, making it difficult for instructors to provide appropriate instruction tailored to each child's abilities. Furthermore, ensuring security when transmitting video data is also a major issue.

[0594] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0595] In this invention, the server includes a camera for analyzing a child's athletic ability, a means for capturing video of the child's athletic activity, a means for transmitting the captured video to a data center, a means for analyzing the video data at the data center and evaluating the child's athletic ability, a means for generating videos and moving images of exercise tips for the child based on the evaluation results, a means for transmitting the generated videos and moving images to a display device, a means for presenting the transmitted videos and moving images to the child, a means for generating support methods for teachers based on the athletic ability evaluation results at the data center, a means for transmitting the generated support methods to a display device, a means for displaying the transmitted support methods to the teacher, a means for the teacher to refer to the evaluation results, a means for utilizing a generative AI model based on the child's athletic ability to provide specific training methods and learning plans, and a means for inputting prompts related to the target exercise into the generative AI model to generate appropriate instruction content. This enables a detailed evaluation of each child's athletic ability and the provision of effective instruction based on the evaluation results in a timely and appropriate manner. Security can also be ensured during the transmission of video data.

[0596] "Photographing device" refers to a device for recording footage of a child's exercise, and includes cameras, video cameras, and the like.

[0597] A "data center" is a centralized management facility for receiving, storing, and analyzing acquired video data.

[0598] "Video data" refers to video files of a child's exercise captured by a camera.

[0599] "Analysis" is the process of evaluating a child's motor skills based on the acquired video data, using computer vision algorithms, etc.

[0600] "Evaluation results" refer to the numerical values ​​and indicators of a child's physical abilities obtained through analysis.

[0601] "Exercise Tips Videos" are videos that provide specific advice and training methods to help children improve their athletic abilities.

[0602] "Moving Images" refers to content that includes animations and visual guides presented to children.

[0603] A "display device" is a device used to present received videos and moving images to children and teachers, and includes tablets, smartphones, and the like.

[0604] "Support methods" refer to teaching strategies and advice provided to teachers to help them provide effective instruction to children.

[0605] A "generative AI model" is an artificial intelligence model that generates specific training methods and learning plans based on a child's athletic ability.

[0606] A "prompt" is a document that describes the instructions or requirements that are input to a generative AI model.

[0607] The present invention is a system for analyzing children's athletic abilities and providing optimal instruction for each child. This system comprises a camera, a data center, a display device, and users (teachers and children).

[0608] The user is a teacher who uses a camera (e.g., a camera built into a tablet or smartphone) to record the target child's exercise (e.g., skipping rope, horizontal bar, vaulting horse). The recorded video data is stored locally on the device (tablet or smartphone) and then sent to a data center using a secure communication protocol (e.g., HTTPS).

[0609] The data center then inputs the received video data into an AI model using computer vision algorithms. The AI ​​model analyzes the video and evaluates the child's athletic ability (e.g., jump height, rhythm, speed, etc.). The evaluation results are stored in a database.

[0610] Based on the assessment results, the data center generates instructional content tailored to each child's specific weaknesses and areas for improvement. Using a generative AI model, it generates videos and animations of exercise tips tailored to each child. This content includes audio guides and text to make it easy for children to understand.

[0611] The generated content is sent to a display device (such as a tablet or smartphone). The content is played on the display device and presented to the child. The playback status is recorded and reflected in future learning plans. Based on the evaluation results, the data center also generates support methods for teachers and sends them to the device. Teachers can check the support methods sent to them on the display device and appropriately convey the content to the children.

[0612] This system provides training methods for advanced athletic skills (e.g., double jumps) to children who are good at it, and video content to encourage children who are not good at it to relearn the basics.In addition, to ensure the security of the video data being transmitted, an encrypted data communication protocol is used.

[0613] For example, if a child has trouble finding the right rhythm when skipping rope, you can use this system to teach them the following steps:

[0614] 1. A teacher records the jump rope video with a camera and sends it to the data center.

[0615] 2. The data center analyzes the video and determines that there is a problem with the rhythm.

[0616] 3. Generate specific animations and audio guides to improve rhythm and send them to the display device.

[0617] 4. Children watch the animation and learn the correct rhythm of skipping rope.

[0618] An example prompt is:

[0619] "Analyze a video of a child skipping rope and assess whether there is a problem with his rhythm. If he is not getting the rhythm right, provide specific areas for improvement and suggest training methods based on those. Your suggestions should include animations and audio guides that are easy for children to understand."

[0620] By using this system in this way, highly efficient instruction tailored to each child's individual motor abilities can be achieved, improving educational effectiveness.

[0621] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0622] Step 1:

[0623] User: A teacher starts a dedicated app on their device to record a child's exercise. They set the exercise to be recorded (e.g., jumping rope) and press the record button to record the child's exercise.

[0624] Input: Device camera, teacher operation

[0625] Output: Recorded exercise video data

[0626] Specific behavior:

[0627] The teacher taps the "Start Recording" button on the app.

[0628] The device's camera will activate and capture video of your workout.

[0629] During video capture, the device temporarily stores the data in a buffer memory.

[0630] After recording is complete, the data is saved to the internal storage.

[0631] Step 2:

[0632] Terminal: Sends stored video data to the data center.

[0633] Input: Recorded exercise video data

[0634] Output: Video data sent to the data center

[0635] Specific behavior:

[0636] The device acquires the video data and uploads it to a data center using a secure communication protocol (HTTPS).

[0637] The data center receives and stores the uploaded data.

[0638] Step 3:

[0639] Server: Prepares for analyzing the received video data. Inputs the data into the AI ​​model and begins analysis.

[0640] Input: Video data stored in the data center

[0641] Output: Evaluation results of athletic ability

[0642] Specific behavior:

[0643] The server inputs the video data into an AI model for analysis.

[0644] The AI ​​model analyzes the video frame by frame and extracts movement parameters (jump height, rhythm, speed, etc.).

[0645] Athletic ability is evaluated based on the extracted data, and the results are stored in a database.

[0646] Step 4:

[0647] Server: Based on the analysis results, it generates videos and moving images of exercise tips for children.

[0648] Input: Athletic ability evaluation results

[0649] Output: Generated exercise tips videos and moving images

[0650] Specific behavior:

[0651] Based on the evaluation results, the server sends a prompt to the generative AI model.

[0652] The generative AI model generates videos and animations with appropriate tips and improvements.

[0653] Audio guides and text are added to the generated videos and moving images.

[0654] Step 5:

[0655] Server: Sends the generated video and moving images to the device.

[0656] Input: Generated exercise tips videos and moving images

[0657] Output: Videos and images sent to the device

[0658] Specific behavior:

[0659] The server retrieves the generated content and transmits it to the terminal.

[0660] The device receives the transmitted data and stores it in local storage.

[0661] Step 6:

[0662] Device: Plays the received videos and images and shows them to the child.

[0663] Input: Received video or moving images

[0664] Output: Played video and moving images

[0665] Specific behavior:

[0666] A teacher or child taps the "Start Playing" button in the app.

[0667] The device plays videos and moving images and shows them to the child.

[0668] Record playback status and reflect it in future learning plans.

[0669] Step 7:

[0670] Server: Generates support methods for teachers and sends them to the devices.

[0671] Input: Athletic ability evaluation results

[0672] Output: Generated support instructions for teachers

[0673] Specific behavior:

[0674] The server generates support policies for teachers based on the evaluation results.

[0675] The generated support policy is sent to the terminal.

[0676] The device displays the support method received and notifies the teacher.

[0677] Step 8:

[0678] User: Teachers refer to the assessment results and provide appropriate guidance to children.

[0679] Input: Evaluation results, generated support methods

[0680] Output: Teacher's teaching content

[0681] Specific behavior:

[0682] The teacher checks the app's evaluation results screen.

[0683] Based on the information obtained, specific guidance is given to the child.

[0684] If necessary, have your child review the training videos and animations provided by the app.

[0685] This makes it possible to evaluate each child's motor skills in detail and provide effective instruction based on that evaluation.

[0686] (Application example 1)

[0687] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0688] It is difficult to provide a system that evaluates the efficiency of workers' actions in a factory and provides appropriate training methods. It is also difficult to generate content for individualized instruction and training that takes into account the strengths and weaknesses of each worker. This creates the challenge of making it difficult to achieve the expected improvement in the production efficiency of the entire factory and the skills of workers.

[0689] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0690] In this invention, the server includes means for acquiring video of the movements of factory workers, means for transmitting the acquired video to a device on the network, means for analyzing the video data in the device on the network and evaluating the movement efficiency of the factory workers, means for generating instructional videos and animations for improving the movements of the factory workers based on the evaluation results, means for transmitting the generated videos and animations to a terminal, means for presenting the transmitted videos and animations to the factory workers, means for generating support methods for a manager based on the evaluation results of the movement efficiency in the device on the network, means for transmitting the generated support methods to the terminal, means for displaying the transmitted support methods to the manager, and means for the manager to refer to the evaluation results. This makes it possible to efficiently analyze the movements of workers in a factory and provide specific instruction methods tailored to each worker.

[0691] "Factory workers" refer to workers engaged in production activities within a factory.

[0692] "Motion video" refers to video data that records the movements of factory workers while they are working.

[0693] "Capture equipment" refers to cameras or other image capture devices used to capture footage of factory workers in action.

[0694] "Devices on a network" refers to computing resources such as servers connected via a network such as the Internet or an intranet.

[0695] "Video data" refers to digital data that records the actions of factory workers.

[0696] "Operational efficiency" is an index that evaluates how efficiently factory workers are performing their work.

[0697] "Instructional videos" refer to video content that visually teaches factory workers how to improve their performance.

[0698] "Animation" refers to visual media consisting of a sequence of image frames that visually represent behavioral improvement techniques.

[0699] "Terminal" refers to a device such as a smartphone or tablet used to play instructional videos and animations.

[0700] "Supervisor" refers to the person responsible for overseeing the activities of factory workers and providing guidance on how to support them.

[0701] "Support methods" refer to specific guidance and improvement measures provided to administrators based on evaluations of operational efficiency.

[0702] 1. System Overview

[0703] The system of the present invention analyzes the operational efficiency of factory workers and provides appropriate guidance methods. The main components of the system are terminals, a server, and users (managers or workers).

[0704] 2. Program processing explanation

[0705] Video acquisition and transmission

[0706] User: The user launches a dedicated app installed on a device (e.g., a tablet or smartphone). The administrator uses a camera to record the actions of factory workers. The device stores the captured video locally and prepares it for further processing. The video data is then sent to a device (server) on the network using a secure communication protocol (e.g., HTTPS).

[0707] Video analysis

[0708] Server: Prepares the received video data for input into an AI model (e.g., computer vision algorithm) for analysis. The AI ​​model analyzes the video and evaluates the efficiency of the factory workers' movements (such as accuracy, speed, and whether there are any unnecessary movements). The evaluation results are stored in a database.

[0709] Teaching the tricks

[0710] Server: Based on the analysis results, the server generates videos and animations that show each factory worker's specific weaknesses and areas for improvement. The generated content includes audio guides and text to make it easy for factory workers to understand. The generated videos and animations are sent to the device.

[0711] Device: The received videos and animations are played back within the app and presented to factory workers. The video playback status is recorded and reflected in the next training plan.

[0712] Support suggestions for administrators

[0713] Server: Generates support methods for managers based on the evaluation results of each factory worker's operational efficiency. For example, it creates a training policy such as, "This worker is slow. Speed ​​training would be effective." The generated support methods are sent to the terminal.

[0714] On the device: The administrator can review the suggestions received through the app, display the suggestions clearly to the administrator, and provide detailed explanations.

[0715] Worker level-up and evaluation

[0716] Server: Generates training methods for advanced skills for skilled factory workers, and provides video content for less skilled factory workers to refresh their knowledge from the basics.

[0717] Devices: For factory workers who need a refresher, video content is presented that explains the basic steps.

[0718] User: The manager checks the evaluation results in the app and assigns an evaluation and plans the next training based on the evaluation of each factory worker's operational efficiency.

[0719] 3. Specific Examples

[0720] For example, if a worker makes a lot of unnecessary movements when carrying equipment, the system can provide the following guidance:

[0721] Terminal: The administrator uses a camera to record footage of the workers' actions and sends it to the server.

[0722] Server: The AI ​​model analyzes the video and determines that there are many unnecessary movements. Based on the analysis results, it generates specific animations and audio guidance to improve the movement and sends them to the device.

[0723] Terminal: Workers watch the animation and learn efficient movements.

[0724] Managers can effectively train workers by referring to the support methods provided by the terminals and using aids and guidelines to promote efficient operations.

[0725] Prompt Sentence Examples

[0726] Please analyze the following video and provide specific guidance on how to improve work efficiency. This video shows Worker A carrying equipment.

[0727] In this way, by using the system of the present invention, it is possible to provide guidance that is tailored to the individual operational efficiency of factory workers, thereby realizing effective and efficient operational guidance.

[0728] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0729] Step 1:

[0730] User: The user launches a dedicated app installed on a device (tablet or smartphone). The administrator uses a camera to record the actions of the factory workers. The input is the video of the factory workers' actions.

[0731] Step 2:

[0732] Terminal: Stores the acquired video locally. Prepares for the next process and sends the video data to the server using a secure communication protocol (e.g., HTTPS). The input is the locally stored video data, and the output is the data to be sent to the server.

[0733] Step 3:

[0734] Server: Prepares the received video data for input into the AI ​​model in order to analyze it. Specifically, it converts the data format and performs preprocessing. The input is the video data sent to the server, and the output is data in a format suitable for the AI ​​model.

[0735] Step 4:

[0736] Server: The AI ​​model analyzes the video and evaluates the efficiency of factory workers' movements. Specifically, it calculates the accuracy, speed, and presence or absence of unnecessary movements. The input is preprocessed video data, and the output is the evaluation result of movement efficiency.

[0737] Step 5:

[0738] Server: The evaluation results are stored in a database. This database is later used to generate instructional videos. The input is the evaluation results of movement efficiency, and the output is the data stored in the database.

[0739] Step 6:

[0740] Server: Based on the analysis results, it generates videos and animations that show each factory worker's specific weaknesses and areas for improvement. The generated content includes audio guides and text to make it easy for factory workers to understand. The input is the evaluation results stored in the database, and the output is the generated instructional videos and animations.

[0741] Step 7:

[0742] Server: Sends the generated instructional videos and animations to the terminal. The input is the generated videos and animations, and the output is the data sent to the terminal.

[0743] Step 8:

[0744] Device: The received videos and animations are played within the app and presented to factory workers. The video playback status is recorded and reflected in the next training plan. The input is the instructional videos and animations sent from the server, and the output is the video playback and a record of the playback status.

[0745] Step 9:

[0746] Server: Generates support methods for managers based on the evaluation results of each factory worker's operational efficiency. Specifically, it creates guidance guidelines such as, "This worker is slow. Speed ​​training would be effective." The input is the evaluation results stored in the database, and the output is the generated support methods.

[0747] Step 10:

[0748] Server: Sends the generated support method to the terminal. The input is the generated support method, and the output is the data sent to the terminal.

[0749] Step 11:

[0750] Terminal: The administrator checks the suggestions received through the app. The suggestions are displayed in an easy-to-understand manner for the administrator, and detailed explanations are provided. The input is the support method sent from the server, and the output is the display of the suggestions.

[0751] Step 12:

[0752] Server: Generates training methods for advanced skills for skilled factory workers, and provides video content for weaker factory workers to refresh their knowledge from the basics. The input is the evaluation results stored in the database, and the output is the training methods and video content.

[0753] Step 13:

[0754] Terminal: For factory workers who need to refresh their knowledge from the basics, video content explaining basic steps is presented. The input is the training method and video content sent from the server, and the output is the video presentation.

[0755] Step 14:

[0756] User: The manager checks the evaluation results on the app and creates an evaluation and next training plan based on the evaluation of each factory worker's operational efficiency. The input is the evaluation results and training methods displayed on the device, and the output is the evaluation and training plan.

[0757] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0758] 1. System Overview

[0759] The system of the present invention analyzes children's athletic abilities and provides appropriate instruction, and furthermore, by combining it with an emotion engine, recognizes the user's emotions and makes the instruction more effective. The main components of the system are a terminal, a server, a user (teacher or student), and an emotion engine. An embodiment of this system is described in detail below.

[0760] 2. Program processing explanation

[0761] Video acquisition and transmission

[0762] User: A teacher or child launches a dedicated app installed on a device (e.g., a tablet or smartphone). The teacher uses the camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[0763] Terminal: The acquired video is stored locally, and the video data is converted into an appropriate format. It also compresses the video data to make it suitable for transmission.

[0764] Terminal: The compressed video data is sent to the server, using a secure communication protocol (e.g., HTTPS) to ensure data security.

[0765] Video analysis

[0766] Server: Prepares the received video data for input into a pre-trained AI model for analysis. Imports the video data and begins image processing.

[0767] Server: Using an AI model, the server tracks the child's movements in the video and calculates evaluation items related to athletic ability (such as jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[0768] Emotion Recognition and Analysis

[0769] Device: During recording, emotional data is collected in real time from the child's facial expressions and voice, including facial expression analysis and voice tone analysis.

[0770] Emotion Engine: Analyzes collected emotion data to determine the child's current emotional state (e.g., excitement, confusion, joy, etc.).

[0771] Device: The determined emotion data is sent to the server, again using a secure communication protocol.

[0772] Tips and emotional feedback

[0773] Server: Based on the analysis results and emotional data, it generates videos and animations with tips tailored to each child's specific weaknesses and areas for improvement. For example, if a child is confused, it will create a video with gentler training methods to resolve the issue.

[0774] Server: Sends the generated video or animation to the device.

[0775] Device: The received videos and animations are played in the app and presented to the child. The video playback status and effectiveness are recorded and reflected in the next lesson plan.

[0776] Support suggestions for teachers

[0777] Server: Based on the results of each child's motor skill assessment and emotional data, the server generates support methods for teachers (e.g., specific approaches for children who are struggling). For example, for a child who is struggling with rhythm, the server suggests a method using music for rhythm training.

[0778] Server: Sends the generated support method to the device.

[0779] Device: The teacher receives suggestions through the app and displays them on the user interface for review.

[0780] Leveling up and assessing your child

[0781] Server: Generates training approaches to help skilled children master advanced techniques (e.g., double jumps), and prepares to provide video content to help weaker children relearn the basics.

[0782] Device: For children who need to relearn the basics, the generated video content for basic learning can be played and they can learn step by step.

[0783] User: Teachers can view the assessment results in the app and use the assessment of each child's motor skills to assign grades and plan the next lesson.

[0784] 3. Specific Examples

[0785] For example, if a child is having trouble finding the right rhythm when jumping rope, you can use this system to teach them the following:

[0786] Device: The teacher records the jump rope with a camera, and at the same time, the emotion engine analyzes the child's facial expressions and tone of voice to determine whether they are confused.

[0787] Server: The AI ​​model analyzes the video and determines that there is a problem with the rhythm. It also receives emotional data indicating confusion.

[0788] Server: Generates specific animations to improve rhythm and gentle, encouraging audio guidance to resolve confusion, and sends them to the device.

[0789] Device: The animation and guide help children learn the correct skipping rhythm and reduce confusion.

[0790] Teachers can refer to the support methods provided by the devices and use rhythm training aids and music to effectively train children.

[0791] In this way, by using the system of the present invention, it is possible to provide instruction that is tailored to each child's individual motor abilities and emotional state, thereby achieving effective and efficient exercise instruction.

[0792] The processing flow will be explained below.

[0793] Step 1:

[0794] User: A teacher or child launches a dedicated app installed on a device (e.g., a tablet or smartphone). The teacher uses the camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[0795] Step 2:

[0796] Terminal: The acquired video is stored locally, and the video data is converted into an appropriate format. It also compresses the video data to make it suitable for transmission.

[0797] Step 3:

[0798] Terminal: The compressed video data is sent to the server, using a secure communication protocol (e.g., HTTPS) to ensure data security.

[0799] Step 4:

[0800] Server: Prepares the received video data for input into a pre-trained AI model for analysis. Imports the video data and begins image processing.

[0801] Step 5:

[0802] Server: Using an AI model, the server tracks the child's movements in the video and calculates evaluation items related to athletic ability (such as jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[0803] Step 6:

[0804] Device: During recording, emotional data is collected in real time from the child's facial expressions and voice, including facial expression analysis and voice tone analysis.

[0805] Step 7:

[0806] Emotion Engine: Analyzes collected emotion data to determine the child's current emotional state (e.g., excitement, confusion, joy, etc.).

[0807] Step 8:

[0808] Device: The determined emotion data is sent to the server, again using a secure communication protocol.

[0809] Step 9:

[0810] Server: Based on the analysis results and emotional data, it generates videos and animations with tips tailored to each child's specific weaknesses and areas for improvement. For example, if a child is confused, it will create a video with gentler training methods to resolve the issue.

[0811] Step 10:

[0812] Server: Sends the generated video or animation to the device, performing checks to ensure the integrity and security of the data in transit.

[0813] Step 11:

[0814] Device: The received video or animation is stored locally and played within the app, with display adjustments and layout settings made, and presented to the child.

[0815] Step 12:

[0816] Server: Based on the results of each child's motor skill assessment and emotional data, the server generates support methods for teachers (e.g., specific approaches for children who are struggling). For example, for a child who is struggling with rhythm, the server suggests a method using music for rhythm training.

[0817] Step 13:

[0818] Server: Sends the generated support method to the device. Checks the integrity of the data when sending and transfers it in the correct format.

[0819] Step 14:

[0820] Device: The teacher receives suggestions via the app and displays them on the user interface. The suggestions are formatted so that they are easy to understand.

[0821] Step 15:

[0822] Server: Generates training approaches to help skilled children master advanced techniques (e.g., double jumps), and prepares to provide video content to help weaker children relearn the basics.

[0823] Step 16:

[0824] Device: For children who need to relearn the basics, the generated video content for basic learning can be played and they can learn step by step.

[0825] Step 17:

[0826] User: Teachers can view the assessment results in the app and grade students and plan the next lesson based on their assessment of each child's motor skills and emotional data.

[0827] In this way, the system performs a series of processes to effectively evaluate and improve children's motor skills, and by combining this with emotional data, it is possible to provide optimal teaching methods for each individual.

[0828] Example 2

[0829] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0830] Conventional motor skill assessment systems rely primarily on visual information to analyze motor skills, and do not take into account the child's emotional state when providing instruction, which has the drawback of not being effective enough. Furthermore, there is a lack of individualized instruction for strong and weak areas, making it difficult to provide follow-up that responds to a child's growth.

[0831] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0832] In this invention, the server includes a means for analyzing video information and evaluating a child's athletic ability, a means for analyzing the child's emotional state using an emotion recognition engine and transmitting the analysis results to a data processing device, and a means for generating exercise instruction videos and visual effects for children based on the evaluation results, thereby enabling effective instruction that takes into account not only the child's athletic ability but also their emotional state.

[0833] "Video capture devices" refer to devices used to capture footage of children's athletic skills, and examples include cameras and smartphones.

[0834] The term "communication device" refers to a device for transmitting acquired video data to a data processing device, and specific examples include a wireless communication module and an internet connection device.

[0835] The "data processing device" refers to a device that analyzes received video information and evaluates a child's motor skills, and specific examples include a server and a computer.

[0836] "Video information" refers to video data that captures a child's athletic skills, specifically information that is saved and transferred in video file format.

[0837] "Motor ability" refers to the ability to assess multiple indicators related to a child's motor skills (e.g., jump height, rhythm, speed, etc.).

[0838] "Evaluation results" refers to the results of the evaluation of athletic ability analyzed by the data processing device, and includes numerical values ​​and text data stored in the database.

[0839] "Exercise method video" refers to a video created for children based on the evaluation results to help them master exercise techniques.

[0840] "Visual effects" refers to animations and graphic elements added to exercise videos, techniques used to present information in a way that is easy for children to understand.

[0841] "Display device" refers to a device for presenting the generated exercise method video and visual effects to a child, and examples include a tablet or smartphone.

[0842] "Educator" refers to a person responsible for teaching children athletic skills, including school teachers and instructors.

[0843] An "emotion recognition engine" refers to algorithms and software that analyze a child's emotional state from their facial expressions, voice, etc.

[0844] "Emotional state" refers to the child's current emotional state (e.g., excited, confused, happy, etc.), and includes information sent to the data processing device as an analysis result.

[0845] "Support measures" refer to specific methods and approaches that educators use to improve children's motor skills, and are created based on the evaluation results.

[0846] "Encrypted communications protocol" refers to the communications method used to ensure the security of transmitted video information, and examples include HTTPS and SSL.

[0847] The system of the present invention analyzes children's athletic abilities and provides appropriate instruction, and furthermore, by combining it with an emotion engine, recognizes the user's emotions and makes the instruction more effective. The main components of the system are a terminal, a server, a user (teacher or student), and an emotion engine. An embodiment of this system is described in detail below.

[0848] The system is equipped with the following hardware and software. Tablets and smartphones are used as terminals, with a dedicated application installed. The server uses a high-performance computer as the data processing device, and a deep learning model is implemented as the emotion recognition engine. OpenPose and FFmpeg are used for video analysis, and Praat is used for audio analysis. HTTPS is used as the communication protocol to ensure data security.

[0849] Video acquisition and transmission

[0850] The user, either a teacher or a child, launches a dedicated app installed on the device and prepares to record. Using the device's built-in camera, the teacher records the child's exercise (for example, skipping rope, horizontal bar, or vaulting box). When recording is complete, the device saves the captured video locally and converts the video data into an appropriate format (for example, MP4). It also performs a compression process to make it suitable for transmission. The compressed video data is sent to the server using the HTTPS protocol.

[0851] Video analysis

[0852] The server prepares the received video data for input into the AI ​​model. FFmpeg is used to break down the video data into frames and analyze each frame. The OpenPose library is used to track the child's movements in the video and calculate evaluation items related to athletic ability (jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[0853] Emotion Recognition and Analysis

[0854] While recording, the device collects emotional data in real time based on the child's facial expressions and voice. It uses OpenCV facial recognition technology and Praat voice tone analysis. The collected emotional data is analyzed by an emotion recognition engine to determine the child's current emotional state (excitement, confusion, joy, etc.). The determined emotional data is then sent back to the server using the HTTPS protocol.

[0855] Tips and emotional feedback

[0856] Based on the analysis results and emotional data, the server generates videos and animations that address each child's specific weaknesses and areas for improvement. Tools such as Adobe After Effects are used here, too. For example, a video containing gentler training methods can be generated for a child who is struggling. These videos and animations are then sent from the server to the device. The device then plays the received videos and animations within the app and presents them to the child. The device also has a function to monitor the child's reactions in real time during playback, and this is reflected in the next lesson plan.

[0857] Support suggestions for teachers

[0858] The server generates support methods for teachers based on each child's motor ability assessment results and emotional data. For children who are struggling with rhythm, it suggests methods such as using rhythmic training music. The generated support methods are sent to the device, which displays the suggestions received through the app on a user interface so that teachers can check them.

[0859] Leveling up and assessing your child

[0860] The server generates training approaches for children who excel at a skill to master advanced techniques (e.g., double jumps), and prepares video content for children who are not so good at a skill to relearn from the basics. For example, a step-by-step training program can be designed using Adobe After Effects. The device plays the generated video content for basic learning for children who need to relearn from the basics, providing a function for them to learn step by step.

[0861] Examples of prompt statements

[0862] "I would like to analyze the video and emotional data of a child skipping rope, and suggest an appropriate training method for a child who is having trouble. Please generate a video and audio guide for this purpose."

[0863] In this way, by using the system of the present invention, it is possible to provide instruction that is tailored to each child's individual motor abilities and emotional state, thereby achieving effective and efficient exercise instruction.

[0864] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0865] Step 1:

[0866] The user launches a dedicated app installed on their device (tablet or smartphone) and prepares for recording. This includes the login procedure and camera startup. The input is the user ID and password, and the output is the home screen of the app.

[0867] Step 2:

[0868] The user uses the device's camera to record their child's exercise (e.g., skipping rope, horizontal bar, vaulting box). During recording, the camera angle and distance are adjusted to capture the entire exercise. The input is the camera image, and the output is the recorded data.

[0869] Step 3:

[0870] The device stores the captured video locally and the video data is converted to an appropriate format (e.g., MP4). The input is the recorded data, and the output is the converted MP4 format video data.

[0871] Step 4:

[0872] The terminal then compresses the converted video data using the H.264 codec to make it suitable for transmission. The input is video data in MP4 format, and the output is compressed video data.

[0873] Step 5:

[0874] The terminal sends the compressed video data to the server using the HTTPS protocol, with the compressed video data as input and a confirmation of successful data transfer to the server as output.

[0875] Step 6:

[0876] The server prepares the received video data for input to the AI ​​model. To do this, it uses FFmpeg to break down the video data into frames. The input is compressed video data, and the output is image data for each frame.

[0877] Step 7:

[0878] The server uses the OpenPose library to track the child's movements frame by frame and calculates evaluation items related to athletic ability (jump height, rhythm, speed, etc.) Image data for each frame is input, and numerical data based on the evaluation items is output.

[0879] Step 8:

[0880] The server saves the calculated evaluation results in a database such as MySQL or PostgreSQL. The input is numerical data based on the evaluation items, and the output is recorded in the database.

[0881] Step 9:

[0882] The device collects emotional data in real time based on the child's facial expressions and voice during recording. This includes facial recognition technology (e.g., OpenCV) and tone analysis (e.g., Praat). The input is captured facial images and voice data, and the output is analyzed emotional data.

[0883] Step 10:

[0884] The emotion recognition engine analyzes the collected emotional data using a deep learning model to determine the child's current emotional state (excitement, confusion, joy, etc.) The input is the analyzed emotional data, and the output is a judgment of the child's emotional state.

[0885] Step 11:

[0886] The device transmits the determined emotion data to the server using the HTTPS protocol. The input is the result of the determination of the emotional state, and the output is confirmation of successful data transfer to the server.

[0887] Step 12:

[0888] Based on the analysis results and emotional data, the server generates videos and animations that provide tips for each child based on their specific weaknesses and areas for improvement. Tools such as Adobe After Effects are used for this. The inputs are the evaluation results and emotional data, and the output is the generated videos and animations.

[0889] Step 13:

[0890] The server sends the generated video or animation to the display device, with the generated video as input and confirmation of successful data transfer to the display device as output.

[0891] Step 14:

[0892] The device then plays the received video or animation within the app and presents it to the child. The input is the received video or animation, and the output is the video playback screen. The app also has a function to monitor the child's reaction in real time during playback.

[0893] Step 15:

[0894] The server generates support methods for educators based on each child's motor ability assessment results and emotional data. For example, for a child who is struggling with rhythm, it suggests a method using rhythm training music. The inputs are the assessment results and emotional data, and the output is the generated support method.

[0895] Step 16:

[0896] The server sends the generated support method to the display device, with the generated support method as input and confirmation of successful data transfer to the display device as output.

[0897] Step 17:

[0898] The device displays the suggestions received through the app on a user interface so that the teacher can check them. The input is the received support method, and the output is a screen showing the suggestions.

[0899] Step 18:

[0900] Teachers, who are users, check the assessment results on the app and use the assessment to grade each child and plan the next lesson. The input is the assessment results and support methods, and the output is a grade report and lesson plan.

[0901] Through the above steps, the system can provide effective and efficient exercise instruction tailored to each child's individual motor abilities and emotional state.

[0902] (Application example 2)

[0903] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0904] Currently, specific instruction is required to enhance children's motor skills, and appropriate feedback and training methods are needed. However, individual instruction tailored to each child's motor skills and emotional state is time-consuming and difficult to provide efficiently. Providing appropriate instruction and feedback to improve the operational efficiency and safety of robotic workers in factories is also difficult. In particular, systems that analyze the emotions and state of robots while they are operating and adjust their operations based on this information are still in the development stage, and there are many challenges before they can be applied in real-world situations.

[0905] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: video acquisition means equipped with a camera for analyzing the motor skills of children and the movements of robot workers; means for transmitting the acquired video to the server; means for analyzing and evaluating the video data in the server; means for transmitting generated training content to the terminal; means for presenting the transmitted training content; means for generating support methods for teachers or work instructors based on the motor skill evaluation results and emotional data in the server; means for transmitting the generated support methods to the terminal; means for displaying the transmitted support methods; means for analyzing emotional states using an emotion engine; means for providing feedback based on the evaluation results and emotional states; and means for improving work efficiency using a motor video analysis system. This enables detailed instruction and feedback to improve the movement efficiency of individual children and robot workers.

[0906] The "camera" is an image capturing device that captures the movements and work actions of the child and robot worker in real time.

[0907] The "server" is a central processing unit that analyzes the collected video data and emotional data and generates evaluation results regarding athletic ability and work efficiency.

[0908] "Image capture means" refers to a method or device for capturing images of a subject's movements or actions using a camera and acquiring the image data.

[0909] The "video transmission means" is a method or device for transmitting captured video data to a server via a network.

[0910] "Video analysis means" refers to a method or device by which the server analyzes video data and evaluates athletic ability and work efficiency.

[0911] The "training content generation means" is a method or device for generating motion programs for exercise training and work improvement for children and robot workers based on the evaluation results.

[0912] The "content transmission means" is a method or device for transmitting the generated training content and feedback to the terminal.

[0913] The "content presentation means" is a method or device for displaying the transmitted training content to children and robot workers.

[0914] The "emotion engine" is a system for analyzing a subject's emotional state from their facial expressions, movements, voice data, etc.

[0915] An "emotion analysis means" is a method or device for analyzing collected emotion data and determining a subject's current emotional state.

[0916] The "feedback providing means" is a method or device for providing appropriate feedback to a subject based on the analysis results and emotional state.

[0917] The "motion improvement means" is a method or device for the server to improve work efficiency based on the evaluation results using the motion video analysis system.

[0918] 1. System Configuration

[0919] This invention is a system that analyzes children's motor skills and the work movements of factory robots, and provides appropriate guidance and feedback. The main components of the system are a camera, a server, a terminal, a user (teacher or work instructor), and an emotion engine.

[0920] 2. Program processing explanation

[0921] Video acquisition and transmission

[0922] Camera: Records the movements and working actions of children and robot workers in real time. The camera has high resolution and clearly captures their movements.

[0923] Device: Stores the captured video locally, converts it into an appropriate format, compresses the video data for transmission, and sends it to the server using a secure communication protocol (HTTPS).

[0924] Video analysis

[0925] Server: Uses a pre-trained AI model to analyze the received video data. Imports the video data and starts image processing. Uses TensorFlow or PyTorch as the AI ​​model framework.

[0926] Server: The AI ​​model tracks the movements of the child or robot worker in the video and calculates evaluation items (speed, accuracy, safety, etc.) related to motor skills and work efficiency. This includes the speed and accuracy of exercise and work movements, as well as fatigue level. The calculated evaluation results are stored in a database.

[0927] Emotion Recognition and Analysis

[0928] Device: During recording, emotional data is collected in real time from the facial expressions and voices of the child and robotic worker, including facial expression analysis and voice tone analysis.

[0929] Emotion engine: Analyzes collected emotion data and determines the subject's current emotional state (e.g., excited, confused, tired, etc.). The emotion engine can use Microsoft Azure Cognitive Services.

[0930] Terminal: The determined emotion data is sent to the server, and a secure communication protocol is used to ensure the security of the data.

[0931] Generating and Providing Feedback

[0932] Server: Based on the analysis results and emotional data, it generates feedback tailored to each subject's specific weaknesses and areas for improvement. For example, if a robot is overworked, it generates feedback recommending appropriate rest or adjustments to its behavior.

[0933] Server: Sends the generated feedback to the device and displays it on the device. The server also generates and sends support methods to teachers and instructors.

[0934] Terminal: The received feedback and support methods are displayed on the user interface and presented to the target.

[0935] 3. Specific Examples

[0936] For example, if a robotic arm in a factory is moving slower than usual, its built-in camera will capture its movements and send them to a server. The server will then use an AI model and emotion engine to recognize that the robot is overworked. It will then generate feedback recommending appropriate breaks or minor adjustments to its movements and send it to the robot. For children, a teacher will record a video of the robot skipping rope with a camera, while the emotion engine will analyze the child's facial expressions and tone of voice to determine whether the child is confused. The server will then analyze the video, determine that there is a problem with the rhythm, and receive the emotional data indicating confusion. Based on this, it will generate specific animations to improve the rhythm and gentle, encouraging audio guidance to resolve the confusion, which will then be sent to the device. The child will then learn the correct skipping rhythm and become less confused.

[0937] Example of a generative AI model prompt:

[0938] To ensure that factory robots are working properly, how can you analyze video footage captured by built-in cameras, recognize the robot's performance evaluation and emotional state, and provide optimal feedback?

[0939] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0940] Step 1:

[0941] Footage collection

[0942] User: A teacher or instructor uses a camera to record the child's exercise and the robot's task movements.

[0943] Input: Child's movement or robot's task movement.

[0944] Processing: The camera captures the action and acquires the video data.

[0945] Output: Captured high-resolution video data.

[0946] Step 2:

[0947] Video storage and conversion

[0948] Device: Stores the captured video in local storage and converts it to the appropriate format.

[0949] Input: Acquired video data.

[0950] Processing: Video data is compressed and converted into a format suitable for transmission.

[0951] Output: Compressed video data.

[0952] Step 3:

[0953] Video data transmission

[0954] Terminal: Compressed video data is sent to the server using a secure protocol (HTTPS).

[0955] Input: Compressed video data.

[0956] Processing: Data transmission via secure communication protocols.

[0957] Output: Video data sent to the server.

[0958] Step 4:

[0959] Preparing video data for analysis

[0960] Server: Prepares to analyze the received video data.

[0961] Input: Video data sent to the server.

[0962] Processing: Preparing the video data in the appropriate format to input into the AI ​​model.

[0963] Output: Video data that can be input into an AI model.

[0964] Step 5:

[0965] Video analysis using AI models

[0966] Server: Uses AI models to analyze movements in the video (e.g., jump height, rhythm, and work efficiency).

[0967] Input: Video data.

[0968] Processing: Apply video analysis algorithms to evaluate subject's movements.

[0969] Output: Evaluation results of motor skills and work efficiency.

[0970] Step 6:

[0971] Emotion data collection and analysis

[0972] Device: Collects emotional data in real time based on the facial expressions and voices of children and robots.

[0973] Input: facial and voice data.

[0974] Processing: Facial expression analysis and voice tone analysis are performed to obtain emotion data.

[0975] Output: The obtained emotion data.

[0976] Step 7:

[0977] Emotion analysis using an emotion engine

[0978] Emotion engine: Analyzes the acquired emotion data and determines the current emotional state.

[0979] Input: Collected emotion data.

[0980] Processing: Determining emotional state.

[0981] Output: Determined emotional state data.

[0982] Step 8:

[0983] Sending emotional data

[0984] Device: Sends the determined emotion data to the server.

[0985] Input: Determined emotional state data.

[0986] Processing: Data transmission via secure communication protocol (HTTPS).

[0987] Output: Emotional state data sent to the server.

[0988] Step 9:

[0989] Generate feedback

[0990] Server: Based on the analysis results and emotional data, it generates feedback tailored to each subject's specific weaknesses and areas for improvement.

[0991] Input: Evaluation results and emotional state data.

[0992] Processing: Feedback generation and customization.

[0993] Output: The generated feedback data.

[0994] Step 10:

[0995] Providing feedback

[0996] Terminal: Display the received feedback on the user interface.

[0997] Input: Feedback data.

[0998] Action: View feedback.

[0999] Output: Feedback presented to the child and robot worker.

[1000] Through the above steps, it is possible to realize a system that properly analyzes the motion data and emotional state of children and robot workers, and provides detailed guidance and feedback.

[1001] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1002] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1003] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1004] [Third embodiment]

[1005] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1006] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1007] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1008] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1009] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1010] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1011] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1012] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1013] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1014] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1015] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1016] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1017] 1. System Overview

[1018] The system of the present invention analyzes the motor skills of children and provides appropriate guidance. The main components of the system are a terminal, a server, and a user (teacher or student). An embodiment of the system will be described in detail below.

[1019] 2. Program processing explanation

[1020] Video acquisition and transmission

[1021] User: The user launches a dedicated app installed on a device (e.g., a tablet or smartphone). A teacher uses a camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[1022] Terminal: The acquired video is stored locally and prepared for the next process. The video data is sent to the server using a secure communication protocol (e.g., HTTPS).

[1023] Video analysis

[1024] Server: Prepares the received video data for input into an AI model (e.g., computer vision algorithm) for analysis. The AI ​​model analyzes the video and evaluates the child's athletic ability (jump height, rhythm, speed, etc.). The evaluation results are stored in a database.

[1025] Teaching the tricks

[1026] Server: Based on the analysis results, the server generates videos and animations that address each child's specific weaknesses and areas for improvement. The generated content includes audio guides and text to make it easy for children to understand.

[1027] Server: Sends the generated video or animation to the device.

[1028] On the device: The received videos and animations are played in the app and presented to the child. The video playback status is recorded and reflected in the next lesson plan.

[1029] Support suggestions for teachers

[1030] Server: Generates support methods for teachers based on the results of each child's motor ability assessment. For example, it creates a teaching policy such as, "This child is not good at rhythm. Rhythm training would be effective."

[1031] Server: Sends the generated support method to the device.

[1032] On the device: The teacher checks the suggestions received through the app, displays the suggestions in an easy-to-understand format for the teacher, and provides detailed explanations.

[1033] Leveling up and assessing your child

[1034] Server: Generates training methods for advanced techniques (e.g. double jumps) for children who are good at the skill, and provides video content for children who are not good at the skill to relearn the basics.

[1035] Device: For children who need to refresh their knowledge, provide video content that explains the basic steps.

[1036] User: Teachers can view the assessment results in the app and use the assessment of each child's motor skills to assign grades and plan the next lesson.

[1037] 3. Specific Examples

[1038] For example, if a child is having trouble finding the right rhythm when jumping rope, you can use this system to teach them the following:

[1039] Device: The teacher records the jump rope video with a camera and sends it to the server.

[1040] Server: The AI ​​model analyzes the footage and determines that there is a problem with the rhythm.

[1041] Server: Generates specific animations and audio guides to improve rhythm and sends them to the device.

[1042] Device: Children watch the animation and learn the correct rhythm for skipping rope.

[1043] Teachers can refer to the support methods provided by the devices and use rhythm training aids and music to effectively train children.

[1044] In this way, by using the system of the present invention, it is possible to provide instruction that is tailored to each child's individual athletic ability, thereby realizing effective and efficient exercise instruction.

[1045] The processing flow will be explained below.

[1046] Step 1:

[1047] User: A teacher or child launches a dedicated app installed on a device (e.g., a tablet or smartphone). The teacher uses the camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[1048] Step 2:

[1049] Terminal: Stores the recorded video locally, converts the video data into an appropriate format, and compresses the video data for transmission.

[1050] Step 3:

[1051] Terminal: The compressed video data is sent to the server, using a secure communication protocol (e.g., HTTPS) to ensure data security.

[1052] Step 4:

[1053] Server: Prepares the received video data for input into a pre-trained AI model for analysis. Imports the video data and begins image processing.

[1054] Step 5:

[1055] Server: Using an AI model, the server tracks the child's movements in the video and calculates evaluation items related to athletic ability (such as jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[1056] Step 6:

[1057] Server: Identifies each child's specific weaknesses and areas for improvement, and generates videos and animations based on those weaknesses and areas for improvement. The generated content is then processed to add audio guides and text that are easy for children to understand.

[1058] Step 7:

[1059] Server: Sends the generated video or animation to the device, performing checks to ensure the integrity and security of the data in transit.

[1060] Step 8:

[1061] Device: The received video or animation is stored locally and played within the app, with display adjustments and layout settings made, and presented to the child.

[1062] Step 9:

[1063] Server: Generates support methods (e.g., specific rhythm training methods) for teachers based on the results of each child's motor ability assessment. The generated support methods are documented and stored in a database.

[1064] Step 10:

[1065] Server: Sends the generated support method to the device. Checks the integrity of the data when sending and transfers it in the correct format.

[1066] Step 11:

[1067] Device: The teacher receives suggestions via the app and displays them on the user interface. The suggestions are formatted so that they are easy to understand.

[1068] Step 12:

[1069] Server: Generates training approaches to help skilled children master advanced techniques (e.g., double jumps), and prepares to provide video content to help weaker children relearn the basics.

[1070] Step 13:

[1071] Device: For children who need to relearn the basics, the generated video content for basic learning can be played and they can learn step by step.

[1072] Step 14:

[1073] User: Teachers can view the assessment results in the app and use the assessment of each child's motor skills to assign grades and plan the next lesson.

[1074] In this way, the system performs a series of processes to effectively evaluate and improve children's motor skills.

[1075] Example 1

[1076] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1077] Currently, it is difficult to properly evaluate children's athletic ability and provide effective instruction based on that evaluation. Conventional athletic ability assessments struggle to grasp the specific characteristics and weaknesses of individual children in detail, making it impossible to provide effective training methods. Furthermore, there is a lack of means to generate specific instructional content based on the evaluation results, making it difficult for instructors to provide appropriate instruction tailored to each child's abilities. Furthermore, ensuring security when transmitting video data is also a major issue.

[1078] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1079] In this invention, the server includes a camera for analyzing a child's athletic ability, a means for capturing video of the child's athletic activity, a means for transmitting the captured video to a data center, a means for analyzing the video data at the data center and evaluating the child's athletic ability, a means for generating videos and moving images of exercise tips for the child based on the evaluation results, a means for transmitting the generated videos and moving images to a display device, a means for presenting the transmitted videos and moving images to the child, a means for generating support methods for teachers based on the athletic ability evaluation results at the data center, a means for transmitting the generated support methods to a display device, a means for displaying the transmitted support methods to the teacher, a means for the teacher to refer to the evaluation results, a means for utilizing a generative AI model based on the child's athletic ability to provide specific training methods and learning plans, and a means for inputting prompts related to the target exercise into the generative AI model to generate appropriate instruction content. This enables a detailed evaluation of each child's athletic ability and the provision of effective instruction based on the evaluation results in a timely and appropriate manner. Security can also be ensured during the transmission of video data.

[1080] "Photographing device" refers to a device for recording footage of a child's exercise, and includes cameras, video cameras, and the like.

[1081] A "data center" is a centralized management facility for receiving, storing, and analyzing acquired video data.

[1082] "Video data" refers to video files of a child's exercise captured by a camera.

[1083] "Analysis" is the process of evaluating a child's motor skills based on the acquired video data, using computer vision algorithms, etc.

[1084] "Evaluation results" refer to the numerical values ​​and indicators of a child's physical abilities obtained through analysis.

[1085] "Exercise Tips Videos" are videos that provide specific advice and training methods to help children improve their athletic abilities.

[1086] "Moving Images" refers to content that includes animations and visual guides presented to children.

[1087] A "display device" is a device used to present received videos and moving images to children and teachers, and includes tablets, smartphones, and the like.

[1088] "Support methods" refer to teaching strategies and advice provided to teachers to help them provide effective instruction to children.

[1089] A "generative AI model" is an artificial intelligence model that generates specific training methods and learning plans based on a child's athletic ability.

[1090] A "prompt" is a document that describes the instructions or requirements that are input to a generative AI model.

[1091] The present invention is a system for analyzing children's athletic abilities and providing optimal instruction for each child. This system comprises a camera, a data center, a display device, and users (teachers and children).

[1092] The user is a teacher who uses a camera (e.g., a camera built into a tablet or smartphone) to record the target child's exercise (e.g., skipping rope, horizontal bar, vaulting horse). The recorded video data is stored locally on the device (tablet or smartphone) and then sent to a data center using a secure communication protocol (e.g., HTTPS).

[1093] The data center then inputs the received video data into an AI model using computer vision algorithms. The AI ​​model analyzes the video and evaluates the child's athletic ability (e.g., jump height, rhythm, speed, etc.). The evaluation results are stored in a database.

[1094] Based on the assessment results, the data center generates instructional content tailored to each child's specific weaknesses and areas for improvement. Using a generative AI model, it generates videos and animations of exercise tips tailored to each child. This content includes audio guides and text to make it easy for children to understand.

[1095] The generated content is sent to a display device (such as a tablet or smartphone). The content is played on the display device and presented to the child. The playback status is recorded and reflected in future learning plans. Based on the evaluation results, the data center also generates support methods for teachers and sends them to the device. Teachers can check the support methods sent to them on the display device and appropriately convey the content to the children.

[1096] This system provides training methods for advanced athletic skills (e.g., double jumps) to children who are good at it, and video content to encourage children who are not good at it to relearn the basics.In addition, to ensure the security of the video data being transmitted, an encrypted data communication protocol is used.

[1097] For example, if a child has trouble finding the right rhythm when skipping rope, you can use this system to teach them the following steps:

[1098] 1. A teacher records the jump rope video with a camera and sends it to the data center.

[1099] 2. The data center analyzes the video and determines that there is a problem with the rhythm.

[1100] 3. Generate specific animations and audio guides to improve rhythm and send them to the display device.

[1101] 4. Children watch the animation and learn the correct rhythm of skipping rope.

[1102] An example prompt is:

[1103] "Analyze a video of a child skipping rope and assess whether there is a problem with his rhythm. If he is not getting the rhythm right, provide specific areas for improvement and suggest training methods based on those. Your suggestions should include animations and audio guides that are easy for children to understand."

[1104] By using this system in this way, highly efficient instruction tailored to each child's individual motor abilities can be achieved, improving educational effectiveness.

[1105] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1106] Step 1:

[1107] User: A teacher starts a dedicated app on their device to record a child's exercise. They set the exercise to be recorded (e.g., jumping rope) and press the record button to record the child's exercise.

[1108] Input: Device camera, teacher operation

[1109] Output: Recorded exercise video data

[1110] Specific behavior:

[1111] The teacher taps the "Start Recording" button on the app.

[1112] The device's camera will activate and capture video of your workout.

[1113] During video capture, the device temporarily stores the data in a buffer memory.

[1114] After recording is complete, the data is saved to the internal storage.

[1115] Step 2:

[1116] Terminal: Sends stored video data to the data center.

[1117] Input: Recorded exercise video data

[1118] Output: Video data sent to the data center

[1119] Specific behavior:

[1120] The device acquires the video data and uploads it to a data center using a secure communication protocol (HTTPS).

[1121] The data center receives and stores the uploaded data.

[1122] Step 3:

[1123] Server: Prepares for analyzing the received video data. Inputs the data into the AI ​​model and begins analysis.

[1124] Input: Video data stored in the data center

[1125] Output: Evaluation results of athletic ability

[1126] Specific behavior:

[1127] The server inputs the video data into an AI model for analysis.

[1128] The AI ​​model analyzes the video frame by frame and extracts movement parameters (jump height, rhythm, speed, etc.).

[1129] Athletic ability is evaluated based on the extracted data, and the results are stored in a database.

[1130] Step 4:

[1131] Server: Based on the analysis results, it generates videos and moving images of exercise tips for children.

[1132] Input: Athletic ability evaluation results

[1133] Output: Generated exercise tips videos and moving images

[1134] Specific behavior:

[1135] Based on the evaluation results, the server sends a prompt to the generative AI model.

[1136] The generative AI model generates videos and animations with appropriate tips and improvements.

[1137] Audio guides and text are added to the generated videos and moving images.

[1138] Step 5:

[1139] Server: Sends the generated video and moving images to the device.

[1140] Input: Generated exercise tips videos and moving images

[1141] Output: Videos and images sent to the device

[1142] Specific behavior:

[1143] The server retrieves the generated content and transmits it to the terminal.

[1144] The device receives the transmitted data and stores it in local storage.

[1145] Step 6:

[1146] Device: Plays the received videos and images and shows them to the child.

[1147] Input: Received video or moving images

[1148] Output: Played video and moving images

[1149] Specific behavior:

[1150] A teacher or child taps the "Start Playing" button in the app.

[1151] The device plays videos and moving images and shows them to the child.

[1152] Record playback status and reflect it in future learning plans.

[1153] Step 7:

[1154] Server: Generates support methods for teachers and sends them to the devices.

[1155] Input: Athletic ability evaluation results

[1156] Output: Generated support instructions for teachers

[1157] Specific behavior:

[1158] The server generates support policies for teachers based on the evaluation results.

[1159] The generated support policy is sent to the terminal.

[1160] The device displays the support method received and notifies the teacher.

[1161] Step 8:

[1162] User: Teachers refer to the assessment results and provide appropriate guidance to children.

[1163] Input: Evaluation results, generated support methods

[1164] Output: Teacher's teaching content

[1165] Specific behavior:

[1166] The teacher checks the app's evaluation results screen.

[1167] Based on the information obtained, specific guidance is given to the child.

[1168] If necessary, have your child review the training videos and animations provided by the app.

[1169] This makes it possible to evaluate each child's motor skills in detail and provide effective instruction based on that evaluation.

[1170] (Application example 1)

[1171] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1172] It is difficult to provide a system that evaluates the efficiency of workers' actions in a factory and provides appropriate training methods. It is also difficult to generate content for individualized instruction and training that takes into account the strengths and weaknesses of each worker. This creates the challenge of making it difficult to achieve the expected improvement in the production efficiency of the entire factory and the skills of workers.

[1173] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1174] In this invention, the server includes means for acquiring video of the movements of factory workers, means for transmitting the acquired video to a device on the network, means for analyzing the video data in the device on the network and evaluating the movement efficiency of the factory workers, means for generating instructional videos and animations for improving the movements of the factory workers based on the evaluation results, means for transmitting the generated videos and animations to a terminal, means for presenting the transmitted videos and animations to the factory workers, means for generating support methods for a manager based on the evaluation results of the movement efficiency in the device on the network, means for transmitting the generated support methods to the terminal, means for displaying the transmitted support methods to the manager, and means for the manager to refer to the evaluation results. This makes it possible to efficiently analyze the movements of workers in a factory and provide specific instruction methods tailored to each worker.

[1175] "Factory workers" refer to workers engaged in production activities within a factory.

[1176] "Motion video" refers to video data that records the movements of factory workers while they are working.

[1177] "Capture equipment" refers to cameras or other image capture devices used to capture footage of factory workers in action.

[1178] "Devices on a network" refers to computing resources such as servers connected via a network such as the Internet or an intranet.

[1179] "Video data" refers to digital data that records the actions of factory workers.

[1180] "Operational efficiency" is an index that evaluates how efficiently factory workers are performing their work.

[1181] "Instructional videos" refer to video content that visually teaches factory workers how to improve their performance.

[1182] "Animation" refers to visual media consisting of a sequence of image frames that visually represent behavioral improvement techniques.

[1183] "Terminal" refers to a device such as a smartphone or tablet used to play instructional videos and animations.

[1184] "Supervisor" refers to the person responsible for overseeing the activities of factory workers and providing guidance on how to support them.

[1185] "Support methods" refer to specific guidance and improvement measures provided to administrators based on evaluations of operational efficiency.

[1186] 1. System Overview

[1187] The system of the present invention analyzes the operational efficiency of factory workers and provides appropriate guidance methods. The main components of the system are terminals, a server, and users (managers or workers).

[1188] 2. Program processing explanation

[1189] Video acquisition and transmission

[1190] User: The user launches a dedicated app installed on a device (e.g., a tablet or smartphone). The administrator uses a camera to record the actions of factory workers. The device stores the captured video locally and prepares it for further processing. The video data is then sent to a device (server) on the network using a secure communication protocol (e.g., HTTPS).

[1191] Video analysis

[1192] Server: Prepares the received video data for input into an AI model (e.g., computer vision algorithm) for analysis. The AI ​​model analyzes the video and evaluates the efficiency of the factory workers' movements (such as accuracy, speed, and whether there are any unnecessary movements). The evaluation results are stored in a database.

[1193] Teaching the tricks

[1194] Server: Based on the analysis results, the server generates videos and animations that show each factory worker's specific weaknesses and areas for improvement. The generated content includes audio guides and text to make it easy for factory workers to understand. The generated videos and animations are sent to the device.

[1195] Device: The received videos and animations are played back within the app and presented to factory workers. The video playback status is recorded and reflected in the next training plan.

[1196] Support suggestions for administrators

[1197] Server: Generates support methods for managers based on the evaluation results of each factory worker's operational efficiency. For example, it creates a training policy such as, "This worker is slow. Speed ​​training would be effective." The generated support methods are sent to the terminal.

[1198] On the device: The administrator can review the suggestions received through the app, display the suggestions clearly to the administrator, and provide detailed explanations.

[1199] Worker level-up and evaluation

[1200] Server: Generates training methods for advanced skills for skilled factory workers, and provides video content for less skilled factory workers to refresh their knowledge from the basics.

[1201] Devices: For factory workers who need a refresher, video content is presented that explains the basic steps.

[1202] User: The manager checks the evaluation results in the app and assigns an evaluation and plans the next training based on the evaluation of each factory worker's operational efficiency.

[1203] 3. Specific Examples

[1204] For example, if a worker makes a lot of unnecessary movements when carrying equipment, the system can provide the following guidance:

[1205] Terminal: The administrator uses a camera to record footage of the workers' actions and sends it to the server.

[1206] Server: The AI ​​model analyzes the video and determines that there are many unnecessary movements. Based on the analysis results, it generates specific animations and audio guidance to improve the movement and sends them to the device.

[1207] Terminal: Workers watch the animation and learn efficient movements.

[1208] Managers can effectively train workers by referring to the support methods provided by the terminals and using aids and guidelines to promote efficient operations.

[1209] Prompt Sentence Examples

[1210] Please analyze the following video and provide specific guidance on how to improve work efficiency. This video shows Worker A carrying equipment.

[1211] In this way, by using the system of the present invention, it is possible to provide guidance that is tailored to the individual operational efficiency of factory workers, thereby realizing effective and efficient operational guidance.

[1212] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1213] Step 1:

[1214] User: The user launches a dedicated app installed on a device (tablet or smartphone). The administrator uses a camera to record the actions of the factory workers. The input is the video of the factory workers' actions.

[1215] Step 2:

[1216] Terminal: Stores the acquired video locally. Prepares for the next process and sends the video data to the server using a secure communication protocol (e.g., HTTPS). The input is the locally stored video data, and the output is the data to be sent to the server.

[1217] Step 3:

[1218] Server: Prepares the received video data for input into the AI ​​model in order to analyze it. Specifically, it converts the data format and performs preprocessing. The input is the video data sent to the server, and the output is data in a format suitable for the AI ​​model.

[1219] Step 4:

[1220] Server: The AI ​​model analyzes the video and evaluates the efficiency of factory workers' movements. Specifically, it calculates the accuracy, speed, and presence or absence of unnecessary movements. The input is preprocessed video data, and the output is the evaluation result of movement efficiency.

[1221] Step 5:

[1222] Server: The evaluation results are stored in a database. This database is later used to generate instructional videos. The input is the evaluation results of movement efficiency, and the output is the data stored in the database.

[1223] Step 6:

[1224] Server: Based on the analysis results, it generates videos and animations that show each factory worker's specific weaknesses and areas for improvement. The generated content includes audio guides and text to make it easy for factory workers to understand. The input is the evaluation results stored in the database, and the output is the generated instructional videos and animations.

[1225] Step 7:

[1226] Server: Sends the generated instructional videos and animations to the terminal. The input is the generated videos and animations, and the output is the data sent to the terminal.

[1227] Step 8:

[1228] Device: The received videos and animations are played within the app and presented to factory workers. The video playback status is recorded and reflected in the next training plan. The input is the instructional videos and animations sent from the server, and the output is the video playback and a record of the playback status.

[1229] Step 9:

[1230] Server: Generates support methods for managers based on the evaluation results of each factory worker's operational efficiency. Specifically, it creates guidance guidelines such as, "This worker is slow. Speed ​​training would be effective." The input is the evaluation results stored in the database, and the output is the generated support methods.

[1231] Step 10:

[1232] Server: Sends the generated support method to the terminal. The input is the generated support method, and the output is the data sent to the terminal.

[1233] Step 11:

[1234] Terminal: The administrator checks the suggestions received through the app. The suggestions are displayed in an easy-to-understand manner for the administrator, and detailed explanations are provided. The input is the support method sent from the server, and the output is the display of the suggestions.

[1235] Step 12:

[1236] Server: Generates training methods for advanced skills for skilled factory workers, and provides video content for weaker factory workers to refresh their knowledge from the basics. The input is the evaluation results stored in the database, and the output is the training methods and video content.

[1237] Step 13:

[1238] Terminal: For factory workers who need to refresh their knowledge from the basics, video content explaining basic steps is presented. The input is the training method and video content sent from the server, and the output is the video presentation.

[1239] Step 14:

[1240] User: The manager checks the evaluation results on the app and creates an evaluation and next training plan based on the evaluation of each factory worker's operational efficiency. The input is the evaluation results and training methods displayed on the device, and the output is the evaluation and training plan.

[1241] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1242] 1. System Overview

[1243] The system of the present invention analyzes children's athletic abilities and provides appropriate instruction, and furthermore, by combining it with an emotion engine, recognizes the user's emotions and makes the instruction more effective. The main components of the system are a terminal, a server, a user (teacher or student), and an emotion engine. An embodiment of this system is described in detail below.

[1244] 2. Program processing explanation

[1245] Video acquisition and transmission

[1246] User: A teacher or child launches a dedicated app installed on a device (e.g., a tablet or smartphone). The teacher uses the camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[1247] Terminal: The acquired video is stored locally, and the video data is converted into an appropriate format. It also compresses the video data to make it suitable for transmission.

[1248] Terminal: The compressed video data is sent to the server, using a secure communication protocol (e.g., HTTPS) to ensure data security.

[1249] Video analysis

[1250] Server: Prepares the received video data for input into a pre-trained AI model for analysis. Imports the video data and begins image processing.

[1251] Server: Using an AI model, the server tracks the child's movements in the video and calculates evaluation items related to athletic ability (such as jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[1252] Emotion Recognition and Analysis

[1253] Device: During recording, emotional data is collected in real time from the child's facial expressions and voice, including facial expression analysis and voice tone analysis.

[1254] Emotion Engine: Analyzes collected emotion data to determine the child's current emotional state (e.g., excitement, confusion, joy, etc.).

[1255] Device: The determined emotion data is sent to the server, again using a secure communication protocol.

[1256] Tips and emotional feedback

[1257] Server: Based on the analysis results and emotional data, it generates videos and animations with tips tailored to each child's specific weaknesses and areas for improvement. For example, if a child is confused, it will create a video with gentler training methods to resolve the issue.

[1258] Server: Sends the generated video or animation to the device.

[1259] Device: The received videos and animations are played in the app and presented to the child. The video playback status and effectiveness are recorded and reflected in the next lesson plan.

[1260] Support suggestions for teachers

[1261] Server: Based on the results of each child's motor skill assessment and emotional data, the server generates support methods for teachers (e.g., specific approaches for children who are struggling). For example, for a child who is struggling with rhythm, the server suggests a method using music for rhythm training.

[1262] Server: Sends the generated support method to the device.

[1263] Device: The teacher receives suggestions through the app and displays them on the user interface for review.

[1264] Leveling up and assessing your child

[1265] Server: Generates training approaches to help skilled children master advanced techniques (e.g., double jumps), and prepares to provide video content to help weaker children relearn the basics.

[1266] Device: For children who need to relearn the basics, the generated video content for basic learning can be played and they can learn step by step.

[1267] User: Teachers can view the assessment results in the app and use the assessment of each child's motor skills to assign grades and plan the next lesson.

[1268] 3. Specific Examples

[1269] For example, if a child is having trouble finding the right rhythm when jumping rope, you can use this system to teach them the following:

[1270] Device: The teacher records the jump rope with a camera, and at the same time, the emotion engine analyzes the child's facial expressions and tone of voice to determine whether they are confused.

[1271] Server: The AI ​​model analyzes the video and determines that there is a problem with the rhythm. It also receives emotional data indicating confusion.

[1272] Server: Generates specific animations to improve rhythm and gentle, encouraging audio guidance to resolve confusion, and sends them to the device.

[1273] Device: The animation and guide help children learn the correct skipping rhythm and reduce confusion.

[1274] Teachers can refer to the support methods provided by the devices and use rhythm training aids and music to effectively train children.

[1275] In this way, by using the system of the present invention, it is possible to provide instruction that is tailored to each child's individual motor abilities and emotional state, thereby achieving effective and efficient exercise instruction.

[1276] The processing flow will be explained below.

[1277] Step 1:

[1278] User: A teacher or child launches a dedicated app installed on a device (e.g., a tablet or smartphone). The teacher uses the camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[1279] Step 2:

[1280] Terminal: The acquired video is stored locally, and the video data is converted into an appropriate format. It also compresses the video data to make it suitable for transmission.

[1281] Step 3:

[1282] Terminal: The compressed video data is sent to the server, using a secure communication protocol (e.g., HTTPS) to ensure data security.

[1283] Step 4:

[1284] Server: Prepares the received video data for input into a pre-trained AI model for analysis. Imports the video data and begins image processing.

[1285] Step 5:

[1286] Server: Using an AI model, the server tracks the child's movements in the video and calculates evaluation items related to athletic ability (such as jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[1287] Step 6:

[1288] Device: During recording, emotional data is collected in real time from the child's facial expressions and voice, including facial expression analysis and voice tone analysis.

[1289] Step 7:

[1290] Emotion Engine: Analyzes collected emotion data to determine the child's current emotional state (e.g., excitement, confusion, joy, etc.).

[1291] Step 8:

[1292] Device: The determined emotion data is sent to the server, again using a secure communication protocol.

[1293] Step 9:

[1294] Server: Based on the analysis results and emotional data, it generates videos and animations with tips tailored to each child's specific weaknesses and areas for improvement. For example, if a child is confused, it will create a video with gentler training methods to resolve the issue.

[1295] Step 10:

[1296] Server: Sends the generated video or animation to the device, performing checks to ensure the integrity and security of the data in transit.

[1297] Step 11:

[1298] Device: The received video or animation is stored locally and played within the app, with display adjustments and layout settings made, and presented to the child.

[1299] Step 12:

[1300] Server: Based on the results of each child's motor skill assessment and emotional data, the server generates support methods for teachers (e.g., specific approaches for children who are struggling). For example, for a child who is struggling with rhythm, the server suggests a method using music for rhythm training.

[1301] Step 13:

[1302] Server: Sends the generated support method to the device. Checks the integrity of the data when sending and transfers it in the correct format.

[1303] Step 14:

[1304] Device: The teacher receives suggestions via the app and displays them on the user interface. The suggestions are formatted so that they are easy to understand.

[1305] Step 15:

[1306] Server: Generates training approaches to help skilled children master advanced techniques (e.g., double jumps), and prepares to provide video content to help weaker children relearn the basics.

[1307] Step 16:

[1308] Device: For children who need to relearn the basics, the generated video content for basic learning can be played and they can learn step by step.

[1309] Step 17:

[1310] User: Teachers can view the assessment results in the app and grade students and plan the next lesson based on their assessment of each child's motor skills and emotional data.

[1311] In this way, the system performs a series of processes to effectively evaluate and improve children's motor skills, and by combining this with emotional data, it is possible to provide optimal teaching methods for each individual.

[1312] Example 2

[1313] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1314] Conventional motor skill assessment systems rely primarily on visual information to analyze motor skills, and do not take into account the child's emotional state when providing instruction, which has the drawback of not being effective enough. Furthermore, there is a lack of individualized instruction for strong and weak areas, making it difficult to provide follow-up that responds to a child's growth.

[1315] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1316] In this invention, the server includes a means for analyzing video information and evaluating a child's athletic ability, a means for analyzing the child's emotional state using an emotion recognition engine and transmitting the analysis results to a data processing device, and a means for generating exercise instruction videos and visual effects for children based on the evaluation results, thereby enabling effective instruction that takes into account not only the child's athletic ability but also their emotional state.

[1317] "Video capture devices" refer to devices used to capture footage of children's athletic skills, and examples include cameras and smartphones.

[1318] The term "communication device" refers to a device for transmitting acquired video data to a data processing device, and specific examples include a wireless communication module and an internet connection device.

[1319] The "data processing device" refers to a device that analyzes received video information and evaluates a child's motor skills, and specific examples include a server and a computer.

[1320] "Video information" refers to video data that captures a child's athletic skills, specifically information that is saved and transferred in video file format.

[1321] "Motor ability" refers to the ability to assess multiple indicators related to a child's motor skills (e.g., jump height, rhythm, speed, etc.).

[1322] "Evaluation results" refers to the results of the evaluation of athletic ability analyzed by the data processing device, and includes numerical values ​​and text data stored in the database.

[1323] "Exercise method video" refers to a video created for children based on the evaluation results to help them master exercise techniques.

[1324] "Visual effects" refers to animations and graphic elements added to exercise videos, techniques used to present information in a way that is easy for children to understand.

[1325] "Display device" refers to a device for presenting the generated exercise method video and visual effects to a child, and examples include a tablet or smartphone.

[1326] "Educator" refers to a person responsible for teaching children athletic skills, including school teachers and instructors.

[1327] An "emotion recognition engine" refers to algorithms and software that analyze a child's emotional state from their facial expressions, voice, etc.

[1328] "Emotional state" refers to the child's current emotional state (e.g., excited, confused, happy, etc.), and includes information sent to the data processing device as an analysis result.

[1329] "Support measures" refer to specific methods and approaches that educators use to improve children's motor skills, and are created based on the evaluation results.

[1330] "Encrypted communications protocol" refers to the communications method used to ensure the security of transmitted video information, and examples include HTTPS and SSL.

[1331] The system of the present invention analyzes children's athletic abilities and provides appropriate instruction, and furthermore, by combining it with an emotion engine, recognizes the user's emotions and makes the instruction more effective. The main components of the system are a terminal, a server, a user (teacher or student), and an emotion engine. An embodiment of this system is described in detail below.

[1332] The system is equipped with the following hardware and software. Tablets and smartphones are used as terminals, with a dedicated application installed. The server uses a high-performance computer as the data processing device, and a deep learning model is implemented as the emotion recognition engine. OpenPose and FFmpeg are used for video analysis, and Praat is used for audio analysis. HTTPS is used as the communication protocol to ensure data security.

[1333] Video acquisition and transmission

[1334] The user, either a teacher or a child, launches a dedicated app installed on the device and prepares to record. Using the device's built-in camera, the teacher records the child's exercise (for example, skipping rope, horizontal bar, or vaulting box). When recording is complete, the device saves the captured video locally and converts the video data into an appropriate format (for example, MP4). It also performs a compression process to make it suitable for transmission. The compressed video data is sent to the server using the HTTPS protocol.

[1335] Video analysis

[1336] The server prepares the received video data for input into the AI ​​model. FFmpeg is used to break down the video data into frames and analyze each frame. The OpenPose library is used to track the child's movements in the video and calculate evaluation items related to athletic ability (jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[1337] Emotion Recognition and Analysis

[1338] While recording, the device collects emotional data in real time based on the child's facial expressions and voice. It uses OpenCV facial recognition technology and Praat voice tone analysis. The collected emotional data is analyzed by an emotion recognition engine to determine the child's current emotional state (excitement, confusion, joy, etc.). The determined emotional data is then sent back to the server using the HTTPS protocol.

[1339] Tips and emotional feedback

[1340] Based on the analysis results and emotional data, the server generates videos and animations that address each child's specific weaknesses and areas for improvement. Tools such as Adobe After Effects are used here, too. For example, a video containing gentler training methods can be generated for a child who is struggling. These videos and animations are then sent from the server to the device. The device then plays the received videos and animations within the app and presents them to the child. The device also has a function to monitor the child's reactions in real time during playback, and this is reflected in the next lesson plan.

[1341] Support suggestions for teachers

[1342] The server generates support methods for teachers based on each child's motor ability assessment results and emotional data. For children who are struggling with rhythm, it suggests methods such as using rhythmic training music. The generated support methods are sent to the device, which displays the suggestions received through the app on a user interface so that teachers can check them.

[1343] Leveling up and assessing your child

[1344] The server generates training approaches for children who excel at a skill to master advanced techniques (e.g., double jumps), and prepares video content for children who are not so good at a skill to relearn from the basics. For example, a step-by-step training program can be designed using Adobe After Effects. The device plays the generated video content for basic learning for children who need to relearn from the basics, providing a function for them to learn step by step.

[1345] Examples of prompt statements

[1346] "I would like to analyze the video and emotional data of a child skipping rope, and suggest an appropriate training method for a child who is having trouble. Please generate a video and audio guide for this purpose."

[1347] In this way, by using the system of the present invention, it is possible to provide instruction that is tailored to each child's individual motor abilities and emotional state, thereby achieving effective and efficient exercise instruction.

[1348] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1349] Step 1:

[1350] The user launches a dedicated app installed on their device (tablet or smartphone) and prepares for recording. This includes the login procedure and camera startup. The input is the user ID and password, and the output is the home screen of the app.

[1351] Step 2:

[1352] The user uses the device's camera to record their child's exercise (e.g., skipping rope, horizontal bar, vaulting box). During recording, the camera angle and distance are adjusted to capture the entire exercise. The input is the camera image, and the output is the recorded data.

[1353] Step 3:

[1354] The device stores the captured video locally and the video data is converted to an appropriate format (e.g., MP4). The input is the recorded data, and the output is the converted MP4 format video data.

[1355] Step 4:

[1356] The terminal then compresses the converted video data using the H.264 codec to make it suitable for transmission. The input is video data in MP4 format, and the output is compressed video data.

[1357] Step 5:

[1358] The terminal sends the compressed video data to the server using the HTTPS protocol, with the compressed video data as input and a confirmation of successful data transfer to the server as output.

[1359] Step 6:

[1360] The server prepares the received video data for input to the AI ​​model. To do this, it uses FFmpeg to break down the video data into frames. The input is compressed video data, and the output is image data for each frame.

[1361] Step 7:

[1362] The server uses the OpenPose library to track the child's movements frame by frame and calculates evaluation items related to athletic ability (jump height, rhythm, speed, etc.) Image data for each frame is input, and numerical data based on the evaluation items is output.

[1363] Step 8:

[1364] The server saves the calculated evaluation results in a database such as MySQL or PostgreSQL. The input is numerical data based on the evaluation items, and the output is recorded in the database.

[1365] Step 9:

[1366] The device collects emotional data in real time based on the child's facial expressions and voice during recording. This includes facial recognition technology (e.g., OpenCV) and tone analysis (e.g., Praat). The input is captured facial images and voice data, and the output is analyzed emotional data.

[1367] Step 10:

[1368] The emotion recognition engine analyzes the collected emotional data using a deep learning model to determine the child's current emotional state (excitement, confusion, joy, etc.) The input is the analyzed emotional data, and the output is a judgment of the child's emotional state.

[1369] Step 11:

[1370] The device transmits the determined emotion data to the server using the HTTPS protocol. The input is the result of the determination of the emotional state, and the output is confirmation of successful data transfer to the server.

[1371] Step 12:

[1372] Based on the analysis results and emotional data, the server generates videos and animations that provide tips for each child based on their specific weaknesses and areas for improvement. Tools such as Adobe After Effects are used for this. The inputs are the evaluation results and emotional data, and the output is the generated videos and animations.

[1373] Step 13:

[1374] The server sends the generated video or animation to the display device, with the generated video as input and confirmation of successful data transfer to the display device as output.

[1375] Step 14:

[1376] The device then plays the received video or animation within the app and presents it to the child. The input is the received video or animation, and the output is the video playback screen. The app also has a function to monitor the child's reaction in real time during playback.

[1377] Step 15:

[1378] The server generates support methods for educators based on each child's motor ability assessment results and emotional data. For example, for a child who is struggling with rhythm, it suggests a method using rhythm training music. The inputs are the assessment results and emotional data, and the output is the generated support method.

[1379] Step 16:

[1380] The server sends the generated support method to the display device, with the generated support method as input and confirmation of successful data transfer to the display device as output.

[1381] Step 17:

[1382] The device displays the suggestions received through the app on a user interface so that the teacher can check them. The input is the received support method, and the output is a screen showing the suggestions.

[1383] Step 18:

[1384] Teachers, who are users, check the assessment results on the app and use the assessment to grade each child and plan the next lesson. The input is the assessment results and support methods, and the output is a grade report and lesson plan.

[1385] Through the above steps, the system can provide effective and efficient exercise instruction tailored to each child's individual motor abilities and emotional state.

[1386] (Application example 2)

[1387] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1388] Currently, specific instruction is required to enhance children's motor skills, and appropriate feedback and training methods are needed. However, individual instruction tailored to each child's motor skills and emotional state is time-consuming and difficult to provide efficiently. Providing appropriate instruction and feedback to improve the operational efficiency and safety of robotic workers in factories is also difficult. In particular, systems that analyze the emotions and state of robots while they are operating and adjust their operations based on this information are still in the development stage, and there are many challenges before they can be applied in real-world situations.

[1389] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: video acquisition means equipped with a camera for analyzing the motor skills of children and the movements of robot workers; means for transmitting the acquired video to the server; means for analyzing and evaluating the video data in the server; means for transmitting generated training content to the terminal; means for presenting the transmitted training content; means for generating support methods for teachers or work instructors based on the motor skill evaluation results and emotional data in the server; means for transmitting the generated support methods to the terminal; means for displaying the transmitted support methods; means for analyzing emotional states using an emotion engine; means for providing feedback based on the evaluation results and emotional states; and means for improving work efficiency using a motor video analysis system. This enables detailed instruction and feedback to improve the movement efficiency of individual children and robot workers.

[1390] The "camera" is an image capturing device that captures the movements and work actions of the child and robot worker in real time.

[1391] The "server" is a central processing unit that analyzes the collected video data and emotional data and generates evaluation results regarding athletic ability and work efficiency.

[1392] "Image capture means" refers to a method or device for capturing images of a subject's movements or actions using a camera and acquiring the image data.

[1393] The "video transmission means" is a method or device for transmitting captured video data to a server via a network.

[1394] "Video analysis means" refers to a method or device by which the server analyzes video data and evaluates athletic ability and work efficiency.

[1395] The "training content generation means" is a method or device for generating motion programs for exercise training and work improvement for children and robot workers based on the evaluation results.

[1396] The "content transmission means" is a method or device for transmitting the generated training content and feedback to the terminal.

[1397] The "content presentation means" is a method or device for displaying the transmitted training content to children and robot workers.

[1398] The "emotion engine" is a system for analyzing a subject's emotional state from their facial expressions, movements, voice data, etc.

[1399] An "emotion analysis means" is a method or device for analyzing collected emotion data and determining a subject's current emotional state.

[1400] The "feedback providing means" is a method or device for providing appropriate feedback to a subject based on the analysis results and emotional state.

[1401] The "motion improvement means" is a method or device for the server to improve work efficiency based on the evaluation results using the motion video analysis system.

[1402] 1. System Configuration

[1403] This invention is a system that analyzes children's motor skills and the work movements of factory robots, and provides appropriate guidance and feedback. The main components of the system are a camera, a server, a terminal, a user (teacher or work instructor), and an emotion engine.

[1404] 2. Program processing explanation

[1405] Video acquisition and transmission

[1406] Camera: Records the movements and working actions of children and robot workers in real time. The camera has high resolution and clearly captures their movements.

[1407] Device: Stores the captured video locally, converts it into an appropriate format, compresses the video data for transmission, and sends it to the server using a secure communication protocol (HTTPS).

[1408] Video analysis

[1409] Server: Uses a pre-trained AI model to analyze the received video data. Imports the video data and starts image processing. Uses TensorFlow or PyTorch as the AI ​​model framework.

[1410] Server: The AI ​​model tracks the movements of the child or robot worker in the video and calculates evaluation items (speed, accuracy, safety, etc.) related to motor skills and work efficiency. This includes the speed and accuracy of exercise and work movements, as well as fatigue level. The calculated evaluation results are stored in a database.

[1411] Emotion Recognition and Analysis

[1412] Device: During recording, emotional data is collected in real time from the facial expressions and voices of the child and robotic worker, including facial expression analysis and voice tone analysis.

[1413] Emotion engine: Analyzes collected emotion data and determines the subject's current emotional state (e.g., excited, confused, tired, etc.). The emotion engine can use Microsoft Azure Cognitive Services.

[1414] Terminal: The determined emotion data is sent to the server, and a secure communication protocol is used to ensure the security of the data.

[1415] Generating and Providing Feedback

[1416] Server: Based on the analysis results and emotional data, it generates feedback tailored to each subject's specific weaknesses and areas for improvement. For example, if a robot is overworked, it generates feedback recommending appropriate rest or adjustments to its behavior.

[1417] Server: Sends the generated feedback to the device and displays it on the device. The server also generates and sends support methods to teachers and instructors.

[1418] Terminal: The received feedback and support methods are displayed on the user interface and presented to the target.

[1419] 3. Specific Examples

[1420] For example, if a robotic arm in a factory is moving slower than usual, its built-in camera will capture its movements and send them to a server. The server will then use an AI model and emotion engine to recognize that the robot is overworked. It will then generate feedback recommending appropriate breaks or minor adjustments to its movements and send it to the robot. For children, a teacher will record a video of the robot skipping rope with a camera, while the emotion engine will analyze the child's facial expressions and tone of voice to determine whether the child is confused. The server will then analyze the video, determine that there is a problem with the rhythm, and receive the emotional data indicating confusion. Based on this, it will generate specific animations to improve the rhythm and gentle, encouraging audio guidance to resolve the confusion, which will then be sent to the device. The child will then learn the correct skipping rhythm and become less confused.

[1421] Example of a generative AI model prompt:

[1422] To ensure that factory robots are working properly, how can you analyze video footage captured by built-in cameras, recognize the robot's performance evaluation and emotional state, and provide optimal feedback?

[1423] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1424] Step 1:

[1425] Footage collection

[1426] User: A teacher or instructor uses a camera to record the child's exercise and the robot's task movements.

[1427] Input: Child's movement or robot's task movement.

[1428] Processing: The camera captures the action and acquires the video data.

[1429] Output: Captured high-resolution video data.

[1430] Step 2:

[1431] Video storage and conversion

[1432] Device: Stores the captured video in local storage and converts it to the appropriate format.

[1433] Input: Acquired video data.

[1434] Processing: Video data is compressed and converted into a format suitable for transmission.

[1435] Output: Compressed video data.

[1436] Step 3:

[1437] Video data transmission

[1438] Terminal: Compressed video data is sent to the server using a secure protocol (HTTPS).

[1439] Input: Compressed video data.

[1440] Processing: Data transmission via secure communication protocols.

[1441] Output: Video data sent to the server.

[1442] Step 4:

[1443] Preparing video data for analysis

[1444] Server: Prepares to analyze the received video data.

[1445] Input: Video data sent to the server.

[1446] Processing: Preparing the video data in the appropriate format to input into the AI ​​model.

[1447] Output: Video data that can be input into an AI model.

[1448] Step 5:

[1449] Video analysis using AI models

[1450] Server: Uses AI models to analyze movements in the video (e.g., jump height, rhythm, and work efficiency).

[1451] Input: Video data.

[1452] Processing: Apply video analysis algorithms to evaluate subject's movements.

[1453] Output: Evaluation results of motor skills and work efficiency.

[1454] Step 6:

[1455] Emotion data collection and analysis

[1456] Device: Collects emotional data in real time based on the facial expressions and voices of children and robots.

[1457] Input: facial and voice data.

[1458] Processing: Facial expression analysis and voice tone analysis are performed to obtain emotion data.

[1459] Output: The obtained emotion data.

[1460] Step 7:

[1461] Emotion analysis using an emotion engine

[1462] Emotion engine: Analyzes the acquired emotion data and determines the current emotional state.

[1463] Input: Collected emotion data.

[1464] Processing: Determining emotional state.

[1465] Output: Determined emotional state data.

[1466] Step 8:

[1467] Sending emotional data

[1468] Device: Sends the determined emotion data to the server.

[1469] Input: Determined emotional state data.

[1470] Processing: Data transmission via secure communication protocol (HTTPS).

[1471] Output: Emotional state data sent to the server.

[1472] Step 9:

[1473] Generate feedback

[1474] Server: Based on the analysis results and emotional data, it generates feedback tailored to each subject's specific weaknesses and areas for improvement.

[1475] Input: Evaluation results and emotional state data.

[1476] Processing: Feedback generation and customization.

[1477] Output: The generated feedback data.

[1478] Step 10:

[1479] Providing feedback

[1480] Terminal: Display the received feedback on the user interface.

[1481] Input: Feedback data.

[1482] Action: View feedback.

[1483] Output: Feedback presented to the child and robot worker.

[1484] Through the above steps, it is possible to realize a system that properly analyzes the motion data and emotional state of children and robot workers, and provides detailed guidance and feedback.

[1485] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1486] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1487] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1488] [Fourth embodiment]

[1489] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1490] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1491] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1492] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1493] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1494] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1495] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1496] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1497] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1498] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1499] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1500] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1501] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1502] 1. System Overview

[1503] The system of the present invention analyzes the motor skills of children and provides appropriate guidance. The main components of the system are a terminal, a server, and a user (teacher or student). An embodiment of the system will be described in detail below.

[1504] 2. Program processing explanation

[1505] Video acquisition and transmission

[1506] User: The user launches a dedicated app installed on a device (e.g., a tablet or smartphone). A teacher uses a camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[1507] Terminal: The acquired video is stored locally and prepared for the next process. The video data is sent to the server using a secure communication protocol (e.g., HTTPS).

[1508] Video analysis

[1509] Server: Prepares the received video data for input into an AI model (e.g., computer vision algorithm) for analysis. The AI ​​model analyzes the video and evaluates the child's athletic ability (jump height, rhythm, speed, etc.). The evaluation results are stored in a database.

[1510] Teaching the tricks

[1511] Server: Based on the analysis results, the server generates videos and animations that address each child's specific weaknesses and areas for improvement. The generated content includes audio guides and text to make it easy for children to understand.

[1512] Server: Sends the generated video or animation to the device.

[1513] On the device: The received videos and animations are played in the app and presented to the child. The video playback status is recorded and reflected in the next lesson plan.

[1514] Support suggestions for teachers

[1515] Server: Generates support methods for teachers based on the results of each child's motor ability assessment. For example, it creates a teaching policy such as, "This child is not good at rhythm. Rhythm training would be effective."

[1516] Server: Sends the generated support method to the device.

[1517] On the device: The teacher checks the suggestions received through the app, displays the suggestions in an easy-to-understand format for the teacher, and provides detailed explanations.

[1518] Leveling up and assessing your child

[1519] Server: Generates training methods for advanced techniques (e.g. double jumps) for children who are good at the skill, and provides video content for children who are not good at the skill to relearn the basics.

[1520] Device: For children who need to refresh their knowledge, provide video content that explains the basic steps.

[1521] User: Teachers can view the assessment results in the app and use the assessment of each child's motor skills to assign grades and plan the next lesson.

[1522] 3. Specific Examples

[1523] For example, if a child is having trouble finding the right rhythm when jumping rope, you can use this system to teach them the following:

[1524] Device: The teacher records the jump rope video with a camera and sends it to the server.

[1525] Server: The AI ​​model analyzes the footage and determines that there is a problem with the rhythm.

[1526] Server: Generates specific animations and audio guides to improve rhythm and sends them to the device.

[1527] Device: Children watch the animation and learn the correct rhythm for skipping rope.

[1528] Teachers can refer to the support methods provided by the devices and use rhythm training aids and music to effectively train children.

[1529] In this way, by using the system of the present invention, it is possible to provide instruction that is tailored to each child's individual athletic ability, thereby realizing effective and efficient exercise instruction.

[1530] The processing flow will be explained below.

[1531] Step 1:

[1532] User: A teacher or child launches a dedicated app installed on a device (e.g., a tablet or smartphone). The teacher uses the camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[1533] Step 2:

[1534] Terminal: Stores the recorded video locally, converts the video data into an appropriate format, and compresses the video data for transmission.

[1535] Step 3:

[1536] Terminal: The compressed video data is sent to the server, using a secure communication protocol (e.g., HTTPS) to ensure data security.

[1537] Step 4:

[1538] Server: Prepares the received video data for input into a pre-trained AI model for analysis. Imports the video data and begins image processing.

[1539] Step 5:

[1540] Server: Using an AI model, the server tracks the child's movements in the video and calculates evaluation items related to athletic ability (such as jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[1541] Step 6:

[1542] Server: Identifies each child's specific weaknesses and areas for improvement, and generates videos and animations based on those weaknesses and areas for improvement. The generated content is then processed to add audio guides and text that are easy for children to understand.

[1543] Step 7:

[1544] Server: Sends the generated video or animation to the device, performing checks to ensure the integrity and security of the data in transit.

[1545] Step 8:

[1546] Device: The received video or animation is stored locally and played within the app, with display adjustments and layout settings made, and presented to the child.

[1547] Step 9:

[1548] Server: Generates support methods (e.g., specific rhythm training methods) for teachers based on the results of each child's motor ability assessment. The generated support methods are documented and stored in a database.

[1549] Step 10:

[1550] Server: Sends the generated support method to the device. Checks the integrity of the data when sending and transfers it in the correct format.

[1551] Step 11:

[1552] Device: The teacher receives suggestions via the app and displays them on the user interface. The suggestions are formatted so that they are easy to understand.

[1553] Step 12:

[1554] Server: Generates training approaches to help skilled children master advanced techniques (e.g., double jumps), and prepares to provide video content to help weaker children relearn the basics.

[1555] Step 13:

[1556] Device: For children who need to relearn the basics, the generated video content for basic learning can be played and they can learn step by step.

[1557] Step 14:

[1558] User: Teachers can view the assessment results in the app and use the assessment of each child's motor skills to assign grades and plan the next lesson.

[1559] In this way, the system performs a series of processes to effectively evaluate and improve children's motor skills.

[1560] Example 1

[1561] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1562] Currently, it is difficult to properly evaluate children's athletic ability and provide effective instruction based on that evaluation. Conventional athletic ability assessments struggle to grasp the specific characteristics and weaknesses of individual children in detail, making it impossible to provide effective training methods. Furthermore, there is a lack of means to generate specific instructional content based on the evaluation results, making it difficult for instructors to provide appropriate instruction tailored to each child's abilities. Furthermore, ensuring security when transmitting video data is also a major issue.

[1563] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1564] In this invention, the server includes a camera for analyzing a child's athletic ability, a means for capturing video of the child's athletic activity, a means for transmitting the captured video to a data center, a means for analyzing the video data at the data center and evaluating the child's athletic ability, a means for generating videos and moving images of exercise tips for the child based on the evaluation results, a means for transmitting the generated videos and moving images to a display device, a means for presenting the transmitted videos and moving images to the child, a means for generating support methods for teachers based on the athletic ability evaluation results at the data center, a means for transmitting the generated support methods to a display device, a means for displaying the transmitted support methods to the teacher, a means for the teacher to refer to the evaluation results, a means for utilizing a generative AI model based on the child's athletic ability to provide specific training methods and learning plans, and a means for inputting prompts related to the target exercise into the generative AI model to generate appropriate instruction content. This enables a detailed evaluation of each child's athletic ability and the provision of effective instruction based on the evaluation results in a timely and appropriate manner. Security can also be ensured during the transmission of video data.

[1565] "Photographing device" refers to a device for recording footage of a child's exercise, and includes cameras, video cameras, and the like.

[1566] A "data center" is a centralized management facility for receiving, storing, and analyzing acquired video data.

[1567] "Video data" refers to video files of a child's exercise captured by a camera.

[1568] "Analysis" is the process of evaluating a child's motor skills based on the acquired video data, using computer vision algorithms, etc.

[1569] "Evaluation results" refer to the numerical values ​​and indicators of a child's physical abilities obtained through analysis.

[1570] "Exercise Tips Videos" are videos that provide specific advice and training methods to help children improve their athletic abilities.

[1571] "Moving Images" refers to content that includes animations and visual guides presented to children.

[1572] A "display device" is a device used to present received videos and moving images to children and teachers, and includes tablets, smartphones, and the like.

[1573] "Support methods" refer to teaching strategies and advice provided to teachers to help them provide effective instruction to children.

[1574] A "generative AI model" is an artificial intelligence model that generates specific training methods and learning plans based on a child's athletic ability.

[1575] A "prompt" is a document that describes the instructions or requirements that are input to a generative AI model.

[1576] The present invention is a system for analyzing children's athletic abilities and providing optimal instruction for each child. This system comprises a camera, a data center, a display device, and users (teachers and children).

[1577] The user is a teacher who uses a camera (e.g., a camera built into a tablet or smartphone) to record the target child's exercise (e.g., skipping rope, horizontal bar, vaulting horse). The recorded video data is stored locally on the device (tablet or smartphone) and then sent to a data center using a secure communication protocol (e.g., HTTPS).

[1578] The data center then inputs the received video data into an AI model using computer vision algorithms. The AI ​​model analyzes the video and evaluates the child's athletic ability (e.g., jump height, rhythm, speed, etc.). The evaluation results are stored in a database.

[1579] Based on the assessment results, the data center generates instructional content tailored to each child's specific weaknesses and areas for improvement. Using a generative AI model, it generates videos and animations of exercise tips tailored to each child. This content includes audio guides and text to make it easy for children to understand.

[1580] The generated content is sent to a display device (such as a tablet or smartphone). The content is played on the display device and presented to the child. The playback status is recorded and reflected in future learning plans. Based on the evaluation results, the data center also generates support methods for teachers and sends them to the device. Teachers can check the support methods sent to them on the display device and appropriately convey the content to the children.

[1581] This system provides training methods for advanced athletic skills (e.g., double jumps) to children who are good at it, and video content to encourage children who are not good at it to relearn the basics.In addition, to ensure the security of the video data being transmitted, an encrypted data communication protocol is used.

[1582] For example, if a child has trouble finding the right rhythm when skipping rope, you can use this system to teach them the following steps:

[1583] 1. A teacher records the jump rope video with a camera and sends it to the data center.

[1584] 2. The data center analyzes the video and determines that there is a problem with the rhythm.

[1585] 3. Generate specific animations and audio guides to improve rhythm and send them to the display device.

[1586] 4. Children watch the animation and learn the correct rhythm of skipping rope.

[1587] An example prompt is:

[1588] "Analyze a video of a child skipping rope and assess whether there is a problem with his rhythm. If he is not getting the rhythm right, provide specific areas for improvement and suggest training methods based on those. Your suggestions should include animations and audio guides that are easy for children to understand."

[1589] By using this system in this way, highly efficient instruction tailored to each child's individual motor abilities can be achieved, improving educational effectiveness.

[1590] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1591] Step 1:

[1592] User: A teacher starts a dedicated app on their device to record a child's exercise. They set the exercise to be recorded (e.g., jumping rope) and press the record button to record the child's exercise.

[1593] Input: Device camera, teacher operation

[1594] Output: Recorded exercise video data

[1595] Specific behavior:

[1596] The teacher taps the "Start Recording" button on the app.

[1597] The device's camera will activate and capture video of your workout.

[1598] During video capture, the device temporarily stores the data in a buffer memory.

[1599] After recording is complete, the data is saved to the internal storage.

[1600] Step 2:

[1601] Terminal: Sends stored video data to the data center.

[1602] Input: Recorded exercise video data

[1603] Output: Video data sent to the data center

[1604] Specific behavior:

[1605] The device acquires the video data and uploads it to a data center using a secure communication protocol (HTTPS).

[1606] The data center receives and stores the uploaded data.

[1607] Step 3:

[1608] Server: Prepares for analyzing the received video data. Inputs the data into the AI ​​model and begins analysis.

[1609] Input: Video data stored in the data center

[1610] Output: Evaluation results of athletic ability

[1611] Specific behavior:

[1612] The server inputs the video data into an AI model for analysis.

[1613] The AI ​​model analyzes the video frame by frame and extracts movement parameters (jump height, rhythm, speed, etc.).

[1614] Athletic ability is evaluated based on the extracted data, and the results are stored in a database.

[1615] Step 4:

[1616] Server: Based on the analysis results, it generates videos and moving images of exercise tips for children.

[1617] Input: Athletic ability evaluation results

[1618] Output: Generated exercise tips videos and moving images

[1619] Specific behavior:

[1620] Based on the evaluation results, the server sends a prompt to the generative AI model.

[1621] The generative AI model generates videos and animations with appropriate tips and improvements.

[1622] Audio guides and text are added to the generated videos and moving images.

[1623] Step 5:

[1624] Server: Sends the generated video and moving images to the device.

[1625] Input: Generated exercise tips videos and moving images

[1626] Output: Videos and images sent to the device

[1627] Specific behavior:

[1628] The server retrieves the generated content and transmits it to the terminal.

[1629] The device receives the transmitted data and stores it in local storage.

[1630] Step 6:

[1631] Device: Plays the received videos and images and shows them to the child.

[1632] Input: Received video or moving images

[1633] Output: Played video and moving images

[1634] Specific behavior:

[1635] A teacher or child taps the "Start Playing" button in the app.

[1636] The device plays videos and moving images and shows them to the child.

[1637] Record playback status and reflect it in future learning plans.

[1638] Step 7:

[1639] Server: Generates support methods for teachers and sends them to the devices.

[1640] Input: Athletic ability evaluation results

[1641] Output: Generated support instructions for teachers

[1642] Specific behavior:

[1643] The server generates support policies for teachers based on the evaluation results.

[1644] The generated support policy is sent to the terminal.

[1645] The device displays the support method received and notifies the teacher.

[1646] Step 8:

[1647] User: Teachers refer to the assessment results and provide appropriate guidance to children.

[1648] Input: Evaluation results, generated support methods

[1649] Output: Teacher's teaching content

[1650] Specific behavior:

[1651] The teacher checks the app's evaluation results screen.

[1652] Based on the information obtained, specific guidance is given to the child.

[1653] If necessary, have your child review the training videos and animations provided by the app.

[1654] This makes it possible to evaluate each child's motor skills in detail and provide effective instruction based on that evaluation.

[1655] (Application example 1)

[1656] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1657] It is difficult to provide a system that evaluates the efficiency of workers' actions in a factory and provides appropriate training methods. It is also difficult to generate content for individualized instruction and training that takes into account the strengths and weaknesses of each worker. This creates the challenge of making it difficult to achieve the expected improvement in the production efficiency of the entire factory and the skills of workers.

[1658] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1659] In this invention, the server includes means for acquiring video of the movements of factory workers, means for transmitting the acquired video to a device on the network, means for analyzing the video data in the device on the network and evaluating the movement efficiency of the factory workers, means for generating instructional videos and animations for improving the movements of the factory workers based on the evaluation results, means for transmitting the generated videos and animations to a terminal, means for presenting the transmitted videos and animations to the factory workers, means for generating support methods for a manager based on the evaluation results of the movement efficiency in the device on the network, means for transmitting the generated support methods to the terminal, means for displaying the transmitted support methods to the manager, and means for the manager to refer to the evaluation results. This makes it possible to efficiently analyze the movements of workers in a factory and provide specific instruction methods tailored to each worker.

[1660] "Factory workers" refer to workers engaged in production activities within a factory.

[1661] "Motion video" refers to video data that records the movements of factory workers while they are working.

[1662] "Capture equipment" refers to cameras or other image capture devices used to capture footage of factory workers in action.

[1663] "Devices on a network" refers to computing resources such as servers connected via a network such as the Internet or an intranet.

[1664] "Video data" refers to digital data that records the actions of factory workers.

[1665] "Operational efficiency" is an index that evaluates how efficiently factory workers are performing their work.

[1666] "Instructional videos" refer to video content that visually teaches factory workers how to improve their performance.

[1667] "Animation" refers to visual media consisting of a sequence of image frames that visually represent behavioral improvement techniques.

[1668] "Terminal" refers to a device such as a smartphone or tablet used to play instructional videos and animations.

[1669] "Supervisor" refers to the person responsible for overseeing the activities of factory workers and providing guidance on how to support them.

[1670] "Support methods" refer to specific guidance and improvement measures provided to administrators based on evaluations of operational efficiency.

[1671] 1. System Overview

[1672] The system of the present invention analyzes the operational efficiency of factory workers and provides appropriate guidance methods. The main components of the system are terminals, a server, and users (managers or workers).

[1673] 2. Program processing explanation

[1674] Video acquisition and transmission

[1675] User: The user launches a dedicated app installed on a device (e.g., a tablet or smartphone). The administrator uses a camera to record the actions of factory workers. The device stores the captured video locally and prepares it for further processing. The video data is then sent to a device (server) on the network using a secure communication protocol (e.g., HTTPS).

[1676] Video analysis

[1677] Server: Prepares the received video data for input into an AI model (e.g., computer vision algorithm) for analysis. The AI ​​model analyzes the video and evaluates the efficiency of the factory workers' movements (such as accuracy, speed, and whether there are any unnecessary movements). The evaluation results are stored in a database.

[1678] Teaching the tricks

[1679] Server: Based on the analysis results, the server generates videos and animations that show each factory worker's specific weaknesses and areas for improvement. The generated content includes audio guides and text to make it easy for factory workers to understand. The generated videos and animations are sent to the device.

[1680] Device: The received videos and animations are played back within the app and presented to factory workers. The video playback status is recorded and reflected in the next training plan.

[1681] Support suggestions for administrators

[1682] Server: Generates support methods for managers based on the evaluation results of each factory worker's operational efficiency. For example, it creates a training policy such as, "This worker is slow. Speed ​​training would be effective." The generated support methods are sent to the terminal.

[1683] On the device: The administrator can review the suggestions received through the app, display the suggestions clearly to the administrator, and provide detailed explanations.

[1684] Worker level-up and evaluation

[1685] Server: Generates training methods for advanced skills for skilled factory workers, and provides video content for less skilled factory workers to refresh their knowledge from the basics.

[1686] Devices: For factory workers who need a refresher, video content is presented that explains the basic steps.

[1687] User: The manager checks the evaluation results in the app and assigns an evaluation and plans the next training based on the evaluation of each factory worker's operational efficiency.

[1688] 3. Specific Examples

[1689] For example, if a worker makes a lot of unnecessary movements when carrying equipment, the system can provide the following guidance:

[1690] Terminal: The administrator uses a camera to record footage of the workers' actions and sends it to the server.

[1691] Server: The AI ​​model analyzes the video and determines that there are many unnecessary movements. Based on the analysis results, it generates specific animations and audio guidance to improve the movement and sends them to the device.

[1692] Terminal: Workers watch the animation and learn efficient movements.

[1693] Managers can effectively train workers by referring to the support methods provided by the terminals and using aids and guidelines to promote efficient operations.

[1694] Prompt Sentence Examples

[1695] Please analyze the following video and provide specific guidance on how to improve work efficiency. This video shows Worker A carrying equipment.

[1696] In this way, by using the system of the present invention, it is possible to provide guidance that is tailored to the individual operational efficiency of factory workers, thereby realizing effective and efficient operational guidance.

[1697] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1698] Step 1:

[1699] User: The user launches a dedicated app installed on a device (tablet or smartphone). The administrator uses a camera to record the actions of the factory workers. The input is the video of the factory workers' actions.

[1700] Step 2:

[1701] Terminal: Stores the acquired video locally. Prepares for the next process and sends the video data to the server using a secure communication protocol (e.g., HTTPS). The input is the locally stored video data, and the output is the data to be sent to the server.

[1702] Step 3:

[1703] Server: Prepares the received video data for input into the AI ​​model in order to analyze it. Specifically, it converts the data format and performs preprocessing. The input is the video data sent to the server, and the output is data in a format suitable for the AI ​​model.

[1704] Step 4:

[1705] Server: The AI ​​model analyzes the video and evaluates the efficiency of factory workers' movements. Specifically, it calculates the accuracy, speed, and presence or absence of unnecessary movements. The input is preprocessed video data, and the output is the evaluation result of movement efficiency.

[1706] Step 5:

[1707] Server: The evaluation results are stored in a database. This database is later used to generate instructional videos. The input is the evaluation results of movement efficiency, and the output is the data stored in the database.

[1708] Step 6:

[1709] Server: Based on the analysis results, it generates videos and animations that show each factory worker's specific weaknesses and areas for improvement. The generated content includes audio guides and text to make it easy for factory workers to understand. The input is the evaluation results stored in the database, and the output is the generated instructional videos and animations.

[1710] Step 7:

[1711] Server: Sends the generated instructional videos and animations to the terminal. The input is the generated videos and animations, and the output is the data sent to the terminal.

[1712] Step 8:

[1713] Device: The received videos and animations are played within the app and presented to factory workers. The video playback status is recorded and reflected in the next training plan. The input is the instructional videos and animations sent from the server, and the output is the video playback and a record of the playback status.

[1714] Step 9:

[1715] Server: Generates support methods for managers based on the evaluation results of each factory worker's operational efficiency. Specifically, it creates guidance guidelines such as, "This worker is slow. Speed ​​training would be effective." The input is the evaluation results stored in the database, and the output is the generated support methods.

[1716] Step 10:

[1717] Server: Sends the generated support method to the terminal. The input is the generated support method, and the output is the data sent to the terminal.

[1718] Step 11:

[1719] Terminal: The administrator checks the suggestions received through the app. The suggestions are displayed in an easy-to-understand manner for the administrator, and detailed explanations are provided. The input is the support method sent from the server, and the output is the display of the suggestions.

[1720] Step 12:

[1721] Server: Generates training methods for advanced skills for skilled factory workers, and provides video content for weaker factory workers to refresh their knowledge from the basics. The input is the evaluation results stored in the database, and the output is the training methods and video content.

[1722] Step 13:

[1723] Terminal: For factory workers who need to refresh their knowledge from the basics, video content explaining basic steps is presented. The input is the training method and video content sent from the server, and the output is the video presentation.

[1724] Step 14:

[1725] User: The manager checks the evaluation results on the app and creates an evaluation and next training plan based on the evaluation of each factory worker's operational efficiency. The input is the evaluation results and training methods displayed on the device, and the output is the evaluation and training plan.

[1726] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1727] 1. System Overview

[1728] The system of the present invention analyzes children's athletic abilities and provides appropriate instruction, and furthermore, by combining it with an emotion engine, recognizes the user's emotions and makes the instruction more effective. The main components of the system are a terminal, a server, a user (teacher or student), and an emotion engine. An embodiment of this system is described in detail below.

[1729] 2. Program processing explanation

[1730] Video acquisition and transmission

[1731] User: A teacher or child launches a dedicated app installed on a device (e.g., a tablet or smartphone). The teacher uses the camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[1732] Terminal: The acquired video is stored locally, and the video data is converted into an appropriate format. It also compresses the video data to make it suitable for transmission.

[1733] Terminal: The compressed video data is sent to the server, using a secure communication protocol (e.g., HTTPS) to ensure data security.

[1734] Video analysis

[1735] Server: Prepares the received video data for input into a pre-trained AI model for analysis. Imports the video data and begins image processing.

[1736] Server: Using an AI model, the server tracks the child's movements in the video and calculates evaluation items related to athletic ability (such as jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[1737] Emotion Recognition and Analysis

[1738] Device: During recording, emotional data is collected in real time from the child's facial expressions and voice, including facial expression analysis and voice tone analysis.

[1739] Emotion Engine: Analyzes collected emotion data to determine the child's current emotional state (e.g., excitement, confusion, joy, etc.).

[1740] Device: The determined emotion data is sent to the server, again using a secure communication protocol.

[1741] Tips and emotional feedback

[1742] Server: Based on the analysis results and emotional data, it generates videos and animations with tips tailored to each child's specific weaknesses and areas for improvement. For example, if a child is confused, it will create a video with gentler training methods to resolve the issue.

[1743] Server: Sends the generated video or animation to the device.

[1744] Device: The received videos and animations are played in the app and presented to the child. The video playback status and effectiveness are recorded and reflected in the next lesson plan.

[1745] Support suggestions for teachers

[1746] Server: Based on the results of each child's motor skill assessment and emotional data, the server generates support methods for teachers (e.g., specific approaches for children who are struggling). For example, for a child who is struggling with rhythm, the server suggests a method using music for rhythm training.

[1747] Server: Sends the generated support method to the device.

[1748] Device: The teacher receives suggestions through the app and displays them on the user interface for review.

[1749] Leveling up and assessing your child

[1750] Server: Generates training approaches to help skilled children master advanced techniques (e.g., double jumps), and prepares to provide video content to help weaker children relearn the basics.

[1751] Device: For children who need to relearn the basics, the generated video content for basic learning can be played and they can learn step by step.

[1752] User: Teachers can view the assessment results in the app and use the assessment of each child's motor skills to assign grades and plan the next lesson.

[1753] 3. Specific Examples

[1754] For example, if a child is having trouble finding the right rhythm when jumping rope, you can use this system to teach them the following:

[1755] Device: The teacher records the jump rope with a camera, and at the same time, the emotion engine analyzes the child's facial expressions and tone of voice to determine whether they are confused.

[1756] Server: The AI ​​model analyzes the video and determines that there is a problem with the rhythm. It also receives emotional data indicating confusion.

[1757] Server: Generates specific animations to improve rhythm and gentle, encouraging audio guidance to resolve confusion, and sends them to the device.

[1758] Device: The animation and guide help children learn the correct skipping rhythm and reduce confusion.

[1759] Teachers can refer to the support methods provided by the devices and use rhythm training aids and music to effectively train children.

[1760] In this way, by using the system of the present invention, it is possible to provide instruction that is tailored to each child's individual motor abilities and emotional state, thereby achieving effective and efficient exercise instruction.

[1761] The processing flow will be explained below.

[1762] Step 1:

[1763] User: A teacher or child launches a dedicated app installed on a device (e.g., a tablet or smartphone). The teacher uses the camera to record the child's exercise (e.g., skipping rope, horizontal bar, vaulting box).

[1764] Step 2:

[1765] Terminal: The acquired video is stored locally, and the video data is converted into an appropriate format. It also compresses the video data to make it suitable for transmission.

[1766] Step 3:

[1767] Terminal: The compressed video data is sent to the server, using a secure communication protocol (e.g., HTTPS) to ensure data security.

[1768] Step 4:

[1769] Server: Prepares the received video data for input into a pre-trained AI model for analysis. Imports the video data and begins image processing.

[1770] Step 5:

[1771] Server: Using an AI model, the server tracks the child's movements in the video and calculates evaluation items related to athletic ability (such as jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[1772] Step 6:

[1773] Device: During recording, emotional data is collected in real time from the child's facial expressions and voice, including facial expression analysis and voice tone analysis.

[1774] Step 7:

[1775] Emotion Engine: Analyzes collected emotion data to determine the child's current emotional state (e.g., excitement, confusion, joy, etc.).

[1776] Step 8:

[1777] Device: The determined emotion data is sent to the server, again using a secure communication protocol.

[1778] Step 9:

[1779] Server: Based on the analysis results and emotional data, it generates videos and animations with tips tailored to each child's specific weaknesses and areas for improvement. For example, if a child is confused, it will create a video with gentler training methods to resolve the issue.

[1780] Step 10:

[1781] Server: Sends the generated video or animation to the device, performing checks to ensure the integrity and security of the data in transit.

[1782] Step 11:

[1783] Device: The received video or animation is stored locally and played within the app, with display adjustments and layout settings made, and presented to the child.

[1784] Step 12:

[1785] Server: Based on the results of each child's motor skill assessment and emotional data, the server generates support methods for teachers (e.g., specific approaches for children who are struggling). For example, for a child who is struggling with rhythm, the server suggests a method using music for rhythm training.

[1786] Step 13:

[1787] Server: Sends the generated support method to the device. Checks the integrity of the data when sending and transfers it in the correct format.

[1788] Step 14:

[1789] Device: The teacher receives suggestions via the app and displays them on the user interface. The suggestions are formatted so that they are easy to understand.

[1790] Step 15:

[1791] Server: Generates training approaches to help skilled children master advanced techniques (e.g., double jumps), and prepares to provide video content to help weaker children relearn the basics.

[1792] Step 16:

[1793] Device: For children who need to relearn the basics, the generated video content for basic learning can be played and they can learn step by step.

[1794] Step 17:

[1795] User: Teachers can view the assessment results in the app and grade students and plan the next lesson based on their assessment of each child's motor skills and emotional data.

[1796] In this way, the system performs a series of processes to effectively evaluate and improve children's motor skills, and by combining this with emotional data, it is possible to provide optimal teaching methods for each individual.

[1797] Example 2

[1798] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1799] Conventional motor skill assessment systems rely primarily on visual information to analyze motor skills, and do not take into account the child's emotional state when providing instruction, which has the drawback of not being effective enough. Furthermore, there is a lack of individualized instruction for strong and weak areas, making it difficult to provide follow-up that responds to a child's growth.

[1800] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1801] In this invention, the server includes a means for analyzing video information and evaluating a child's athletic ability, a means for analyzing the child's emotional state using an emotion recognition engine and transmitting the analysis results to a data processing device, and a means for generating exercise instruction videos and visual effects for children based on the evaluation results, thereby enabling effective instruction that takes into account not only the child's athletic ability but also their emotional state.

[1802] "Video capture devices" refer to devices used to capture footage of children's athletic skills, and examples include cameras and smartphones.

[1803] The term "communication device" refers to a device for transmitting acquired video data to a data processing device, and specific examples include a wireless communication module and an internet connection device.

[1804] The "data processing device" refers to a device that analyzes received video information and evaluates a child's motor skills, and specific examples include a server and a computer.

[1805] "Video information" refers to video data that captures a child's athletic skills, specifically information that is saved and transferred in video file format.

[1806] "Motor ability" refers to the ability to assess multiple indicators related to a child's motor skills (e.g., jump height, rhythm, speed, etc.).

[1807] "Evaluation results" refers to the results of the evaluation of athletic ability analyzed by the data processing device, and includes numerical values ​​and text data stored in the database.

[1808] "Exercise method video" refers to a video created for children based on the evaluation results to help them master exercise techniques.

[1809] "Visual effects" refers to animations and graphic elements added to exercise videos, techniques used to present information in a way that is easy for children to understand.

[1810] "Display device" refers to a device for presenting the generated exercise method video and visual effects to a child, and examples include a tablet or smartphone.

[1811] "Educator" refers to a person responsible for teaching children athletic skills, including school teachers and instructors.

[1812] An "emotion recognition engine" refers to algorithms and software that analyze a child's emotional state from their facial expressions, voice, etc.

[1813] "Emotional state" refers to the child's current emotional state (e.g., excited, confused, happy, etc.), and includes information sent to the data processing device as an analysis result.

[1814] "Support measures" refer to specific methods and approaches that educators use to improve children's motor skills, and are created based on the evaluation results.

[1815] "Encrypted communications protocol" refers to the communications method used to ensure the security of transmitted video information, and examples include HTTPS and SSL.

[1816] The system of the present invention analyzes children's athletic abilities and provides appropriate instruction, and furthermore, by combining it with an emotion engine, recognizes the user's emotions and makes the instruction more effective. The main components of the system are a terminal, a server, a user (teacher or student), and an emotion engine. An embodiment of this system is described in detail below.

[1817] The system is equipped with the following hardware and software. Tablets and smartphones are used as terminals, with a dedicated application installed. The server uses a high-performance computer as the data processing device, and a deep learning model is implemented as the emotion recognition engine. OpenPose and FFmpeg are used for video analysis, and Praat is used for audio analysis. HTTPS is used as the communication protocol to ensure data security.

[1818] Video acquisition and transmission

[1819] The user, either a teacher or a child, launches a dedicated app installed on the device and prepares to record. Using the device's built-in camera, the teacher records the child's exercise (for example, skipping rope, horizontal bar, or vaulting box). When recording is complete, the device saves the captured video locally and converts the video data into an appropriate format (for example, MP4). It also performs a compression process to make it suitable for transmission. The compressed video data is sent to the server using the HTTPS protocol.

[1820] Video analysis

[1821] The server prepares the received video data for input into the AI ​​model. FFmpeg is used to break down the video data into frames and analyze each frame. The OpenPose library is used to track the child's movements in the video and calculate evaluation items related to athletic ability (jump height, rhythm, speed, etc.). The calculated evaluation results are stored in a database.

[1822] Emotion Recognition and Analysis

[1823] While recording, the device collects emotional data in real time based on the child's facial expressions and voice. It uses OpenCV facial recognition technology and Praat voice tone analysis. The collected emotional data is analyzed by an emotion recognition engine to determine the child's current emotional state (excitement, confusion, joy, etc.). The determined emotional data is then sent back to the server using the HTTPS protocol.

[1824] Tips and emotional feedback

[1825] Based on the analysis results and emotional data, the server generates videos and animations that address each child's specific weaknesses and areas for improvement. Tools such as Adobe After Effects are used here, too. For example, a video containing gentler training methods can be generated for a child who is struggling. These videos and animations are then sent from the server to the device. The device then plays the received videos and animations within the app and presents them to the child. The device also has a function to monitor the child's reactions in real time during playback, and this is reflected in the next lesson plan.

[1826] Support suggestions for teachers

[1827] The server generates support methods for teachers based on each child's motor ability assessment results and emotional data. For children who are struggling with rhythm, it suggests methods such as using rhythmic training music. The generated support methods are sent to the device, which displays the suggestions received through the app on a user interface so that teachers can check them.

[1828] Leveling up and assessing your child

[1829] The server generates training approaches for children who excel at a skill to master advanced techniques (e.g., double jumps), and prepares video content for children who are not so good at a skill to relearn from the basics. For example, a step-by-step training program can be designed using Adobe After Effects. The device plays the generated video content for basic learning for children who need to relearn from the basics, providing a function for them to learn step by step.

[1830] Examples of prompt statements

[1831] "I would like to analyze the video and emotional data of a child skipping rope, and suggest an appropriate training method for a child who is having trouble. Please generate a video and audio guide for this purpose."

[1832] In this way, by using the system of the present invention, it is possible to provide instruction that is tailored to each child's individual motor abilities and emotional state, thereby achieving effective and efficient exercise instruction.

[1833] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1834] Step 1:

[1835] The user launches a dedicated app installed on their device (tablet or smartphone) and prepares for recording. This includes the login procedure and camera startup. The input is the user ID and password, and the output is the home screen of the app.

[1836] Step 2:

[1837] The user uses the device's camera to record their child's exercise (e.g., skipping rope, horizontal bar, vaulting box). During recording, the camera angle and distance are adjusted to capture the entire exercise. The input is the camera image, and the output is the recorded data.

[1838] Step 3:

[1839] The device stores the captured video locally and the video data is converted to an appropriate format (e.g., MP4). The input is the recorded data, and the output is the converted MP4 format video data.

[1840] Step 4:

[1841] The terminal then compresses the converted video data using the H.264 codec to make it suitable for transmission. The input is video data in MP4 format, and the output is compressed video data.

[1842] Step 5:

[1843] The terminal sends the compressed video data to the server using the HTTPS protocol, with the compressed video data as input and a confirmation of successful data transfer to the server as output.

[1844] Step 6:

[1845] The server prepares the received video data for input to the AI ​​model. To do this, it uses FFmpeg to break down the video data into frames. The input is compressed video data, and the output is image data for each frame.

[1846] Step 7:

[1847] The server uses the OpenPose library to track the child's movements frame by frame and calculates evaluation items related to athletic ability (jump height, rhythm, speed, etc.) Image data for each frame is input, and numerical data based on the evaluation items is output.

[1848] Step 8:

[1849] The server saves the calculated evaluation results in a database such as MySQL or PostgreSQL. The input is numerical data based on the evaluation items, and the output is recorded in the database.

[1850] Step 9:

[1851] The device collects emotional data in real time based on the child's facial expressions and voice during recording. This includes facial recognition technology (e.g., OpenCV) and tone analysis (e.g., Praat). The input is captured facial images and voice data, and the output is analyzed emotional data.

[1852] Step 10:

[1853] The emotion recognition engine analyzes the collected emotional data using a deep learning model to determine the child's current emotional state (excitement, confusion, joy, etc.) The input is the analyzed emotional data, and the output is a judgment of the child's emotional state.

[1854] Step 11:

[1855] The device transmits the determined emotion data to the server using the HTTPS protocol. The input is the result of the determination of the emotional state, and the output is confirmation of successful data transfer to the server.

[1856] Step 12:

[1857] Based on the analysis results and emotional data, the server generates videos and animations that provide tips for each child based on their specific weaknesses and areas for improvement. Tools such as Adobe After Effects are used for this. The inputs are the evaluation results and emotional data, and the output is the generated videos and animations.

[1858] Step 13:

[1859] The server sends the generated video or animation to the display device, with the generated video as input and confirmation of successful data transfer to the display device as output.

[1860] Step 14:

[1861] The device then plays the received video or animation within the app and presents it to the child. The input is the received video or animation, and the output is the video playback screen. The app also has a function to monitor the child's reaction in real time during playback.

[1862] Step 15:

[1863] The server generates support methods for educators based on each child's motor ability assessment results and emotional data. For example, for a child who is struggling with rhythm, it suggests a method using rhythm training music. The inputs are the assessment results and emotional data, and the output is the generated support method.

[1864] Step 16:

[1865] The server sends the generated support method to the display device, with the generated support method as input and confirmation of successful data transfer to the display device as output.

[1866] Step 17:

[1867] The device displays the suggestions received through the app on a user interface so that the teacher can check them. The input is the received support method, and the output is a screen showing the suggestions.

[1868] Step 18:

[1869] Teachers, who are users, check the assessment results on the app and use the assessment to grade each child and plan the next lesson. The input is the assessment results and support methods, and the output is a grade report and lesson plan.

[1870] Through the above steps, the system can provide effective and efficient exercise instruction tailored to each child's individual motor abilities and emotional state.

[1871] (Application example 2)

[1872] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1873] Currently, specific instruction is required to enhance children's motor skills, and appropriate feedback and training methods are needed. However, individual instruction tailored to each child's motor skills and emotional state is time-consuming and difficult to provide efficiently. Providing appropriate instruction and feedback to improve the operational efficiency and safety of robotic workers in factories is also difficult. In particular, systems that analyze the emotions and state of robots while they are operating and adjust their operations based on this information are still in the development stage, and there are many challenges before they can be applied in real-world situations.

[1874] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: video acquisition means equipped with a camera for analyzing the motor skills of children and the movements of robot workers; means for transmitting the acquired video to the server; means for analyzing and evaluating the video data in the server; means for transmitting generated training content to the terminal; means for presenting the transmitted training content; means for generating support methods for teachers or work instructors based on the motor skill evaluation results and emotional data in the server; means for transmitting the generated support methods to the terminal; means for displaying the transmitted support methods; means for analyzing emotional states using an emotion engine; means for providing feedback based on the evaluation results and emotional states; and means for improving work efficiency using a motor video analysis system. This enables detailed instruction and feedback to improve the movement efficiency of individual children and robot workers.

[1875] The "camera" is an image capturing device that captures the movements and work actions of the child and robot worker in real time.

[1876] The "server" is a central processing unit that analyzes the collected video data and emotional data and generates evaluation results regarding athletic ability and work efficiency.

[1877] "Image capture means" refers to a method or device for capturing images of a subject's movements or actions using a camera and acquiring the image data.

[1878] The "video transmission means" is a method or device for transmitting captured video data to a server via a network.

[1879] "Video analysis means" refers to a method or device by which the server analyzes video data and evaluates athletic ability and work efficiency.

[1880] The "training content generation means" is a method or device for generating motion programs for exercise training and work improvement for children and robot workers based on the evaluation results.

[1881] The "content transmission means" is a method or device for transmitting the generated training content and feedback to the terminal.

[1882] The "content presentation means" is a method or device for displaying the transmitted training content to children and robot workers.

[1883] The "emotion engine" is a system for analyzing a subject's emotional state from their facial expressions, movements, voice data, etc.

[1884] An "emotion analysis means" is a method or device for analyzing collected emotion data and determining a subject's current emotional state.

[1885] The "feedback providing means" is a method or device for providing appropriate feedback to a subject based on the analysis results and emotional state.

[1886] The "motion improvement means" is a method or device for the server to improve work efficiency based on the evaluation results using the motion video analysis system.

[1887] 1. System Configuration

[1888] This invention is a system that analyzes children's motor skills and the work movements of factory robots, and provides appropriate guidance and feedback. The main components of the system are a camera, a server, a terminal, a user (teacher or work instructor), and an emotion engine.

[1889] 2. Program processing explanation

[1890] Video acquisition and transmission

[1891] Camera: Records the movements and working actions of children and robot workers in real time. The camera has high resolution and clearly captures their movements.

[1892] Device: Stores the captured video locally, converts it into an appropriate format, compresses the video data for transmission, and sends it to the server using a secure communication protocol (HTTPS).

[1893] Video analysis

[1894] Server: Uses a pre-trained AI model to analyze the received video data. Imports the video data and starts image processing. Uses TensorFlow or PyTorch as the AI ​​model framework.

[1895] Server: The AI ​​model tracks the movements of the child or robot worker in the video and calculates evaluation items (speed, accuracy, safety, etc.) related to motor skills and work efficiency. This includes the speed and accuracy of exercise and work movements, as well as fatigue level. The calculated evaluation results are stored in a database.

[1896] Emotion Recognition and Analysis

[1897] Device: During recording, emotional data is collected in real time from the facial expressions and voices of the child and robotic worker, including facial expression analysis and voice tone analysis.

[1898] Emotion engine: Analyzes collected emotion data and determines the subject's current emotional state (e.g., excited, confused, tired, etc.). The emotion engine can use Microsoft Azure Cognitive Services.

[1899] Terminal: The determined emotion data is sent to the server, and a secure communication protocol is used to ensure the security of the data.

[1900] Generating and Providing Feedback

[1901] Server: Based on the analysis results and emotional data, it generates feedback tailored to each subject's specific weaknesses and areas for improvement. For example, if a robot is overworked, it generates feedback recommending appropriate rest or adjustments to its behavior.

[1902] Server: Sends the generated feedback to the device and displays it on the device. The server also generates and sends support methods to teachers and instructors.

[1903] Terminal: The received feedback and support methods are displayed on the user interface and presented to the target.

[1904] 3. Specific Examples

[1905] For example, if a robotic arm in a factory is moving slower than usual, its built-in camera will capture its movements and send them to a server. The server will then use an AI model and emotion engine to recognize that the robot is overworked. It will then generate feedback recommending appropriate breaks or minor adjustments to its movements and send it to the robot. For children, a teacher will record a video of the robot skipping rope with a camera, while the emotion engine will analyze the child's facial expressions and tone of voice to determine whether the child is confused. The server will then analyze the video, determine that there is a problem with the rhythm, and receive the emotional data indicating confusion. Based on this, it will generate specific animations to improve the rhythm and gentle, encouraging audio guidance to resolve the confusion, which will then be sent to the device. The child will then learn the correct skipping rhythm and become less confused.

[1906] Example of a generative AI model prompt:

[1907] To ensure that factory robots are working properly, how can you analyze video footage captured by built-in cameras, recognize the robot's performance evaluation and emotional state, and provide optimal feedback?

[1908] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1909] Step 1:

[1910] Footage collection

[1911] User: A teacher or instructor uses a camera to record the child's exercise and the robot's task movements.

[1912] Input: Child's movement or robot's task movement.

[1913] Processing: The camera captures the action and acquires the video data.

[1914] Output: Captured high-resolution video data.

[1915] Step 2:

[1916] Video storage and conversion

[1917] Device: Stores the captured video in local storage and converts it to the appropriate format.

[1918] Input: Acquired video data.

[1919] Processing: Video data is compressed and converted into a format suitable for transmission.

[1920] Output: Compressed video data.

[1921] Step 3:

[1922] Video data transmission

[1923] Terminal: Compressed video data is sent to the server using a secure protocol (HTTPS).

[1924] Input: Compressed video data.

[1925] Processing: Data transmission via secure communication protocols.

[1926] Output: Video data sent to the server.

[1927] Step 4:

[1928] Preparing video data for analysis

[1929] Server: Prepares to analyze the received video data.

[1930] Input: Video data sent to the server.

[1931] Processing: Preparing the video data in the appropriate format to input into the AI ​​model.

[1932] Output: Video data that can be input into an AI model.

[1933] Step 5:

[1934] Video analysis using AI models

[1935] Server: Uses AI models to analyze movements in the video (e.g., jump height, rhythm, and work efficiency).

[1936] Input: Video data.

[1937] Processing: Apply video analysis algorithms to evaluate subject's movements.

[1938] Output: Evaluation results of motor skills and work efficiency.

[1939] Step 6:

[1940] Emotion data collection and analysis

[1941] Device: Collects emotional data in real time based on the facial expressions and voices of children and robots.

[1942] Input: facial and voice data.

[1943] Processing: Facial expression analysis and voice tone analysis are performed to obtain emotion data.

[1944] Output: The obtained emotion data.

[1945] Step 7:

[1946] Emotion analysis using an emotion engine

[1947] Emotion engine: Analyzes the acquired emotion data and determines the current emotional state.

[1948] Input: Collected emotion data.

[1949] Processing: Determining emotional state.

[1950] Output: Determined emotional state data.

[1951] Step 8:

[1952] Sending emotional data

[1953] Device: Sends the determined emotion data to the server.

[1954] Input: Determined emotional state data.

[1955] Processing: Data transmission via secure communication protocol (HTTPS).

[1956] Output: Emotional state data sent to the server.

[1957] Step 9:

[1958] Generate feedback

[1959] Server: Based on the analysis results and emotional data, it generates feedback tailored to each subject's specific weaknesses and areas for improvement.

[1960] Input: Evaluation results and emotional state data.

[1961] Processing: Feedback generation and customization.

[1962] Output: The generated feedback data.

[1963] Step 10:

[1964] Providing feedback

[1965] Terminal: Display the received feedback on the user interface.

[1966] Input: Feedback data.

[1967] Action: View feedback.

[1968] Output: Feedback presented to the child and robot worker.

[1969] Through the above steps, it is possible to realize a system that properly analyzes the motion data and emotional state of children and robot workers, and provides detailed guidance and feedback.

[1970] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1971] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1972] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1973] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may de...

Claims

1. a means for acquiring a video of a child's exercise, the video being provided with a camera for analyzing the child's athletic ability; means for transmitting the acquired video to a server; a means for analyzing the video data in the server and evaluating the motor skills of the child; A means for generating exercise tips videos and animations for children based on the evaluation results; A means for transmitting the generated video or animation to a terminal; a means for presenting the transmitted video or animation to the child; A means for generating a support method for teachers based on the results of the motor ability evaluation in the server; means for transmitting the generated support method to the terminal; A means for the teacher to view the support methods sent; A means for teachers to refer to the evaluation results, and A system including:

2. The system according to claim 1, further comprising means for generating training approaches for skilled children to acquire advanced skills and providing video content for weaker children to relearn from the basics.

3. 10. The system of claim 1, further comprising means for using an encrypted communications protocol for data transmission to ensure security of the transmitted video data.

Images