system

Abstract

Provide a system. [Solution] A means for receiving videos of practice or games taken by a user; A means of analyzing the received video to track player movements and ball trajectory, A means to identify technical issues and areas for improvement for players based on the analysis results, and A means of generating specific advice based on identified issues and areas for improvement; The system includes means for transmitting the generated advice to a user terminal.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, for children to improve their soccer skills, they need not only regular practice but also appropriate feedback and coaching. However, it is difficult for busy parents and coaches to always provide appropriate advice. Furthermore, it is difficult for children to understand their own technical problems and areas for improvement, which means it takes time for their performance to improve. This presents a challenge that hinders children's efficient skill development. [Means for solving the problem]

[0005] We propose a system that receives videos of practice and matches filmed by users, analyzes the received videos, and tracks player movements and the trajectory of the ball. This system identifies technical issues and areas for improvement for the player based on the analysis results, and generates specific advice for those issues and areas for improvement. The generated advice is sent to the user's device, where the user can review it and use it in the next practice or match. In particular, the system has a means for dividing the video at a fixed frame rate, analyzing each frame, storing the analysis results in a database, and furthermore, a means for converting the advice into written text using natural language generation technology. It also includes computer vision technology for analyzing player movements and the trajectory of the ball. In this way, users can efficiently and effectively obtain feedback to improve their soccer skills.

[0006] "User" refers to the entity that uploads the video they have taken to the system.

[0007] "Terminal" refers to the device that a user uses to record video and receive advice from the system.

[0008] "Server" refers to the central processing system that receives videos uploaded by users and provides analysis and feedback.

[0009] "Video" refers to video data of practice or games filmed by the user.

[0010] "Analysis" refers to the process of tracking player movements and ball trajectories from video to identify technical issues and areas for improvement.

[0011] A "frame" refers to an individual still image that makes up a video.

[0012] "Tracking" refers to tracking the position and movement of players and the ball in a video.

[0013] "Issues" refer to areas in a player's technique or movement that need improvement.

[0014] "Advice" refers to specific improvement suggestions and guidance generated based on the analysis results.

[0015] "Computer vision technology" refers to algorithms and methods for extracting meaningful information from images and videos.

[0016] "Natural language generation technology" refers to the technology that allows a computer to generate understandable sentences based on data.

[0017] "Database" refers to a system for systematically storing analysis results and related information. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention provides a system for supporting players in improving their soccer skills by using videos of practice sessions and matches taken by users. Specific embodiments will be described below based on the claims.

[0040] Recording and uploading videos

[0041] When a user wants to record a soccer practice or game, they use a device such as a smartphone or tablet. After shooting, the video file is saved on the device. The user opens the application, selects the video they shot, and uploads it to the server. This upload function sends the video to the server.

[0042] Video Analysis

[0043] When the server receives the video, it first splits it into frames at a fixed frame rate, generating individual frame images that are then analyzed. Computer vision technology is used to track the movements of players and the trajectory of the ball. The position and movement of each player and the ball in the video are identified, and the analysis results are stored in a database.

[0044] Generating Advice

[0045] The server uses the video analysis results to identify the player's technical issues and areas for improvement. This includes evaluating how the player kicks and stops the ball, as well as technical elements such as their positioning and passing options during the game. Next, natural language generation technology is used to translate the analyzed data into written advice. For example, advice such as "To kick the ball more powerfully, it would be better to position your stepping foot a little further forward" is generated.

[0046] Providing advice

[0047] The generated advice is sent from the server to the user's device, which is designed to display the received advice in an easy-to-read format. The user can review the displayed advice and implement the points they need to improve in their next practice or game.

[0048] Specific examples

[0049] Examples of kicking and stopping a ball

[0050] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[0051] 2. The server receives the video, divides it into frames, and analyzes it.

[0052] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[0053] 4. The server generates advice such as "To kick the ball harder, it would be better to position your stepping foot a little further forward" and sends it to the device.

[0054] 5. The user checks the advice in the application and puts it into practice in the next practice session.

[0055] Examples of positions during a match

[0056] 1. A user records their child's gameplay and uploads the video to a server.

[0057] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[0058] 3. The server evaluates the player's position and the ball's movement and generates advice such as, "In this scene, there is open space on the left, so it would be better to move there and receive the ball."

[0059] 4. The generated advice is sent to the terminal and confirmed by the user.

[0060] 5. The user can use this advice to improve their moves in their next game and aim for better play.

[0061] The above is a specific embodiment for carrying out the present invention. This system allows users to receive effective feedback and efficiently improve their soccer skills.

[0062] The processing flow will be explained below.

[0063] Step 1:

[0064] The user uses the device to record videos of practice and games.

[0065] Specifically, you can use the camera function of your smartphone or tablet to record the necessary scenes and save the video files on the device.

[0066] Step 2:

[0067] The user opens the application, selects the video they have taken, and uploads it to the server.

[0068] Specifically, the video is sent to the server by selecting a video file using the upload function and pressing the upload button.

[0069] Step 3:

[0070] The server receives the uploaded video.

[0071] Specifically, the received video file is stored in a predetermined storage area.

[0072] Step 4:

[0073] The server splits the video into chunks at a fixed frame rate.

[0074] Specifically, detailed analysis becomes possible by dividing a video into frames and generating individual frame images.

[0075] Step 5:

[0076] The server analyzes the player's movements and the ball's trajectory for each divided frame.

[0077] Specifically, computer vision technology is used to track the position, movement, and trajectory of players and the ball, and collect analytical data.

[0078] Step 6:

[0079] The server stores the analysis results in a database.

[0080] Specifically, the tracking results are recorded in a database and used for subsequent processing.

[0081] Step 7:

[0082] The server identifies technical issues and areas for improvement based on the analysis data.

[0083] Specifically, it runs algorithms that evaluate how the ball is kicked and stopped, as well as positioning and passing options during the game.

[0084] Step 8:

[0085] The server generates specific advice based on the identified issues and areas for improvement.

[0086] Specifically, natural language generation technology is used to generate sentences such as "To kick the ball harder, it is better to position your stepping foot a little further forward" from the analyzed data.

[0087] Step 9:

[0088] The server transmits the generated advice to the user's terminal.

[0089] Specifically, the generated advice content is structured as a response in JSON format or the like and sent to the terminal.

[0090] Step 10:

[0091] The terminal displays the advice received from the server.

[0092] Specifically, the application will display the advice received in an easy-to-read format so that users can put it into practice in their next practice or game.

[0093] Example 1

[0094] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0095] Conventional soccer skill improvement systems have had difficulty effectively analyzing videos of practice and matches taken by users and providing specific technical advice. In particular, they lacked a means to accurately track player movements and ball trajectories and provide specific areas for improvement based on that information. This forced users to analyze the videos themselves, making it difficult to obtain effective feedback.

[0096] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0097] In this invention, the server includes means for receiving videos of practice or games filmed by a user, means for dividing the videos at a fixed frame rate, means for tracking the player's movements and the ball's trajectory for each divided frame, means for storing the tracking results in a database, means for identifying technical issues and areas for improvement for the player based on the stored data, means for generating specific advice using a generative AI model based on the identified issues and areas for improvement, and means for transmitting the generated advice to the user's device, thereby enabling the user to receive pulsating feedback and efficiently improve their skills.

[0098] A "user" is someone who takes videos of soccer practice or matches and uploads the videos to the system.

[0099] A "server" is a computer system that receives videos uploaded by users, analyzes and stores them, and generates and provides advice.

[0100] "Device" means the device used by the User to record video, upload it to the server, and receive and display analysis results and advice.

[0101] "Video" refers to a video file in which a user records a practice or game.

[0102] "Frame rate" is a numerical value that indicates the display speed of still images that make up a moving image.

[0103] A "frame" refers to an individual still image that makes up a video.

[0104] "Tracking" is the process of tracking the player's movements and the ball's trajectory in the video and identifying their position and movement.

[0105] A "database" is a data collection point where analysis results are stored and used for subsequent processing and advice generation.

[0106] A "generative AI model" is an artificial intelligence algorithm that automatically generates specific advice based on tracking data.

[0107] The "analysis results" are the data obtained after dividing the video into frames and tracking the movements of the players and the ball.

[0108] "Specific advice" refers to instructions and advice for players to improve their skills, generated by the generative AI model based on the analysis results.

[0109] The present invention is a system that analyzes videos of soccer practice and matches taken by users and supports players in improving their skills. Specific embodiments will be described below.

[0110] Recording and uploading videos

[0111] Users use devices such as smartphones and tablets to record soccer practice and matches. After recording, the video files are saved on the device. Users then open a dedicated application, select the video they have taken, and upload it to the server. This upload function sends the video file to the server.

[0112] Video Analysis

[0113] The server divides the received video file into chunks at a fixed frame rate. For example, a 30 fps video generates 30 frame images per second. The server then analyzes each frame using computer vision technology (e.g., OpenCV). Player movements and the ball's trajectory are tracked, and the position and movement of each player and ball in the video are identified. These analysis results are stored in a database.

[0114] Generating Advice

[0115] Based on the video analysis, the server identifies the player's technical issues and areas for improvement. This identification process includes evaluating the player's kicking and stopping technique, as well as their positioning and passing options during the match. Based on the identified issues and areas for improvement, the server uses a generative AI model (e.g., GPT-3 (registered trademark)) to generate specific advice. For example, the server might generate advice such as, "To kick the ball more powerfully, you should position your stepping foot a little further forward."

[0116] Providing advice

[0117] The generated advice is sent from the server to the user's device, which then displays it in an easy-to-read format. The user can then review the advice and implement it to improve their game in the next practice or match.

[0118] Specific examples

[0119] A specific example is given below.

[0120] Examples of kicking and stopping a ball

[0121] 1. The user records their son's ball kicking practice on their smartphone and uploads the video to the server via the application.

[0122] 2. The server receives the video, divides it into frames, and analyzes it.

[0123] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[0124] 4. The server generates advice such as "To kick the ball harder, it would be better to position your stepping foot a little further forward" and sends it to the device.

[0125] 5. The user checks the advice in the application and puts it into practice in the next practice session.

[0126] Examples of positions during a match

[0127] 1. The user records their child's gameplay and uploads the video to the server.

[0128] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[0129] 3. The server evaluates the player's position and the ball's movement and generates advice such as, "In this scene, there is open space on the left, so it would be better to move there and receive the ball."

[0130] 4. The generated advice is sent to the terminal and confirmed by the user.

[0131] 5. The user can use this advice to improve their moves in their next game and aim for better play.

[0132] This system allows users to receive effective feedback and improve their soccer skills efficiently.

[0133] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0134] Step 1: Record and save the video

[0135] Description: Users record soccer practice and games using their smartphones or tablets. The recorded videos are saved on the device.

[0136] Input: Soccer practice and game scenes

[0137] Output: Recorded video file

[0138] How it works: The user opens the camera app, taps the record button to record a scene from practice or a game, and after recording is complete, the video file is automatically saved to the device's storage.

[0139] Step 2: Upload your video

[0140] Description: The user opens the dedicated application and uploads the video they have taken to the server. The device then sends the selected video file to the server.

[0141] Input: Saved video file

[0142] Output: Video file uploaded to the server

[0143] Specific operation: The user opens the application and taps the "Upload video" button. A file selection screen will appear, and the user can select a saved video file and perform the upload.

[0144] Step 3: Split the video into frames

[0145] Description: The server splits the received video file at a fixed frame rate, which converts the video into individual frame images.

[0146] Input: Uploaded video file

[0147] Output: Split frame images

[0148] Specific operation: The server analyzes the video at a fixed frame rate (for example, 30 fps) and generates and saves 30 frame images per second.

[0149] Step 4: Video Tracking Analysis

[0150] Description: The server uses computer vision technology to track player and ball movements frame by frame, determining player positions, actions, and ball trajectory.

[0151] Input: Segmented frame images

[0152] Output: Tracking data (player position, movement, ball trajectory, etc.)

[0153] Specific operation: The server uses OpenCV to recognize the player and ball for each frame image, record their positions and movements, and connect them to generate consistent tracking data.

[0154] Step 5: Identifying the technical challenges

[0155] Description: The server analyzes the player's technical issues and areas for improvement based on the tracking data, for example, identifying issues with the foot position to kick the ball hard.

[0156] Input: Tracking data

[0157] Output: Analysis results (technical issues and areas for improvement)

[0158] Specific operation: The server analyzes the tracking data and detects problems with the way players kick and stop the ball, their positioning during the game, and their pass selection.

[0159] Step 6: Generating Advice

[0160] Description: Based on the identified technical challenges, the server uses a generative AI model to generate specific advice, such as "To kick the ball harder, position your stepping foot a little further forward."

[0161] Input: Analysis results

[0162] Output: Generated advice

[0163] Specific operation: The server inputs the analysis results into a generative AI model (e.g., GPT-3) to generate specific, actionable advice in natural language.

[0164] Step 7: Providing advice

[0165] Description: The server provides the generated advice to the user's terminal, which displays the received advice in an easy-to-read format for the user to review.

[0166] Input: Generated advice

[0167] Output: Advice displayed to the user

[0168] Specific operation: The server sends the generated advice to the user's device, which displays the advice through a dedicated application. The user can then check the advice in the application and use it in their next practice or game.

[0169] (Application example 1)

[0170] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0171] Conventional factory robots face the challenge of not being able to obtain specific feedback to improve the efficiency and quality of their operations. In particular, there is a lack of means to identify subtle improvements and technical issues in the robot's operation or object handling, and to provide effective advice. This makes it difficult for operators to quickly receive specific, data-based advice to optimize the robot's operation.

[0172] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0173] In this invention, the server includes means for receiving video of a task or process filmed by a user, means for analyzing the received video to track the robot's movements and the trajectories of objects, means for identifying technical issues and improvements to the robot based on the analysis results, means for generating specific advice based on the identified issues and improvements, and means for transmitting the generated advice to the user's terminal, thereby enabling the operator to quickly receive specific improvement advice regarding the robot's movements and processes and to optimize the movements efficiently and effectively.

[0174] "User" refers to the individual or institution that operates the robot in the factory and is responsible for monitoring and improving its operation.

[0175] "Video of work or processes" refers to video data that records specific operations or work processes performed by factory robots.

[0176] "Means for receiving" refers to the communication interface and software used to transfer and store videos taken by users to a server or cloud.

[0177] "Means for analyzing" refers to a system or algorithm that uses computer vision technology to decompose and analyze received video and extract information about movements, object positions, and trajectories.

[0178] "Robot movement or object trajectory" refers to the movement of a factory robot's arm or other moving part, and the path of an object moved or handled by that movement.

[0179] "Means for identifying technical issues and areas for improvement" refers to a system that uses analyzed data to identify factors that reduce the efficiency and quality of robot operations and determine areas that need improvement.

[0180] "Means for generating specific advice" refers to a system or algorithm that generates detailed recommendations or instructions in natural language to improve the robot's behavior based on identified issues and areas for improvement.

[0181] "Means for sending to a user's terminal" refers to the communications infrastructure and software for sending the generated advice to a digital device such as a user's smartphone, tablet, or computer.

[0182] This invention is a system that analyzes videos of factory robot operations taken by users, and identifies and provides technical issues and areas for improvement. Specific embodiments are described below.

[0183] System Program

[0184] This system works in cooperation with a server and a user terminal. The server implements functions for receiving video, analyzing, generating and sending advice. The user terminal is responsible for recording and playing video, and receiving and displaying advice.

[0185] Program processing explanation

[0186] 1. Recording and uploading videos

[0187] Users can record videos of factory robots' work and processes using a smartphone or tablet, and then upload the videos to a server using an application on the user's device.

[0188] 2. Video Analysis

[0189] The server analyzes the received video using computer vision technology such as OpenCV. Specifically, it divides the video into frames at a fixed rate and tracks the robot's movements and object trajectories in each frame. The results of this analysis are stored in a database.

[0190] 3. Generating Advice

[0191] The server identifies technical issues and areas for improvement based on the analyzed data. It then uses generative AI models and natural language generation technology to translate specific advice into text form. For example, the server could generate advice such as "You'll get a better finish if you slow down the welding speed a little" based on the analysis results.

[0192] 4. Providing advice

[0193] Finally, the generated advice is sent to the user's device, where the user can review the advice and implement the improvements in the next task or process.

[0194] Hardware and software used

[0195] Hardware

[0196] Smartphones, tablets, servers

[0197] software

[0198] Flask (Python web framework)

[0199] OpenCV (computer vision library)

[0200] Generative AI models (libraries for natural language generation)

[0201] Specific examples

[0202] Examples of work improvements

[0203] 1. A user records the welding work of a factory robot on a smartphone and uploads the video to a server via an application.

[0204] 2. The server divides the received video into frames, analyzes them, and tracks the welding operations.

[0205] 3. Based on the analysis results, the deterioration in quality that occurs when the welding speed is too fast is identified, and advice such as "If you slow down the welding speed a little, the finish will be better" is generated.

[0206] 4. This advice is sent to the user's device, and the user puts this advice into practice the next time they work.

[0207] Prompt Sentence Examples

[0208] "Please analyze the video of the robot's movements and point out areas for improvement."

[0209] Example: "The welding speed is too fast, resulting in a rough finish. Slowing the welding speed by 25% will improve the quality."

[0210] The above is a specific embodiment for carrying out the present invention, which allows users to efficiently and effectively optimize the operation of factory robots and improve the quality and efficiency of work.

[0211] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0212] Step 1:

[0213] Users take videos of factory robots' work and processes using their smartphones or tablets.

[0214] Input: Video of factory robot in action

[0215] Output: Recorded video file

[0216] In this step, the user actually takes a video and saves it on a digital device (smartphone or tablet).

[0217] Step 2:

[0218] The user uploads the video they have taken to the server through the application.

[0219] Input: Recorded video file

[0220] Output: Video file stored on the server

[0221] In this step, the user opens the application, selects a video file, and uploads it to the server. The server receives the uploaded video and saves it in storage.

[0222] Step 3:

[0223] The server divides the video it receives at a fixed frame rate and tracks the robot's movements and object trajectories in each frame.

[0224] Input: Video file stored on the server

[0225] Output: Frame-by-frame data showing tracking results

[0226] This is done using computer vision techniques such as OpenCV. The server splits the video file into frames and analyzes the robot's movements and object positions in each frame. The results of this analysis are stored in a database.

[0227] Step 4:

[0228] The server identifies technical issues and areas for improvement based on the analysis results stored in the database.

[0229] Input: Frame-by-frame data showing tracking results

[0230] Output: Identified technical issues and areas for improvement

[0231] In this step, the server evaluates the analysis data and identifies problems and areas that need improvement in the robot's operation.

[0232] Step 5:

[0233] Based on the identified issues and areas for improvement, the server uses generative AI models and natural language generation technology to generate specific advice.

[0234] Input: Identified technical issues and improvements

[0235] Output: Advice statement

[0236] Here, a generative AI model is used to automatically generate advice, such as specific instructions such as "You'll get a better finish if you slow down the welding speed a little."

[0237] Step 6:

[0238] The server transmits the generated advice to the user's terminal.

[0239] Input: Advice sentence

[0240] Output: Advice displayed on the user's terminal

[0241] In this step, the server sends the generated advice to the user's smartphone or tablet, which receives the advice and displays it on the application.

[0242] Through the above processing steps, the user can quickly receive specific feedback for optimizing the operation of the factory robot and put it into practice.

[0243] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0244] This invention is a system that supports players in improving their soccer skills by using videos of practice and matches taken by the user, and also provides more personalized advice by combining it with an emotion engine that recognizes the user's emotions. Specific embodiments will be described below based on the claims.

[0245] Recording and uploading videos

[0246] When a user wants to record a soccer practice or game, they use a device such as a smartphone or tablet. After shooting, the video file is saved on the device. The user opens the application, selects the video they shot, and uploads it to the server. This upload function sends the video to the server.

[0247] Video Analysis

[0248] When the server receives the video, it first splits it into frames at a fixed frame rate, generating individual frame images that are then analyzed. Computer vision technology is used to track the movements of players and the trajectory of the ball. The position and movement of each player and the ball in the video are identified, and the analysis results are stored in a database.

[0249] Generating Advice

[0250] The server uses the video analysis results to identify the player's technical issues and areas for improvement. This includes evaluating how the ball is kicked and stopped, as well as technical elements such as positioning and passing options during the game. Next, natural language generation technology is used to translate the analyzed data into written advice. For example, the generated advice might be, "To kick the ball more powerfully, it would be better to position your stepping foot a little further forward."

[0251] Providing advice

[0252] The generated advice is sent from the server to the user's device, which is designed to display the received advice in an easy-to-read format. The user can review the displayed advice and implement the points they need to improve in their next practice or game.

[0253] Introducing the Emotion Engine

[0254] This system incorporates an emotion engine that recognizes users' emotions. The emotion engine analyzes the user's tone of voice and facial expressions to identify their emotions. When a user uploads a video, the emotion engine analyzes the user's voice and facial expressions in real time and sends the emotional data to the server.

[0255] Use of Emotional Data

[0256] The server combines the emotion data sent from the emotion engine with the video analysis results to generate more personalized advice. For example, if the user is feeling discouraged, the server generates advice that includes mental support, such as "Try not to rush during your next practice, and take your steps slowly." The content and tone of the generated advice are adjusted according to the user's emotional state, and sent to the device in an appropriate format.

[0257] Specific examples

[0258] Example of kicking and stopping a ball using emotion recognition

[0259] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[0260] 2. The server receives the video, divides it into frames, and analyzes it.

[0261] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[0262] 4. When users express their thoughts vocally during upload, the emotion engine analyzes their tone of voice to identify their emotional state.

[0263] 5. The server combines the video analysis results with the emotional data and generates advice such as, "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing."

[0264] 6. The advice is sent to the device, and the user checks it in the application and puts it into practice the next time they practice.

[0265] Example of a match position using emotion recognition

[0266] 1. A user records their child's gameplay and uploads the video to a server.

[0267] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[0268] 3. When users express their emotions through facial expressions during upload, the emotion engine analyzes the facial expressions and identifies the emotional state.

[0269] 4. The server evaluates the player's position and the ball's movement and generates advice such as "In this scene, it would be better to move to the space on the left."

[0270] 5. Consider the user's emotional state and include encouragement such as, "Try this advice next time and play with confidence."

[0271] 6. The generated advice is sent to the terminal and confirmed by the user.

[0272] 7. The user can use this advice to improve their moves in their next game and aim for better play.

[0273] The above is a specific embodiment for carrying out the present invention. With this system, users can receive effective feedback and support that takes into consideration their emotions, enabling them to efficiently improve their soccer skills.

[0274] The processing flow will be explained below.

[0275] Step 1:

[0276] The user uses the device to record videos of practice and games.

[0277] Specifically, you can use the camera function of your smartphone or tablet to record the necessary scenes and save the video files on the device.

[0278] Step 2:

[0279] The user opens the application, selects the video they have taken, and uploads it to the server.

[0280] Specifically, the video is sent to the server by selecting a video file using the upload function and pressing the upload button.

[0281] Step 3:

[0282] The server receives the uploaded video.

[0283] Specifically, the received video file is stored in a predetermined storage area.

[0284] Step 4:

[0285] The server splits the video into chunks at a fixed frame rate.

[0286] Specifically, detailed analysis becomes possible by dividing a video into frames and generating individual frame images.

[0287] Step 5:

[0288] The server analyzes the player's movements and the ball's trajectory for each divided frame.

[0289] Specifically, computer vision technology is used to track the position, movement, and trajectory of players and the ball, and collect analytical data.

[0290] Step 6:

[0291] The server stores the analysis results in a database.

[0292] Specifically, the tracking results are recorded in a database and used for subsequent processing.

[0293] Step 7:

[0294] The server identifies technical issues and areas for improvement based on the analysis data.

[0295] Specifically, it runs algorithms that evaluate how the ball is kicked and stopped, as well as positioning and passing options during the game.

[0296] Step 8:

[0297] The server generates specific advice based on the identified issues and areas for improvement.

[0298] Specifically, natural language generation technology is used to generate sentences such as "To kick the ball harder, it is better to position your stepping foot a little further forward" from the analyzed data.

[0299] Step 9:

[0300] The device analyzes the user's tone of voice and facial expressions using an emotion engine.

[0301] Specifically, the system captures the user's voice and facial expressions in real time when they upload a video and identifies their emotions.

[0302] Step 10:

[0303] The server receives the emotion data sent from the emotion engine.

[0304] Specifically, the emotional data is combined with the analysis results and used as data to generate advice based on the user's emotions.

[0305] Step 11:

[0306] The server takes into account the user's emotional state and adjusts the content and tone of the advice.

[0307] Specifically, if the user is feeling discouraged, the system generates advice that includes mental support, such as "Try not to rush during the next practice session, and take your steps slowly."

[0308] Step 12:

[0309] The server transmits the generated advice to the user's terminal.

[0310] Specifically, the generated advice content is structured as a response in JSON format or the like and sent to the terminal.

[0311] Step 13:

[0312] The terminal displays the advice received from the server.

[0313] Specifically, the application will display the advice received in an easy-to-read format so that users can put it into practice in their next practice or game.

[0314] Specific examples

[0315] Examples of kicking and stopping a ball

[0316] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[0317] 2. The server receives the video, divides it into frames, and analyzes it.

[0318] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[0319] 4. When users express their thoughts vocally during upload, the emotion engine analyzes their tone of voice to identify their emotional state.

[0320] 5. The server combines the video analysis results with the emotional data and generates advice such as, "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing."

[0321] 6. The advice is sent to the device, and the user checks it in the application and puts it into practice the next time they practice.

[0322] Examples of positions during a match

[0323] 1. A user records their child's gameplay and uploads the video to a server.

[0324] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[0325] 3. When users express their emotions through facial expressions during upload, the emotion engine analyzes the facial expressions and identifies the emotional state.

[0326] 4. The server evaluates the player's position and the ball's movement and generates advice such as "In this scene, it would be better to move to the space on the left."

[0327] 5. Consider the user's emotional state and include encouragement such as, "Try this advice next time and play with confidence."

[0328] 6. The generated advice is sent to the terminal and confirmed by the user.

[0329] 7. The user can use this advice to improve their moves in their next game and aim for better play.

[0330] Example 2

[0331] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0332] Conventional feedback systems for soccer practice and matches are limited to analyzing the player's physical movements and the ball's trajectory, and are unable to provide personalized advice that takes into account the user's emotional state. Therefore, there is a need for comprehensive feedback that includes mental support for users in addition to technical improvements.

[0333] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0334] In this invention, the server includes means for receiving videos of practice and games filmed by the user, means for analyzing the received videos to track the player's movements and the ball's trajectory, means for identifying technical issues and areas for improvement for the player based on the analysis results, means for generating specific advice based on the identified issues and areas for improvement, means for transmitting the generated advice to the user's device, means for analyzing emotion data using an emotion engine that recognizes the user's emotions, and means for generating personalized advice that combines the emotion data and the video analysis results. This enables personalized feedback that includes not only technical feedback but also mental support based on the user's emotions.

[0335] "User" refers to an individual who uses this system to film and upload videos of soccer practice and matches and receive feedback.

[0336] "Means for receiving video" refers to a system or method that has the function of receiving video files sent from a user to a server.

[0337] "Means for analyzing video to track player movements and ball trajectory" refers to systems and methods for using computer vision technology to track the position and movement of players and the ball in video.

[0338] "Means for identifying technical issues and areas for improvement" refers to a system or method for identifying problems in a player's actions and areas for technical improvement based on the analysis results.

[0339] "Means for generating advice" refers to a system or method for generating specific advice in written form for identified technical issues or areas for improvement.

[0340] "Means for transmitting advice to the user's terminal" refers to a communication means for delivering the generated advice to the user's terminal.

[0341] "Emotion engine" refers to a system or method for analyzing a user's voice and facial expressions to identify their emotional state.

[0342] "Means for analyzing emotion data" refers to a system or method for analyzing emotion data obtained by an emotion engine.

[0343] "Means for generating personalized advice by combining emotional data and video analysis results" refers to a system or method for integrating a user's emotional state with video analysis results to generate individually customized advice.

[0344] This invention is a system that analyzes videos of soccer practice and matches taken by users to help players improve their skills. Furthermore, by combining this with an emotion engine that recognizes the user's emotions, it provides personalized advice. A specific embodiment of this system will be described.

[0345] Recording and uploading videos

[0346] A user uses a smartphone or tablet to record soccer practice or a game. After recording, the video file is saved on the device, and the user opens a dedicated application, selects the video, and uploads it to a server. During this process, the application on the device sends the video file to the server via the network, allowing the server to receive the video recorded by the user.

[0347] Video Analysis

[0348] When the server receives the video, it divides it into frames at a fixed frame rate. For example, it uses a computer vision library such as OpenCV. The server tracks the player's movements and the ball's trajectory from the images of each frame and stores these analysis results in a database.

[0349] Generating Advice

[0350] Based on the analysis results, the server identifies the player's technical issues and areas for improvement. Specifically, it uses a machine learning model to evaluate the player's movements. Next, it uses natural language generation technology (for example, a generative AI model such as GPT-4 (registered trademark)) to translate specific advice into text. For example, it generates advice such as, "To kick the ball harder, it would be better to position your stepping foot a little further forward."

[0351] Providing advice

[0352] The generated advice is sent from the server to the user's device, which displays it in an easy-to-read format, allowing the user to review the advice and put it into practice in their next practice or game.

[0353] Introducing the Emotion Engine

[0354] This system incorporates an emotion engine that recognizes the user's emotions. The emotion engine analyzes the user's tone of voice and facial expressions to identify their emotions. For example, it uses emotion recognition software such as "Microsoft® Azure® Cognitive Services." When a user uploads a video, the emotion engine analyzes it in real time and sends the results to a server.

[0355] Use of Emotional Data

[0356] The server combines the emotion data sent from the emotion engine with the video analysis results to generate more personalized advice. For example, if the user is feeling discouraged, the server generates advice that includes mental support, such as "Try not to rush during the next practice session, and take your steps slowly." The content and tone of this advice are also adjusted to suit the user's emotional state. The generated advice is then sent to the user's device.

[0357] Specific examples

[0358] Example of kicking and stopping a ball using emotion recognition

[0359] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[0360] 2. The server receives the video, divides it into frames, and analyzes it.

[0361] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[0362] 4. When users express their thoughts vocally while uploading a video, the emotion engine analyzes their tone of voice to identify their emotional state.

[0363] 5. The server combines the video analysis results with the emotional data and generates advice such as, "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing."

[0364] 6. The advice is sent to the device, and the user checks it in the application and puts it into practice the next time they practice.

[0365] Example of a match position using emotion recognition

[0366] 1. A user records their child's gameplay and uploads the video to a server.

[0367] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[0368] 3. When users express emotions through facial expressions while uploading videos, the emotion engine analyzes the facial expressions and identifies the emotional state.

[0369] 4. The server evaluates the player's position and the ball's movement and generates advice such as "In this scene, it would be better to move to the space on the left."

[0370] 5. Consider the user's emotional state and include encouragement such as, "Try this advice next time and play with confidence."

[0371] 6. The generated advice is sent to the terminal and confirmed by the user.

[0372] 7. The user can use this advice to improve their moves in their next game and aim for better play.

[0373] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0374] The flow of this system's program processing

[0375] Step 1:

[0376] Users record soccer practice and matches on their smartphones or tablets, and the video files are saved on the device.

[0377] Input: Video data (videos of practice and games)

[0378] Output: Video file saved on your device

[0379] Specific operation: The user launches the camera app on their smartphone and records practice or a game. After recording is complete, the video is saved on the device as "Game Video.mp4."

[0380] Step 2:

[0381] The user opens a dedicated application, selects the video they have taken, and uploads it to the server. The application on the device then sends the video file to the server via the network.

[0382] Input: Video files saved on the device

[0383] Output: Video file uploaded to the server

[0384] Specific operation: The user opens the application and taps the "Upload" button. The application selects "MatchVideo.mp4" on the device and sends it to the server.

[0385] Step 3:

[0386] The server divides the received video at a fixed frame rate to generate individual frame images, using computer vision technology to track the player's movements and the ball's trajectory, and stores the analysis results in a database.

[0387] Input: Video file uploaded to the server

[0388] Output: Analysis results (player and ball position data)

[0389] Specific operation: The server uses "OpenCV" to split the video into frames at 30 fps, extracts the player's position and the ball's position from each frame, and saves the analysis results in "Analysis Result DB".

[0390] Step 4:

[0391] The server analyzes the video to identify the player's technical issues and areas for improvement, then uses a generative AI model to write specific advice.

[0392] Input: Analysis results (player and ball position data)

[0393] Output: Written advice

[0394] Specific operation: The server uses a machine learning model to evaluate the player's actions, and then uses a generative AI model (e.g., GPT-4) to generate advice such as "To kick the ball harder, you should position your stepping foot a little further forward."

[0395] Step 5:

[0396] The server sends the generated advice in JSON format to the user's device, which displays the received advice in an easy-to-read format, and the user confirms the advice.

[0397] Input: Written advice

[0398] Output: Advice displayed on terminal

[0399] Specific operation: The server sends the generated advice in JSON format to the device. The device parses the received JSON data and displays it in the advice section of the application. The user confirms the displayed advice.

[0400] Step 6:

[0401] When a user expresses their feelings aloud while uploading a video, the emotion engine analyzes their tone of voice and facial expressions to identify their emotions, and the emotion data is sent to the server.

[0402] Input: User's emotional data (tone of voice and facial expression)

[0403] Output: Emotion data sent to the server

[0404] Specific operation: When uploading a video, the user says out loud, "Today didn't go well." The emotion engine analyzes the voice in real time and recognizes it as "disappointment." The emotion data is then sent to the server.

[0405] Step 7:

[0406] The server combines the emotion data with the video analysis results to generate personalized advice, with content and tone tailored to the user's emotional state, and sends it to the user's device.

[0407] Input: Analysis results (player and ball position data), emotion data

[0408] Output: Personalized advice

[0409] Specific operation: The server receives the emotion data "disappointment" and combines it with the video analysis results. The generative AI model inputs "advice for a disappointed user" as a prompt, generating advice such as "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing." The advice is then sent to the device and confirmed by the user.

[0410] (Application example 2)

[0411] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0412] Conventional soccer skill improvement support systems are limited to providing technical advice and do not provide personalized support based on the user's mental state or emotions. This can make it difficult to address the user's mental challenges and effectively improve their skills. In addition, there is a lack of a method for generating advice that utilizes emotional data.

[0413] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving videos of practice or games filmed by the user, means for analyzing the received videos to track the player's movements and the ball's trajectory, means for identifying technical issues and areas for improvement for the player based on the analysis results, means for generating specific advice based on the identified issues and areas for improvement, means for transmitting the generated advice to the user's terminal, means for generating emotional data by analyzing the user's tone of voice and facial expressions, and means for generating personalized advice including mental support based on the emotional data. This enables comprehensive support that is not only technical but also responsive to the user's emotional state.

[0414] 1. "User" refers to a person who uses the system to record videos of soccer practice and matches and receives analysis results and advice.

[0415] 2. "Means for receiving videos" refers to a function that allows users to upload videos of practice or games they have taken to the server and receive them.

[0416] 3. "Means for tracking player movements and ball trajectory" means technology for tracking player movements and the position and trajectory of the ball through video analysis.

[0417] 4. "Means for identifying technical issues and areas for improvement" is a function that uses the analysis results to identify problems and areas that need improvement in a player's soccer skills.

[0418] 5. "Means for generating specific advice" is a function for providing specific improvement methods and practice methods in written form based on technical issues and areas for improvement.

[0419] 6. "Means for sending advice to the user's device" refers to a function for sending the generated advice to the user's device, such as a smartphone or tablet, and making it viewable.

[0420] 7. "Means for generating emotional data by analyzing tone of voice and facial expressions" refers to technology that analyzes the tone of voice and facial expressions of users when they upload videos, and then digitizes the emotional state of the users based on that analysis.

[0421] 8. "Means for generating personalized advice including mental support" refers to a function that provides advice including support and encouragement according to the user's mental state based on emotional data.

[0422] 9. "Means for analyzing each frame" refers to a technique for dividing a video into parts at a fixed frame rate and performing analysis on each frame.

[0423] 10. "Means for saving in a database" refers to the function for recording and saving the analyzed results in a database.

[0424] 11. "Means for using natural language generation technology to translate advice into written form" means technology that uses a generative AI model to output analyzed data as advice in written form.

[0425] 12. "Computer vision technology" is a technology for tracking and recognizing specific objects by analyzing images and videos.

[0426] The present invention is a method for combining a system that uses videos of soccer practice and matches taken by users to help players improve their skills with an emotion engine that recognizes the emotions of users. A specific form of this method is described below.

[0427] Recording and uploading videos

[0428] First, the user records soccer practice or a match using a device such as a smartphone. The recorded video is saved on the device and then uploaded to a server via an application. At this time, the user can express their thoughts about the video by voice, and the emotion engine analyzes the voice data.

[0429] Video Analysis

[0430] The server splits the uploaded video at a fixed frame rate to generate individual frame images. Computer vision techniques (e.g., OpenCV) are used for video analysis to track player movements and ball trajectories. The analysis results are stored in a database.

[0431] Generating Advice

[0432] The server uses the video analysis to identify players' technical issues and areas for improvement, such as how they kick and stop the ball, their positioning and passing options during the game, etc. It then uses natural language generation technology (e.g., a generative AI model) to generate specific advice sentences from the analyzed data.

[0433] Introducing the Emotion Engine

[0434] When a user uploads a video, the emotion engine analyzes the user's tone of voice and facial expressions to generate emotional data, which is then sent to a server and combined with technical analysis results to generate more personalized advice.

[0435] Providing advice

[0436] The generated advice is sent from the server to the user's smartphone or other device, where the user can check it using the application and use it for their next practice or game.

[0437] Hardware and software used

[0438] Hardware: Smartphone (with camera and microphone), server

[0439] software:

[0440] OpenCV: A computer vision library for video analysis

[0441] Emotion Recognition Module: A custom module for emotion recognition (e.g., an extension of a Python sentiment analysis library)

[0442] Video Analysis Module: A custom module for player movement analysis

[0443] Specific examples

[0444] For example, consider a case where a user films their son's soccer practice with their smartphone and uploads the video to a server via an application. When uploading the video, if the user expresses their opinion by saying, "Today's practice didn't go well," the emotion engine analyzes the tone of the voice and recognizes that the user is disappointed. Based on the results of the video analysis and the emotion data, the server generates advice that includes mental support, such as, "Next time, try practicing your steps without rushing." The generated advice is sent to the user's smartphone and implemented at the next practice or game.

[0445] Prompt Sentence Examples

[0446] I analyzed a video of my son's soccer practice and uploaded it to a server. Please provide technical advice on kicking the ball while recognizing the user's emotions. If the user is feeling discouraged, please also provide advice that includes mental support.

[0447] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0448] Step 1:

[0449] A user films a soccer practice or game with their smartphone. The input here is a video of the soccer practice or game. The user saves the video to their device. The output of this step is a video file saved to their device.

[0450] Step 2:

[0451] The user opens the application, selects a saved video, and uploads it to the server. The input here is the video file, which is the output of step 1. The uploaded video is sent to and received by the server. The output of this step is the video file saved on the server.

[0452] Step 3:

[0453] The video received by the server is split at a constant frame rate. The input here is a video file stored on the server. Using computer vision techniques such as OpenCV, the video is split into frames and individual frame images are generated. The output of this step is a set of split frame images.

[0454] Step 4:

[0455] The server analyzes the frame images and tracks the player movements and ball trajectory. The input here is the set of frame images output from step 3. Using computer vision techniques, the server identifies the player's position and movements in each frame and stores them in a database. The output of this step is the analyzed data stored in the database.

[0456] Step 5:

[0457] When uploading a video, the user expresses their feelings by voice. The input here is the user's voice data. The emotion engine analyzes the tone and content of the user's voice and generates emotion data. The output of this step is the emotion data that is sent to the server.

[0458] Step 6:

[0459] The server uses the video analysis data and emotion data to identify the player's technical issues and areas for improvement and generate specific advice. The input here is the analysis data from step 4 and the emotion data from step 5. A generative AI model is used to convert the analysis data into advice in natural language. The output of this step is the generated text of the advice.

[0460] Step 7:

[0461] The server sends the generated advice to the user's device. The input here is the text of the advice from step 6. The advice is sent to the user's device, such as a smartphone, and displayed on the device. The output of this step is the advice received by the user.

[0462] Step 8:

[0463] The user checks the advice received through the application and puts it into practice in the next practice or match. The input here is the advice received in step 7. The user attempts to improve their technique based on the advice. The output of this step is the improvement of the user's soccer skills and the effect of mental support.

[0464] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0465] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0466] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0467] [Second embodiment]

[0468] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0469] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0470] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0471] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0472] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0473] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0474] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0475] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0476] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0477] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0478] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0479] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0480] The present invention provides a system for supporting players in improving their soccer skills by using videos of practice sessions and matches taken by users. Specific embodiments will be described below based on the claims.

[0481] Recording and uploading videos

[0482] When a user wants to record a soccer practice or game, they use a device such as a smartphone or tablet. After shooting, the video file is saved on the device. The user opens the application, selects the video they shot, and uploads it to the server. This upload function sends the video to the server.

[0483] Video Analysis

[0484] When the server receives the video, it first splits it into frames at a fixed frame rate, generating individual frame images that are then analyzed. Computer vision technology is used to track the movements of players and the trajectory of the ball. The position and movement of each player and the ball in the video are identified, and the analysis results are stored in a database.

[0485] Generating Advice

[0486] The server uses the video analysis results to identify the player's technical issues and areas for improvement. This includes evaluating how the player kicks and stops the ball, as well as technical elements such as their positioning and passing options during the game. Next, natural language generation technology is used to translate the analyzed data into written advice. For example, advice such as "To kick the ball more powerfully, it would be better to position your stepping foot a little further forward" is generated.

[0487] Providing advice

[0488] The generated advice is sent from the server to the user's device, which is designed to display the received advice in an easy-to-read format. The user can review the displayed advice and implement the points they need to improve in their next practice or game.

[0489] Specific examples

[0490] Examples of kicking and stopping a ball

[0491] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[0492] 2. The server receives the video, divides it into frames, and analyzes it.

[0493] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[0494] 4. The server generates advice such as "To kick the ball harder, it would be better to position your stepping foot a little further forward" and sends it to the device.

[0495] 5. The user checks the advice in the application and puts it into practice in the next practice session.

[0496] Examples of positions during a match

[0497] 1. A user records their child's gameplay and uploads the video to a server.

[0498] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[0499] 3. The server evaluates the player's position and the ball's movement and generates advice such as, "In this scene, there is open space on the left, so it would be better to move there and receive the ball."

[0500] 4. The generated advice is sent to the terminal and confirmed by the user.

[0501] 5. The user can use this advice to improve their moves in their next game and aim for better play.

[0502] The above is a specific embodiment for carrying out the present invention. This system allows users to receive effective feedback and efficiently improve their soccer skills.

[0503] The processing flow will be explained below.

[0504] Step 1:

[0505] The user uses the device to record videos of practice and games.

[0506] Specifically, you can use the camera function of your smartphone or tablet to record the necessary scenes and save the video files on the device.

[0507] Step 2:

[0508] The user opens the application, selects the video they have taken, and uploads it to the server.

[0509] Specifically, the video is sent to the server by selecting a video file using the upload function and pressing the upload button.

[0510] Step 3:

[0511] The server receives the uploaded video.

[0512] Specifically, the received video file is stored in a predetermined storage area.

[0513] Step 4:

[0514] The server splits the video into chunks at a fixed frame rate.

[0515] Specifically, detailed analysis becomes possible by dividing a video into frames and generating individual frame images.

[0516] Step 5:

[0517] The server analyzes the player's movements and the ball's trajectory for each divided frame.

[0518] Specifically, computer vision technology is used to track the position, movement, and trajectory of players and the ball, and collect analytical data.

[0519] Step 6:

[0520] The server stores the analysis results in a database.

[0521] Specifically, the tracking results are recorded in a database and used for subsequent processing.

[0522] Step 7:

[0523] The server identifies technical issues and areas for improvement based on the analysis data.

[0524] Specifically, it runs algorithms that evaluate how the ball is kicked and stopped, as well as positioning and passing options during the game.

[0525] Step 8:

[0526] The server generates specific advice based on the identified issues and areas for improvement.

[0527] Specifically, natural language generation technology is used to generate sentences such as "To kick the ball harder, it is better to position your stepping foot a little further forward" from the analyzed data.

[0528] Step 9:

[0529] The server transmits the generated advice to the user's terminal.

[0530] Specifically, the generated advice content is structured as a response in JSON format or the like and sent to the terminal.

[0531] Step 10:

[0532] The terminal displays the advice received from the server.

[0533] Specifically, the application will display the advice received in an easy-to-read format so that users can put it into practice in their next practice or game.

[0534] Example 1

[0535] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0536] Conventional soccer skill improvement systems have had difficulty effectively analyzing videos of practice and matches taken by users and providing specific technical advice. In particular, they lacked a means to accurately track player movements and ball trajectories and provide specific areas for improvement based on that information. This forced users to analyze the videos themselves, making it difficult to obtain effective feedback.

[0537] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0538] In this invention, the server includes means for receiving videos of practice or games filmed by a user, means for dividing the videos at a fixed frame rate, means for tracking the player's movements and the ball's trajectory for each divided frame, means for storing the tracking results in a database, means for identifying technical issues and areas for improvement for the player based on the stored data, means for generating specific advice using a generative AI model based on the identified issues and areas for improvement, and means for transmitting the generated advice to the user's device, thereby enabling the user to receive pulsating feedback and efficiently improve their skills.

[0539] A "user" is someone who takes videos of soccer practice or matches and uploads the videos to the system.

[0540] A "server" is a computer system that receives videos uploaded by users, analyzes and stores them, and generates and provides advice.

[0541] "Device" means the device used by the User to record video, upload it to the server, and receive and display analysis results and advice.

[0542] "Video" refers to a video file in which a user records a practice or game.

[0543] "Frame rate" is a numerical value that indicates the display speed of still images that make up a moving image.

[0544] A "frame" refers to an individual still image that makes up a video.

[0545] "Tracking" is the process of tracking the player's movements and the ball's trajectory in the video and identifying their position and movement.

[0546] A "database" is a data collection point where analysis results are stored and used for subsequent processing and advice generation.

[0547] A "generative AI model" is an artificial intelligence algorithm that automatically generates specific advice based on tracking data.

[0548] The "analysis results" are the data obtained after dividing the video into frames and tracking the movements of the players and the ball.

[0549] "Specific advice" refers to instructions and advice for players to improve their skills, generated by the generative AI model based on the analysis results.

[0550] The present invention is a system that analyzes videos of soccer practice and matches taken by users and supports players in improving their skills. Specific embodiments will be described below.

[0551] Recording and uploading videos

[0552] Users use devices such as smartphones and tablets to record soccer practice and matches. After recording, the video files are saved on the device. Users then open a dedicated application, select the video they have taken, and upload it to the server. This upload function sends the video file to the server.

[0553] Video Analysis

[0554] The server divides the received video file into chunks at a fixed frame rate. For example, a 30 fps video generates 30 frame images per second. The server then analyzes each frame using computer vision technology (e.g., OpenCV). Player movements and the ball's trajectory are tracked, and the position and movement of each player and ball in the video are identified. These analysis results are stored in a database.

[0555] Generating Advice

[0556] Based on the video analysis, the server identifies the player's technical issues and areas for improvement. This identification process includes evaluating the player's kicking and stopping technique, as well as their positioning and passing options during the match. Based on the identified issues and areas for improvement, the server uses a generative AI model (e.g., GPT-3) to generate specific advice. For example, the server might suggest, "To kick the ball more powerfully, you should position your stepping foot a little further forward."

[0557] Providing advice

[0558] The generated advice is sent from the server to the user's device, which then displays it in an easy-to-read format. The user can then review the advice and implement it to improve their game in the next practice or match.

[0559] Specific examples

[0560] A specific example is given below.

[0561] Examples of kicking and stopping a ball

[0562] 1. The user records their son's ball kicking practice on their smartphone and uploads the video to the server via the application.

[0563] 2. The server receives the video, divides it into frames, and analyzes it.

[0564] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[0565] 4. The server generates advice such as "To kick the ball harder, it would be better to position your stepping foot a little further forward" and sends it to the device.

[0566] 5. The user checks the advice in the application and puts it into practice in the next practice session.

[0567] Examples of positions during a match

[0568] 1. The user records their child's gameplay and uploads the video to the server.

[0569] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[0570] 3. The server evaluates the player's position and the ball's movement and generates advice such as, "In this scene, there is open space on the left, so it would be better to move there and receive the ball."

[0571] 4. The generated advice is sent to the terminal and confirmed by the user.

[0572] 5. The user can use this advice to improve their moves in their next game and aim for better play.

[0573] This system allows users to receive effective feedback and improve their soccer skills efficiently.

[0574] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0575] Step 1: Record and save the video

[0576] Description: Users record soccer practice and games using their smartphones or tablets. The recorded videos are saved on the device.

[0577] Input: Soccer practice and game scenes

[0578] Output: Recorded video file

[0579] How it works: The user opens the camera app, taps the record button to record a scene from practice or a game, and after recording is complete, the video file is automatically saved to the device's storage.

[0580] Step 2: Upload your video

[0581] Description: The user opens the dedicated application and uploads the video they have taken to the server. The device then sends the selected video file to the server.

[0582] Input: Saved video file

[0583] Output: Video file uploaded to the server

[0584] Specific operation: The user opens the application and taps the "Upload video" button. A file selection screen will appear, and the user can select a saved video file and perform the upload.

[0585] Step 3: Split the video into frames

[0586] Description: The server splits the received video file at a fixed frame rate, which converts the video into individual frame images.

[0587] Input: Uploaded video file

[0588] Output: Split frame images

[0589] Specific operation: The server analyzes the video at a fixed frame rate (for example, 30 fps) and generates and saves 30 frame images per second.

[0590] Step 4: Video Tracking Analysis

[0591] Description: The server uses computer vision technology to track player and ball movements frame by frame, determining player positions, actions, and ball trajectory.

[0592] Input: Segmented frame images

[0593] Output: Tracking data (player position, movement, ball trajectory, etc.)

[0594] Specific operation: The server uses OpenCV to recognize the player and ball for each frame image, record their positions and movements, and connect them to generate consistent tracking data.

[0595] Step 5: Identifying the technical challenges

[0596] Description: The server analyzes the player's technical issues and areas for improvement based on the tracking data, for example, identifying issues with the foot position to kick the ball hard.

[0597] Input: Tracking data

[0598] Output: Analysis results (technical issues and areas for improvement)

[0599] Specific operation: The server analyzes the tracking data and detects problems with the way players kick and stop the ball, their positioning during the game, and their pass selection.

[0600] Step 6: Generating Advice

[0601] Description: Based on the identified technical challenges, the server uses a generative AI model to generate specific advice, such as "To kick the ball harder, position your stepping foot a little further forward."

[0602] Input: Analysis results

[0603] Output: Generated advice

[0604] Specific operation: The server inputs the analysis results into a generative AI model (e.g., GPT-3) to generate specific, actionable advice in natural language.

[0605] Step 7: Providing advice

[0606] Description: The server provides the generated advice to the user's terminal, which displays the received advice in an easy-to-read format for the user to review.

[0607] Input: Generated advice

[0608] Output: Advice displayed to the user

[0609] Specific operation: The server sends the generated advice to the user's device, which displays the advice through a dedicated application. The user can then check the advice in the application and use it in their next practice or game.

[0610] (Application example 1)

[0611] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0612] Conventional factory robots face the challenge of not being able to obtain specific feedback to improve the efficiency and quality of their operations. In particular, there is a lack of means to identify subtle improvements and technical issues in the robot's operation or object handling, and to provide effective advice. This makes it difficult for operators to quickly receive specific, data-based advice to optimize the robot's operation.

[0613] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0614] In this invention, the server includes means for receiving video of a task or process filmed by a user, means for analyzing the received video to track the robot's movements and the trajectories of objects, means for identifying technical issues and improvements to the robot based on the analysis results, means for generating specific advice based on the identified issues and improvements, and means for transmitting the generated advice to the user's terminal, thereby enabling the operator to quickly receive specific improvement advice regarding the robot's movements and processes and to optimize the movements efficiently and effectively.

[0615] "User" refers to the individual or institution that operates the robot in the factory and is responsible for monitoring and improving its operation.

[0616] "Video of work or processes" refers to video data that records specific operations or work processes performed by factory robots.

[0617] "Means for receiving" refers to the communication interface and software used to transfer and store videos taken by users to a server or cloud.

[0618] "Means for analyzing" refers to a system or algorithm that uses computer vision technology to decompose and analyze received video and extract information about movements, object positions, and trajectories.

[0619] "Robot movement or object trajectory" refers to the movement of a factory robot's arm or other moving part, and the path of an object moved or handled by that movement.

[0620] "Means for identifying technical issues and areas for improvement" refers to a system that uses analyzed data to identify factors that reduce the efficiency and quality of robot operations and determine areas that need improvement.

[0621] "Means for generating specific advice" refers to a system or algorithm that generates detailed recommendations or instructions in natural language to improve the robot's behavior based on identified issues and areas for improvement.

[0622] "Means for sending to a user's terminal" refers to the communications infrastructure and software for sending the generated advice to a digital device such as a user's smartphone, tablet, or computer.

[0623] This invention is a system that analyzes videos of factory robot operations taken by users, and identifies and provides technical issues and areas for improvement. Specific embodiments are described below.

[0624] System Program

[0625] This system works in cooperation with a server and a user terminal. The server implements functions for receiving video, analyzing, generating and sending advice. The user terminal is responsible for recording and playing video, and receiving and displaying advice.

[0626] Program processing explanation

[0627] 1. Recording and uploading videos

[0628] Users can record videos of factory robots' work and processes using a smartphone or tablet, and then upload the videos to a server using an application on the user's device.

[0629] 2. Video Analysis

[0630] The server analyzes the received video using computer vision technology such as OpenCV. Specifically, it divides the video into frames at a fixed rate and tracks the robot's movements and object trajectories in each frame. The results of this analysis are stored in a database.

[0631] 3. Generating Advice

[0632] The server identifies technical issues and areas for improvement based on the analyzed data. It then uses generative AI models and natural language generation technology to translate specific advice into text form. For example, the server could generate advice such as "You'll get a better finish if you slow down the welding speed a little" based on the analysis results.

[0633] 4. Providing advice

[0634] Finally, the generated advice is sent to the user's device, where the user can review the advice and implement the improvements in the next task or process.

[0635] Hardware and software used

[0636] Hardware

[0637] Smartphones, tablets, servers

[0638] software

[0639] Flask (Python web framework)

[0640] OpenCV (computer vision library)

[0641] Generative AI models (libraries for natural language generation)

[0642] Specific examples

[0643] Examples of work improvements

[0644] 1. A user records the welding work of a factory robot on a smartphone and uploads the video to a server via an application.

[0645] 2. The server divides the received video into frames, analyzes them, and tracks the welding operations.

[0646] 3. Based on the analysis results, the deterioration in quality that occurs when the welding speed is too fast is identified, and advice such as "If you slow down the welding speed a little, the finish will be better" is generated.

[0647] 4. This advice is sent to the user's device, and the user puts this advice into practice the next time they work.

[0648] Prompt Sentence Examples

[0649] "Please analyze the video of the robot's movements and point out areas for improvement."

[0650] Example: "The welding speed is too fast, resulting in a rough finish. Slowing the welding speed by 25% will improve the quality."

[0651] The above is a specific embodiment for carrying out the present invention, which allows users to efficiently and effectively optimize the operation of factory robots and improve the quality and efficiency of work.

[0652] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0653] Step 1:

[0654] Users take videos of factory robots' work and processes using their smartphones or tablets.

[0655] Input: Video of factory robot in action

[0656] Output: Recorded video file

[0657] In this step, the user actually takes a video and saves it on a digital device (smartphone or tablet).

[0658] Step 2:

[0659] The user uploads the video they have taken to the server through the application.

[0660] Input: Recorded video file

[0661] Output: Video file stored on the server

[0662] In this step, the user opens the application, selects a video file, and uploads it to the server. The server receives the uploaded video and saves it in storage.

[0663] Step 3:

[0664] The server divides the video it receives at a fixed frame rate and tracks the robot's movements and object trajectories in each frame.

[0665] Input: Video file stored on the server

[0666] Output: Frame-by-frame data showing tracking results

[0667] This is done using computer vision techniques such as OpenCV. The server splits the video file into frames and analyzes the robot's movements and object positions in each frame. The results of this analysis are stored in a database.

[0668] Step 4:

[0669] The server identifies technical issues and areas for improvement based on the analysis results stored in the database.

[0670] Input: Frame-by-frame data showing tracking results

[0671] Output: Identified technical issues and areas for improvement

[0672] In this step, the server evaluates the analysis data and identifies problems and areas that need improvement in the robot's operation.

[0673] Step 5:

[0674] Based on the identified issues and areas for improvement, the server uses generative AI models and natural language generation technology to generate specific advice.

[0675] Input: Identified technical issues and improvements

[0676] Output: Advice statement

[0677] Here, a generative AI model is used to automatically generate advice, such as specific instructions such as "You'll get a better finish if you slow down the welding speed a little."

[0678] Step 6:

[0679] The server transmits the generated advice to the user's terminal.

[0680] Input: Advice sentence

[0681] Output: Advice displayed on the user's terminal

[0682] In this step, the server sends the generated advice to the user's smartphone or tablet, which receives the advice and displays it on the application.

[0683] Through the above processing steps, the user can quickly receive specific feedback for optimizing the operation of the factory robot and put it into practice.

[0684] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0685] This invention is a system that supports players in improving their soccer skills by using videos of practice and matches taken by the user, and also provides more personalized advice by combining it with an emotion engine that recognizes the user's emotions. Specific embodiments will be described below based on the claims.

[0686] Recording and uploading videos

[0687] When a user wants to record a soccer practice or game, they use a device such as a smartphone or tablet. After shooting, the video file is saved on the device. The user opens the application, selects the video they shot, and uploads it to the server. This upload function sends the video to the server.

[0688] Video Analysis

[0689] When the server receives the video, it first splits it into frames at a fixed frame rate, generating individual frame images that are then analyzed. Computer vision technology is used to track the movements of players and the trajectory of the ball. The position and movement of each player and the ball in the video are identified, and the analysis results are stored in a database.

[0690] Generating Advice

[0691] The server uses the video analysis results to identify the player's technical issues and areas for improvement. This includes evaluating how the ball is kicked and stopped, as well as technical elements such as positioning and passing options during the game. Next, natural language generation technology is used to translate the analyzed data into written advice. For example, the generated advice might be, "To kick the ball more powerfully, it would be better to position your stepping foot a little further forward."

[0692] Providing advice

[0693] The generated advice is sent from the server to the user's device, which is designed to display the received advice in an easy-to-read format. The user can review the displayed advice and implement the points they need to improve in their next practice or game.

[0694] Introducing the Emotion Engine

[0695] This system incorporates an emotion engine that recognizes users' emotions. The emotion engine analyzes the user's tone of voice and facial expressions to identify their emotions. When a user uploads a video, the emotion engine analyzes the user's voice and facial expressions in real time and sends the emotional data to the server.

[0696] Use of Emotional Data

[0697] The server combines the emotion data sent from the emotion engine with the video analysis results to generate more personalized advice. For example, if the user is feeling discouraged, the server generates advice that includes mental support, such as "Try not to rush during your next practice, and take your steps slowly." The content and tone of the generated advice are adjusted according to the user's emotional state, and sent to the device in an appropriate format.

[0698] Specific examples

[0699] Example of kicking and stopping a ball using emotion recognition

[0700] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[0701] 2. The server receives the video, divides it into frames, and analyzes it.

[0702] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[0703] 4. When users express their thoughts vocally during upload, the emotion engine analyzes their tone of voice to identify their emotional state.

[0704] 5. The server combines the video analysis results with the emotional data and generates advice such as, "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing."

[0705] 6. The advice is sent to the device, and the user checks it in the application and puts it into practice the next time they practice.

[0706] Example of a match position using emotion recognition

[0707] 1. A user records their child's gameplay and uploads the video to a server.

[0708] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[0709] 3. When users express their emotions through facial expressions during upload, the emotion engine analyzes the facial expressions and identifies the emotional state.

[0710] 4. The server evaluates the player's position and the ball's movement and generates advice such as "In this scene, it would be better to move to the space on the left."

[0711] 5. Consider the user's emotional state and include encouragement such as, "Try this advice next time and play with confidence."

[0712] 6. The generated advice is sent to the terminal and confirmed by the user.

[0713] 7. The user can use this advice to improve their moves in their next game and aim for better play.

[0714] The above is a specific embodiment for carrying out the present invention. With this system, users can receive effective feedback and support that takes into consideration their emotions, enabling them to efficiently improve their soccer skills.

[0715] The processing flow will be explained below.

[0716] Step 1:

[0717] The user uses the device to record videos of practice and games.

[0718] Specifically, you can use the camera function of your smartphone or tablet to record the necessary scenes and save the video files on the device.

[0719] Step 2:

[0720] The user opens the application, selects the video they have taken, and uploads it to the server.

[0721] Specifically, the video is sent to the server by selecting a video file using the upload function and pressing the upload button.

[0722] Step 3:

[0723] The server receives the uploaded video.

[0724] Specifically, the received video file is stored in a predetermined storage area.

[0725] Step 4:

[0726] The server splits the video into chunks at a fixed frame rate.

[0727] Specifically, detailed analysis becomes possible by dividing a video into frames and generating individual frame images.

[0728] Step 5:

[0729] The server analyzes the player's movements and the ball's trajectory for each divided frame.

[0730] Specifically, computer vision technology is used to track the position, movement, and trajectory of players and the ball, and collect analytical data.

[0731] Step 6:

[0732] The server stores the analysis results in a database.

[0733] Specifically, the tracking results are recorded in a database and used for subsequent processing.

[0734] Step 7:

[0735] The server identifies technical issues and areas for improvement based on the analysis data.

[0736] Specifically, it runs algorithms that evaluate how the ball is kicked and stopped, as well as positioning and passing options during the game.

[0737] Step 8:

[0738] The server generates specific advice based on the identified issues and areas for improvement.

[0739] Specifically, natural language generation technology is used to generate sentences such as "To kick the ball harder, it is better to position your stepping foot a little further forward" from the analyzed data.

[0740] Step 9:

[0741] The device analyzes the user's tone of voice and facial expressions using an emotion engine.

[0742] Specifically, the system captures the user's voice and facial expressions in real time when they upload a video and identifies their emotions.

[0743] Step 10:

[0744] The server receives the emotion data sent from the emotion engine.

[0745] Specifically, the emotional data is combined with the analysis results and used as data to generate advice based on the user's emotions.

[0746] Step 11:

[0747] The server takes into account the user's emotional state and adjusts the content and tone of the advice.

[0748] Specifically, if the user is feeling discouraged, the system generates advice that includes mental support, such as "Try not to rush during the next practice session, and take your steps slowly."

[0749] Step 12:

[0750] The server transmits the generated advice to the user's terminal.

[0751] Specifically, the generated advice content is structured as a response in JSON format or the like and sent to the terminal.

[0752] Step 13:

[0753] The terminal displays the advice received from the server.

[0754] Specifically, the application will display the advice received in an easy-to-read format so that users can put it into practice in their next practice or game.

[0755] Specific examples

[0756] Examples of kicking and stopping a ball

[0757] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[0758] 2. The server receives the video, divides it into frames, and analyzes it.

[0759] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[0760] 4. When users express their thoughts vocally during upload, the emotion engine analyzes their tone of voice to identify their emotional state.

[0761] 5. The server combines the video analysis results with the emotional data and generates advice such as, "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing."

[0762] 6. The advice is sent to the device, and the user checks it in the application and puts it into practice the next time they practice.

[0763] Examples of positions during a match

[0764] 1. A user records their child's gameplay and uploads the video to a server.

[0765] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[0766] 3. When users express their emotions through facial expressions during upload, the emotion engine analyzes the facial expressions and identifies the emotional state.

[0767] 4. The server evaluates the player's position and the ball's movement and generates advice such as "In this scene, it would be better to move to the space on the left."

[0768] 5. Consider the user's emotional state and include encouragement such as, "Try this advice next time and play with confidence."

[0769] 6. The generated advice is sent to the terminal and confirmed by the user.

[0770] 7. The user can use this advice to improve their moves in their next game and aim for better play.

[0771] Example 2

[0772] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0773] Conventional feedback systems for soccer practice and matches are limited to analyzing the player's physical movements and the ball's trajectory, and are unable to provide personalized advice that takes into account the user's emotional state. Therefore, there is a need for comprehensive feedback that includes mental support for users in addition to technical improvements.

[0774] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0775] In this invention, the server includes means for receiving videos of practice and games filmed by the user, means for analyzing the received videos to track the player's movements and the ball's trajectory, means for identifying technical issues and areas for improvement for the player based on the analysis results, means for generating specific advice based on the identified issues and areas for improvement, means for transmitting the generated advice to the user's device, means for analyzing emotion data using an emotion engine that recognizes the user's emotions, and means for generating personalized advice that combines the emotion data and the video analysis results. This enables personalized feedback that includes not only technical feedback but also mental support based on the user's emotions.

[0776] "User" refers to an individual who uses this system to film and upload videos of soccer practice and matches and receive feedback.

[0777] "Means for receiving video" refers to a system or method that has the function of receiving video files sent from a user to a server.

[0778] "Means for analyzing video to track player movements and ball trajectory" refers to systems and methods for using computer vision technology to track the position and movement of players and the ball in video.

[0779] "Means for identifying technical issues and areas for improvement" refers to a system or method for identifying problems in a player's actions and areas for technical improvement based on the analysis results.

[0780] "Means for generating advice" refers to a system or method for generating specific advice in written form for identified technical issues or areas for improvement.

[0781] "Means for transmitting advice to the user's terminal" refers to a communication means for delivering the generated advice to the user's terminal.

[0782] "Emotion engine" refers to a system or method for analyzing a user's voice and facial expressions to identify their emotional state.

[0783] "Means for analyzing emotion data" refers to a system or method for analyzing emotion data obtained by an emotion engine.

[0784] "Means for generating personalized advice by combining emotional data and video analysis results" refers to a system or method for integrating a user's emotional state with video analysis results to generate individually customized advice.

[0785] This invention is a system that analyzes videos of soccer practice and matches taken by users to help players improve their skills. Furthermore, by combining this with an emotion engine that recognizes the user's emotions, it provides personalized advice. A specific embodiment of this system will be described.

[0786] Recording and uploading videos

[0787] A user uses a smartphone or tablet to record soccer practice or a game. After recording, the video file is saved on the device, and the user opens a dedicated application, selects the video, and uploads it to a server. During this process, the application on the device sends the video file to the server via the network, allowing the server to receive the video recorded by the user.

[0788] Video Analysis

[0789] When the server receives the video, it divides it into frames at a fixed frame rate. For example, it uses a computer vision library such as OpenCV. The server tracks the player's movements and the ball's trajectory from the images of each frame and stores these analysis results in a database.

[0790] Generating Advice

[0791] Based on the analysis results, the server identifies the player's technical issues and areas for improvement. Specifically, it uses a machine learning model to evaluate the player's movements. Next, it uses natural language generation technology (for example, a generative AI model such as GPT-4) to translate specific advice into written text. For example, it generates advice such as, "To kick the ball harder, it would be better to position your stepping foot a little further forward."

[0792] Providing advice

[0793] The generated advice is sent from the server to the user's device, which displays it in an easy-to-read format, allowing the user to review the advice and put it into practice in their next practice or game.

[0794] Introducing the Emotion Engine

[0795] The system incorporates an emotion engine that recognizes the user's emotions. The emotion engine analyzes the user's tone of voice and facial expressions to identify their emotions. For example, it uses emotion recognition software such as Microsoft Azure Cognitive Services. When a user uploads a video, the emotion engine analyzes it in real time and sends the results to a server.

[0796] Use of Emotional Data

[0797] The server combines the emotion data sent from the emotion engine with the video analysis results to generate more personalized advice. For example, if the user is feeling discouraged, the server generates advice that includes mental support, such as "Try not to rush during the next practice session, and take your steps slowly." The content and tone of this advice are also adjusted to suit the user's emotional state. The generated advice is then sent to the user's device.

[0798] Specific examples

[0799] Example of kicking and stopping a ball using emotion recognition

[0800] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[0801] 2. The server receives the video, divides it into frames, and analyzes it.

[0802] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[0803] 4. When users express their thoughts vocally while uploading a video, the emotion engine analyzes their tone of voice to identify their emotional state.

[0804] 5. The server combines the video analysis results with the emotional data and generates advice such as, "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing."

[0805] 6. The advice is sent to the device, and the user checks it in the application and puts it into practice the next time they practice.

[0806] Example of a match position using emotion recognition

[0807] 1. A user records their child's gameplay and uploads the video to a server.

[0808] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[0809] 3. When users express emotions through facial expressions while uploading videos, the emotion engine analyzes the facial expressions and identifies the emotional state.

[0810] 4. The server evaluates the player's position and the ball's movement and generates advice such as "In this scene, it would be better to move to the space on the left."

[0811] 5. Consider the user's emotional state and include encouragement such as, "Try this advice next time and play with confidence."

[0812] 6. The generated advice is sent to the terminal and confirmed by the user.

[0813] 7. The user can use this advice to improve their moves in their next game and aim for better play.

[0814] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0815] The flow of this system's program processing

[0816] Step 1:

[0817] Users record soccer practice and matches on their smartphones or tablets, and the video files are saved on the device.

[0818] Input: Video data (videos of practice and games)

[0819] Output: Video file saved on your device

[0820] Specific operation: The user launches the camera app on their smartphone and records practice or a game. After recording is complete, the video is saved on the device as "Game Video.mp4."

[0821] Step 2:

[0822] The user opens a dedicated application, selects the video they have taken, and uploads it to the server. The application on the device then sends the video file to the server via the network.

[0823] Input: Video files saved on the device

[0824] Output: Video file uploaded to the server

[0825] Specific operation: The user opens the application and taps the "Upload" button. The application selects "MatchVideo.mp4" on the device and sends it to the server.

[0826] Step 3:

[0827] The server divides the received video at a fixed frame rate to generate individual frame images, using computer vision technology to track the player's movements and the ball's trajectory, and stores the analysis results in a database.

[0828] Input: Video file uploaded to the server

[0829] Output: Analysis results (player and ball position data)

[0830] Specific operation: The server uses "OpenCV" to split the video into frames at 30 fps, extracts the player's position and the ball's position from each frame, and saves the analysis results in "Analysis Result DB".

[0831] Step 4:

[0832] The server analyzes the video to identify the player's technical issues and areas for improvement, then uses a generative AI model to write specific advice.

[0833] Input: Analysis results (player and ball position data)

[0834] Output: Written advice

[0835] Specific operation: The server uses a machine learning model to evaluate the player's actions, and then uses a generative AI model (e.g., GPT-4) to generate advice such as "To kick the ball harder, you should position your stepping foot a little further forward."

[0836] Step 5:

[0837] The server sends the generated advice in JSON format to the user's device, which displays the received advice in an easy-to-read format, and the user confirms the advice.

[0838] Input: Written advice

[0839] Output: Advice displayed on terminal

[0840] Specific operation: The server sends the generated advice in JSON format to the device. The device parses the received JSON data and displays it in the advice section of the application. The user confirms the displayed advice.

[0841] Step 6:

[0842] When a user expresses their feelings aloud while uploading a video, the emotion engine analyzes their tone of voice and facial expressions to identify their emotions, and the emotion data is sent to the server.

[0843] Input: User's emotional data (tone of voice and facial expression)

[0844] Output: Emotion data sent to the server

[0845] Specific operation: When uploading a video, the user says out loud, "Today didn't go well." The emotion engine analyzes the voice in real time and recognizes it as "disappointment." The emotion data is then sent to the server.

[0846] Step 7:

[0847] The server combines the emotion data with the video analysis results to generate personalized advice, with content and tone tailored to the user's emotional state, and sends it to the user's device.

[0848] Input: Analysis results (player and ball position data), emotion data

[0849] Output: Personalized advice

[0850] Specific operation: The server receives the emotion data "disappointment" and combines it with the video analysis results. The generative AI model inputs "advice for a disappointed user" as a prompt, generating advice such as "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing." The advice is then sent to the device and confirmed by the user.

[0851] (Application example 2)

[0852] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0853] Conventional soccer skill improvement support systems are limited to providing technical advice and do not provide personalized support based on the user's mental state or emotions. This can make it difficult to address the user's mental challenges and effectively improve their skills. In addition, there is a lack of a method for generating advice that utilizes emotional data.

[0854] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving videos of practice or games filmed by the user, means for analyzing the received videos to track the player's movements and the ball's trajectory, means for identifying technical issues and areas for improvement for the player based on the analysis results, means for generating specific advice based on the identified issues and areas for improvement, means for transmitting the generated advice to the user's terminal, means for generating emotional data by analyzing the user's tone of voice and facial expressions, and means for generating personalized advice including mental support based on the emotional data. This enables comprehensive support that is not only technical but also responsive to the user's emotional state.

[0855] 1. "User" refers to a person who uses the system to record videos of soccer practice and matches and receives analysis results and advice.

[0856] 2. "Means for receiving videos" refers to a function that allows users to upload videos of practice or games they have taken to the server and receive them.

[0857] 3. "Means for tracking player movements and ball trajectory" means technology for tracking player movements and the position and trajectory of the ball through video analysis.

[0858] 4. "Means for identifying technical issues and areas for improvement" is a function that uses the analysis results to identify problems and areas that need improvement in a player's soccer skills.

[0859] 5. "Means for generating specific advice" is a function for providing specific improvement methods and practice methods in written form based on technical issues and areas for improvement.

[0860] 6. "Means for sending advice to the user's device" refers to a function for sending the generated advice to the user's device, such as a smartphone or tablet, and making it viewable.

[0861] 7. "Means for generating emotional data by analyzing tone of voice and facial expressions" refers to technology that analyzes the tone of voice and facial expressions of users when they upload videos, and then digitizes the emotional state of the users based on that analysis.

[0862] 8. "Means for generating personalized advice including mental support" refers to a function that provides advice including support and encouragement according to the user's mental state based on emotional data.

[0863] 9. "Means for analyzing each frame" refers to a technique for dividing a video into parts at a fixed frame rate and performing analysis on each frame.

[0864] 10. "Means for saving in a database" refers to the function for recording and saving the analyzed results in a database.

[0865] 11. "Means for using natural language generation technology to translate advice into written form" means technology that uses a generative AI model to output analyzed data as advice in written form.

[0866] 12. "Computer vision technology" is a technology for tracking and recognizing specific objects by analyzing images and videos.

[0867] The present invention is a method for combining a system that uses videos of soccer practice and matches taken by users to help players improve their skills with an emotion engine that recognizes the emotions of users. A specific form of this method is described below.

[0868] Recording and uploading videos

[0869] First, the user records soccer practice or a match using a device such as a smartphone. The recorded video is saved on the device and then uploaded to a server via an application. At this time, the user can express their thoughts about the video by voice, and the emotion engine analyzes the voice data.

[0870] Video Analysis

[0871] The server splits the uploaded video at a fixed frame rate to generate individual frame images. Computer vision techniques (e.g., OpenCV) are used for video analysis to track player movements and ball trajectories. The analysis results are stored in a database.

[0872] Generating Advice

[0873] The server uses the video analysis to identify players' technical issues and areas for improvement, such as how they kick and stop the ball, their positioning and passing options during the game, etc. It then uses natural language generation technology (e.g., a generative AI model) to generate specific advice sentences from the analyzed data.

[0874] Introducing the Emotion Engine

[0875] When a user uploads a video, the emotion engine analyzes the user's tone of voice and facial expressions to generate emotional data, which is then sent to a server and combined with technical analysis results to generate more personalized advice.

[0876] Providing advice

[0877] The generated advice is sent from the server to the user's smartphone or other device, where the user can check it using the application and use it for their next practice or game.

[0878] Hardware and software used

[0879] Hardware: Smartphone (with camera and microphone), server

[0880] software:

[0881] OpenCV: A computer vision library for video analysis

[0882] Emotion Recognition Module: A custom module for emotion recognition (e.g., an extension of a Python sentiment analysis library)

[0883] Video Analysis Module: A custom module for player movement analysis

[0884] Specific examples

[0885] For example, consider a case where a user films their son's soccer practice with their smartphone and uploads the video to a server via an application. When uploading the video, if the user expresses their opinion by saying, "Today's practice didn't go well," the emotion engine analyzes the tone of the voice and recognizes that the user is disappointed. Based on the results of the video analysis and the emotion data, the server generates advice that includes mental support, such as, "Next time, try practicing your steps without rushing." The generated advice is sent to the user's smartphone and implemented at the next practice or game.

[0886] Prompt Sentence Examples

[0887] I analyzed a video of my son's soccer practice and uploaded it to a server. Please provide technical advice on kicking the ball while recognizing the user's emotions. If the user is feeling discouraged, please also provide advice that includes mental support.

[0888] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0889] Step 1:

[0890] A user films a soccer practice or game with their smartphone. The input here is a video of the soccer practice or game. The user saves the video to their device. The output of this step is a video file saved to their device.

[0891] Step 2:

[0892] The user opens the application, selects a saved video, and uploads it to the server. The input here is the video file, which is the output of step 1. The uploaded video is sent to and received by the server. The output of this step is the video file saved on the server.

[0893] Step 3:

[0894] The video received by the server is split at a constant frame rate. The input here is a video file stored on the server. Using computer vision techniques such as OpenCV, the video is split into frames and individual frame images are generated. The output of this step is a set of split frame images.

[0895] Step 4:

[0896] The server analyzes the frame images and tracks the player movements and ball trajectory. The input here is the set of frame images output from step 3. Using computer vision techniques, the server identifies the player's position and movements in each frame and stores them in a database. The output of this step is the analyzed data stored in the database.

[0897] Step 5:

[0898] When uploading a video, the user expresses their feelings by voice. The input here is the user's voice data. The emotion engine analyzes the tone and content of the user's voice and generates emotion data. The output of this step is the emotion data that is sent to the server.

[0899] Step 6:

[0900] The server uses the video analysis data and emotion data to identify the player's technical issues and areas for improvement and generate specific advice. The input here is the analysis data from step 4 and the emotion data from step 5. A generative AI model is used to convert the analysis data into advice in natural language. The output of this step is the generated text of the advice.

[0901] Step 7:

[0902] The server sends the generated advice to the user's device. The input here is the text of the advice from step 6. The advice is sent to the user's device, such as a smartphone, and displayed on the device. The output of this step is the advice received by the user.

[0903] Step 8:

[0904] The user checks the advice received through the application and puts it into practice in the next practice or match. The input here is the advice received in step 7. The user attempts to improve their technique based on the advice. The output of this step is the improvement of the user's soccer skills and the effect of mental support.

[0905] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0906] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0907] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0908] [Third embodiment]

[0909] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0910] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0911] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0912] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0913] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0914] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0915] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0916] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0917] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0918] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0919] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0920] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0921] The present invention provides a system for supporting players in improving their soccer skills by using videos of practice sessions and matches taken by users. Specific embodiments will be described below based on the claims.

[0922] Recording and uploading videos

[0923] When a user wants to record a soccer practice or game, they use a device such as a smartphone or tablet. After shooting, the video file is saved on the device. The user opens the application, selects the video they shot, and uploads it to the server. This upload function sends the video to the server.

[0924] Video Analysis

[0925] When the server receives the video, it first splits it into frames at a fixed frame rate, generating individual frame images that are then analyzed. Computer vision technology is used to track the movements of players and the trajectory of the ball. The position and movement of each player and the ball in the video are identified, and the analysis results are stored in a database.

[0926] Generating Advice

[0927] The server uses the video analysis results to identify the player's technical issues and areas for improvement. This includes evaluating how the player kicks and stops the ball, as well as technical elements such as their positioning and passing options during the game. Next, natural language generation technology is used to translate the analyzed data into written advice. For example, advice such as "To kick the ball more powerfully, it would be better to position your stepping foot a little further forward" is generated.

[0928] Providing advice

[0929] The generated advice is sent from the server to the user's device, which is designed to display the received advice in an easy-to-read format. The user can review the displayed advice and implement the points they need to improve in their next practice or game.

[0930] Specific examples

[0931] Examples of kicking and stopping a ball

[0932] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[0933] 2. The server receives the video, divides it into frames, and analyzes it.

[0934] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[0935] 4. The server generates advice such as "To kick the ball harder, it would be better to position your stepping foot a little further forward" and sends it to the device.

[0936] 5. The user checks the advice in the application and puts it into practice in the next practice session.

[0937] Examples of positions during a match

[0938] 1. A user records their child's gameplay and uploads the video to a server.

[0939] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[0940] 3. The server evaluates the player's position and the ball's movement and generates advice such as, "In this scene, there is open space on the left, so it would be better to move there and receive the ball."

[0941] 4. The generated advice is sent to the terminal and confirmed by the user.

[0942] 5. The user can use this advice to improve their moves in their next game and aim for better play.

[0943] The above is a specific embodiment for carrying out the present invention. This system allows users to receive effective feedback and efficiently improve their soccer skills.

[0944] The processing flow will be explained below.

[0945] Step 1:

[0946] The user uses the device to record videos of practice and games.

[0947] Specifically, you can use the camera function of your smartphone or tablet to record the necessary scenes and save the video files on the device.

[0948] Step 2:

[0949] The user opens the application, selects the video they have taken, and uploads it to the server.

[0950] Specifically, the video is sent to the server by selecting a video file using the upload function and pressing the upload button.

[0951] Step 3:

[0952] The server receives the uploaded video.

[0953] Specifically, the received video file is stored in a predetermined storage area.

[0954] Step 4:

[0955] The server splits the video into chunks at a fixed frame rate.

[0956] Specifically, detailed analysis becomes possible by dividing a video into frames and generating individual frame images.

[0957] Step 5:

[0958] The server analyzes the player's movements and the ball's trajectory for each divided frame.

[0959] Specifically, computer vision technology is used to track the position, movement, and trajectory of players and the ball, and collect analytical data.

[0960] Step 6:

[0961] The server stores the analysis results in a database.

[0962] Specifically, the tracking results are recorded in a database and used for subsequent processing.

[0963] Step 7:

[0964] The server identifies technical issues and areas for improvement based on the analysis data.

[0965] Specifically, it runs algorithms that evaluate how the ball is kicked and stopped, as well as positioning and passing options during the game.

[0966] Step 8:

[0967] The server generates specific advice based on the identified issues and areas for improvement.

[0968] Specifically, natural language generation technology is used to generate sentences such as "To kick the ball harder, it is better to position your stepping foot a little further forward" from the analyzed data.

[0969] Step 9:

[0970] The server transmits the generated advice to the user's terminal.

[0971] Specifically, the generated advice content is structured as a response in JSON format or the like and sent to the terminal.

[0972] Step 10:

[0973] The terminal displays the advice received from the server.

[0974] Specifically, the application will display the advice received in an easy-to-read format so that users can put it into practice in their next practice or game.

[0975] Example 1

[0976] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0977] Conventional soccer skill improvement systems have had difficulty effectively analyzing videos of practice and matches taken by users and providing specific technical advice. In particular, they lacked a means to accurately track player movements and ball trajectories and provide specific areas for improvement based on that information. This forced users to analyze the videos themselves, making it difficult to obtain effective feedback.

[0978] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0979] In this invention, the server includes means for receiving videos of practice or games filmed by a user, means for dividing the videos at a fixed frame rate, means for tracking the player's movements and the ball's trajectory for each divided frame, means for storing the tracking results in a database, means for identifying technical issues and areas for improvement for the player based on the stored data, means for generating specific advice using a generative AI model based on the identified issues and areas for improvement, and means for transmitting the generated advice to the user's device, thereby enabling the user to receive pulsating feedback and efficiently improve their skills.

[0980] A "user" is someone who takes videos of soccer practice or matches and uploads the videos to the system.

[0981] A "server" is a computer system that receives videos uploaded by users, analyzes and stores them, and generates and provides advice.

[0982] "Device" means the device used by the User to record video, upload it to the server, and receive and display analysis results and advice.

[0983] "Video" refers to a video file in which a user records a practice or game.

[0984] "Frame rate" is a numerical value that indicates the display speed of still images that make up a moving image.

[0985] A "frame" refers to an individual still image that makes up a video.

[0986] "Tracking" is the process of tracking the player's movements and the ball's trajectory in the video and identifying their position and movement.

[0987] A "database" is a data collection point where analysis results are stored and used for subsequent processing and advice generation.

[0988] A "generative AI model" is an artificial intelligence algorithm that automatically generates specific advice based on tracking data.

[0989] The "analysis results" are the data obtained after dividing the video into frames and tracking the movements of the players and the ball.

[0990] "Specific advice" refers to instructions and advice for players to improve their skills, generated by the generative AI model based on the analysis results.

[0991] The present invention is a system that analyzes videos of soccer practice and matches taken by users and supports players in improving their skills. Specific embodiments will be described below.

[0992] Recording and uploading videos

[0993] Users use devices such as smartphones and tablets to record soccer practice and matches. After recording, the video files are saved on the device. Users then open a dedicated application, select the video they have taken, and upload it to the server. This upload function sends the video file to the server.

[0994] Video Analysis

[0995] The server divides the received video file into chunks at a fixed frame rate. For example, a 30 fps video generates 30 frame images per second. The server then analyzes each frame using computer vision technology (e.g., OpenCV). Player movements and the ball's trajectory are tracked, and the position and movement of each player and ball in the video are identified. These analysis results are stored in a database.

[0996] Generating Advice

[0997] Based on the video analysis, the server identifies the player's technical issues and areas for improvement. This identification process includes evaluating the player's kicking and stopping technique, as well as their positioning and passing options during the match. Based on the identified issues and areas for improvement, the server uses a generative AI model (e.g., GPT-3) to generate specific advice. For example, the server might suggest, "To kick the ball more powerfully, you should position your stepping foot a little further forward."

[0998] Providing advice

[0999] The generated advice is sent from the server to the user's device, which then displays it in an easy-to-read format. The user can then review the advice and implement it to improve their game in the next practice or match.

[1000] Specific examples

[1001] A specific example is given below.

[1002] Examples of kicking and stopping a ball

[1003] 1. The user records their son's ball kicking practice on their smartphone and uploads the video to the server via the application.

[1004] 2. The server receives the video, divides it into frames, and analyzes it.

[1005] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[1006] 4. The server generates advice such as "To kick the ball harder, it would be better to position your stepping foot a little further forward" and sends it to the device.

[1007] 5. The user checks the advice in the application and puts it into practice in the next practice session.

[1008] Examples of positions during a match

[1009] 1. The user records their child's gameplay and uploads the video to the server.

[1010] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[1011] 3. The server evaluates the player's position and the ball's movement and generates advice such as, "In this scene, there is open space on the left, so it would be better to move there and receive the ball."

[1012] 4. The generated advice is sent to the terminal and confirmed by the user.

[1013] 5. The user can use this advice to improve their moves in their next game and aim for better play.

[1014] This system allows users to receive effective feedback and improve their soccer skills efficiently.

[1015] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1016] Step 1: Record and save the video

[1017] Description: Users record soccer practice and games using their smartphones or tablets. The recorded videos are saved on the device.

[1018] Input: Soccer practice and game scenes

[1019] Output: Recorded video file

[1020] How it works: The user opens the camera app, taps the record button to record a scene from practice or a game, and after recording is complete, the video file is automatically saved to the device's storage.

[1021] Step 2: Upload your video

[1022] Description: The user opens the dedicated application and uploads the video they have taken to the server. The device then sends the selected video file to the server.

[1023] Input: Saved video file

[1024] Output: Video file uploaded to the server

[1025] Specific operation: The user opens the application and taps the "Upload video" button. A file selection screen will appear, and the user can select a saved video file and perform the upload.

[1026] Step 3: Split the video into frames

[1027] Description: The server splits the received video file at a fixed frame rate, which converts the video into individual frame images.

[1028] Input: Uploaded video file

[1029] Output: Split frame images

[1030] Specific operation: The server analyzes the video at a fixed frame rate (for example, 30 fps) and generates and saves 30 frame images per second.

[1031] Step 4: Video Tracking Analysis

[1032] Description: The server uses computer vision technology to track player and ball movements frame by frame, determining player positions, actions, and ball trajectory.

[1033] Input: Segmented frame images

[1034] Output: Tracking data (player position, movement, ball trajectory, etc.)

[1035] Specific operation: The server uses OpenCV to recognize the player and ball for each frame image, record their positions and movements, and connect them to generate consistent tracking data.

[1036] Step 5: Identifying the technical challenges

[1037] Description: The server analyzes the player's technical issues and areas for improvement based on the tracking data, for example, identifying issues with the foot position to kick the ball hard.

[1038] Input: Tracking data

[1039] Output: Analysis results (technical issues and areas for improvement)

[1040] Specific operation: The server analyzes the tracking data and detects problems with the way players kick and stop the ball, their positioning during the game, and their pass selection.

[1041] Step 6: Generating Advice

[1042] Description: Based on the identified technical challenges, the server uses a generative AI model to generate specific advice, such as "To kick the ball harder, position your stepping foot a little further forward."

[1043] Input: Analysis results

[1044] Output: Generated advice

[1045] Specific operation: The server inputs the analysis results into a generative AI model (e.g., GPT-3) to generate specific, actionable advice in natural language.

[1046] Step 7: Providing advice

[1047] Description: The server provides the generated advice to the user's terminal, which displays the received advice in an easy-to-read format for the user to review.

[1048] Input: Generated advice

[1049] Output: Advice displayed to the user

[1050] Specific operation: The server sends the generated advice to the user's device, which displays the advice through a dedicated application. The user can then check the advice in the application and use it in their next practice or game.

[1051] (Application example 1)

[1052] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1053] Conventional factory robots face the challenge of not being able to obtain specific feedback to improve the efficiency and quality of their operations. In particular, there is a lack of means to identify subtle improvements and technical issues in the robot's operation or object handling, and to provide effective advice. This makes it difficult for operators to quickly receive specific, data-based advice to optimize the robot's operation.

[1054] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1055] In this invention, the server includes means for receiving video of a task or process filmed by a user, means for analyzing the received video to track the robot's movements and the trajectories of objects, means for identifying technical issues and improvements to the robot based on the analysis results, means for generating specific advice based on the identified issues and improvements, and means for transmitting the generated advice to the user's terminal, thereby enabling the operator to quickly receive specific improvement advice regarding the robot's movements and processes and to optimize the movements efficiently and effectively.

[1056] "User" refers to the individual or institution that operates the robot in the factory and is responsible for monitoring and improving its operation.

[1057] "Video of work or processes" refers to video data that records specific operations or work processes performed by factory robots.

[1058] "Means for receiving" refers to the communication interface and software used to transfer and store videos taken by users to a server or cloud.

[1059] "Means for analyzing" refers to a system or algorithm that uses computer vision technology to decompose and analyze received video and extract information about movements, object positions, and trajectories.

[1060] "Robot movement or object trajectory" refers to the movement of a factory robot's arm or other moving part, and the path of an object moved or handled by that movement.

[1061] "Means for identifying technical issues and areas for improvement" refers to a system that uses analyzed data to identify factors that reduce the efficiency and quality of robot operations and determine areas that need improvement.

[1062] "Means for generating specific advice" refers to a system or algorithm that generates detailed recommendations or instructions in natural language to improve the robot's behavior based on identified issues and areas for improvement.

[1063] "Means for sending to a user's terminal" refers to the communications infrastructure and software for sending the generated advice to a digital device such as a user's smartphone, tablet, or computer.

[1064] This invention is a system that analyzes videos of factory robot operations taken by users, and identifies and provides technical issues and areas for improvement. Specific embodiments are described below.

[1065] System Program

[1066] This system works in cooperation with a server and a user terminal. The server implements functions for receiving video, analyzing, generating and sending advice. The user terminal is responsible for recording and playing video, and receiving and displaying advice.

[1067] Program processing explanation

[1068] 1. Recording and uploading videos

[1069] Users can record videos of factory robots' work and processes using a smartphone or tablet, and then upload the videos to a server using an application on the user's device.

[1070] 2. Video Analysis

[1071] The server analyzes the received video using computer vision technology such as OpenCV. Specifically, it divides the video into frames at a fixed rate and tracks the robot's movements and object trajectories in each frame. The results of this analysis are stored in a database.

[1072] 3. Generating Advice

[1073] The server identifies technical issues and areas for improvement based on the analyzed data. It then uses generative AI models and natural language generation technology to translate specific advice into text form. For example, the server could generate advice such as "You'll get a better finish if you slow down the welding speed a little" based on the analysis results.

[1074] 4. Providing advice

[1075] Finally, the generated advice is sent to the user's device, where the user can review the advice and implement the improvements in the next task or process.

[1076] Hardware and software used

[1077] Hardware

[1078] Smartphones, tablets, servers

[1079] software

[1080] Flask (Python web framework)

[1081] OpenCV (computer vision library)

[1082] Generative AI models (libraries for natural language generation)

[1083] Specific examples

[1084] Examples of work improvements

[1085] 1. A user records the welding work of a factory robot on a smartphone and uploads the video to a server via an application.

[1086] 2. The server divides the received video into frames, analyzes them, and tracks the welding operations.

[1087] 3. Based on the analysis results, the deterioration in quality that occurs when the welding speed is too fast is identified, and advice such as "If you slow down the welding speed a little, the finish will be better" is generated.

[1088] 4. This advice is sent to the user's device, and the user puts this advice into practice the next time they work.

[1089] Prompt Sentence Examples

[1090] "Please analyze the video of the robot's movements and point out areas for improvement."

[1091] Example: "The welding speed is too fast, resulting in a rough finish. Slowing the welding speed by 25% will improve the quality."

[1092] The above is a specific embodiment for carrying out the present invention, which allows users to efficiently and effectively optimize the operation of factory robots and improve the quality and efficiency of work.

[1093] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1094] Step 1:

[1095] Users take videos of factory robots' work and processes using their smartphones or tablets.

[1096] Input: Video of factory robot in action

[1097] Output: Recorded video file

[1098] In this step, the user actually takes a video and saves it on a digital device (smartphone or tablet).

[1099] Step 2:

[1100] The user uploads the video they have taken to the server through the application.

[1101] Input: Recorded video file

[1102] Output: Video file stored on the server

[1103] In this step, the user opens the application, selects a video file, and uploads it to the server. The server receives the uploaded video and saves it in storage.

[1104] Step 3:

[1105] The server divides the video it receives at a fixed frame rate and tracks the robot's movements and object trajectories in each frame.

[1106] Input: Video file stored on the server

[1107] Output: Frame-by-frame data showing tracking results

[1108] This is done using computer vision techniques such as OpenCV. The server splits the video file into frames and analyzes the robot's movements and object positions in each frame. The results of this analysis are stored in a database.

[1109] Step 4:

[1110] The server identifies technical issues and areas for improvement based on the analysis results stored in the database.

[1111] Input: Frame-by-frame data showing tracking results

[1112] Output: Identified technical issues and areas for improvement

[1113] In this step, the server evaluates the analysis data and identifies problems and areas that need improvement in the robot's operation.

[1114] Step 5:

[1115] Based on the identified issues and areas for improvement, the server uses generative AI models and natural language generation technology to generate specific advice.

[1116] Input: Identified technical issues and improvements

[1117] Output: Advice statement

[1118] Here, a generative AI model is used to automatically generate advice, such as specific instructions such as "You'll get a better finish if you slow down the welding speed a little."

[1119] Step 6:

[1120] The server transmits the generated advice to the user's terminal.

[1121] Input: Advice sentence

[1122] Output: Advice displayed on the user's terminal

[1123] In this step, the server sends the generated advice to the user's smartphone or tablet, which receives the advice and displays it on the application.

[1124] Through the above processing steps, the user can quickly receive specific feedback for optimizing the operation of the factory robot and put it into practice.

[1125] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1126] This invention is a system that supports players in improving their soccer skills by using videos of practice and matches taken by the user, and also provides more personalized advice by combining it with an emotion engine that recognizes the user's emotions. Specific embodiments will be described below based on the claims.

[1127] Recording and uploading videos

[1128] When a user wants to record a soccer practice or game, they use a device such as a smartphone or tablet. After shooting, the video file is saved on the device. The user opens the application, selects the video they shot, and uploads it to the server. This upload function sends the video to the server.

[1129] Video Analysis

[1130] When the server receives the video, it first splits it into frames at a fixed frame rate, generating individual frame images that are then analyzed. Computer vision technology is used to track the movements of players and the trajectory of the ball. The position and movement of each player and the ball in the video are identified, and the analysis results are stored in a database.

[1131] Generating Advice

[1132] The server uses the video analysis results to identify the player's technical issues and areas for improvement. This includes evaluating how the ball is kicked and stopped, as well as technical elements such as positioning and passing options during the game. Next, natural language generation technology is used to translate the analyzed data into written advice. For example, the generated advice might be, "To kick the ball more powerfully, it would be better to position your stepping foot a little further forward."

[1133] Providing advice

[1134] The generated advice is sent from the server to the user's device, which is designed to display the received advice in an easy-to-read format. The user can review the displayed advice and implement the points they need to improve in their next practice or game.

[1135] Introducing the Emotion Engine

[1136] This system incorporates an emotion engine that recognizes users' emotions. The emotion engine analyzes the user's tone of voice and facial expressions to identify their emotions. When a user uploads a video, the emotion engine analyzes the user's voice and facial expressions in real time and sends the emotional data to the server.

[1137] Use of Emotional Data

[1138] The server combines the emotion data sent from the emotion engine with the video analysis results to generate more personalized advice. For example, if the user is feeling discouraged, the server generates advice that includes mental support, such as "Try not to rush during your next practice, and take your steps slowly." The content and tone of the generated advice are adjusted according to the user's emotional state, and sent to the device in an appropriate format.

[1139] Specific examples

[1140] Example of kicking and stopping a ball using emotion recognition

[1141] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[1142] 2. The server receives the video, divides it into frames, and analyzes it.

[1143] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[1144] 4. When users express their thoughts vocally during upload, the emotion engine analyzes their tone of voice to identify their emotional state.

[1145] 5. The server combines the video analysis results with the emotional data and generates advice such as, "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing."

[1146] 6. The advice is sent to the device, and the user checks it in the application and puts it into practice the next time they practice.

[1147] Example of a match position using emotion recognition

[1148] 1. A user records their child's gameplay and uploads the video to a server.

[1149] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[1150] 3. When users express their emotions through facial expressions during upload, the emotion engine analyzes the facial expressions and identifies the emotional state.

[1151] 4. The server evaluates the player's position and the ball's movement and generates advice such as "In this scene, it would be better to move to the space on the left."

[1152] 5. Consider the user's emotional state and include encouragement such as, "Try this advice next time and play with confidence."

[1153] 6. The generated advice is sent to the terminal and confirmed by the user.

[1154] 7. The user can use this advice to improve their moves in their next game and aim for better play.

[1155] The above is a specific embodiment for carrying out the present invention. With this system, users can receive effective feedback and support that takes into consideration their emotions, enabling them to efficiently improve their soccer skills.

[1156] The processing flow will be explained below.

[1157] Step 1:

[1158] The user uses the device to record videos of practice and games.

[1159] Specifically, you can use the camera function of your smartphone or tablet to record the necessary scenes and save the video files on the device.

[1160] Step 2:

[1161] The user opens the application, selects the video they have taken, and uploads it to the server.

[1162] Specifically, the video is sent to the server by selecting a video file using the upload function and pressing the upload button.

[1163] Step 3:

[1164] The server receives the uploaded video.

[1165] Specifically, the received video file is stored in a predetermined storage area.

[1166] Step 4:

[1167] The server splits the video into chunks at a fixed frame rate.

[1168] Specifically, detailed analysis becomes possible by dividing a video into frames and generating individual frame images.

[1169] Step 5:

[1170] The server analyzes the player's movements and the ball's trajectory for each divided frame.

[1171] Specifically, computer vision technology is used to track the position, movement, and trajectory of players and the ball, and collect analytical data.

[1172] Step 6:

[1173] The server stores the analysis results in a database.

[1174] Specifically, the tracking results are recorded in a database and used for subsequent processing.

[1175] Step 7:

[1176] The server identifies technical issues and areas for improvement based on the analysis data.

[1177] Specifically, it runs algorithms that evaluate how the ball is kicked and stopped, as well as positioning and passing options during the game.

[1178] Step 8:

[1179] The server generates specific advice based on the identified issues and areas for improvement.

[1180] Specifically, natural language generation technology is used to generate sentences such as "To kick the ball harder, it is better to position your stepping foot a little further forward" from the analyzed data.

[1181] Step 9:

[1182] The device analyzes the user's tone of voice and facial expressions using an emotion engine.

[1183] Specifically, the system captures the user's voice and facial expressions in real time when they upload a video and identifies their emotions.

[1184] Step 10:

[1185] The server receives the emotion data sent from the emotion engine.

[1186] Specifically, the emotional data is combined with the analysis results and used as data to generate advice based on the user's emotions.

[1187] Step 11:

[1188] The server takes into account the user's emotional state and adjusts the content and tone of the advice.

[1189] Specifically, if the user is feeling discouraged, the system generates advice that includes mental support, such as "Try not to rush during the next practice session, and take your steps slowly."

[1190] Step 12:

[1191] The server transmits the generated advice to the user's terminal.

[1192] Specifically, the generated advice content is structured as a response in JSON format or the like and sent to the terminal.

[1193] Step 13:

[1194] The terminal displays the advice received from the server.

[1195] Specifically, the application will display the advice received in an easy-to-read format so that users can put it into practice in their next practice or game.

[1196] Specific examples

[1197] Examples of kicking and stopping a ball

[1198] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[1199] 2. The server receives the video, divides it into frames, and analyzes it.

[1200] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[1201] 4. When users express their thoughts vocally during upload, the emotion engine analyzes their tone of voice to identify their emotional state.

[1202] 5. The server combines the video analysis results with the emotional data and generates advice such as, "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing."

[1203] 6. The advice is sent to the device, and the user checks it in the application and puts it into practice the next time they practice.

[1204] Examples of positions during a match

[1205] 1. A user records their child's gameplay and uploads the video to a server.

[1206] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[1207] 3. When users express their emotions through facial expressions during upload, the emotion engine analyzes the facial expressions and identifies the emotional state.

[1208] 4. The server evaluates the player's position and the ball's movement and generates advice such as "In this scene, it would be better to move to the space on the left."

[1209] 5. Consider the user's emotional state and include encouragement such as, "Try this advice next time and play with confidence."

[1210] 6. The generated advice is sent to the terminal and confirmed by the user.

[1211] 7. The user can use this advice to improve their moves in their next game and aim for better play.

[1212] Example 2

[1213] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1214] Conventional feedback systems for soccer practice and matches are limited to analyzing the player's physical movements and the ball's trajectory, and are unable to provide personalized advice that takes into account the user's emotional state. Therefore, there is a need for comprehensive feedback that includes mental support for users in addition to technical improvements.

[1215] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1216] In this invention, the server includes means for receiving videos of practice and games filmed by the user, means for analyzing the received videos to track the player's movements and the ball's trajectory, means for identifying technical issues and areas for improvement for the player based on the analysis results, means for generating specific advice based on the identified issues and areas for improvement, means for transmitting the generated advice to the user's device, means for analyzing emotion data using an emotion engine that recognizes the user's emotions, and means for generating personalized advice that combines the emotion data and the video analysis results. This enables personalized feedback that includes not only technical feedback but also mental support based on the user's emotions.

[1217] "User" refers to an individual who uses this system to film and upload videos of soccer practice and matches and receive feedback.

[1218] "Means for receiving video" refers to a system or method that has the function of receiving video files sent from a user to a server.

[1219] "Means for analyzing video to track player movements and ball trajectory" refers to systems and methods for using computer vision technology to track the position and movement of players and the ball in video.

[1220] "Means for identifying technical issues and areas for improvement" refers to a system or method for identifying problems in a player's actions and areas for technical improvement based on the analysis results.

[1221] "Means for generating advice" refers to a system or method for generating specific advice in written form for identified technical issues or areas for improvement.

[1222] "Means for transmitting advice to the user's terminal" refers to a communication means for delivering the generated advice to the user's terminal.

[1223] "Emotion engine" refers to a system or method for analyzing a user's voice and facial expressions to identify their emotional state.

[1224] "Means for analyzing emotion data" refers to a system or method for analyzing emotion data obtained by an emotion engine.

[1225] "Means for generating personalized advice by combining emotional data and video analysis results" refers to a system or method for integrating a user's emotional state with video analysis results to generate individually customized advice.

[1226] This invention is a system that analyzes videos of soccer practice and matches taken by users to help players improve their skills. Furthermore, by combining this with an emotion engine that recognizes the user's emotions, it provides personalized advice. A specific embodiment of this system will be described.

[1227] Recording and uploading videos

[1228] A user uses a smartphone or tablet to record soccer practice or a game. After recording, the video file is saved on the device, and the user opens a dedicated application, selects the video, and uploads it to a server. During this process, the application on the device sends the video file to the server via the network, allowing the server to receive the video recorded by the user.

[1229] Video Analysis

[1230] When the server receives the video, it divides it into frames at a fixed frame rate. For example, it uses a computer vision library such as OpenCV. The server tracks the player's movements and the ball's trajectory from the images of each frame and stores these analysis results in a database.

[1231] Generating Advice

[1232] Based on the analysis results, the server identifies the player's technical issues and areas for improvement. Specifically, it uses a machine learning model to evaluate the player's movements. Next, it uses natural language generation technology (for example, a generative AI model such as GPT-4) to translate specific advice into written text. For example, it generates advice such as, "To kick the ball harder, it would be better to position your stepping foot a little further forward."

[1233] Providing advice

[1234] The generated advice is sent from the server to the user's device, which displays it in an easy-to-read format, allowing the user to review the advice and put it into practice in their next practice or game.

[1235] Introducing the Emotion Engine

[1236] The system incorporates an emotion engine that recognizes the user's emotions. The emotion engine analyzes the user's tone of voice and facial expressions to identify their emotions. For example, it uses emotion recognition software such as Microsoft Azure Cognitive Services. When a user uploads a video, the emotion engine analyzes it in real time and sends the results to a server.

[1237] Use of Emotional Data

[1238] The server combines the emotion data sent from the emotion engine with the video analysis results to generate more personalized advice. For example, if the user is feeling discouraged, the server generates advice that includes mental support, such as "Try not to rush during the next practice session, and take your steps slowly." The content and tone of this advice are also adjusted to suit the user's emotional state. The generated advice is then sent to the user's device.

[1239] Specific examples

[1240] Example of kicking and stopping a ball using emotion recognition

[1241] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[1242] 2. The server receives the video, divides it into frames, and analyzes it.

[1243] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[1244] 4. When users express their thoughts vocally while uploading a video, the emotion engine analyzes their tone of voice to identify their emotional state.

[1245] 5. The server combines the video analysis results with the emotional data and generates advice such as, "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing."

[1246] 6. The advice is sent to the device, and the user checks it in the application and puts it into practice the next time they practice.

[1247] Example of a match position using emotion recognition

[1248] 1. A user records their child's gameplay and uploads the video to a server.

[1249] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[1250] 3. When users express emotions through facial expressions while uploading videos, the emotion engine analyzes the facial expressions and identifies the emotional state.

[1251] 4. The server evaluates the player's position and the ball's movement and generates advice such as "In this scene, it would be better to move to the space on the left."

[1252] 5. Consider the user's emotional state and include encouragement such as, "Try this advice next time and play with confidence."

[1253] 6. The generated advice is sent to the terminal and confirmed by the user.

[1254] 7. The user can use this advice to improve their moves in their next game and aim for better play.

[1255] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1256] The flow of this system's program processing

[1257] Step 1:

[1258] Users record soccer practice and matches on their smartphones or tablets, and the video files are saved on the device.

[1259] Input: Video data (videos of practice and games)

[1260] Output: Video file saved on your device

[1261] Specific operation: The user launches the camera app on their smartphone and records practice or a game. After recording is complete, the video is saved on the device as "Game Video.mp4."

[1262] Step 2:

[1263] The user opens a dedicated application, selects the video they have taken, and uploads it to the server. The application on the device then sends the video file to the server via the network.

[1264] Input: Video files saved on the device

[1265] Output: Video file uploaded to the server

[1266] Specific operation: The user opens the application and taps the "Upload" button. The application selects "MatchVideo.mp4" on the device and sends it to the server.

[1267] Step 3:

[1268] The server divides the received video at a fixed frame rate to generate individual frame images, using computer vision technology to track the player's movements and the ball's trajectory, and stores the analysis results in a database.

[1269] Input: Video file uploaded to the server

[1270] Output: Analysis results (player and ball position data)

[1271] Specific operation: The server uses "OpenCV" to split the video into frames at 30 fps, extracts the player's position and the ball's position from each frame, and saves the analysis results in "Analysis Result DB".

[1272] Step 4:

[1273] The server analyzes the video to identify the player's technical issues and areas for improvement, then uses a generative AI model to write specific advice.

[1274] Input: Analysis results (player and ball position data)

[1275] Output: Written advice

[1276] Specific operation: The server uses a machine learning model to evaluate the player's actions, and then uses a generative AI model (e.g., GPT-4) to generate advice such as "To kick the ball harder, you should position your stepping foot a little further forward."

[1277] Step 5:

[1278] The server sends the generated advice in JSON format to the user's device, which displays the received advice in an easy-to-read format, and the user confirms the advice.

[1279] Input: Written advice

[1280] Output: Advice displayed on terminal

[1281] Specific operation: The server sends the generated advice in JSON format to the device. The device parses the received JSON data and displays it in the advice section of the application. The user confirms the displayed advice.

[1282] Step 6:

[1283] When a user expresses their feelings aloud while uploading a video, the emotion engine analyzes their tone of voice and facial expressions to identify their emotions, and the emotion data is sent to the server.

[1284] Input: User's emotional data (tone of voice and facial expression)

[1285] Output: Emotion data sent to the server

[1286] Specific operation: When uploading a video, the user says out loud, "Today didn't go well." The emotion engine analyzes the voice in real time and recognizes it as "disappointment." The emotion data is then sent to the server.

[1287] Step 7:

[1288] The server combines the emotion data with the video analysis results to generate personalized advice, with content and tone tailored to the user's emotional state, and sends it to the user's device.

[1289] Input: Analysis results (player and ball position data), emotion data

[1290] Output: Personalized advice

[1291] Specific operation: The server receives the emotion data "disappointment" and combines it with the video analysis results. The generative AI model inputs "advice for a disappointed user" as a prompt, generating advice such as "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing." The advice is then sent to the device and confirmed by the user.

[1292] (Application example 2)

[1293] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1294] Conventional soccer skill improvement support systems are limited to providing technical advice and do not provide personalized support based on the user's mental state or emotions. This can make it difficult to address the user's mental challenges and effectively improve their skills. In addition, there is a lack of a method for generating advice that utilizes emotional data.

[1295] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving videos of practice or games filmed by the user, means for analyzing the received videos to track the player's movements and the ball's trajectory, means for identifying technical issues and areas for improvement for the player based on the analysis results, means for generating specific advice based on the identified issues and areas for improvement, means for transmitting the generated advice to the user's terminal, means for generating emotional data by analyzing the user's tone of voice and facial expressions, and means for generating personalized advice including mental support based on the emotional data. This enables comprehensive support that is not only technical but also responsive to the user's emotional state.

[1296] 1. "User" refers to a person who uses the system to record videos of soccer practice and matches and receives analysis results and advice.

[1297] 2. "Means for receiving videos" refers to a function that allows users to upload videos of practice or games they have taken to the server and receive them.

[1298] 3. "Means for tracking player movements and ball trajectory" means technology for tracking player movements and the position and trajectory of the ball through video analysis.

[1299] 4. "Means for identifying technical issues and areas for improvement" is a function that uses the analysis results to identify problems and areas that need improvement in a player's soccer skills.

[1300] 5. "Means for generating specific advice" is a function for providing specific improvement methods and practice methods in written form based on technical issues and areas for improvement.

[1301] 6. "Means for sending advice to the user's device" refers to a function for sending the generated advice to the user's device, such as a smartphone or tablet, and making it viewable.

[1302] 7. "Means for generating emotional data by analyzing tone of voice and facial expressions" refers to technology that analyzes the tone of voice and facial expressions of users when they upload videos, and then digitizes the emotional state of the users based on that analysis.

[1303] 8. "Means for generating personalized advice including mental support" refers to a function that provides advice including support and encouragement according to the user's mental state based on emotional data.

[1304] 9. "Means for analyzing each frame" refers to a technique for dividing a video into parts at a fixed frame rate and performing analysis on each frame.

[1305] 10. "Means for saving in a database" refers to the function for recording and saving the analyzed results in a database.

[1306] 11. "Means for using natural language generation technology to translate advice into written form" means technology that uses a generative AI model to output analyzed data as advice in written form.

[1307] 12. "Computer vision technology" is a technology for tracking and recognizing specific objects by analyzing images and videos.

[1308] The present invention is a method for combining a system that uses videos of soccer practice and matches taken by users to help players improve their skills with an emotion engine that recognizes the emotions of users. A specific form of this method is described below.

[1309] Recording and uploading videos

[1310] First, the user records soccer practice or a match using a device such as a smartphone. The recorded video is saved on the device and then uploaded to a server via an application. At this time, the user can express their thoughts about the video by voice, and the emotion engine analyzes the voice data.

[1311] Video Analysis

[1312] The server splits the uploaded video at a fixed frame rate to generate individual frame images. Computer vision techniques (e.g., OpenCV) are used for video analysis to track player movements and ball trajectories. The analysis results are stored in a database.

[1313] Generating Advice

[1314] The server uses the video analysis to identify players' technical issues and areas for improvement, such as how they kick and stop the ball, their positioning and passing options during the game, etc. It then uses natural language generation technology (e.g., a generative AI model) to generate specific advice sentences from the analyzed data.

[1315] Introducing the Emotion Engine

[1316] When a user uploads a video, the emotion engine analyzes the user's tone of voice and facial expressions to generate emotional data, which is then sent to a server and combined with technical analysis results to generate more personalized advice.

[1317] Providing advice

[1318] The generated advice is sent from the server to the user's smartphone or other device, where the user can check it using the application and use it for their next practice or game.

[1319] Hardware and software used

[1320] Hardware: Smartphone (with camera and microphone), server

[1321] software:

[1322] OpenCV: A computer vision library for video analysis

[1323] Emotion Recognition Module: A custom module for emotion recognition (e.g., an extension of a Python sentiment analysis library)

[1324] Video Analysis Module: A custom module for player movement analysis

[1325] Specific examples

[1326] For example, consider a case where a user films their son's soccer practice with their smartphone and uploads the video to a server via an application. When uploading the video, if the user expresses their opinion by saying, "Today's practice didn't go well," the emotion engine analyzes the tone of the voice and recognizes that the user is disappointed. Based on the results of the video analysis and the emotion data, the server generates advice that includes mental support, such as, "Next time, try practicing your steps without rushing." The generated advice is sent to the user's smartphone and implemented at the next practice or game.

[1327] Prompt Sentence Examples

[1328] I analyzed a video of my son's soccer practice and uploaded it to a server. Please provide technical advice on kicking the ball while recognizing the user's emotions. If the user is feeling discouraged, please also provide advice that includes mental support.

[1329] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1330] Step 1:

[1331] A user films a soccer practice or game with their smartphone. The input here is a video of the soccer practice or game. The user saves the video to their device. The output of this step is a video file saved to their device.

[1332] Step 2:

[1333] The user opens the application, selects a saved video, and uploads it to the server. The input here is the video file, which is the output of step 1. The uploaded video is sent to and received by the server. The output of this step is the video file saved on the server.

[1334] Step 3:

[1335] The video received by the server is split at a constant frame rate. The input here is a video file stored on the server. Using computer vision techniques such as OpenCV, the video is split into frames and individual frame images are generated. The output of this step is a set of split frame images.

[1336] Step 4:

[1337] The server analyzes the frame images and tracks the player movements and ball trajectory. The input here is the set of frame images output from step 3. Using computer vision techniques, the server identifies the player's position and movements in each frame and stores them in a database. The output of this step is the analyzed data stored in the database.

[1338] Step 5:

[1339] When uploading a video, the user expresses their feelings by voice. The input here is the user's voice data. The emotion engine analyzes the tone and content of the user's voice and generates emotion data. The output of this step is the emotion data that is sent to the server.

[1340] Step 6:

[1341] The server uses the video analysis data and emotion data to identify the player's technical issues and areas for improvement and generate specific advice. The input here is the analysis data from step 4 and the emotion data from step 5. A generative AI model is used to convert the analysis data into advice in natural language. The output of this step is the generated text of the advice.

[1342] Step 7:

[1343] The server sends the generated advice to the user's device. The input here is the text of the advice from step 6. The advice is sent to the user's device, such as a smartphone, and displayed on the device. The output of this step is the advice received by the user.

[1344] Step 8:

[1345] The user checks the advice received through the application and puts it into practice in the next practice or match. The input here is the advice received in step 7. The user attempts to improve their technique based on the advice. The output of this step is the improvement of the user's soccer skills and the effect of mental support.

[1346] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1347] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1348] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1349] [Fourth embodiment]

[1350] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1351] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1352] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1353] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1354] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1355] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1356] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1357] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1358] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1359] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1360] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1361] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1362] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1363] The present invention provides a system for supporting players in improving their soccer skills by using videos of practice sessions and matches taken by users. Specific embodiments will be described below based on the claims.

[1364] Recording and uploading videos

[1365] When a user wants to record a soccer practice or game, they use a device such as a smartphone or tablet. After shooting, the video file is saved on the device. The user opens the application, selects the video they shot, and uploads it to the server. This upload function sends the video to the server.

[1366] Video Analysis

[1367] When the server receives the video, it first splits it into frames at a fixed frame rate, generating individual frame images that are then analyzed. Computer vision technology is used to track the movements of players and the trajectory of the ball. The position and movement of each player and the ball in the video are identified, and the analysis results are stored in a database.

[1368] Generating Advice

[1369] The server uses the video analysis results to identify the player's technical issues and areas for improvement. This includes evaluating how the player kicks and stops the ball, as well as technical elements such as their positioning and passing options during the game. Next, natural language generation technology is used to translate the analyzed data into written advice. For example, advice such as "To kick the ball more powerfully, it would be better to position your stepping foot a little further forward" is generated.

[1370] Providing advice

[1371] The generated advice is sent from the server to the user's device, which is designed to display the received advice in an easy-to-read format. The user can review the displayed advice and implement the points they need to improve in their next practice or game.

[1372] Specific examples

[1373] Examples of kicking and stopping a ball

[1374] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[1375] 2. The server receives the video, divides it into frames, and analyzes it.

[1376] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[1377] 4. The server generates advice such as "To kick the ball harder, it would be better to position your stepping foot a little further forward" and sends it to the device.

[1378] 5. The user checks the advice in the application and puts it into practice in the next practice session.

[1379] Examples of positions during a match

[1380] 1. A user records their child's gameplay and uploads the video to a server.

[1381] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[1382] 3. The server evaluates the player's position and the ball's movement and generates advice such as, "In this scene, there is open space on the left, so it would be better to move there and receive the ball."

[1383] 4. The generated advice is sent to the terminal and confirmed by the user.

[1384] 5. The user can use this advice to improve their moves in their next game and aim for better play.

[1385] The above is a specific embodiment for carrying out the present invention. This system allows users to receive effective feedback and efficiently improve their soccer skills.

[1386] The processing flow will be explained below.

[1387] Step 1:

[1388] The user uses the device to record videos of practice and games.

[1389] Specifically, you can use the camera function of your smartphone or tablet to record the necessary scenes and save the video files on the device.

[1390] Step 2:

[1391] The user opens the application, selects the video they have taken, and uploads it to the server.

[1392] Specifically, the video is sent to the server by selecting a video file using the upload function and pressing the upload button.

[1393] Step 3:

[1394] The server receives the uploaded video.

[1395] Specifically, the received video file is stored in a predetermined storage area.

[1396] Step 4:

[1397] The server splits the video into chunks at a fixed frame rate.

[1398] Specifically, detailed analysis becomes possible by dividing a video into frames and generating individual frame images.

[1399] Step 5:

[1400] The server analyzes the player's movements and the ball's trajectory for each divided frame.

[1401] Specifically, computer vision technology is used to track the position, movement, and trajectory of players and the ball, and collect analytical data.

[1402] Step 6:

[1403] The server stores the analysis results in a database.

[1404] Specifically, the tracking results are recorded in a database and used for subsequent processing.

[1405] Step 7:

[1406] The server identifies technical issues and areas for improvement based on the analysis data.

[1407] Specifically, it runs algorithms that evaluate how the ball is kicked and stopped, as well as positioning and passing options during the game.

[1408] Step 8:

[1409] The server generates specific advice based on the identified issues and areas for improvement.

[1410] Specifically, natural language generation technology is used to generate sentences such as "To kick the ball harder, it is better to position your stepping foot a little further forward" from the analyzed data.

[1411] Step 9:

[1412] The server transmits the generated advice to the user's terminal.

[1413] Specifically, the generated advice content is structured as a response in JSON format or the like and sent to the terminal.

[1414] Step 10:

[1415] The terminal displays the advice received from the server.

[1416] Specifically, the application will display the advice received in an easy-to-read format so that users can put it into practice in their next practice or game.

[1417] Example 1

[1418] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1419] Conventional soccer skill improvement systems have had difficulty effectively analyzing videos of practice and matches taken by users and providing specific technical advice. In particular, they lacked a means to accurately track player movements and ball trajectories and provide specific areas for improvement based on that information. This forced users to analyze the videos themselves, making it difficult to obtain effective feedback.

[1420] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1421] In this invention, the server includes means for receiving videos of practice or games filmed by a user, means for dividing the videos at a fixed frame rate, means for tracking the player's movements and the ball's trajectory for each divided frame, means for storing the tracking results in a database, means for identifying technical issues and areas for improvement for the player based on the stored data, means for generating specific advice using a generative AI model based on the identified issues and areas for improvement, and means for transmitting the generated advice to the user's device, thereby enabling the user to receive pulsating feedback and efficiently improve their skills.

[1422] A "user" is someone who takes videos of soccer practice or matches and uploads the videos to the system.

[1423] A "server" is a computer system that receives videos uploaded by users, analyzes and stores them, and generates and provides advice.

[1424] "Device" means the device used by the User to record video, upload it to the server, and receive and display analysis results and advice.

[1425] "Video" refers to a video file in which a user records a practice or game.

[1426] "Frame rate" is a numerical value that indicates the display speed of still images that make up a moving image.

[1427] A "frame" refers to an individual still image that makes up a video.

[1428] "Tracking" is the process of tracking the player's movements and the ball's trajectory in the video and identifying their position and movement.

[1429] A "database" is a data collection point where analysis results are stored and used for subsequent processing and advice generation.

[1430] A "generative AI model" is an artificial intelligence algorithm that automatically generates specific advice based on tracking data.

[1431] The "analysis results" are the data obtained after dividing the video into frames and tracking the movements of the players and the ball.

[1432] "Specific advice" refers to instructions and advice for players to improve their skills, generated by the generative AI model based on the analysis results.

[1433] The present invention is a system that analyzes videos of soccer practice and matches taken by users and supports players in improving their skills. Specific embodiments will be described below.

[1434] Recording and uploading videos

[1435] Users use devices such as smartphones and tablets to record soccer practice and matches. After recording, the video files are saved on the device. Users then open a dedicated application, select the video they have taken, and upload it to the server. This upload function sends the video file to the server.

[1436] Video Analysis

[1437] The server divides the received video file into chunks at a fixed frame rate. For example, a 30 fps video generates 30 frame images per second. The server then analyzes each frame using computer vision technology (e.g., OpenCV). Player movements and the ball's trajectory are tracked, and the position and movement of each player and ball in the video are identified. These analysis results are stored in a database.

[1438] Generating Advice

[1439] Based on the video analysis, the server identifies the player's technical issues and areas for improvement. This identification process includes evaluating the player's kicking and stopping technique, as well as their positioning and passing options during the match. Based on the identified issues and areas for improvement, the server uses a generative AI model (e.g., GPT-3) to generate specific advice. For example, the server might suggest, "To kick the ball more powerfully, you should position your stepping foot a little further forward."

[1440] Providing advice

[1441] The generated advice is sent from the server to the user's device, which then displays it in an easy-to-read format. The user can then review the advice and implement it to improve their game in the next practice or match.

[1442] Specific examples

[1443] A specific example is given below.

[1444] Examples of kicking and stopping a ball

[1445] 1. The user records their son's ball kicking practice on their smartphone and uploads the video to the server via the application.

[1446] 2. The server receives the video, divides it into frames, and analyzes it.

[1447] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[1448] 4. The server generates advice such as "To kick the ball harder, it would be better to position your stepping foot a little further forward" and sends it to the device.

[1449] 5. The user checks the advice in the application and puts it into practice in the next practice session.

[1450] Examples of positions during a match

[1451] 1. The user records their child's gameplay and uploads the video to the server.

[1452] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[1453] 3. The server evaluates the player's position and the ball's movement and generates advice such as, "In this scene, there is open space on the left, so it would be better to move there and receive the ball."

[1454] 4. The generated advice is sent to the terminal and confirmed by the user.

[1455] 5. The user can use this advice to improve their moves in their next game and aim for better play.

[1456] This system allows users to receive effective feedback and improve their soccer skills efficiently.

[1457] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1458] Step 1: Record and save the video

[1459] Description: Users record soccer practice and games using their smartphones or tablets. The recorded videos are saved on the device.

[1460] Input: Soccer practice and game scenes

[1461] Output: Recorded video file

[1462] How it works: The user opens the camera app, taps the record button to record a scene from practice or a game, and after recording is complete, the video file is automatically saved to the device's storage.

[1463] Step 2: Upload your video

[1464] Description: The user opens the dedicated application and uploads the video they have taken to the server. The device then sends the selected video file to the server.

[1465] Input: Saved video file

[1466] Output: Video file uploaded to the server

[1467] Specific operation: The user opens the application and taps the "Upload video" button. A file selection screen will appear, and the user can select a saved video file and perform the upload.

[1468] Step 3: Split the video into frames

[1469] Description: The server splits the received video file at a fixed frame rate, which converts the video into individual frame images.

[1470] Input: Uploaded video file

[1471] Output: Split frame images

[1472] Specific operation: The server analyzes the video at a fixed frame rate (for example, 30 fps) and generates and saves 30 frame images per second.

[1473] Step 4: Video Tracking Analysis

[1474] Description: The server uses computer vision technology to track player and ball movements frame by frame, determining player positions, actions, and ball trajectory.

[1475] Input: Segmented frame images

[1476] Output: Tracking data (player position, movement, ball trajectory, etc.)

[1477] Specific operation: The server uses OpenCV to recognize the player and ball for each frame image, record their positions and movements, and connect them to generate consistent tracking data.

[1478] Step 5: Identifying the technical challenges

[1479] Description: The server analyzes the player's technical issues and areas for improvement based on the tracking data, for example, identifying issues with the foot position to kick the ball hard.

[1480] Input: Tracking data

[1481] Output: Analysis results (technical issues and areas for improvement)

[1482] Specific operation: The server analyzes the tracking data and detects problems with the way players kick and stop the ball, their positioning during the game, and their pass selection.

[1483] Step 6: Generating Advice

[1484] Description: Based on the identified technical challenges, the server uses a generative AI model to generate specific advice, such as "To kick the ball harder, position your stepping foot a little further forward."

[1485] Input: Analysis results

[1486] Output: Generated advice

[1487] Specific operation: The server inputs the analysis results into a generative AI model (e.g., GPT-3) to generate specific, actionable advice in natural language.

[1488] Step 7: Providing advice

[1489] Description: The server provides the generated advice to the user's terminal, which displays the received advice in an easy-to-read format for the user to review.

[1490] Input: Generated advice

[1491] Output: Advice displayed to the user

[1492] Specific operation: The server sends the generated advice to the user's device, which displays the advice through a dedicated application. The user can then check the advice in the application and use it in their next practice or game.

[1493] (Application example 1)

[1494] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1495] Conventional factory robots face the challenge of not being able to obtain specific feedback to improve the efficiency and quality of their operations. In particular, there is a lack of means to identify subtle improvements and technical issues in the robot's operation or object handling, and to provide effective advice. This makes it difficult for operators to quickly receive specific, data-based advice to optimize the robot's operation.

[1496] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1497] In this invention, the server includes means for receiving video of a task or process filmed by a user, means for analyzing the received video to track the robot's movements and the trajectories of objects, means for identifying technical issues and improvements to the robot based on the analysis results, means for generating specific advice based on the identified issues and improvements, and means for transmitting the generated advice to the user's terminal, thereby enabling the operator to quickly receive specific improvement advice regarding the robot's movements and processes and to optimize the movements efficiently and effectively.

[1498] "User" refers to the individual or institution that operates the robot in the factory and is responsible for monitoring and improving its operation.

[1499] "Video of work or processes" refers to video data that records specific operations or work processes performed by factory robots.

[1500] "Means for receiving" refers to the communication interface and software used to transfer and store videos taken by users to a server or cloud.

[1501] "Means for analyzing" refers to a system or algorithm that uses computer vision technology to decompose and analyze received video and extract information about movements, object positions, and trajectories.

[1502] "Robot movement or object trajectory" refers to the movement of a factory robot's arm or other moving part, and the path of an object moved or handled by that movement.

[1503] "Means for identifying technical issues and areas for improvement" refers to a system that uses analyzed data to identify factors that reduce the efficiency and quality of robot operations and determine areas that need improvement.

[1504] "Means for generating specific advice" refers to a system or algorithm that generates detailed recommendations or instructions in natural language to improve the robot's behavior based on identified issues and areas for improvement.

[1505] "Means for sending to a user's terminal" refers to the communications infrastructure and software for sending the generated advice to a digital device such as a user's smartphone, tablet, or computer.

[1506] This invention is a system that analyzes videos of factory robot operations taken by users, and identifies and provides technical issues and areas for improvement. Specific embodiments are described below.

[1507] System Program

[1508] This system works in cooperation with a server and a user terminal. The server implements functions for receiving video, analyzing, generating and sending advice. The user terminal is responsible for recording and playing video, and receiving and displaying advice.

[1509] Program processing explanation

[1510] 1. Recording and uploading videos

[1511] Users can record videos of factory robots' work and processes using a smartphone or tablet, and then upload the videos to a server using an application on the user's device.

[1512] 2. Video Analysis

[1513] The server analyzes the received video using computer vision technology such as OpenCV. Specifically, it divides the video into frames at a fixed rate and tracks the robot's movements and object trajectories in each frame. The results of this analysis are stored in a database.

[1514] 3. Generating Advice

[1515] The server identifies technical issues and areas for improvement based on the analyzed data. It then uses generative AI models and natural language generation technology to translate specific advice into text form. For example, the server could generate advice such as "You'll get a better finish if you slow down the welding speed a little" based on the analysis results.

[1516] 4. Providing advice

[1517] Finally, the generated advice is sent to the user's device, where the user can review the advice and implement the improvements in the next task or process.

[1518] Hardware and software used

[1519] Hardware

[1520] Smartphones, tablets, servers

[1521] software

[1522] Flask (Python web framework)

[1523] OpenCV (computer vision library)

[1524] Generative AI models (libraries for natural language generation)

[1525] Specific examples

[1526] Examples of work improvements

[1527] 1. A user records the welding work of a factory robot on a smartphone and uploads the video to a server via an application.

[1528] 2. The server divides the received video into frames, analyzes them, and tracks the welding operations.

[1529] 3. Based on the analysis results, the deterioration in quality that occurs when the welding speed is too fast is identified, and advice such as "If you slow down the welding speed a little, the finish will be better" is generated.

[1530] 4. This advice is sent to the user's device, and the user puts this advice into practice the next time they work.

[1531] Prompt Sentence Examples

[1532] "Please analyze the video of the robot's movements and point out areas for improvement."

[1533] Example: "The welding speed is too fast, resulting in a rough finish. Slowing the welding speed by 25% will improve the quality."

[1534] The above is a specific embodiment for carrying out the present invention, which allows users to efficiently and effectively optimize the operation of factory robots and improve the quality and efficiency of work.

[1535] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1536] Step 1:

[1537] Users take videos of factory robots' work and processes using their smartphones or tablets.

[1538] Input: Video of factory robot in action

[1539] Output: Recorded video file

[1540] In this step, the user actually takes a video and saves it on a digital device (smartphone or tablet).

[1541] Step 2:

[1542] The user uploads the video they have taken to the server through the application.

[1543] Input: Recorded video file

[1544] Output: Video file stored on the server

[1545] In this step, the user opens the application, selects a video file, and uploads it to the server. The server receives the uploaded video and saves it in storage.

[1546] Step 3:

[1547] The server divides the video it receives at a fixed frame rate and tracks the robot's movements and object trajectories in each frame.

[1548] Input: Video file stored on the server

[1549] Output: Frame-by-frame data showing tracking results

[1550] This is done using computer vision techniques such as OpenCV. The server splits the video file into frames and analyzes the robot's movements and object positions in each frame. The results of this analysis are stored in a database.

[1551] Step 4:

[1552] The server identifies technical issues and areas for improvement based on the analysis results stored in the database.

[1553] Input: Frame-by-frame data showing tracking results

[1554] Output: Identified technical issues and areas for improvement

[1555] In this step, the server evaluates the analysis data and identifies problems and areas that need improvement in the robot's operation.

[1556] Step 5:

[1557] Based on the identified issues and areas for improvement, the server uses generative AI models and natural language generation technology to generate specific advice.

[1558] Input: Identified technical issues and improvements

[1559] Output: Advice statement

[1560] Here, a generative AI model is used to automatically generate advice, such as specific instructions such as "You'll get a better finish if you slow down the welding speed a little."

[1561] Step 6:

[1562] The server transmits the generated advice to the user's terminal.

[1563] Input: Advice sentence

[1564] Output: Advice displayed on the user's terminal

[1565] In this step, the server sends the generated advice to the user's smartphone or tablet, which receives the advice and displays it on the application.

[1566] Through the above processing steps, the user can quickly receive specific feedback for optimizing the operation of the factory robot and put it into practice.

[1567] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1568] This invention is a system that supports players in improving their soccer skills by using videos of practice and matches taken by the user, and also provides more personalized advice by combining it with an emotion engine that recognizes the user's emotions. Specific embodiments will be described below based on the claims.

[1569] Recording and uploading videos

[1570] When a user wants to record a soccer practice or game, they use a device such as a smartphone or tablet. After shooting, the video file is saved on the device. The user opens the application, selects the video they shot, and uploads it to the server. This upload function sends the video to the server.

[1571] Video Analysis

[1572] When the server receives the video, it first splits it into frames at a fixed frame rate, generating individual frame images that are then analyzed. Computer vision technology is used to track the movements of players and the trajectory of the ball. The position and movement of each player and the ball in the video are identified, and the analysis results are stored in a database.

[1573] Generating Advice

[1574] The server uses the video analysis results to identify the player's technical issues and areas for improvement. This includes evaluating how the ball is kicked and stopped, as well as technical elements such as positioning and passing options during the game. Next, natural language generation technology is used to translate the analyzed data into written advice. For example, the generated advice might be, "To kick the ball more powerfully, it would be better to position your stepping foot a little further forward."

[1575] Providing advice

[1576] The generated advice is sent from the server to the user's device, which is designed to display the received advice in an easy-to-read format. The user can review the displayed advice and implement the points they need to improve in their next practice or game.

[1577] Introducing the Emotion Engine

[1578] This system incorporates an emotion engine that recognizes users' emotions. The emotion engine analyzes the user's tone of voice and facial expressions to identify their emotions. When a user uploads a video, the emotion engine analyzes the user's voice and facial expressions in real time and sends the emotional data to the server.

[1579] Use of Emotional Data

[1580] The server combines the emotion data sent from the emotion engine with the video analysis results to generate more personalized advice. For example, if the user is feeling discouraged, the server generates advice that includes mental support, such as "Try not to rush during your next practice, and take your steps slowly." The content and tone of the generated advice are adjusted according to the user's emotional state, and sent to the device in an appropriate format.

[1581] Specific examples

[1582] Example of kicking and stopping a ball using emotion recognition

[1583] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[1584] 2. The server receives the video, divides it into frames, and analyzes it.

[1585] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[1586] 4. When users express their thoughts vocally during upload, the emotion engine analyzes their tone of voice to identify their emotional state.

[1587] 5. The server combines the video analysis results with the emotional data and generates advice such as, "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing."

[1588] 6. The advice is sent to the device, and the user checks it in the application and puts it into practice the next time they practice.

[1589] Example of a match position using emotion recognition

[1590] 1. A user records their child's gameplay and uploads the video to a server.

[1591] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[1592] 3. When users express their emotions through facial expressions during upload, the emotion engine analyzes the facial expressions and identifies the emotional state.

[1593] 4. The server evaluates the player's position and the ball's movement and generates advice such as "In this scene, it would be better to move to the space on the left."

[1594] 5. Consider the user's emotional state and include encouragement such as, "Try this advice next time and play with confidence."

[1595] 6. The generated advice is sent to the terminal and confirmed by the user.

[1596] 7. The user can use this advice to improve their moves in their next game and aim for better play.

[1597] The above is a specific embodiment for carrying out the present invention. With this system, users can receive effective feedback and support that takes into consideration their emotions, enabling them to efficiently improve their soccer skills.

[1598] The processing flow will be explained below.

[1599] Step 1:

[1600] The user uses the device to record videos of practice and games.

[1601] Specifically, you can use the camera function of your smartphone or tablet to record the necessary scenes and save the video files on the device.

[1602] Step 2:

[1603] The user opens the application, selects the video they have taken, and uploads it to the server.

[1604] Specifically, the video is sent to the server by selecting a video file using the upload function and pressing the upload button.

[1605] Step 3:

[1606] The server receives the uploaded video.

[1607] Specifically, the received video file is stored in a predetermined storage area.

[1608] Step 4:

[1609] The server splits the video into chunks at a fixed frame rate.

[1610] Specifically, detailed analysis becomes possible by dividing a video into frames and generating individual frame images.

[1611] Step 5:

[1612] The server analyzes the player's movements and the ball's trajectory for each divided frame.

[1613] Specifically, computer vision technology is used to track the position, movement, and trajectory of players and the ball, and collect analytical data.

[1614] Step 6:

[1615] The server stores the analysis results in a database.

[1616] Specifically, the tracking results are recorded in a database and used for subsequent processing.

[1617] Step 7:

[1618] The server identifies technical issues and areas for improvement based on the analysis data.

[1619] Specifically, it runs algorithms that evaluate how the ball is kicked and stopped, as well as positioning and passing options during the game.

[1620] Step 8:

[1621] The server generates specific advice based on the identified issues and areas for improvement.

[1622] Specifically, natural language generation technology is used to generate sentences such as "To kick the ball harder, it is better to position your stepping foot a little further forward" from the analyzed data.

[1623] Step 9:

[1624] The device analyzes the user's tone of voice and facial expressions using an emotion engine.

[1625] Specifically, the system captures the user's voice and facial expressions in real time when they upload a video and identifies their emotions.

[1626] Step 10:

[1627] The server receives the emotion data sent from the emotion engine.

[1628] Specifically, the emotional data is combined with the analysis results and used as data to generate advice based on the user's emotions.

[1629] Step 11:

[1630] The server takes into account the user's emotional state and adjusts the content and tone of the advice.

[1631] Specifically, if the user is feeling discouraged, the system generates advice that includes mental support, such as "Try not to rush during the next practice session, and take your steps slowly."

[1632] Step 12:

[1633] The server transmits the generated advice to the user's terminal.

[1634] Specifically, the generated advice content is structured as a response in JSON format or the like and sent to the terminal.

[1635] Step 13:

[1636] The terminal displays the advice received from the server.

[1637] Specifically, the application will display the advice received in an easy-to-read format so that users can put it into practice in their next practice or game.

[1638] Specific examples

[1639] Examples of kicking and stopping a ball

[1640] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[1641] 2. The server receives the video, divides it into frames, and analyzes it.

[1642] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[1643] 4. When users express their thoughts vocally during upload, the emotion engine analyzes their tone of voice to identify their emotional state.

[1644] 5. The server combines the video analysis results with the emotional data and generates advice such as, "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing."

[1645] 6. The advice is sent to the device, and the user checks it in the application and puts it into practice the next time they practice.

[1646] Examples of positions during a match

[1647] 1. A user records their child's gameplay and uploads the video to a server.

[1648] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[1649] 3. When users express their emotions through facial expressions during upload, the emotion engine analyzes the facial expressions and identifies the emotional state.

[1650] 4. The server evaluates the player's position and the ball's movement and generates advice such as "In this scene, it would be better to move to the space on the left."

[1651] 5. Consider the user's emotional state and include encouragement such as, "Try this advice next time and play with confidence."

[1652] 6. The generated advice is sent to the terminal and confirmed by the user.

[1653] 7. The user can use this advice to improve their moves in their next game and aim for better play.

[1654] Example 2

[1655] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1656] Conventional feedback systems for soccer practice and matches are limited to analyzing the player's physical movements and the ball's trajectory, and are unable to provide personalized advice that takes into account the user's emotional state. Therefore, there is a need for comprehensive feedback that includes mental support for users in addition to technical improvements.

[1657] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1658] In this invention, the server includes means for receiving videos of practice and games filmed by the user, means for analyzing the received videos to track the player's movements and the ball's trajectory, means for identifying technical issues and areas for improvement for the player based on the analysis results, means for generating specific advice based on the identified issues and areas for improvement, means for transmitting the generated advice to the user's device, means for analyzing emotion data using an emotion engine that recognizes the user's emotions, and means for generating personalized advice that combines the emotion data and the video analysis results. This enables personalized feedback that includes not only technical feedback but also mental support based on the user's emotions.

[1659] "User" refers to an individual who uses this system to film and upload videos of soccer practice and matches and receive feedback.

[1660] "Means for receiving video" refers to a system or method that has the function of receiving video files sent from a user to a server.

[1661] "Means for analyzing video to track player movements and ball trajectory" refers to systems and methods for using computer vision technology to track the position and movement of players and the ball in video.

[1662] "Means for identifying technical issues and areas for improvement" refers to a system or method for identifying problems in a player's actions and areas for technical improvement based on the analysis results.

[1663] "Means for generating advice" refers to a system or method for generating specific advice in written form for identified technical issues or areas for improvement.

[1664] "Means for transmitting advice to the user's terminal" refers to a communication means for delivering the generated advice to the user's terminal.

[1665] "Emotion engine" refers to a system or method for analyzing a user's voice and facial expressions to identify their emotional state.

[1666] "Means for analyzing emotion data" refers to a system or method for analyzing emotion data obtained by an emotion engine.

[1667] "Means for generating personalized advice by combining emotional data and video analysis results" refers to a system or method for integrating a user's emotional state with video analysis results to generate individually customized advice.

[1668] This invention is a system that analyzes videos of soccer practice and matches taken by users to help players improve their skills. Furthermore, by combining this with an emotion engine that recognizes the user's emotions, it provides personalized advice. A specific embodiment of this system will be described.

[1669] Recording and uploading videos

[1670] A user uses a smartphone or tablet to record soccer practice or a game. After recording, the video file is saved on the device, and the user opens a dedicated application, selects the video, and uploads it to a server. During this process, the application on the device sends the video file to the server via the network, allowing the server to receive the video recorded by the user.

[1671] Video Analysis

[1672] When the server receives the video, it divides it into frames at a fixed frame rate. For example, it uses a computer vision library such as OpenCV. The server tracks the player's movements and the ball's trajectory from the images of each frame and stores these analysis results in a database.

[1673] Generating Advice

[1674] Based on the analysis results, the server identifies the player's technical issues and areas for improvement. Specifically, it uses a machine learning model to evaluate the player's movements. Next, it uses natural language generation technology (for example, a generative AI model such as GPT-4) to translate specific advice into written text. For example, it generates advice such as, "To kick the ball harder, it would be better to position your stepping foot a little further forward."

[1675] Providing advice

[1676] The generated advice is sent from the server to the user's device, which displays it in an easy-to-read format, allowing the user to review the advice and put it into practice in their next practice or game.

[1677] Introducing the Emotion Engine

[1678] The system incorporates an emotion engine that recognizes the user's emotions. The emotion engine analyzes the user's tone of voice and facial expressions to identify their emotions. For example, it uses emotion recognition software such as Microsoft Azure Cognitive Services. When a user uploads a video, the emotion engine analyzes it in real time and sends the results to a server.

[1679] Use of Emotional Data

[1680] The server combines the emotion data sent from the emotion engine with the video analysis results to generate more personalized advice. For example, if the user is feeling discouraged, the server generates advice that includes mental support, such as "Try not to rush during the next practice session, and take your steps slowly." The content and tone of this advice are also adjusted to suit the user's emotional state. The generated advice is then sent to the user's device.

[1681] Specific examples

[1682] Example of kicking and stopping a ball using emotion recognition

[1683] 1. A user records their son's ball practice on their smartphone and uploads the video to the server via the application.

[1684] 2. The server receives the video, divides it into frames, and analyzes it.

[1685] 3. Using computer vision technology, we identify the angle of the foot and the position of the foot at the moment of kicking the ball.

[1686] 4. When users express their thoughts vocally while uploading a video, the emotion engine analyzes their tone of voice to identify their emotional state.

[1687] 5. The server combines the video analysis results with the emotional data and generates advice such as, "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing."

[1688] 6. The advice is sent to the device, and the user checks it in the application and puts it into practice the next time they practice.

[1689] Example of a match position using emotion recognition

[1690] 1. A user records their child's gameplay and uploads the video to a server.

[1691] 2. The server divides the video into frames and analyzes the player's movements and positioning.

[1692] 3. When users express emotions through facial expressions while uploading videos, the emotion engine analyzes the facial expressions and identifies the emotional state.

[1693] 4. The server evaluates the player's position and the ball's movement and generates advice such as "In this scene, it would be better to move to the space on the left."

[1694] 5. Consider the user's emotional state and include encouragement such as, "Try this advice next time and play with confidence."

[1695] 6. The generated advice is sent to the terminal and confirmed by the user.

[1696] 7. The user can use this advice to improve their moves in their next game and aim for better play.

[1697] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1698] The flow of this system's program processing

[1699] Step 1:

[1700] Users record soccer practice and matches on their smartphones or tablets, and the video files are saved on the device.

[1701] Input: Video data (videos of practice and games)

[1702] Output: Video file saved on your device

[1703] Specific operation: The user launches the camera app on their smartphone and records practice or a game. After recording is complete, the video is saved on the device as "Game Video.mp4."

[1704] Step 2:

[1705] The user opens a dedicated application, selects the video they have taken, and uploads it to the server. The application on the device then sends the video file to the server via the network.

[1706] Input: Video files saved on the device

[1707] Output: Video file uploaded to the server

[1708] Specific operation: The user opens the application and taps the "Upload" button. The application selects "MatchVideo.mp4" on the device and sends it to the server.

[1709] Step 3:

[1710] The server divides the received video at a fixed frame rate to generate individual frame images, using computer vision technology to track the player's movements and the ball's trajectory, and stores the analysis results in a database.

[1711] Input: Video file uploaded to the server

[1712] Output: Analysis results (player and ball position data)

[1713] Specific operation: The server uses "OpenCV" to split the video into frames at 30 fps, extracts the player's position and the ball's position from each frame, and saves the analysis results in "Analysis Result DB".

[1714] Step 4:

[1715] The server analyzes the video to identify the player's technical issues and areas for improvement, then uses a generative AI model to write specific advice.

[1716] Input: Analysis results (player and ball position data)

[1717] Output: Written advice

[1718] Specific operation: The server uses a machine learning model to evaluate the player's actions, and then uses a generative AI model (e.g., GPT-4) to generate advice such as "To kick the ball harder, you should position your stepping foot a little further forward."

[1719] Step 5:

[1720] The server sends the generated advice in JSON format to the user's device, which displays the received advice in an easy-to-read format, and the user confirms the advice.

[1721] Input: Written advice

[1722] Output: Advice displayed on terminal

[1723] Specific operation: The server sends the generated advice in JSON format to the device. The device parses the received JSON data and displays it in the advice section of the application. The user confirms the displayed advice.

[1724] Step 6:

[1725] When a user expresses their feelings aloud while uploading a video, the emotion engine analyzes their tone of voice and facial expressions to identify their emotions, and the emotion data is sent to the server.

[1726] Input: User's emotional data (tone of voice and facial expression)

[1727] Output: Emotion data sent to the server

[1728] Specific operation: When uploading a video, the user says out loud, "Today didn't go well." The emotion engine analyzes the voice in real time and recognizes it as "disappointment." The emotion data is then sent to the server.

[1729] Step 7:

[1730] The server combines the emotion data with the video analysis results to generate personalized advice, with content and tone tailored to the user's emotional state, and sends it to the user's device.

[1731] Input: Analysis results (player and ball position data), emotion data

[1732] Output: Personalized advice

[1733] Specific operation: The server receives the emotion data "disappointment" and combines it with the video analysis results. The generative AI model inputs "advice for a disappointed user" as a prompt, generating advice such as "It would be better to position your stepping foot a little further forward, but don't rush, just keep practicing." The advice is then sent to the device and confirmed by the user.

[1734] (Application example 2)

[1735] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1736] Conventional soccer skill improvement support systems are limited to providing technical advice and do not provide personalized support based on the user's mental state or emotions. This can make it difficult to address the user's mental challenges and effectively improve their skills. In addition, there is a lack of a method for generating advice that utilizes emotional data.

[1737] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving videos of practice or games filmed by the user, means for analyzing the received videos to track the player's movements and the ball's trajectory, means for identifying technical issues and areas for improvement for the player based on the analysis results, means for generating specific advice based on the identified issues and areas for improvement, means for transmitting the generated advice to the user's terminal, means for generating emotional data by analyzing the user's tone of voice and facial expressions, and means for generating personalized advice including mental support based on the emotional data. This enables comprehensive support that is not only technical but also responsive to the user's emotional state.

[1738] 1. "User" refers to a person who uses the system to record videos of soccer practice and matches and receives analysis results and advice.

[1739] 2. "Means for receiving videos" refers to a function that allows users to upload videos of practice or games they have taken to the server and receive them.

[1740] 3. "Means for tracking player movements and ball trajectory" means technology for tracking player movements and the position and trajectory of the ball through video analysis.

[1741] 4. "Means for identifying technical issues and areas for improvement" is a function that uses the analysis results to identify problems and areas that need improvement in a player's soccer skills.

[1742] 5. "Means for generating specific advice" is a function for providing specific improvement methods and practice methods in written form based on technical issues and areas for improvement.

[1743] 6. "Means for sending advice to the user's device" refers to a function for sending the generated advice to the user's device, such as a smartphone or tablet, and making it viewable.

[1744] 7. "Means for generating emotional data by analyzing tone of voice and facial expressions" refers to technology that analyzes the tone of voice and facial expressions of users when they upload videos, and then digitizes the emotional state of the users based on that analysis.

[1745] 8. "Means for generating personalized advice including mental support" refers to a function that provides advice including support and encouragement according to the user's mental state based on emotional data.

[1746] 9. "Means for analyzing each frame" refers to a technique for dividing a video into parts at a fixed frame rate and performing analysis on each frame.

[1747] 10. "Means for saving in a database" refers to the function for recording and saving the analyzed results in a database.

[1748] 11. "Means for using natural language generation technology to translate advice into written form" means technology that uses a generative AI model to output analyzed data as advice in written form.

[1749] 12. "Computer vision technology" is a technology for tracking and recognizing specific objects by analyzing images and videos.

[1750] The present invention is a method for combining a system that uses videos of soccer practice and matches taken by users to help players improve their skills with an emotion engine that recognizes the emotions of users. A specific form of this method is described below.

[1751] Recording and uploading videos

[1752] First, the user records soccer practice or a match using a device such as a smartphone. The recorded video is saved on the device and then uploaded to a server via an application. At this time, the user can express their thoughts about the video by voice, and the emotion engine analyzes the voice data.

[1753] Video Analysis

[1754] The server splits the uploaded video at a fixed frame rate to generate individual frame images. Computer vision techniques (e.g., OpenCV) are used for video analysis to track player movements and ball trajectories. The analysis results are stored in a database.

[1755] Generating Advice

[1756] The server uses the video analysis to identify players' technical issues and areas for improvement, such as how they kick and stop the ball, their positioning and passing options during the game, etc. It then uses natural language generation technology (e.g., a generative AI model) to generate specific advice sentences from the analyzed data.

[1757] Introducing the Emotion Engine

[1758] When a user uploads a video, the emotion engine analyzes the user's tone of voice and facial expressions to generate emotional data, which is then sent to a server and combined with technical analysis results to generate more personalized advice.

[1759] Providing advice

[1760] The generated advice is sent from the server to the user's smartphone or other device, where the user can check it using the application and use it for their next practice or game.

[1761] Hardware and software used

[1762] Hardware: Smartphone (with camera and microphone), server

[1763] software:

[1764] OpenCV: A computer vision library for video analysis

[1765] Emotion Recognition Module: A custom module for emotion recognition (e.g., an extension of a Python sentiment analysis library)

[1766] Video Analysis Module: A custom module for player movement analysis

[1767] Specific examples

[1768] For example, consider a case where a user films their son's soccer practice with their smartphone and uploads the video to a server via an application. When uploading the video, if the user expresses their opinion by saying, "Today's practice didn't go well," the emotion engine analyzes the tone of the voice and recognizes that the user is disappointed. Based on the results of the video analysis and the emotion data, the server generates advice that includes mental support, such as, "Next time, try practicing your steps without rushing." The generated advice is sent to the user's smartphone and implemented at the next practice or game.

[1769] Prompt Sentence Examples

[1770] I analyzed a video of my son's soccer practice and uploaded it to a server. Please provide technical advice on kicking the ball while recognizing the user's emotions. If the user is feeling discouraged, please also provide advice that includes mental support.

[1771] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1772] Step 1:

[1773] A user films a soccer practice or game with their smartphone. The input here is a video of the soccer practice or game. The user saves the video to their device. The output of this step is a video file saved to their device.

[1774] Step 2:

[1775] The user opens the application, selects a saved video, and uploads it to the server. The input here is the video file, which is the output of step 1. The uploaded video is sent to and received by the server. The output of this step is the video file saved on the server.

[1776] Step 3:

[1777] The video received by the server is split at a constant frame rate. The input here is a video file stored on the server. Using computer vision techniques such as OpenCV, the video is split into frames and individual frame images are generated. The output of this step is a set of split frame images.

[1778] Step 4:

[1779] The server analyzes the frame images and tracks the player movements and ball trajectory. The input here is the set of frame images output from step 3. Using computer vision techniques, the server identifies the player's position and movements in each frame and stores them in a database. The output of this step is the analyzed data stored in the database.

[1780] Step 5:

[1781] When uploading a video, the user expresses their feelings by voice. The input here is the user's voice data. The emotion engine analyzes the tone and content of the user's voice and generates emotion data. The output of this step is the emotion data that is sent to the server.

[1782] Step 6:

[1783] The server uses the video analysis data and emotion data to identify the player's technical issues and areas for improvement and generate specific advice. The input here is the analysis data from step 4 and the emotion data from step 5. A generative AI model is used to convert the analysis data into advice in natural language. The output of this step is the generated text of the advice.

[1784] Step 7:

[1785] The server sends the generated advice to the user's device. The input here is the text of the advice from step 6. The advice is sent to the user's device, such as a smartphone, and displayed on the device. The output of this step is the advice received by the user.

[1786] Step 8:

[1787] The user checks the advice received through the application and puts it into practice in the next practice or match. The input here is the advice received in step 7. The user attempts to improve their technique based on the advice. The output of this step is the improvement of the user's soccer skills and the effect of mental support.

[1788] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1789] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1790] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1791] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1792] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1793] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1794] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1795] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1796] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1797] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1798] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1799] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1800] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1801] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1802] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1803] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1804] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1805] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1806] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1807] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1808] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1809] The following is further disclosed regarding the above embodiment.

[1810] (Claim 1)

[1811] A means for receiving videos of practice or games taken by a user;

[1812] A means of analyzing the received video to track player movements and ball trajectory,

[1813] A means to identify technical issues and areas for improvement for players based on the analysis results, and

[1814] A means of generating specific advice based on identified issues and areas for improvement;

[1815] The system includes means for transmitting the generated advice to a user terminal.

[1816] (Claim 2)

[1817] A method for dividing videos of practice and matches taken by users into a certain frame rate and analyzing each frame.

[1818] a means for storing the analysis results in a database;

[1819] The system of claim 1, further comprising means for using natural language generation technology to write advice based on the analysis data stored in the database.

[1820] (Claim 3)

[1821] 10. The system of claim 1, including computer vision technology for analyzing player movements and ball trajectories.

[1822] "Example 1"

[1823] (Claim 1)

[1824] A means for receiving videos of practice or games taken by a user;

[1825] A means for dividing the received video at a constant frame rate;

[1826] A means of tracking player movements and ball trajectories for each divided frame,

[1827] a means for storing the tracking results in a database;

[1828] A means to identify technical issues and areas for improvement for players based on the stored data, and

[1829] A means to generate specific advice using generative AI models based on identified issues and areas for improvement; and

[1830] The system includes means for transmitting the generated advice to a user terminal.

[1831] (Claim 2)

[1832] A means for dividing the received video at a fixed frame rate and analyzing each frame;

[1833] a means for storing the analysis results in a database;

[1834] 10. The system of claim 1, further comprising means for generating advice using natural language generation techniques based on the stored data.

[1835] (Claim 3)

[1836] 10. The system of claim 1, further comprising means for using computer vision techniques to analyze player movements and ball trajectories.

[1837] "Application Example 1"

[1838] (Claim 1)

[1839] A means for receiving a video of a task or process taken by a user;

[1840] A means of analyzing the received video to track the robot's movements and the trajectory of the object;

[1841] A means to identify technical issues and areas for improvement for the robot based on the analysis results;

[1842] A means of generating specific advice based on identified issues and areas for improvement;

[1843] The system includes means for transmitting the generated advice to a user terminal.

[1844] (Claim 2)

[1845] A method for dividing videos of work or processes taken by users at a fixed frame rate and analyzing each frame.

[1846] a means for storing the analysis results in a database;

[1847] The system of claim 1, further comprising means for using natural language generation technology to write advice based on the analysis data stored in the database.

[1848] (Claim 3)

[1849] 10. The system of claim 1, including computer vision techniques for analyzing robot movements and object trajectories.

[1850] "Example 2: Combining Emotion Engines"

[1851] (Claim 1)

[1852] A means for receiving videos of practice or games taken by a user;

[1853] A means of analyzing the received video to track player movements and ball trajectory,

[1854] A means to identify technical issues and areas for improvement for players based on the analysis results, and

[1855] A means of generating specific advice based on identified issues and areas for improvement;

[1856] means for transmitting the generated advice to a user terminal;

[1857] means for analyzing emotion data using an emotion engine that recognizes the emotion of a user;

[1858] A system including a means for generating personalized advice that combines emotion data and video analysis results.

[1859] (Claim 2)

[1860] A method for dividing videos of practice and matches taken by users into a certain frame rate and analyzing each frame.

[1861] a means for storing the analysis results in a database;

[1862] The system of claim 1, further comprising means for using natural language generation technology to write advice based on the analysis data stored in the database.

[1863] (Claim 3)

[1864] 10. The system of claim 1, including computer vision technology for analyzing player movements and ball trajectories.

[1865] "Application example 2 when combining emotion engines"

[1866] (Claim 1)

[1867] A means for receiving videos of practice or games taken by a user;

[1868] A means of analyzing the received video to track player movements and ball trajectory,

[1869] A means to identify technical issues and areas for improvement for players based on the analysis results, and

[1870] A means of generating specific advice based on identified issues and areas for improvement;

[1871] means for transmitting the generated advice to a user terminal;

[1872] A means for generating emotion data by analyzing the tone of voice and facial expression of a user;

[1873] A system including means for generating personalized advice, including mental support, based on emotional data.

[1874] (Claim 2)

[1875] A method for dividing videos of practice and matches taken by users into a certain frame rate and analyzing each frame.

[1876] a means for storing the analysis results in a database;

[1877] A means of using natural language generation technology to write advice based on the analysis data stored in the database;

[1878] 2. The system according to claim 1, further comprising means for adjusting the content and tone of the advice generated based on the user's emotional data.

[1879] (Claim 3)

[1880] 10. The system of claim 1, including computer vision technology for analyzing player movements and ball trajectories. [Explanation of symbols]

[1881] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for receiving videos of practice or games taken by a user; A means of analyzing the received video to track player movements and ball trajectory, A means to identify technical issues and areas for improvement for players based on the analysis results, and A means of generating specific advice based on identified issues and areas for improvement; The system includes means for transmitting the generated advice to a user terminal.

2. A method for dividing videos of practice and matches taken by users into a certain frame rate and analyzing each frame. a means for storing the analysis results in a database; The system according to claim 1, further comprising means for generating written advice using natural language generation techniques based on the analysis data stored in the database.

3. 10. The system of claim 1, including computer vision techniques for analyzing player movements and ball trajectories.

Images