system

Abstract

Provide a system. A means for capturing an image of a user's golf swing; An artificial intelligence means for analyzing the captured swing video; visual and audio feedback means for providing the analysis results to the user; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention aims to provide a system that allows users who want to practice their golf swing but are unable to practice effectively due to lack of time, distance to a driving range, high lesson costs, etc. to receive appropriate feedback anytime, anywhere. Specifically, the aim is to create an environment where users can easily receive swing diagnosis and lessons using a smartphone and smart glasses. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems with a system that includes a means for filming a user's golf swing, an artificial intelligence means for analyzing the filmed swing video, and a visual and audio feedback means for providing the user with the analysis results. Specifically, the user films their swing with a smartphone, and the video is analyzed by an artificial intelligence to identify swing flaws and areas for improvement. The analysis results are fed back visually through smart glasses and audio via Bluetooth. The system also includes a means for providing video lessons by receiving lesson videos via streaming from a server, a billing management means for managing usage fees as a monthly service, and a means for collecting user feedback and improving the service. This allows users to practice their golf swing effectively, regardless of location or time.

[0006] "User" refers to an individual who utilizes this system to receive golf swing diagnosis and lessons.

[0007] A "golf swing" refers to the series of movements that occur when hitting a ball with a golf club.

[0008] The term "means for recording" refers to a device or function for recording a user's golf swing as a video.

[0009] A "smartphone" refers to a mobile phone that has advanced information processing capabilities and can run applications.

[0010] "Artificial intelligence means" refers to technology that uses software or algorithms to analyze a user's golf swing and automatically evaluate it and identify areas for improvement.

[0011] "Analysis results" refers to the evaluation of a golf swing and information on areas for improvement obtained by artificial intelligence means.

[0012] "Visual feedback means" refers to a means for displaying the analysis results to the user on a screen or display.

[0013] "Voice feedback means" refers to a means for providing analysis results and advice to the user by voice.

[0014] "Smart glasses" refers to wearable devices that can display information through visual feedback.

[0015] "Bluetooth" refers to a short-range wireless communication technology that provides a means for devices to exchange data.

[0016] "Video lesson means" refers to a function for providing golf swing lesson videos to users.

[0017] "Server" refers to a computing device that receives, analyzes, and transmits data over a network.

[0018] "Streaming" refers to the technology of providing video and audio data to users in real time via the Internet.

[0019] "Charge management means" refers to a system that manages user usage fees and processes monthly service fees.

[0020] "User feedback" refers to opinions and reactions provided by users, and is information used to improve the service. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] The present invention is a system that captures a user's golf swing, analyzes it using artificial intelligence, and provides the results through visual and audio feedback. Using a smartphone and smart glasses, users can receive golf swing diagnosis and lessons anywhere, anytime. The following describes in detail the embodiments of the present invention.

[0043] 1. Initial Setup

[0044] First, users download a compatible golf app onto their smartphone. Next, they create an account and enter their profile information. Then, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the system confirms the connection status and notifies the user.

[0045] 2. Swing photography

[0046] The user places their smartphone in a suitable position and films their golf swing. They tap the "Start Recording" button in the app to begin recording the swing video. The smartphone camera captures the user's swing video in real time and temporarily saves the video data.

[0047] 3. Data transmission and AI analysis

[0048] The device sends the captured swing video to a server. The server then passes the received swing video to an AI algorithm, which begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, to identify swing flaws and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[0049] 4. Swing Feedback

[0050] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[0051] 5. Video lessons

[0052] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons.

[0053] 6. Monthly service and billing management

[0054] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that users always receive the latest lessons.

[0055] 7. Continued Use and Update Provision

[0056] The server collects user feedback and considers service improvements and feature additions based on that feedback. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally.

[0057] With the above configuration, the system of the present invention allows users to receive golf swing diagnosis and lessons regardless of location or time, thereby achieving effective training.

[0058] The processing flow will be explained below.

[0059] Step 1:

[0060] The user downloads the golf app on their smartphone and creates an account. The user completes the process by entering their profile information.

[0061] Step 2:

[0062] The user pairs the smart glasses with their smartphone via Bluetooth. The device notifies them that pairing was successful.

[0063] Step 3:

[0064] The user places the smartphone in the appropriate position and prepares to record their swing. The user then taps the "Start Recording" button in the app to begin recording.

[0065] Step 4:

[0066] The device captures the user's golf swing in real time using a smartphone camera and temporarily saves the captured swing video.

[0067] Step 5:

[0068] The device sends the captured swing video to the server, which receives the video and passes it to the AI ​​algorithm.

[0069] Step 6:

[0070] The server uses an AI algorithm to analyze each element of the swing (posture, timing, speed, etc.) and compiles the analysis results in a report format.

[0071] Step 7:

[0072] The server sends a report of the analysis results back to the device, which receives the analysis results.

[0073] Step 8:

[0074] The device displays the analysis results on the smartphone screen, where the user can check them.

[0075] Step 9:

[0076] The device visually feeds back the analysis results through the smart glasses, and the user receives the visual feedback.

[0077] Step 10:

[0078] The device provides voice feedback via Bluetooth, and the user receives voice advice.

[0079] Step 11:

[0080] The user selects "Watch Video Lessons" within the app and specifies the desired lesson content.

[0081] Step 12:

[0082] The device sends the appropriate video request to the server, which selects the corresponding lesson video in response to the request.

[0083] Step 13:

[0084] The server streams the lesson video to the device, and the device receives the lesson video.

[0085] Step 14:

[0086] The lesson video received by the device is played simultaneously on the smartphone and smart glasses, allowing the user to receive a visual lesson.

[0087] Step 15:

[0088] A user signs up for a monthly service. The user enters payment information and begins the subscription.

[0089] Step 16:

[0090] The server manages billing for the monthly service and periodically delivers new lesson content and updates to users.

[0091] Step 17:

[0092] The server collects user feedback, considers service improvements and feature additions, and provides firmware updates for the app and smart glasses as needed.

[0093] Example 1

[0094] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0095] Conventional golf swing analysis systems require users to spend a lot of time obtaining analysis results and often do not provide results in real time. Furthermore, they do not adequately integrate visual and audio feedback, and do not provide an effective training environment for users. Furthermore, there is a lack of systems that allow users to train anywhere, anytime using their own devices.

[0096] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0097] In this invention, the server includes a means for capturing images of the user's movements, an artificial intelligence means for analyzing the captured images of the movements, a visual and audio feedback means for providing the user with the analysis results, a terminal for the user, and a means for displaying the analysis results and providing advice using the terminal. This allows the user to obtain the analysis results in real time, enabling training regardless of location or time. Furthermore, by providing integrated visual and audio feedback, the user can train more effectively.

[0098] "User behavior" refers to a specific action or behavior generally undertaken by an individual.

[0099] "Photographing means" refers to a device or method for recording actions or movements using a camera or related photographic device.

[0100] "Motion footage" refers to video data of a captured movement.

[0101] "Artificial intelligence means for analysis" refers to AI technology that analyzes the characteristics and shortcomings of behavior based on collected data and extracts areas for improvement.

[0102] "Analysis results" refers to the data and reports obtained after analyzing motion footage.

[0103] "Means for providing visual and audio feedback to the user" refers to a method or device for conveying the analysis results to the user by visual information or audio.

[0104] "Terminal" refers to any electronic device used by a User to access the System, including, for example, a smartphone or tablet.

[0105] "Video display device" refers to any device for displaying images or video, including, for example, smart glasses and monitors.

[0106] "Means for providing advice" refers to a method or device for informing the user of improvements based on the analysis results.

[0107] The present invention is a system that allows a user to receive golf swing diagnosis and lessons regardless of location or time. The system includes a means for the user to film their movements, an artificial intelligence means for analyzing the filmed video of their movements, a visual and audio feedback means for providing the analysis results to the user, a terminal for use by the user, and a means for displaying the analysis results and providing advice using the terminal.

[0108] Initial Setup

[0109] First, the user downloads a compatible golf app onto their smartphone. In this example, the app name is "GolfSwingAnalyzer." Next, they create an account and enter their profile information. After that, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the device checks the connection status and notifies the user.

[0110] Swing shooting

[0111] The user places their smartphone in a suitable position and taps the "Start Recording" button in the app to record their golf swing. The device uses the smartphone camera to capture a video of the user's swing in real time and temporarily stores the video data. For example, the user can mount their smartphone on a tripod placed on the ground and adjust the screen so that the swing area fits within the screen.

[0112] Data transmission and AI analysis

[0113] The device sends the recorded swing video to the server. The server passes the received swing video to an AI algorithm (e.g., TENSORFLOW (registered trademark)) and begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, and identifies swing flaws and areas for improvement. The analysis results are compiled in a report format and returned to the device in real time. An example of a prompt is, "Analyze the user's swing video and generate specific improvement suggestions regarding shoulder movement."

[0114] Swing Feedback

[0115] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result might say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement might be, "To improve your shoulder movement, try practicing the following."

[0116] Video lessons

[0117] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons.

[0118] Monthly service and billing management

[0119] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that the user always receives the latest lessons.

[0120] Continued use and updates

[0121] The server collects user feedback and uses it to improve the service and add new features. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally.

[0122] In this way, the system of the present invention allows the user to receive golf swing diagnosis and lessons regardless of location or time, thereby achieving effective training.

[0123] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0124] Step 1:

[0125] The user downloads and installs the golf app "GolfSwingAnalyzer" onto their smartphone. At this time, the user enters search keywords into the smartphone's App Store or GOOGLE PLAY® store, and the app is found and installed based on those keywords.

[0126] Step 2:

[0127] A user launches the app and creates an account. Here, they enter their name, email address, and password. An account is created based on the information they entered. As an output, a user profile is generated and saved in the database. Specifically, the user enters the required information in each input field and taps the "Register" button.

[0128] Step 3:

[0129] The user pairs the smart glasses with their smartphone via Bluetooth. Input includes the smart glasses' device name and pairing code. The device detects the smart glasses based on this and performs pairing. If successful, the device checks the connection status and sends a notification to the user. Specifically, the user sets the smart glasses to connection mode and taps the "Device Pairing" button in the app.

[0130] Step 4:

[0131] The user places the smartphone in an appropriate position and taps the "Start Recording" button in the app to record their golf swing. The input includes the user's movement information (swing movement). The device uses the smartphone camera to record this movement and temporarily saves it as video data. Specific actions include the user fixing the smartphone to a tripod or other device and adjusting it so that the swing area fits on the screen.

[0132] Step 5:

[0133] The device sends the recorded swing video to the server. The input is the temporarily stored video data, which is sent via Wi-Fi or mobile data communication. The output is the generated video data stored on the server. The specific operation requires the user to tap the "Send Data" button.

[0134] Step 6:

[0135] The server passes the received swing video to an AI algorithm and begins analyzing the golf swing. The input is video data. The AI ​​algorithm (for example, TensorFlow) analyzes this and performs data calculations on swing posture, timing, speed, etc. to identify flaws and areas for improvement. The output is an analysis report. Specifically, the server passes the video data to the analysis module and executes the analysis process.

[0136] Step 7:

[0137] The server compiles the analysis results into a report format and sends it back to the device in real time. The input is the data analyzed by the AI. The output is an analysis report in JSON format that is generated and sent to the device. Specifically, the server formats the analysis data and sends it to the device.

[0138] Step 8:

[0139] The device displays the received analysis results on the smartphone screen. The input is an analysis report. The output is a graph or text display of the analysis results. Specifically, the app UI on the device performs processing to visually display the analysis results.

[0140] Step 9:

[0141] The device provides visual feedback to the smart glasses and specific audio advice via Bluetooth. The input is the analysis results. The output is visual and audio feedback. Specific operations include an overlay of the analysis results summary on the smart glasses and audio guidance.

[0142] Step 10:

[0143] The user selects "Watch a video lesson" within the app and selects a lesson video related to their swing. The input is the user's request for the lesson video they selected. The device receives the corresponding lesson video from the server and plays it simultaneously on the smartphone and smart glasses. The output is the playback of the lesson video. Specifically, the user taps the video play button, and the video plays synchronously on both devices.

[0144] Step 11:

[0145] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The input is the user's credit card information and registration information. The output is the completion of the monthly billing and the continuation of service use. Specifically, the user taps the "Sign up for monthly service" button and enters their credit card information.

[0146] Step 12:

[0147] The server monitors user usage and delivers new lesson content and updated information at appropriate times. The input is user usage data. The output is the provision of new lesson content. Specifically, the server periodically checks usage and delivers new content.

[0148] Step 13:

[0149] The server collects feedback from users and considers improving the service or adding new features based on that information. The input is user feedback information. The output is the development of new features and improvements to the service. Specifically, the user enters and submits their opinions and requests using the "Feedback Submission" form within the app.

[0150] Step 14:

[0151] The server provides firmware updates for the app and smart glasses as needed. The input is the latest version information. The output is the provision of updated firmware. Specifically, when a new version is released, the app automatically checks for updates and sends an update notification to the user. When the user taps the "Update Now" button, the update begins.

[0152] Through the above processing steps, the system enables users to receive golf swing diagnosis and lessons regardless of location or time, thereby achieving effective training.

[0153] (Application example 1)

[0154] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0155] Conventional motion analysis systems make it difficult for users to perform detailed analysis of their motions and conduct effective training regardless of location or time. Furthermore, they do not provide real-time feedback or simulations in virtual environments, limiting improvements based solely on analysis results. Furthermore, they lack the ability to learn and improve motivation by comparing data with other users.

[0156] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0157] In this invention, the server includes a means for filming the user's exercise movements, an artificial intelligence means for analyzing the filmed exercise video, a visual and audio feedback means for providing the user with the analysis results, a means for simulating exercise in a virtual environment, and a means for comparing with other users and providing lesson content. This allows the user to receive effective training regardless of location or time, and through real-time analysis and feedback, as well as simulation in a virtual environment and comparison with other users, the user can achieve exercise improvement with higher motivation.

[0158] "User" refers to an individual who utilizes the system to analyze and improve athletic performance.

[0159] "Athletic movements" refer to movements associated with specific sports or fitness, such as the golf swing.

[0160] "Means for taking photographs" refers to a function for recording the user's exercise movements using a device such as a smartphone or camera.

[0161] "Exercise video" refers to video data of a user's exercise movements that have been captured.

[0162] "Artificial intelligence means" refers to a system that uses AI algorithms to analyze exercise footage and extract and evaluate data about the user's movements.

[0163] "Analysis results" refers to information regarding the evaluation of exercise movements and areas for improvement generated by artificial intelligence means.

[0164] "Visual feedback means" refers to a function that provides visual information to the user through smart glasses or a smartphone screen.

[0165] "Audio feedback means" refers to a function that provides advice and information to the user by voice through a speaker or headphones.

[0166] "Virtual environment" refers to a computer-generated simulated environment.

[0167] "Movement simulation means" refers to a function that reproduces the user's movement within a virtual environment and provides a simulation based on the analysis results.

[0168] "Comparison means" refers to the functionality used to compare a user's athletic data with that of other users.

[0169] "Means for providing lesson content" refers to a function for providing users with educational content aimed at improving or learning exercise movements.

[0170] The present invention is a system that captures a user's exercise movements, analyzes the video using artificial intelligence, and provides the analysis results as visual and audio feedback. The realization of this system will be described in detail below.

[0171] First, users download a specific application onto their smartphone and create an account, then pair the smart glasses with the smartphone via Bluetooth, which enables the visual feedback mechanism.

[0172] When a user performs an exercise, the smartphone records the video. For example, when a user performs a golf swing, the user places the smartphone in an appropriate position, taps the "Start Recording" button, and records the swing video. At this time, the smartphone camera records the user's movements in real time and temporarily stores the video data.

[0173] The device then sends the captured video to a server, which then passes it to an AI algorithm to analyze the exercise. The server analyzes factors such as posture, timing, and speed to identify weaknesses and areas for improvement. The analysis results are compiled into a report and sent back to the device.

[0174] The device displays the analysis results on the smartphone screen, provides visual feedback through smart glasses, and provides specific audio advice via Bluetooth, such as "Your right shoulder is dropping too much at the top of your swing."

[0175] Users can also select "Simulation Mode" within the app to simulate athletic movements in a virtual environment, which provides a visual demonstration of how users can improve their movements in real time.

[0176] Furthermore, users can compare their data with that of other users to create more effective training plans, and can watch lesson content to learn specific training methods related to their movements.

[0177] This allows users to receive effective training regardless of location or time, and achieve improvements in their athletic performance through real-time analysis and feedback, as well as simulation in a virtual environment and comparison with other users.

[0178] As a concrete example, consider a user practicing their golf swing. In this case, the user wears smart glasses and films their swing with a smartphone. The captured video of their swing is analyzed by AI, and feedback such as "Your right shoulder is too low at the top of your swing" is displayed via the smart glasses and smartphone. After that, a simulation is performed on a virtual golf course, and further areas for improvement can be identified by comparing the data with other users' data.

[0179] Examples of prompts to input to a generative AI model include:

[0180] "Analyze the user's athletic movements and output improvements. The elements to analyze are posture, timing, and speed."

[0181] As described above, the present invention is a system that enables a user to effectively analyze and improve exercise movements regardless of location or time.

[0182] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0183] Step 1:

[0184] A user downloads a golf swing analysis application onto their smartphone and creates an account. They then enter their profile information and pair the smart glasses with their smartphone via Bluetooth. This information is sent to the server, which then creates an account and identifies the user's device.

[0185] Step 2:

[0186] The user performs an exercise (e.g., a golf swing) and captures it with the smartphone camera. The user taps the "Start Recording" button in the smartphone application to begin recording the exercise video. The captured video is temporarily saved in the smartphone's local storage. The input is the video of the user's movements, and the output is the saved video data.

[0187] Step 3:

[0188] The smartphone sends the saved video data to a server. The server receives this video data and passes it to an artificial intelligence tool to begin analysis. The input is the video data, and the output is the analysis results by the AI. Specifically, the video file is uploaded to the server via the internet, and then the data is input into the AI ​​model.

[0189] Step 4:

[0190] The server analyzes the received video data using an AI algorithm. Specifically, it processes and calculates data on each element of the movement, such as posture, timing, and speed, to identify shortcomings and areas for improvement. The input is the video data, and data processing involves extracting elements from each frame, followed by the actual analysis process. The output is a report of the analysis results.

[0191] Step 5:

[0192] The analysis results obtained from the server are sent back to the smartphone. The smartphone application receives these analysis results and provides visual and audio feedback to the user. The input is the analysis results, and the output is feedback to the user. Specifically, the application visualizes the analysis results and notifies the user in an easy-to-understand manner using text and audio.

[0193] Step 6:

[0194] The user selects "Simulation Mode" within the application and simulates their own athletic movements in a virtual environment. The device simulates the movement in real time and visually indicates areas for improvement. The input is the analysis results and current movement data, and the output is the simulation results. Specifically, a simulation begins within the app, highlighting areas for improvement.

[0195] Step 7:

[0196] Users compare their own exercise data with that of other users and watch lesson content. The application provides the comparison results and appropriate lesson videos. The input is the user's own and other users' exercise data, and the output is a comparison report and lesson videos. Specifically, the application retrieves other users' data from the database and performs a relative comparison.

[0197] In this way, this system supports effective training by analyzing the user's exercise movements in detail and suggesting areas for improvement.The following is an example of a prompt sentence input to the generative AI model.

[0198] "Analyze the user's athletic movements and output improvements. The elements to analyze are posture, timing, and speed."

[0199] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0200] The present invention is a system that films a user's golf swing, analyzes it using artificial intelligence, and provides the results through visual and audio feedback. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the feedback is adjusted based on the user's emotional state. The following describes in detail the embodiments of the present invention.

[0201] 1. Initial Setup

[0202] First, users download a compatible golf app onto their smartphone and create an account. Next, they enter their profile information. After that, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the system confirms the connection status and notifies the user.

[0203] 2. Swing photography

[0204] The user places their smartphone in a suitable position and films their golf swing. They tap the "Start Recording" button in the app to begin recording the swing video. The smartphone camera captures the user's swing video in real time and temporarily saves the video data.

[0205] 3. Data transmission and AI analysis

[0206] The device sends the captured swing video to a server. The server then passes the received swing video to an AI algorithm, which begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, to identify swing flaws and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[0207] 4. Swing Feedback

[0208] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[0209] 5. Feedback adjustment by emotion engine

[0210] The emotion engine recognizes the user's emotions. Emotion recognition is performed using facial expression analysis and voice tone analysis. The recognized emotional state is used to adjust the feedback content and method. For example, if the user looks tired, the feedback content can be changed to a more encouraging tone. Also, if the user is concentrating, more detailed technical instructions can be provided.

[0211] 6. Video lessons

[0212] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons. The content of the video is also customized based on the emotions recognized by the emotion engine.

[0213] 7. Monthly service and billing management

[0214] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that users always receive the latest lessons.

[0215] 8. Continued Use and Update Provision

[0216] The server collects user feedback and considers service improvements and feature additions based on that feedback. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally.

[0217] With the above configuration, the system of the present invention enables users to receive golf swing diagnosis and lessons regardless of location or time, achieving effective training. In addition, by combining it with an emotion engine, it provides feedback according to the user's emotional state, providing a more personalized training experience.

[0218] The processing flow will be explained below.

[0219] Step 1:

[0220] The user downloads the golf app on their smartphone and creates an account. The user enters the required profile information and completes the setup.

[0221] Step 2:

[0222] The user pairs the smart glasses with their smartphone via Bluetooth. The smartphone confirms the successful pairing and notifies the user.

[0223] Step 3:

[0224] The user positions the smartphone in the appropriate position and prepares to record their golf swing. The user taps the "Start Recording" button in the app to begin recording.

[0225] Step 4:

[0226] The device captures the user's golf swing in real time using a smartphone camera and temporarily stores the captured swing video.

[0227] Step 5:

[0228] The device sends the captured swing video to the server, which receives the video and passes it to the AI ​​algorithm.

[0229] Step 6:

[0230] The server uses an AI algorithm to analyze each element of the swing (posture, timing, speed, etc.) and compiles the analysis results in a report format.

[0231] Step 7:

[0232] The server sends a report of the analysis results back to the device, which receives the results and prepares them for display.

[0233] Step 8:

[0234] The device displays the analysis results on the smartphone screen, and the user checks them.

[0235] Step 9:

[0236] The device visually feeds back the analysis results through the smart glasses, which display the results.

[0237] Step 10:

[0238] The device provides voice feedback via Bluetooth, and the user receives voice advice.

[0239] Step 11:

[0240] The device activates an emotion engine and performs facial expression analysis and voice tone analysis to recognize the user's emotions.

[0241] Step 12:

[0242] The emotion engine identifies the user's emotional state and passes the results on to the rest of the system. For example, if the user is tired, they will receive the recognition result.

[0243] Step 13:

[0244] The device will adjust the feedback content based on the results of the emotion engine, for example adding an encouraging message if the user is tired.

[0245] Step 14:

[0246] The user selects "Watch Video Lessons" within the app and specifies the desired lesson content. The app prepares to retrieve the lesson video based on the request.

[0247] Step 15:

[0248] The device sends the appropriate video request to the server, which selects the appropriate lesson video in response to the request.

[0249] Step 16:

[0250] The server streams the lesson video to the device, which then plays the received video simultaneously on the smartphone and smart glasses.

[0251] Step 17:

[0252] Customize video content based on emotions recognized by the emotion engine, for example, providing detailed technical instructions if the user is distracted.

[0253] Step 18:

[0254] A user signs up for a monthly service, enters payment information, and begins their subscription.

[0255] Step 19:

[0256] The server manages billing for the monthly service, monitors user usage, and periodically delivers new lesson content and updates.

[0257] Step 20:

[0258] The server collects user feedback, considers service improvements and feature additions, and provides firmware updates for the app and smart glasses as needed.

[0259] Example 2

[0260] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0261] Conventional golf swing analysis systems have the ability to film a user's swing and provide the analysis results as visual and audio feedback, but they are unable to provide feedback that takes into account the user's emotional state. Furthermore, they lack the ability to provide real-time analysis and content based on the user's usage. Therefore, there is a need to provide users with a more effective and personalized training experience.

[0262] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0263] In this invention, the server includes a device that captures a user's actions, an artificial intelligence function that analyzes the captured video of the actions, a visual and audio feedback function that provides the user with the analysis results, an emotion recognition engine that recognizes the user's emotions and adjusts the feedback, and a content provision service. This makes it possible to analyze a user's actions in real time and provide personalized feedback according to the user's emotional state. It is also possible to deliver new content at an appropriate time based on the user's usage status.

[0264] The "device for capturing images of user actions" is a device for recording user actions in real time, and includes a smartphone camera, a camcorder, and the like.

[0265] "Artificial intelligence function that analyzes filmed video of movements" refers to artificial intelligence algorithms and software for analyzing filmed video of movements, and is a function that generates analysis results by evaluating the posture, timing, speed, etc. of the movement.

[0266] The "visual and audio feedback function for providing the analysis results to the user" is a function for presenting the analysis results to the user visually and audio-wise, and for providing information via a display or speaker.

[0267] The "emotion recognition engine that recognizes the user's emotions and adjusts feedback" is an engine that analyzes the user's facial expressions and tone of voice to recognize emotions, and appropriately adjusts the feedback content based on that information.

[0268] A "content provision service" is a service that delivers new learning and lesson content at appropriate times based on the user's usage status, and is managed and delivered on the server side.

[0269] A "head-mounted display" is a display device worn by a user on the head, which provides visual information.

[0270] "Real-time analysis" refers to the process of analyzing captured video instantly with almost no delay.

[0271] "Personalized feedback" refers to providing feedback optimized according to the characteristics and status of each individual user.

[0272] The present invention is a system that captures a user's actions, analyzes them using artificial intelligence, and provides the results through visual and audio feedback. Furthermore, by combining an emotion recognition engine that recognizes the user's emotions, the system also has the function of adjusting the feedback based on the user's emotional state. Specific embodiments for implementing the present invention are described in detail below.

[0273] 1. Initial Setup

[0274] First, users download a compatible golf app onto their smartphone and create an account. Next, they enter their profile information. After that, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the device will confirm the connection status and notify the user.

[0275] 2. Swing photography

[0276] The user places their smartphone in a suitable position and films their golf swing. They tap the "Start Recording" button in the app to begin recording the swing video. The smartphone camera captures the user's swing video in real time and temporarily saves the video data.

[0277] 3. Data transmission and AI analysis

[0278] The device sends the captured swing video to a server. The server then passes the received swing video to the AI ​​algorithm "SwingAI," which begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, to identify flaws and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[0279] 4. Swing Feedback

[0280] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[0281] 5. Feedback adjustment by emotion engine

[0282] The emotion recognition engine "EmotionAI" recognizes the user's emotions. Emotion recognition is performed by analyzing the user's facial expressions and vocal tone. The recognized emotional state is used to adjust the content and method of feedback. For example, if the user looks tired, the feedback content will be changed to a more encouraging tone. Also, if the user is concentrating, more detailed technical instructions will be provided.

[0283] 6. Video lessons

[0284] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons. The content of the video is also customized based on emotions recognized by the emotion recognition engine.

[0285] For example, if a user were to film their swing and use AI analysis, the prompt would look like this:

[0286] "Please provide me with feedback through the smart glasses so that I can improve my swing."

[0287] 7. Monthly service and billing management

[0288] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that users always receive the latest lessons.

[0289] 8. Continued Use and Update Provision

[0290] The server collects user feedback and considers service improvements and feature additions based on it. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally. For example, if the user is tired, it adjusts the feedback it provides to them in a more encouraging tone.

[0291] This system allows users to receive golf swing diagnosis and lessons regardless of location or time, enabling effective training. In addition, by combining it with an emotion recognition engine, it is possible to provide feedback according to the user's emotional state, providing a more personalized training experience.

[0292] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0293] Step 1:

[0294] Initial Setup

[0295] A user downloads a golf app onto their smartphone and creates an account by entering their username, email address, and password. This registers the account information in the app.

[0296] Next, users enter profile information (e.g., years of golf experience and handicap), which generates the basis for customized analysis and feedback.

[0297] Pair the smart glasses with the smartphone via Bluetooth, using the smart glasses' identification information and Bluetooth connection data as input, and notify the smartphone once pairing is successful.

[0298] Step 2:

[0299] Swing shooting

[0300] The user places the smartphone in an appropriate position and records the golf swing. The input is the smartphone camera position and shooting angle information.

[0301] When you tap the "Start Recording" button, the smartphone camera will start recording your swing video. The real-time video is saved temporarily during recording, and the data is saved after the recording ends. The output is a recorded swing video file.

[0302] Step 3:

[0303] Data transmission and AI analysis

[0304] The device sends the saved swing video to the server, using the video file path and the server URL as input.

[0305] The server passes the received swing video to the AI ​​algorithm "SwingAI" and begins analysis. The video data and user profile information are used as input. The AI ​​algorithm analyzes the swing posture, timing, speed, etc., and generates a report of the analysis results as output.

[0306] The analysis results are compiled in a report format and sent back to the terminal. An analysis report is sent to the terminal as output.

[0307] Step 4:

[0308] Swing Feedback

[0309] The terminal displays the received analysis results on the smartphone screen. It reads the analysis report as input and generates screen data to be displayed on the smartphone as output.

[0310] It also provides visual feedback through smart glasses and sends audio advice via Bluetooth. It uses text and audio analysis results as input. It provides visual and audio feedback as output. For example, it displays a message like, "Your right shoulder is dropping too low at the top of your swing."

[0311] Step 5:

[0312] Feedback adjustment by emotion engine

[0313] The emotion recognition engine "EmotionAI" uses the user's facial expression images and voice tone data as input to recognize the user's emotions.

[0314] The emotion engine analyzes these data and outputs the user's emotional state. The output is the user's current emotional state (e.g., tired, focused).

[0315] The system adjusts the feedback content and method based on the recognized emotional state. It uses the emotional state data and analysis result data as input and generates adjusted feedback as output. For example, if the user is tired, the system adjusts the feedback to be provided in an encouraging tone.

[0316] Step 6:

[0317] Video lessons

[0318] The user selects "Watch a video lesson" within the app and specifies the relevant lesson video. The user's selection information is used as input.

[0319] The device receives the relevant lesson video from the server and plays it on the smartphone or smart glasses. The URL or path of the video file is used as input. The lesson video is played as output.

[0320] The content of the video is also customized based on the emotions recognized by the emotion engine. Emotional state data and video data are used as input, and customized video content is provided as output.

[0321] Step 7:

[0322] Monthly service and billing management

[0323] When a user signs up for a monthly service, the user's registration information and billing information are used as input.

[0324] The server processes monthly charges and monitors usage using a billing management mechanism, generating a billing completion notice and usage report as outputs.

[0325] Delivering new lesson content and updates at the appropriate time, using usage data as input and generating new content delivery data as output.

[0326] Step 8:

[0327] Continued use and updates

[0328] The server collects user feedback and considers improving the service or adding new features based on it. The feedback data is used as input.

[0329] Provides app and smartglasses firmware updates as needed, using update data as input and providing updated software and firmware as output.

[0330] This ensures that your system always performs optimally.

[0331] (Application example 2)

[0332] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0333] The present invention aims to solve the problems of conventional golf swing analysis systems, such as limited returns when users try out equipment, feedback that does not take into account emotional state, and a lack of specific support for users to select appropriate equipment.

[0334] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a device means for filming the user's hitting motion, a machine learning means for analyzing the filmed hitting motion video, a visual and audio feedback means for providing the analysis results to the user, an emotion engine means for recognizing the user's emotional state and adjusting the feedback, and a means for recommending products for the user to try in a real-world store environment. This provides feedback according to the user's emotional state, enabling hitting motion analysis and appropriate product selection in a real store.

[0335] The term "user's hitting motion" refers to any swing or hitting motion performed with a golf club or other sporting equipment.

[0336] The term "photography device" refers to an electronic device or device, such as a camera or smartphone, that records a user's hitting motion as video.

[0337] "Machine learning means" refers to a technical means that uses artificial intelligence or machine learning algorithms to analyze and evaluate filmed footage of hitting movements.

[0338] "Visual and audio feedback means" refers to a display device or audio output device for providing the analysis results to the user visually or audio.

[0339] "Emotion engine" refers to a technical means for recognizing a user's emotional state by analyzing their facial expressions and tone of voice, and adjusting the feedback provided based on that state.

[0340] "Recommendation method" refers to the algorithms and systems used to select and recommend products for users to try out in physical stores.

[0341] "Display device" refers to a device that provides information visually to a user, such as smart glasses or a head-mounted display.

[0342] The system captures a user's hitting motion, analyzes it using machine learning, and provides the results through visual and audio feedback. By combining it with an emotion engine that recognizes the user's emotional state, the system can tailor the feedback based on the user's emotional state. Furthermore, the system has the ability to recommend products for the user to try in a brick-and-mortar store environment.

[0343] 1. Initial Setup

[0344] Users first download the corresponding app onto their smartphones, create an account, enter their profile information, and pair their smart glasses or other display device with their smartphone via Bluetooth. Once pairing is successful, the system will confirm the connection status and notify the user.

[0345] 2. Filming the hitting motion

[0346] The user places their smartphone in an appropriate position and films their batting motion. They tap the "Start Recording" button in the app to begin recording the video of their batting motion. The smartphone camera captures video of the user's batting motion in real time and temporarily saves the video data.

[0347] 3. Data transmission and AI analysis

[0348] The device sends the captured video of the batting motion to a server. The server then passes the video to a machine learning algorithm, which begins analysis. The machine learning algorithm analyzes each element of the batting motion, such as posture, timing, and speed, and identifies weaknesses and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[0349] 4. Providing Feedback

[0350] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through the display device and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is lowered too much at the top of your hitting motion," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[0351] 5. Feedback adjustment by emotion engine

[0352] The server uses an emotion engine to recognize the user's emotional state. Emotion recognition is performed by analyzing the user's facial expressions and voice tone. The recognized emotional state is used to adjust the content and method of feedback. For example, if the user looks tired, the feedback content can be changed to a more encouraging tone. If the user is concentrating, more detailed technical instructions can be provided.

[0353] Hardware and software used

[0354] Hardware: Smartphone camera, smart glasses, Bluetooth module

[0355] Software: Machine learning algorithms, emotion recognition engine, Bluetooth connectivity software

[0356] Specific examples

[0357] For example, imagine a scenario where a user is trying out golf clubs at a brick-and-mortar store. The user takes a video of their swing with their smartphone and sends it to a server. A machine learning algorithm identifies flaws in the swing, and an emotion engine recognizes the user's level of fatigue. Based on this, real-time feedback such as "Your swing angle isn't good. Take a short break to refresh yourself" is displayed on the smart glasses.

[0358] Example prompts for generative AI models

[0359] "Please point out areas for improvement when the golf swing angle is poor. Also, add encouraging comments if the user is tired."

[0360] The above is a detailed description of the embodiment of the present invention. This system is very useful for users in physical stores to analyze their hitting motion and select the most suitable equipment.

[0361] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0362] Step 1:

[0363] A user downloads the app to their smartphone and creates an account. They enter their profile information and pair their smart glasses or other display device with the smartphone via Bluetooth. A confirmation of successful pairing appears on the device.

[0364] Step 2:

[0365] The user places their smartphone in an appropriate position and records their hitting motion. When the user taps the "Start Recording" button in the app, the smartphone camera begins recording their hitting motion in real time. This video data is temporarily saved on the smartphone.

[0366] Step 3:

[0367] The device sends the captured video of the batting motion to the server. The input data is a video file of the batting motion. After receiving this, the server passes the video data to a machine learning algorithm and begins analysis. The output is the analysis results in JSON format data.

[0368] Step 4:

[0369] The server uses machine learning algorithms to analyze each element of the hitting motion, including posture, timing, and speed. The analysis results include swing flaws and areas for improvement. After data processing and calculations, the results are compiled into a report.

[0370] Step 5:

[0371] The server sends the analysis results back to the device, which then visually displays the received analysis results on the app's user interface. Additionally, the device provides visual feedback via a display device and specific advice via audio via Bluetooth. At this time, feedback based on the analysis results is output as text and audio files.

[0372] Step 6:

[0373] The server uses an emotion engine to recognize the user's emotional state. The input data is the user's facial expression and voice tone, which are analyzed by the emotion recognition algorithm. The output is tag information that indicates the user's emotional state.

[0374] Step 7:

[0375] The server adjusts the feedback content and method based on the user's recognized emotional state. For example, if the user looks tired, the feedback tone is changed to be more encouraging. If the user is concentrating, detailed technical instructions are added. The text and audio files with the adjusted feedback content and tone are output.

[0376] Step 8:

[0377] The server recommends products for the user to try in a physical store environment. The input data is the analysis results and the user's profile information, which are analyzed by the recommendation algorithm. The output is a list of optimal products. The device presents this list to the user within the app.

[0378] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0379] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0380] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0381] [Second embodiment]

[0382] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0383] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0384] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0385] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0386] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0387] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0388] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0389] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0390] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0391] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0392] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0393] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0394] The present invention is a system that captures a user's golf swing, analyzes it using artificial intelligence, and provides the results through visual and audio feedback. Using a smartphone and smart glasses, users can receive golf swing diagnosis and lessons anywhere, anytime. The following describes in detail the embodiments of the present invention.

[0395] 1. Initial Setup

[0396] First, users download a compatible golf app onto their smartphone. Next, they create an account and enter their profile information. Then, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the system confirms the connection status and notifies the user.

[0397] 2. Swing photography

[0398] The user places their smartphone in a suitable position and films their golf swing. They tap the "Start Recording" button in the app to begin recording the swing video. The smartphone camera captures the user's swing video in real time and temporarily saves the video data.

[0399] 3. Data transmission and AI analysis

[0400] The device sends the captured swing video to a server. The server then passes the received swing video to an AI algorithm, which begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, to identify swing flaws and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[0401] 4. Swing Feedback

[0402] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[0403] 5. Video lessons

[0404] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons.

[0405] 6. Monthly service and billing management

[0406] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that users always receive the latest lessons.

[0407] 7. Continued Use and Update Provision

[0408] The server collects user feedback and considers service improvements and feature additions based on that feedback. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally.

[0409] With the above configuration, the system of the present invention allows users to receive golf swing diagnosis and lessons regardless of location or time, thereby achieving effective training.

[0410] The processing flow will be explained below.

[0411] Step 1:

[0412] The user downloads the golf app on their smartphone and creates an account. The user completes the process by entering their profile information.

[0413] Step 2:

[0414] The user pairs the smart glasses with their smartphone via Bluetooth. The device notifies them that pairing was successful.

[0415] Step 3:

[0416] The user places the smartphone in the appropriate position and prepares to record their swing. The user then taps the "Start Recording" button in the app to begin recording.

[0417] Step 4:

[0418] The device captures the user's golf swing in real time using a smartphone camera and temporarily saves the captured swing video.

[0419] Step 5:

[0420] The device sends the captured swing video to the server, which receives the video and passes it to the AI ​​algorithm.

[0421] Step 6:

[0422] The server uses an AI algorithm to analyze each element of the swing (posture, timing, speed, etc.) and compiles the analysis results in a report format.

[0423] Step 7:

[0424] The server sends a report of the analysis results back to the device, which receives the analysis results.

[0425] Step 8:

[0426] The device displays the analysis results on the smartphone screen, where the user can check them.

[0427] Step 9:

[0428] The device visually feeds back the analysis results through the smart glasses, and the user receives the visual feedback.

[0429] Step 10:

[0430] The device provides voice feedback via Bluetooth, and the user receives voice advice.

[0431] Step 11:

[0432] The user selects "Watch Video Lessons" within the app and specifies the desired lesson content.

[0433] Step 12:

[0434] The device sends the appropriate video request to the server, which selects the corresponding lesson video in response to the request.

[0435] Step 13:

[0436] The server streams the lesson video to the device, and the device receives the lesson video.

[0437] Step 14:

[0438] The lesson video received by the device is played simultaneously on the smartphone and smart glasses, allowing the user to receive a visual lesson.

[0439] Step 15:

[0440] A user signs up for a monthly service. The user enters payment information and begins the subscription.

[0441] Step 16:

[0442] The server manages billing for the monthly service and periodically delivers new lesson content and updates to users.

[0443] Step 17:

[0444] The server collects user feedback, considers service improvements and feature additions, and provides firmware updates for the app and smart glasses as needed.

[0445] Example 1

[0446] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0447] Conventional golf swing analysis systems require users to spend a lot of time obtaining analysis results and often do not provide results in real time. Furthermore, they do not adequately integrate visual and audio feedback, and do not provide an effective training environment for users. Furthermore, there is a lack of systems that allow users to train anywhere, anytime using their own devices.

[0448] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0449] In this invention, the server includes a means for capturing images of the user's movements, an artificial intelligence means for analyzing the captured images of the movements, a visual and audio feedback means for providing the user with the analysis results, a terminal for the user, and a means for displaying the analysis results and providing advice using the terminal. This allows the user to obtain the analysis results in real time, enabling training regardless of location or time. Furthermore, by providing integrated visual and audio feedback, the user can train more effectively.

[0450] "User behavior" refers to a specific action or behavior generally undertaken by an individual.

[0451] "Photographing means" refers to a device or method for recording actions or movements using a camera or related photographic device.

[0452] "Motion footage" refers to video data of a captured movement.

[0453] "Artificial intelligence means for analysis" refers to AI technology that analyzes the characteristics and shortcomings of behavior based on collected data and extracts areas for improvement.

[0454] "Analysis results" refers to the data and reports obtained after analyzing motion footage.

[0455] "Means for providing visual and audio feedback to the user" refers to a method or device for conveying the analysis results to the user by visual information or audio.

[0456] "Terminal" refers to any electronic device used by a User to access the System, including, for example, a smartphone or tablet.

[0457] "Video display device" refers to any device for displaying images or video, including, for example, smart glasses and monitors.

[0458] "Means for providing advice" refers to a method or device for informing the user of improvements based on the analysis results.

[0459] The present invention is a system that allows a user to receive golf swing diagnosis and lessons regardless of location or time. The system includes a means for the user to film their movements, an artificial intelligence means for analyzing the filmed video of their movements, a visual and audio feedback means for providing the analysis results to the user, a terminal for use by the user, and a means for displaying the analysis results and providing advice using the terminal.

[0460] Initial Setup

[0461] First, the user downloads a compatible golf app onto their smartphone. In this example, the app name is "GolfSwingAnalyzer." Next, they create an account and enter their profile information. After that, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the device checks the connection status and notifies the user.

[0462] Swing shooting

[0463] The user places their smartphone in a suitable position and taps the "Start Recording" button in the app to record their golf swing. The device uses the smartphone camera to capture a video of the user's swing in real time and temporarily stores the video data. For example, the user can mount their smartphone on a tripod placed on the ground and adjust the screen so that the swing area fits within the screen.

[0464] Data transmission and AI analysis

[0465] The device sends the recorded swing video to a server. The server then passes the received swing video to an AI algorithm (e.g., TensorFlow) and begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, to identify swing flaws and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time. An example prompt is, "Analyze the user's swing video and generate specific improvement suggestions for shoulder movement."

[0466] Swing Feedback

[0467] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result might say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement might be, "To improve your shoulder movement, try practicing the following."

[0468] Video lessons

[0469] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons.

[0470] Monthly service and billing management

[0471] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that the user always receives the latest lessons.

[0472] Continued use and updates

[0473] The server collects user feedback and uses it to improve the service and add new features. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally.

[0474] In this way, the system of the present invention allows the user to receive golf swing diagnosis and lessons regardless of location or time, thereby achieving effective training.

[0475] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0476] Step 1:

[0477] The user downloads and installs the golf app "GolfSwingAnalyzer" onto their smartphone. At this time, the user enters search keywords into the smartphone's App Store or Google Play Store, and the app is found and installed based on those keywords.

[0478] Step 2:

[0479] A user launches the app and creates an account. Here, they enter their name, email address, and password. An account is created based on the information they entered. As an output, a user profile is generated and saved in the database. Specifically, the user enters the required information in each input field and taps the "Register" button.

[0480] Step 3:

[0481] The user pairs the smart glasses with their smartphone via Bluetooth. Input includes the smart glasses' device name and pairing code. The device detects the smart glasses based on this and performs pairing. If successful, the device checks the connection status and sends a notification to the user. Specifically, the user sets the smart glasses to connection mode and taps the "Device Pairing" button in the app.

[0482] Step 4:

[0483] The user places the smartphone in an appropriate position and taps the "Start Recording" button in the app to record their golf swing. The input includes the user's movement information (swing movement). The device uses the smartphone camera to record this movement and temporarily saves it as video data. Specific actions include the user fixing the smartphone to a tripod or other device and adjusting it so that the swing area fits on the screen.

[0484] Step 5:

[0485] The device sends the recorded swing video to the server. The input is the temporarily stored video data, which is sent via Wi-Fi or mobile data communication. The output is the generated video data stored on the server. The specific operation requires the user to tap the "Send Data" button.

[0486] Step 6:

[0487] The server passes the received swing video to an AI algorithm and begins analyzing the golf swing. The input is video data. The AI ​​algorithm (for example, TensorFlow) analyzes this and performs data calculations on swing posture, timing, speed, etc. to identify flaws and areas for improvement. The output is an analysis report. Specifically, the server passes the video data to the analysis module and executes the analysis process.

[0488] Step 7:

[0489] The server compiles the analysis results into a report format and sends it back to the device in real time. The input is the data analyzed by the AI. The output is an analysis report in JSON format that is generated and sent to the device. Specifically, the server formats the analysis data and sends it to the device.

[0490] Step 8:

[0491] The device displays the received analysis results on the smartphone screen. The input is an analysis report. The output is a graph or text display of the analysis results. Specifically, the app UI on the device performs processing to visually display the analysis results.

[0492] Step 9:

[0493] The device provides visual feedback to the smart glasses and specific audio advice via Bluetooth. The input is the analysis results. The output is visual and audio feedback. Specific operations include an overlay of the analysis results summary on the smart glasses and audio guidance.

[0494] Step 10:

[0495] The user selects "Watch a video lesson" within the app and selects a lesson video related to their swing. The input is the user's request for the lesson video they selected. The device receives the corresponding lesson video from the server and plays it simultaneously on the smartphone and smart glasses. The output is the playback of the lesson video. Specifically, the user taps the video play button, and the video plays synchronously on both devices.

[0496] Step 11:

[0497] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The input is the user's credit card information and registration information. The output is the completion of the monthly billing and the continuation of service use. Specifically, the user taps the "Sign up for monthly service" button and enters their credit card information.

[0498] Step 12:

[0499] The server monitors user usage and delivers new lesson content and updated information at appropriate times. The input is user usage data. The output is the provision of new lesson content. Specifically, the server periodically checks usage and delivers new content.

[0500] Step 13:

[0501] The server collects feedback from users and considers improving the service or adding new features based on that information. The input is user feedback information. The output is the development of new features and improvements to the service. Specifically, the user enters and submits their opinions and requests using the "Feedback Submission" form within the app.

[0502] Step 14:

[0503] The server provides firmware updates for the app and smart glasses as needed. The input is the latest version information. The output is the provision of updated firmware. Specifically, when a new version is released, the app automatically checks for updates and sends an update notification to the user. When the user taps the "Update Now" button, the update begins.

[0504] Through the above processing steps, the system enables users to receive golf swing diagnosis and lessons regardless of location or time, thereby achieving effective training.

[0505] (Application example 1)

[0506] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0507] Conventional motion analysis systems make it difficult for users to perform detailed analysis of their motions and conduct effective training regardless of location or time. Furthermore, they do not provide real-time feedback or simulations in virtual environments, limiting improvements based solely on analysis results. Furthermore, they lack the ability to learn and improve motivation by comparing data with other users.

[0508] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0509] In this invention, the server includes a means for filming the user's exercise movements, an artificial intelligence means for analyzing the filmed exercise video, a visual and audio feedback means for providing the user with the analysis results, a means for simulating exercise in a virtual environment, and a means for comparing with other users and providing lesson content. This allows the user to receive effective training regardless of location or time, and through real-time analysis and feedback, as well as simulation in a virtual environment and comparison with other users, the user can achieve exercise improvement with higher motivation.

[0510] "User" refers to an individual who utilizes the system to analyze and improve athletic performance.

[0511] "Athletic movements" refer to movements associated with specific sports or fitness, such as the golf swing.

[0512] "Means for taking photographs" refers to a function for recording the user's exercise movements using a device such as a smartphone or camera.

[0513] "Exercise video" refers to video data of a user's exercise movements that have been captured.

[0514] "Artificial intelligence means" refers to a system that uses AI algorithms to analyze exercise footage and extract and evaluate data about the user's movements.

[0515] "Analysis results" refers to information regarding the evaluation of exercise movements and areas for improvement generated by artificial intelligence means.

[0516] "Visual feedback means" refers to a function that provides visual information to the user through smart glasses or a smartphone screen.

[0517] "Audio feedback means" refers to a function that provides advice and information to the user by voice through a speaker or headphones.

[0518] "Virtual environment" refers to a computer-generated simulated environment.

[0519] "Movement simulation means" refers to a function that reproduces the user's movement within a virtual environment and provides a simulation based on the analysis results.

[0520] "Comparison means" refers to the functionality used to compare a user's athletic data with that of other users.

[0521] "Means for providing lesson content" refers to a function for providing users with educational content aimed at improving or learning exercise movements.

[0522] The present invention is a system that captures a user's exercise movements, analyzes the video using artificial intelligence, and provides the analysis results as visual and audio feedback. The realization of this system will be described in detail below.

[0523] First, users download a specific application onto their smartphone and create an account, then pair the smart glasses with the smartphone via Bluetooth, which enables the visual feedback mechanism.

[0524] When a user performs an exercise, the smartphone records the video. For example, when a user performs a golf swing, the user places the smartphone in an appropriate position, taps the "Start Recording" button, and records the swing video. At this time, the smartphone camera records the user's movements in real time and temporarily stores the video data.

[0525] The device then sends the captured video to a server, which then passes it to an AI algorithm to analyze the exercise. The server analyzes factors such as posture, timing, and speed to identify weaknesses and areas for improvement. The analysis results are compiled into a report and sent back to the device.

[0526] The device displays the analysis results on the smartphone screen, provides visual feedback through smart glasses, and provides specific audio advice via Bluetooth, such as "Your right shoulder is dropping too much at the top of your swing."

[0527] Users can also select "Simulation Mode" within the app to simulate athletic movements in a virtual environment, which provides a visual demonstration of how users can improve their movements in real time.

[0528] Furthermore, users can compare their data with that of other users to create more effective training plans, and can watch lesson content to learn specific training methods related to their movements.

[0529] This allows users to receive effective training regardless of location or time, and achieve improvements in their athletic performance through real-time analysis and feedback, as well as simulation in a virtual environment and comparison with other users.

[0530] As a concrete example, consider a user practicing their golf swing. In this case, the user wears smart glasses and films their swing with a smartphone. The captured video of their swing is analyzed by AI, and feedback such as "Your right shoulder is too low at the top of your swing" is displayed via the smart glasses and smartphone. After that, a simulation is performed on a virtual golf course, and further areas for improvement can be identified by comparing the data with other users' data.

[0531] Examples of prompts to input to a generative AI model include:

[0532] "Analyze the user's athletic movements and output improvements. The elements to analyze are posture, timing, and speed."

[0533] As described above, the present invention is a system that enables a user to effectively analyze and improve exercise movements regardless of location or time.

[0534] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0535] Step 1:

[0536] A user downloads a golf swing analysis application onto their smartphone and creates an account. They then enter their profile information and pair the smart glasses with their smartphone via Bluetooth. This information is sent to the server, which then creates an account and identifies the user's device.

[0537] Step 2:

[0538] The user performs an exercise (e.g., a golf swing) and captures it with the smartphone camera. The user taps the "Start Recording" button in the smartphone application to begin recording the exercise video. The captured video is temporarily saved in the smartphone's local storage. The input is the video of the user's movements, and the output is the saved video data.

[0539] Step 3:

[0540] The smartphone sends the saved video data to a server. The server receives this video data and passes it to an artificial intelligence tool to begin analysis. The input is the video data, and the output is the analysis results by the AI. Specifically, the video file is uploaded to the server via the internet, and then the data is input into the AI ​​model.

[0541] Step 4:

[0542] The server analyzes the received video data using an AI algorithm. Specifically, it processes and calculates data on each element of the movement, such as posture, timing, and speed, to identify shortcomings and areas for improvement. The input is the video data, and data processing involves extracting elements from each frame, followed by the actual analysis process. The output is a report of the analysis results.

[0543] Step 5:

[0544] The analysis results obtained from the server are sent back to the smartphone. The smartphone application receives these analysis results and provides visual and audio feedback to the user. The input is the analysis results, and the output is feedback to the user. Specifically, the application visualizes the analysis results and notifies the user in an easy-to-understand manner using text and audio.

[0545] Step 6:

[0546] The user selects "Simulation Mode" within the application and simulates their own athletic movements in a virtual environment. The device simulates the movement in real time and visually indicates areas for improvement. The input is the analysis results and current movement data, and the output is the simulation results. Specifically, a simulation begins within the app, highlighting areas for improvement.

[0547] Step 7:

[0548] Users compare their own exercise data with that of other users and watch lesson content. The application provides the comparison results and appropriate lesson videos. The input is the user's own and other users' exercise data, and the output is a comparison report and lesson videos. Specifically, the application retrieves other users' data from the database and performs a relative comparison.

[0549] In this way, this system supports effective training by analyzing the user's exercise movements in detail and suggesting areas for improvement.The following is an example of a prompt sentence input to the generative AI model.

[0550] "Analyze the user's athletic movements and output improvements. The elements to analyze are posture, timing, and speed."

[0551] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0552] The present invention is a system that films a user's golf swing, analyzes it using artificial intelligence, and provides the results through visual and audio feedback. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the feedback is adjusted based on the user's emotional state. The following describes in detail the embodiments of the present invention.

[0553] 1. Initial Setup

[0554] First, users download a compatible golf app onto their smartphone and create an account. Next, they enter their profile information. After that, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the system confirms the connection status and notifies the user.

[0555] 2. Swing photography

[0556] The user places their smartphone in a suitable position and films their golf swing. They tap the "Start Recording" button in the app to begin recording the swing video. The smartphone camera captures the user's swing video in real time and temporarily saves the video data.

[0557] 3. Data transmission and AI analysis

[0558] The device sends the captured swing video to a server. The server then passes the received swing video to an AI algorithm, which begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, to identify swing flaws and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[0559] 4. Swing Feedback

[0560] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[0561] 5. Feedback adjustment by emotion engine

[0562] The emotion engine recognizes the user's emotions. Emotion recognition is performed using facial expression analysis and voice tone analysis. The recognized emotional state is used to adjust the feedback content and method. For example, if the user looks tired, the feedback content can be changed to a more encouraging tone. Also, if the user is concentrating, more detailed technical instructions can be provided.

[0563] 6. Video lessons

[0564] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons. The content of the video is also customized based on the emotions recognized by the emotion engine.

[0565] 7. Monthly service and billing management

[0566] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that users always receive the latest lessons.

[0567] 8. Continued Use and Update Provision

[0568] The server collects user feedback and considers service improvements and feature additions based on that feedback. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally.

[0569] With the above configuration, the system of the present invention enables users to receive golf swing diagnosis and lessons regardless of location or time, achieving effective training. In addition, by combining it with an emotion engine, it provides feedback according to the user's emotional state, providing a more personalized training experience.

[0570] The processing flow will be explained below.

[0571] Step 1:

[0572] The user downloads the golf app on their smartphone and creates an account. The user enters the required profile information and completes the setup.

[0573] Step 2:

[0574] The user pairs the smart glasses with their smartphone via Bluetooth. The smartphone confirms the successful pairing and notifies the user.

[0575] Step 3:

[0576] The user positions the smartphone in the appropriate position and prepares to record their golf swing. The user taps the "Start Recording" button in the app to begin recording.

[0577] Step 4:

[0578] The device captures the user's golf swing in real time using a smartphone camera and temporarily stores the captured swing video.

[0579] Step 5:

[0580] The device sends the captured swing video to the server, which receives the video and passes it to the AI ​​algorithm.

[0581] Step 6:

[0582] The server uses an AI algorithm to analyze each element of the swing (posture, timing, speed, etc.) and compiles the analysis results in a report format.

[0583] Step 7:

[0584] The server sends a report of the analysis results back to the device, which receives the results and prepares them for display.

[0585] Step 8:

[0586] The device displays the analysis results on the smartphone screen, and the user checks them.

[0587] Step 9:

[0588] The device visually feeds back the analysis results through the smart glasses, which display the results.

[0589] Step 10:

[0590] The device provides voice feedback via Bluetooth, and the user receives voice advice.

[0591] Step 11:

[0592] The device activates an emotion engine and performs facial expression analysis and voice tone analysis to recognize the user's emotions.

[0593] Step 12:

[0594] The emotion engine identifies the user's emotional state and passes the results on to the rest of the system. For example, if the user is tired, they will receive the recognition result.

[0595] Step 13:

[0596] The device will adjust the feedback content based on the results of the emotion engine, for example adding an encouraging message if the user is tired.

[0597] Step 14:

[0598] The user selects "Watch Video Lessons" within the app and specifies the desired lesson content. The app prepares to retrieve the lesson video based on the request.

[0599] Step 15:

[0600] The device sends the appropriate video request to the server, which selects the appropriate lesson video in response to the request.

[0601] Step 16:

[0602] The server streams the lesson video to the device, which then plays the received video simultaneously on the smartphone and smart glasses.

[0603] Step 17:

[0604] Customize video content based on emotions recognized by the emotion engine, for example, providing detailed technical instructions if the user is distracted.

[0605] Step 18:

[0606] A user signs up for a monthly service, enters payment information, and begins their subscription.

[0607] Step 19:

[0608] The server manages billing for the monthly service, monitors user usage, and periodically delivers new lesson content and updates.

[0609] Step 20:

[0610] The server collects user feedback, considers service improvements and feature additions, and provides firmware updates for the app and smart glasses as needed.

[0611] Example 2

[0612] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0613] Conventional golf swing analysis systems have the ability to film a user's swing and provide the analysis results as visual and audio feedback, but they are unable to provide feedback that takes into account the user's emotional state. Furthermore, they lack the ability to provide real-time analysis and content based on the user's usage. Therefore, there is a need to provide users with a more effective and personalized training experience.

[0614] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0615] In this invention, the server includes a device that captures a user's actions, an artificial intelligence function that analyzes the captured video of the actions, a visual and audio feedback function that provides the user with the analysis results, an emotion recognition engine that recognizes the user's emotions and adjusts the feedback, and a content provision service. This makes it possible to analyze a user's actions in real time and provide personalized feedback according to the user's emotional state. It is also possible to deliver new content at an appropriate time based on the user's usage status.

[0616] The "device for capturing images of user actions" is a device for recording user actions in real time, and includes a smartphone camera, a camcorder, and the like.

[0617] "Artificial intelligence function that analyzes filmed video of movements" refers to artificial intelligence algorithms and software for analyzing filmed video of movements, and is a function that generates analysis results by evaluating the posture, timing, speed, etc. of the movement.

[0618] The "visual and audio feedback function for providing the analysis results to the user" is a function for presenting the analysis results to the user visually and audio-wise, and for providing information via a display or speaker.

[0619] The "emotion recognition engine that recognizes the user's emotions and adjusts feedback" is an engine that analyzes the user's facial expressions and tone of voice to recognize emotions, and appropriately adjusts the feedback content based on that information.

[0620] A "content provision service" is a service that delivers new learning and lesson content at appropriate times based on the user's usage status, and is managed and delivered on the server side.

[0621] A "head-mounted display" is a display device worn by a user on the head, which provides visual information.

[0622] "Real-time analysis" refers to the process of analyzing captured video instantly with almost no delay.

[0623] "Personalized feedback" refers to providing feedback optimized according to the characteristics and status of each individual user.

[0624] The present invention is a system that captures a user's actions, analyzes them using artificial intelligence, and provides the results through visual and audio feedback. Furthermore, by combining an emotion recognition engine that recognizes the user's emotions, the system also has the function of adjusting the feedback based on the user's emotional state. Specific embodiments for implementing the present invention are described in detail below.

[0625] 1. Initial Setup

[0626] First, users download a compatible golf app onto their smartphone and create an account. Next, they enter their profile information. After that, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the device will confirm the connection status and notify the user.

[0627] 2. Swing photography

[0628] The user places their smartphone in a suitable position and films their golf swing. They tap the "Start Recording" button in the app to begin recording the swing video. The smartphone camera captures the user's swing video in real time and temporarily saves the video data.

[0629] 3. Data transmission and AI analysis

[0630] The device sends the captured swing video to a server. The server then passes the received swing video to the AI ​​algorithm "SwingAI," which begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, to identify flaws and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[0631] 4. Swing Feedback

[0632] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[0633] 5. Feedback adjustment by emotion engine

[0634] The emotion recognition engine "EmotionAI" recognizes the user's emotions. Emotion recognition is performed by analyzing the user's facial expressions and vocal tone. The recognized emotional state is used to adjust the content and method of feedback. For example, if the user looks tired, the feedback content will be changed to a more encouraging tone. Also, if the user is concentrating, more detailed technical instructions will be provided.

[0635] 6. Video lessons

[0636] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons. The content of the video is also customized based on emotions recognized by the emotion recognition engine.

[0637] For example, if a user were to film their swing and use AI analysis, the prompt would look like this:

[0638] "Please provide me with feedback through the smart glasses so that I can improve my swing."

[0639] 7. Monthly service and billing management

[0640] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that users always receive the latest lessons.

[0641] 8. Continued Use and Update Provision

[0642] The server collects user feedback and considers service improvements and feature additions based on it. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally. For example, if the user is tired, it adjusts the feedback it provides to them in a more encouraging tone.

[0643] This system allows users to receive golf swing diagnosis and lessons regardless of location or time, enabling effective training. In addition, by combining it with an emotion recognition engine, it is possible to provide feedback according to the user's emotional state, providing a more personalized training experience.

[0644] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0645] Step 1:

[0646] Initial Setup

[0647] A user downloads a golf app onto their smartphone and creates an account by entering their username, email address, and password. This registers the account information in the app.

[0648] Next, users enter profile information (e.g., years of golf experience and handicap), which generates the basis for customized analysis and feedback.

[0649] Pair the smart glasses with the smartphone via Bluetooth, using the smart glasses' identification information and Bluetooth connection data as input, and notify the smartphone once pairing is successful.

[0650] Step 2:

[0651] Swing shooting

[0652] The user places the smartphone in an appropriate position and records the golf swing. The input is the smartphone camera position and shooting angle information.

[0653] When you tap the "Start Recording" button, the smartphone camera will start recording your swing video. The real-time video is saved temporarily during recording, and the data is saved after the recording ends. The output is a recorded swing video file.

[0654] Step 3:

[0655] Data transmission and AI analysis

[0656] The device sends the saved swing video to the server, using the video file path and the server URL as input.

[0657] The server passes the received swing video to the AI ​​algorithm "SwingAI" and begins analysis. The video data and user profile information are used as input. The AI ​​algorithm analyzes the swing posture, timing, speed, etc., and generates a report of the analysis results as output.

[0658] The analysis results are compiled in a report format and sent back to the terminal. An analysis report is sent to the terminal as output.

[0659] Step 4:

[0660] Swing Feedback

[0661] The terminal displays the received analysis results on the smartphone screen. It reads the analysis report as input and generates screen data to be displayed on the smartphone as output.

[0662] It also provides visual feedback through smart glasses and sends audio advice via Bluetooth. It uses text and audio analysis results as input. It provides visual and audio feedback as output. For example, it displays a message like, "Your right shoulder is dropping too low at the top of your swing."

[0663] Step 5:

[0664] Feedback adjustment by emotion engine

[0665] The emotion recognition engine "EmotionAI" uses the user's facial expression images and voice tone data as input to recognize the user's emotions.

[0666] The emotion engine analyzes these data and outputs the user's emotional state. The output is the user's current emotional state (e.g., tired, focused).

[0667] The system adjusts the feedback content and method based on the recognized emotional state. It uses the emotional state data and analysis result data as input and generates adjusted feedback as output. For example, if the user is tired, the system adjusts the feedback to be provided in an encouraging tone.

[0668] Step 6:

[0669] Video lessons

[0670] The user selects "Watch a video lesson" within the app and specifies the relevant lesson video. The user's selection information is used as input.

[0671] The device receives the relevant lesson video from the server and plays it on the smartphone or smart glasses. The URL or path of the video file is used as input. The lesson video is played as output.

[0672] The content of the video is also customized based on the emotions recognized by the emotion engine. Emotional state data and video data are used as input, and customized video content is provided as output.

[0673] Step 7:

[0674] Monthly service and billing management

[0675] When a user signs up for a monthly service, the user's registration information and billing information are used as input.

[0676] The server processes monthly charges and monitors usage using a billing management mechanism, generating a billing completion notice and usage report as outputs.

[0677] Delivering new lesson content and updates at the appropriate time, using usage data as input and generating new content delivery data as output.

[0678] Step 8:

[0679] Continued use and updates

[0680] The server collects user feedback and considers improving the service or adding new features based on it. The feedback data is used as input.

[0681] Provides app and smartglasses firmware updates as needed, using update data as input and providing updated software and firmware as output.

[0682] This ensures that your system always performs optimally.

[0683] (Application example 2)

[0684] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0685] The present invention aims to solve the problems of conventional golf swing analysis systems, such as limited returns when users try out equipment, feedback that does not take into account emotional state, and a lack of specific support for users to select appropriate equipment.

[0686] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a device means for filming the user's hitting motion, a machine learning means for analyzing the filmed hitting motion video, a visual and audio feedback means for providing the analysis results to the user, an emotion engine means for recognizing the user's emotional state and adjusting the feedback, and a means for recommending products for the user to try in a real-world store environment. This provides feedback according to the user's emotional state, enabling hitting motion analysis and appropriate product selection in a real store.

[0687] The term "user's hitting motion" refers to any swing or hitting motion performed with a golf club or other sporting equipment.

[0688] The term "photography device" refers to an electronic device or device, such as a camera or smartphone, that records a user's hitting motion as video.

[0689] "Machine learning means" refers to a technical means that uses artificial intelligence or machine learning algorithms to analyze and evaluate filmed footage of hitting movements.

[0690] "Visual and audio feedback means" refers to a display device or audio output device for providing the analysis results to the user visually or audio.

[0691] "Emotion engine" refers to a technical means for recognizing a user's emotional state by analyzing their facial expressions and tone of voice, and adjusting the feedback provided based on that state.

[0692] "Recommendation method" refers to the algorithms and systems used to select and recommend products for users to try out in physical stores.

[0693] "Display device" refers to a device that provides information visually to a user, such as smart glasses or a head-mounted display.

[0694] The system captures a user's hitting motion, analyzes it using machine learning, and provides the results through visual and audio feedback. By combining it with an emotion engine that recognizes the user's emotional state, the system can tailor the feedback based on the user's emotional state. Furthermore, the system has the ability to recommend products for the user to try in a brick-and-mortar store environment.

[0695] 1. Initial Setup

[0696] Users first download the corresponding app onto their smartphones, create an account, enter their profile information, and pair their smart glasses or other display device with their smartphone via Bluetooth. Once pairing is successful, the system will confirm the connection status and notify the user.

[0697] 2. Filming the hitting motion

[0698] The user places their smartphone in an appropriate position and films their batting motion. They tap the "Start Recording" button in the app to begin recording the video of their batting motion. The smartphone camera captures video of the user's batting motion in real time and temporarily saves the video data.

[0699] 3. Data transmission and AI analysis

[0700] The device sends the captured video of the batting motion to a server. The server then passes the video to a machine learning algorithm, which begins analysis. The machine learning algorithm analyzes each element of the batting motion, such as posture, timing, and speed, and identifies weaknesses and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[0701] 4. Providing Feedback

[0702] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through the display device and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is lowered too much at the top of your hitting motion," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[0703] 5. Feedback adjustment by emotion engine

[0704] The server uses an emotion engine to recognize the user's emotional state. Emotion recognition is performed by analyzing the user's facial expressions and voice tone. The recognized emotional state is used to adjust the content and method of feedback. For example, if the user looks tired, the feedback content can be changed to a more encouraging tone. If the user is concentrating, more detailed technical instructions can be provided.

[0705] Hardware and software used

[0706] Hardware: Smartphone camera, smart glasses, Bluetooth module

[0707] Software: Machine learning algorithms, emotion recognition engine, Bluetooth connectivity software

[0708] Specific examples

[0709] For example, imagine a scenario where a user is trying out golf clubs at a brick-and-mortar store. The user takes a video of their swing with their smartphone and sends it to a server. A machine learning algorithm identifies flaws in the swing, and an emotion engine recognizes the user's level of fatigue. Based on this, real-time feedback such as "Your swing angle isn't good. Take a short break to refresh yourself" is displayed on the smart glasses.

[0710] Example prompts for generative AI models

[0711] "Please point out areas for improvement when the golf swing angle is poor. Also, add encouraging comments if the user is tired."

[0712] The above is a detailed description of the embodiment of the present invention. This system is very useful for users in physical stores to analyze their hitting motion and select the most suitable equipment.

[0713] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0714] Step 1:

[0715] A user downloads the app to their smartphone and creates an account. They enter their profile information and pair their smart glasses or other display device with the smartphone via Bluetooth. A confirmation of successful pairing appears on the device.

[0716] Step 2:

[0717] The user places their smartphone in an appropriate position and records their hitting motion. When the user taps the "Start Recording" button in the app, the smartphone camera begins recording their hitting motion in real time. This video data is temporarily saved on the smartphone.

[0718] Step 3:

[0719] The device sends the captured video of the batting motion to the server. The input data is a video file of the batting motion. After receiving this, the server passes the video data to a machine learning algorithm and begins analysis. The output is the analysis results in JSON format data.

[0720] Step 4:

[0721] The server uses machine learning algorithms to analyze each element of the hitting motion, including posture, timing, and speed. The analysis results include swing flaws and areas for improvement. After data processing and calculations, the results are compiled into a report.

[0722] Step 5:

[0723] The server sends the analysis results back to the device, which then visually displays the received analysis results on the app's user interface. Additionally, the device provides visual feedback via a display device and specific advice via audio via Bluetooth. At this time, feedback based on the analysis results is output as text and audio files.

[0724] Step 6:

[0725] The server uses an emotion engine to recognize the user's emotional state. The input data is the user's facial expression and voice tone, which are analyzed by the emotion recognition algorithm. The output is tag information that indicates the user's emotional state.

[0726] Step 7:

[0727] The server adjusts the feedback content and method based on the user's recognized emotional state. For example, if the user looks tired, the feedback tone is changed to be more encouraging. If the user is concentrating, detailed technical instructions are added. The text and audio files with the adjusted feedback content and tone are output.

[0728] Step 8:

[0729] The server recommends products for the user to try in a physical store environment. The input data is the analysis results and the user's profile information, which are analyzed by the recommendation algorithm. The output is a list of optimal products. The device presents this list to the user within the app.

[0730] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0731] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0732] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0733] [Third embodiment]

[0734] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0735] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0736] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0737] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0738] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0739] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0740] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0741] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0742] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0743] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0744] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0745] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0746] The present invention is a system that captures a user's golf swing, analyzes it using artificial intelligence, and provides the results through visual and audio feedback. Using a smartphone and smart glasses, users can receive golf swing diagnosis and lessons anywhere, anytime. The following describes in detail the embodiments of the present invention.

[0747] 1. Initial Setup

[0748] First, users download a compatible golf app onto their smartphone. Next, they create an account and enter their profile information. Then, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the system confirms the connection status and notifies the user.

[0749] 2. Swing photography

[0750] The user places their smartphone in a suitable position and films their golf swing. They tap the "Start Recording" button in the app to begin recording the swing video. The smartphone camera captures the user's swing video in real time and temporarily saves the video data.

[0751] 3. Data transmission and AI analysis

[0752] The device sends the captured swing video to a server. The server then passes the received swing video to an AI algorithm, which begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, to identify swing flaws and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[0753] 4. Swing Feedback

[0754] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[0755] 5. Video lessons

[0756] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons.

[0757] 6. Monthly service and billing management

[0758] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that users always receive the latest lessons.

[0759] 7. Continued Use and Update Provision

[0760] The server collects user feedback and considers service improvements and feature additions based on that feedback. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally.

[0761] With the above configuration, the system of the present invention allows users to receive golf swing diagnosis and lessons regardless of location or time, thereby achieving effective training.

[0762] The processing flow will be explained below.

[0763] Step 1:

[0764] The user downloads the golf app on their smartphone and creates an account. The user completes the process by entering their profile information.

[0765] Step 2:

[0766] The user pairs the smart glasses with their smartphone via Bluetooth. The device notifies them that pairing was successful.

[0767] Step 3:

[0768] The user places the smartphone in the appropriate position and prepares to record their swing. The user then taps the "Start Recording" button in the app to begin recording.

[0769] Step 4:

[0770] The device captures the user's golf swing in real time using a smartphone camera and temporarily saves the captured swing video.

[0771] Step 5:

[0772] The device sends the captured swing video to the server, which receives the video and passes it to the AI ​​algorithm.

[0773] Step 6:

[0774] The server uses an AI algorithm to analyze each element of the swing (posture, timing, speed, etc.) and compiles the analysis results in a report format.

[0775] Step 7:

[0776] The server sends a report of the analysis results back to the device, which receives the analysis results.

[0777] Step 8:

[0778] The device displays the analysis results on the smartphone screen, where the user can check them.

[0779] Step 9:

[0780] The device visually feeds back the analysis results through the smart glasses, and the user receives the visual feedback.

[0781] Step 10:

[0782] The device provides voice feedback via Bluetooth, and the user receives voice advice.

[0783] Step 11:

[0784] The user selects "Watch Video Lessons" within the app and specifies the desired lesson content.

[0785] Step 12:

[0786] The device sends the appropriate video request to the server, which selects the corresponding lesson video in response to the request.

[0787] Step 13:

[0788] The server streams the lesson video to the device, and the device receives the lesson video.

[0789] Step 14:

[0790] The lesson video received by the device is played simultaneously on the smartphone and smart glasses, allowing the user to receive a visual lesson.

[0791] Step 15:

[0792] A user signs up for a monthly service. The user enters payment information and begins the subscription.

[0793] Step 16:

[0794] The server manages billing for the monthly service and periodically delivers new lesson content and updates to users.

[0795] Step 17:

[0796] The server collects user feedback, considers service improvements and feature additions, and provides firmware updates for the app and smart glasses as needed.

[0797] Example 1

[0798] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0799] Conventional golf swing analysis systems require users to spend a lot of time obtaining analysis results and often do not provide results in real time. Furthermore, they do not adequately integrate visual and audio feedback, and do not provide an effective training environment for users. Furthermore, there is a lack of systems that allow users to train anywhere, anytime using their own devices.

[0800] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0801] In this invention, the server includes a means for capturing images of the user's movements, an artificial intelligence means for analyzing the captured images of the movements, a visual and audio feedback means for providing the user with the analysis results, a terminal for the user, and a means for displaying the analysis results and providing advice using the terminal. This allows the user to obtain the analysis results in real time, enabling training regardless of location or time. Furthermore, by providing integrated visual and audio feedback, the user can train more effectively.

[0802] "User behavior" refers to a specific action or behavior generally undertaken by an individual.

[0803] "Photographing means" refers to a device or method for recording actions or movements using a camera or related photographic device.

[0804] "Motion footage" refers to video data of a captured movement.

[0805] "Artificial intelligence means for analysis" refers to AI technology that analyzes the characteristics and shortcomings of behavior based on collected data and extracts areas for improvement.

[0806] "Analysis results" refers to the data and reports obtained after analyzing motion footage.

[0807] "Means for providing visual and audio feedback to the user" refers to a method or device for conveying the analysis results to the user by visual information or audio.

[0808] "Terminal" refers to any electronic device used by a User to access the System, including, for example, a smartphone or tablet.

[0809] "Video display device" refers to any device for displaying images or video, including, for example, smart glasses and monitors.

[0810] "Means for providing advice" refers to a method or device for informing the user of improvements based on the analysis results.

[0811] The present invention is a system that allows a user to receive golf swing diagnosis and lessons regardless of location or time. The system includes a means for the user to film their movements, an artificial intelligence means for analyzing the filmed video of their movements, a visual and audio feedback means for providing the analysis results to the user, a terminal for use by the user, and a means for displaying the analysis results and providing advice using the terminal.

[0812] Initial Setup

[0813] First, the user downloads a compatible golf app onto their smartphone. In this example, the app name is "GolfSwingAnalyzer." Next, they create an account and enter their profile information. After that, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the device checks the connection status and notifies the user.

[0814] Swing shooting

[0815] The user places their smartphone in a suitable position and taps the "Start Recording" button in the app to record their golf swing. The device uses the smartphone camera to capture a video of the user's swing in real time and temporarily stores the video data. For example, the user can mount their smartphone on a tripod placed on the ground and adjust the screen so that the swing area fits within the screen.

[0816] Data transmission and AI analysis

[0817] The device sends the recorded swing video to a server. The server then passes the received swing video to an AI algorithm (e.g., TensorFlow) and begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, to identify swing flaws and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time. An example prompt is, "Analyze the user's swing video and generate specific improvement suggestions for shoulder movement."

[0818] Swing Feedback

[0819] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result might say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement might be, "To improve your shoulder movement, try practicing the following."

[0820] Video lessons

[0821] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons.

[0822] Monthly service and billing management

[0823] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that the user always receives the latest lessons.

[0824] Continued use and updates

[0825] The server collects user feedback and uses it to improve the service and add new features. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally.

[0826] In this way, the system of the present invention allows the user to receive golf swing diagnosis and lessons regardless of location or time, thereby achieving effective training.

[0827] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0828] Step 1:

[0829] The user downloads and installs the golf app "GolfSwingAnalyzer" onto their smartphone. At this time, the user enters search keywords into the smartphone's App Store or Google Play Store, and the app is found and installed based on those keywords.

[0830] Step 2:

[0831] A user launches the app and creates an account. Here, they enter their name, email address, and password. An account is created based on the information they entered. As an output, a user profile is generated and saved in the database. Specifically, the user enters the required information in each input field and taps the "Register" button.

[0832] Step 3:

[0833] The user pairs the smart glasses with their smartphone via Bluetooth. Input includes the smart glasses' device name and pairing code. The device detects the smart glasses based on this and performs pairing. If successful, the device checks the connection status and sends a notification to the user. Specifically, the user sets the smart glasses to connection mode and taps the "Device Pairing" button in the app.

[0834] Step 4:

[0835] The user places the smartphone in an appropriate position and taps the "Start Recording" button in the app to record their golf swing. The input includes the user's movement information (swing movement). The device uses the smartphone camera to record this movement and temporarily saves it as video data. Specific actions include the user fixing the smartphone to a tripod or other device and adjusting it so that the swing area fits on the screen.

[0836] Step 5:

[0837] The device sends the recorded swing video to the server. The input is the temporarily stored video data, which is sent via Wi-Fi or mobile data communication. The output is the generated video data stored on the server. The specific operation requires the user to tap the "Send Data" button.

[0838] Step 6:

[0839] The server passes the received swing video to an AI algorithm and begins analyzing the golf swing. The input is video data. The AI ​​algorithm (for example, TensorFlow) analyzes this and performs data calculations on swing posture, timing, speed, etc. to identify flaws and areas for improvement. The output is an analysis report. Specifically, the server passes the video data to the analysis module and executes the analysis process.

[0840] Step 7:

[0841] The server compiles the analysis results into a report format and sends it back to the device in real time. The input is the data analyzed by the AI. The output is an analysis report in JSON format that is generated and sent to the device. Specifically, the server formats the analysis data and sends it to the device.

[0842] Step 8:

[0843] The device displays the received analysis results on the smartphone screen. The input is an analysis report. The output is a graph or text display of the analysis results. Specifically, the app UI on the device performs processing to visually display the analysis results.

[0844] Step 9:

[0845] The device provides visual feedback to the smart glasses and specific audio advice via Bluetooth. The input is the analysis results. The output is visual and audio feedback. Specific operations include an overlay of the analysis results summary on the smart glasses and audio guidance.

[0846] Step 10:

[0847] The user selects "Watch a video lesson" within the app and selects a lesson video related to their swing. The input is the user's request for the lesson video they selected. The device receives the corresponding lesson video from the server and plays it simultaneously on the smartphone and smart glasses. The output is the playback of the lesson video. Specifically, the user taps the video play button, and the video plays synchronously on both devices.

[0848] Step 11:

[0849] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The input is the user's credit card information and registration information. The output is the completion of the monthly billing and the continuation of service use. Specifically, the user taps the "Sign up for monthly service" button and enters their credit card information.

[0850] Step 12:

[0851] The server monitors user usage and delivers new lesson content and updated information at appropriate times. The input is user usage data. The output is the provision of new lesson content. Specifically, the server periodically checks usage and delivers new content.

[0852] Step 13:

[0853] The server collects feedback from users and considers improving the service or adding new features based on that information. The input is user feedback information. The output is the development of new features and improvements to the service. Specifically, the user enters and submits their opinions and requests using the "Feedback Submission" form within the app.

[0854] Step 14:

[0855] The server provides firmware updates for the app and smart glasses as needed. The input is the latest version information. The output is the provision of updated firmware. Specifically, when a new version is released, the app automatically checks for updates and sends an update notification to the user. When the user taps the "Update Now" button, the update begins.

[0856] Through the above processing steps, the system enables users to receive golf swing diagnosis and lessons regardless of location or time, thereby achieving effective training.

[0857] (Application example 1)

[0858] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0859] Conventional motion analysis systems make it difficult for users to perform detailed analysis of their motions and conduct effective training regardless of location or time. Furthermore, they do not provide real-time feedback or simulations in virtual environments, limiting improvements based solely on analysis results. Furthermore, they lack the ability to learn and improve motivation by comparing data with other users.

[0860] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0861] In this invention, the server includes a means for filming the user's exercise movements, an artificial intelligence means for analyzing the filmed exercise video, a visual and audio feedback means for providing the user with the analysis results, a means for simulating exercise in a virtual environment, and a means for comparing with other users and providing lesson content. This allows the user to receive effective training regardless of location or time, and through real-time analysis and feedback, as well as simulation in a virtual environment and comparison with other users, the user can achieve exercise improvement with higher motivation.

[0862] "User" refers to an individual who utilizes the system to analyze and improve athletic performance.

[0863] "Athletic movements" refer to movements associated with specific sports or fitness, such as the golf swing.

[0864] "Means for taking photographs" refers to a function for recording the user's exercise movements using a device such as a smartphone or camera.

[0865] "Exercise video" refers to video data of a user's exercise movements that have been captured.

[0866] "Artificial intelligence means" refers to a system that uses AI algorithms to analyze exercise footage and extract and evaluate data about the user's movements.

[0867] "Analysis results" refers to information regarding the evaluation of exercise movements and areas for improvement generated by artificial intelligence means.

[0868] "Visual feedback means" refers to a function that provides visual information to the user through smart glasses or a smartphone screen.

[0869] "Audio feedback means" refers to a function that provides advice and information to the user by voice through a speaker or headphones.

[0870] "Virtual environment" refers to a computer-generated simulated environment.

[0871] "Movement simulation means" refers to a function that reproduces the user's movement within a virtual environment and provides a simulation based on the analysis results.

[0872] "Comparison means" refers to the functionality used to compare a user's athletic data with that of other users.

[0873] "Means for providing lesson content" refers to a function for providing users with educational content aimed at improving or learning exercise movements.

[0874] The present invention is a system that captures a user's exercise movements, analyzes the video using artificial intelligence, and provides the analysis results as visual and audio feedback. The realization of this system will be described in detail below.

[0875] First, users download a specific application onto their smartphone and create an account, then pair the smart glasses with the smartphone via Bluetooth, which enables the visual feedback mechanism.

[0876] When a user performs an exercise, the smartphone records the video. For example, when a user performs a golf swing, the user places the smartphone in an appropriate position, taps the "Start Recording" button, and records the swing video. At this time, the smartphone camera records the user's movements in real time and temporarily stores the video data.

[0877] The device then sends the captured video to a server, which then passes it to an AI algorithm to analyze the exercise. The server analyzes factors such as posture, timing, and speed to identify weaknesses and areas for improvement. The analysis results are compiled into a report and sent back to the device.

[0878] The device displays the analysis results on the smartphone screen, provides visual feedback through smart glasses, and provides specific audio advice via Bluetooth, such as "Your right shoulder is dropping too much at the top of your swing."

[0879] Users can also select "Simulation Mode" within the app to simulate athletic movements in a virtual environment, which provides a visual demonstration of how users can improve their movements in real time.

[0880] Furthermore, users can compare their data with that of other users to create more effective training plans, and can watch lesson content to learn specific training methods related to their movements.

[0881] This allows users to receive effective training regardless of location or time, and achieve improvements in their athletic performance through real-time analysis and feedback, as well as simulation in a virtual environment and comparison with other users.

[0882] As a concrete example, consider a user practicing their golf swing. In this case, the user wears smart glasses and films their swing with a smartphone. The captured video of their swing is analyzed by AI, and feedback such as "Your right shoulder is too low at the top of your swing" is displayed via the smart glasses and smartphone. After that, a simulation is performed on a virtual golf course, and further areas for improvement can be identified by comparing the data with other users' data.

[0883] Examples of prompts to input to a generative AI model include:

[0884] "Analyze the user's athletic movements and output improvements. The elements to analyze are posture, timing, and speed."

[0885] As described above, the present invention is a system that enables a user to effectively analyze and improve exercise movements regardless of location or time.

[0886] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0887] Step 1:

[0888] A user downloads a golf swing analysis application onto their smartphone and creates an account. They then enter their profile information and pair the smart glasses with their smartphone via Bluetooth. This information is sent to the server, which then creates an account and identifies the user's device.

[0889] Step 2:

[0890] The user performs an exercise (e.g., a golf swing) and captures it with the smartphone camera. The user taps the "Start Recording" button in the smartphone application to begin recording the exercise video. The captured video is temporarily saved in the smartphone's local storage. The input is the video of the user's movements, and the output is the saved video data.

[0891] Step 3:

[0892] The smartphone sends the saved video data to a server. The server receives this video data and passes it to an artificial intelligence tool to begin analysis. The input is the video data, and the output is the analysis results by the AI. Specifically, the video file is uploaded to the server via the internet, and then the data is input into the AI ​​model.

[0893] Step 4:

[0894] The server analyzes the received video data using an AI algorithm. Specifically, it processes and calculates data on each element of the movement, such as posture, timing, and speed, to identify shortcomings and areas for improvement. The input is the video data, and data processing involves extracting elements from each frame, followed by the actual analysis process. The output is a report of the analysis results.

[0895] Step 5:

[0896] The analysis results obtained from the server are sent back to the smartphone. The smartphone application receives these analysis results and provides visual and audio feedback to the user. The input is the analysis results, and the output is feedback to the user. Specifically, the application visualizes the analysis results and notifies the user in an easy-to-understand manner using text and audio.

[0897] Step 6:

[0898] The user selects "Simulation Mode" within the application and simulates their own athletic movements in a virtual environment. The device simulates the movement in real time and visually indicates areas for improvement. The input is the analysis results and current movement data, and the output is the simulation results. Specifically, a simulation begins within the app, highlighting areas for improvement.

[0899] Step 7:

[0900] Users compare their own exercise data with that of other users and watch lesson content. The application provides the comparison results and appropriate lesson videos. The input is the user's own and other users' exercise data, and the output is a comparison report and lesson videos. Specifically, the application retrieves other users' data from the database and performs a relative comparison.

[0901] In this way, this system supports effective training by analyzing the user's exercise movements in detail and suggesting areas for improvement.The following is an example of a prompt sentence input to the generative AI model.

[0902] "Analyze the user's athletic movements and output improvements. The elements to analyze are posture, timing, and speed."

[0903] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0904] The present invention is a system that films a user's golf swing, analyzes it using artificial intelligence, and provides the results through visual and audio feedback. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the feedback is adjusted based on the user's emotional state. The following describes in detail the embodiments of the present invention.

[0905] 1. Initial Setup

[0906] First, users download a compatible golf app onto their smartphone and create an account. Next, they enter their profile information. After that, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the system confirms the connection status and notifies the user.

[0907] 2. Swing photography

[0908] The user places their smartphone in a suitable position and films their golf swing. They tap the "Start Recording" button in the app to begin recording the swing video. The smartphone camera captures the user's swing video in real time and temporarily saves the video data.

[0909] 3. Data transmission and AI analysis

[0910] The device sends the captured swing video to a server. The server then passes the received swing video to an AI algorithm, which begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, to identify swing flaws and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[0911] 4. Swing Feedback

[0912] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[0913] 5. Feedback adjustment by emotion engine

[0914] The emotion engine recognizes the user's emotions. Emotion recognition is performed using facial expression analysis and voice tone analysis. The recognized emotional state is used to adjust the feedback content and method. For example, if the user looks tired, the feedback content can be changed to a more encouraging tone. Also, if the user is concentrating, more detailed technical instructions can be provided.

[0915] 6. Video lessons

[0916] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons. The content of the video is also customized based on the emotions recognized by the emotion engine.

[0917] 7. Monthly service and billing management

[0918] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that users always receive the latest lessons.

[0919] 8. Continued Use and Update Provision

[0920] The server collects user feedback and considers service improvements and feature additions based on that feedback. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally.

[0921] With the above configuration, the system of the present invention enables users to receive golf swing diagnosis and lessons regardless of location or time, achieving effective training. In addition, by combining it with an emotion engine, it provides feedback according to the user's emotional state, providing a more personalized training experience.

[0922] The processing flow will be explained below.

[0923] Step 1:

[0924] The user downloads the golf app on their smartphone and creates an account. The user enters the required profile information and completes the setup.

[0925] Step 2:

[0926] The user pairs the smart glasses with their smartphone via Bluetooth. The smartphone confirms the successful pairing and notifies the user.

[0927] Step 3:

[0928] The user positions the smartphone in the appropriate position and prepares to record their golf swing. The user taps the "Start Recording" button in the app to begin recording.

[0929] Step 4:

[0930] The device captures the user's golf swing in real time using a smartphone camera and temporarily stores the captured swing video.

[0931] Step 5:

[0932] The device sends the captured swing video to the server, which receives the video and passes it to the AI ​​algorithm.

[0933] Step 6:

[0934] The server uses an AI algorithm to analyze each element of the swing (posture, timing, speed, etc.) and compiles the analysis results in a report format.

[0935] Step 7:

[0936] The server sends a report of the analysis results back to the device, which receives the results and prepares them for display.

[0937] Step 8:

[0938] The device displays the analysis results on the smartphone screen, and the user checks them.

[0939] Step 9:

[0940] The device visually feeds back the analysis results through the smart glasses, which display the results.

[0941] Step 10:

[0942] The device provides voice feedback via Bluetooth, and the user receives voice advice.

[0943] Step 11:

[0944] The device activates an emotion engine and performs facial expression analysis and voice tone analysis to recognize the user's emotions.

[0945] Step 12:

[0946] The emotion engine identifies the user's emotional state and passes the results on to the rest of the system. For example, if the user is tired, they will receive the recognition result.

[0947] Step 13:

[0948] The device will adjust the feedback content based on the results of the emotion engine, for example adding an encouraging message if the user is tired.

[0949] Step 14:

[0950] The user selects "Watch Video Lessons" within the app and specifies the desired lesson content. The app prepares to retrieve the lesson video based on the request.

[0951] Step 15:

[0952] The device sends the appropriate video request to the server, which selects the appropriate lesson video in response to the request.

[0953] Step 16:

[0954] The server streams the lesson video to the device, which then plays the received video simultaneously on the smartphone and smart glasses.

[0955] Step 17:

[0956] Customize video content based on emotions recognized by the emotion engine, for example, providing detailed technical instructions if the user is distracted.

[0957] Step 18:

[0958] A user signs up for a monthly service, enters payment information, and begins their subscription.

[0959] Step 19:

[0960] The server manages billing for the monthly service, monitors user usage, and periodically delivers new lesson content and updates.

[0961] Step 20:

[0962] The server collects user feedback, considers service improvements and feature additions, and provides firmware updates for the app and smart glasses as needed.

[0963] Example 2

[0964] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0965] Conventional golf swing analysis systems have the ability to film a user's swing and provide the analysis results as visual and audio feedback, but they are unable to provide feedback that takes into account the user's emotional state. Furthermore, they lack the ability to provide real-time analysis and content based on the user's usage. Therefore, there is a need to provide users with a more effective and personalized training experience.

[0966] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0967] In this invention, the server includes a device that captures a user's actions, an artificial intelligence function that analyzes the captured video of the actions, a visual and audio feedback function that provides the user with the analysis results, an emotion recognition engine that recognizes the user's emotions and adjusts the feedback, and a content provision service. This makes it possible to analyze a user's actions in real time and provide personalized feedback according to the user's emotional state. It is also possible to deliver new content at an appropriate time based on the user's usage status.

[0968] The "device for capturing images of user actions" is a device for recording user actions in real time, and includes a smartphone camera, a camcorder, and the like.

[0969] "Artificial intelligence function that analyzes filmed video of movements" refers to artificial intelligence algorithms and software for analyzing filmed video of movements, and is a function that generates analysis results by evaluating the posture, timing, speed, etc. of the movement.

[0970] The "visual and audio feedback function for providing the analysis results to the user" is a function for presenting the analysis results to the user visually and audio-wise, and for providing information via a display or speaker.

[0971] The "emotion recognition engine that recognizes the user's emotions and adjusts feedback" is an engine that analyzes the user's facial expressions and tone of voice to recognize emotions, and appropriately adjusts the feedback content based on that information.

[0972] A "content provision service" is a service that delivers new learning and lesson content at appropriate times based on the user's usage status, and is managed and delivered on the server side.

[0973] A "head-mounted display" is a display device worn by a user on the head, which provides visual information.

[0974] "Real-time analysis" refers to the process of analyzing captured video instantly with almost no delay.

[0975] "Personalized feedback" refers to providing feedback optimized according to the characteristics and status of each individual user.

[0976] The present invention is a system that captures a user's actions, analyzes them using artificial intelligence, and provides the results through visual and audio feedback. Furthermore, by combining an emotion recognition engine that recognizes the user's emotions, the system also has the function of adjusting the feedback based on the user's emotional state. Specific embodiments for implementing the present invention are described in detail below.

[0977] 1. Initial Setup

[0978] First, users download a compatible golf app onto their smartphone and create an account. Next, they enter their profile information. After that, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the device will confirm the connection status and notify the user.

[0979] 2. Swing photography

[0980] The user places their smartphone in a suitable position and films their golf swing. They tap the "Start Recording" button in the app to begin recording the swing video. The smartphone camera captures the user's swing video in real time and temporarily saves the video data.

[0981] 3. Data transmission and AI analysis

[0982] The device sends the captured swing video to a server. The server then passes the received swing video to the AI ​​algorithm "SwingAI," which begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, to identify flaws and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[0983] 4. Swing Feedback

[0984] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[0985] 5. Feedback adjustment by emotion engine

[0986] The emotion recognition engine "EmotionAI" recognizes the user's emotions. Emotion recognition is performed by analyzing the user's facial expressions and vocal tone. The recognized emotional state is used to adjust the content and method of feedback. For example, if the user looks tired, the feedback content will be changed to a more encouraging tone. Also, if the user is concentrating, more detailed technical instructions will be provided.

[0987] 6. Video lessons

[0988] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons. The content of the video is also customized based on emotions recognized by the emotion recognition engine.

[0989] For example, if a user were to film their swing and use AI analysis, the prompt would look like this:

[0990] "Please provide me with feedback through the smart glasses so that I can improve my swing."

[0991] 7. Monthly service and billing management

[0992] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that users always receive the latest lessons.

[0993] 8. Continued Use and Update Provision

[0994] The server collects user feedback and considers service improvements and feature additions based on it. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally. For example, if the user is tired, it adjusts the feedback it provides to them in a more encouraging tone.

[0995] This system allows users to receive golf swing diagnosis and lessons regardless of location or time, enabling effective training. In addition, by combining it with an emotion recognition engine, it is possible to provide feedback according to the user's emotional state, providing a more personalized training experience.

[0996] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0997] Step 1:

[0998] Initial Setup

[0999] A user downloads a golf app onto their smartphone and creates an account by entering their username, email address, and password. This registers the account information in the app.

[1000] Next, users enter profile information (e.g., years of golf experience and handicap), which generates the basis for customized analysis and feedback.

[1001] Pair the smart glasses with the smartphone via Bluetooth, using the smart glasses' identification information and Bluetooth connection data as input, and notify the smartphone once pairing is successful.

[1002] Step 2:

[1003] Swing shooting

[1004] The user places the smartphone in an appropriate position and records the golf swing. The input is the smartphone camera position and shooting angle information.

[1005] When you tap the "Start Recording" button, the smartphone camera will start recording your swing video. The real-time video is saved temporarily during recording, and the data is saved after the recording ends. The output is a recorded swing video file.

[1006] Step 3:

[1007] Data transmission and AI analysis

[1008] The device sends the saved swing video to the server, using the video file path and the server URL as input.

[1009] The server passes the received swing video to the AI ​​algorithm "SwingAI" and begins analysis. The video data and user profile information are used as input. The AI ​​algorithm analyzes the swing posture, timing, speed, etc., and generates a report of the analysis results as output.

[1010] The analysis results are compiled in a report format and sent back to the terminal. An analysis report is sent to the terminal as output.

[1011] Step 4:

[1012] Swing Feedback

[1013] The terminal displays the received analysis results on the smartphone screen. It reads the analysis report as input and generates screen data to be displayed on the smartphone as output.

[1014] It also provides visual feedback through smart glasses and sends audio advice via Bluetooth. It uses text and audio analysis results as input. It provides visual and audio feedback as output. For example, it displays a message like, "Your right shoulder is dropping too low at the top of your swing."

[1015] Step 5:

[1016] Feedback adjustment by emotion engine

[1017] The emotion recognition engine "EmotionAI" uses the user's facial expression images and voice tone data as input to recognize the user's emotions.

[1018] The emotion engine analyzes these data and outputs the user's emotional state. The output is the user's current emotional state (e.g., tired, focused).

[1019] The system adjusts the feedback content and method based on the recognized emotional state. It uses the emotional state data and analysis result data as input and generates adjusted feedback as output. For example, if the user is tired, the system adjusts the feedback to be provided in an encouraging tone.

[1020] Step 6:

[1021] Video lessons

[1022] The user selects "Watch a video lesson" within the app and specifies the relevant lesson video. The user's selection information is used as input.

[1023] The device receives the relevant lesson video from the server and plays it on the smartphone or smart glasses. The URL or path of the video file is used as input. The lesson video is played as output.

[1024] The content of the video is also customized based on the emotions recognized by the emotion engine. Emotional state data and video data are used as input, and customized video content is provided as output.

[1025] Step 7:

[1026] Monthly service and billing management

[1027] When a user signs up for a monthly service, the user's registration information and billing information are used as input.

[1028] The server processes monthly charges and monitors usage using a billing management mechanism, generating a billing completion notice and usage report as outputs.

[1029] Delivering new lesson content and updates at the appropriate time, using usage data as input and generating new content delivery data as output.

[1030] Step 8:

[1031] Continued use and updates

[1032] The server collects user feedback and considers improving the service or adding new features based on it. The feedback data is used as input.

[1033] Provides app and smartglasses firmware updates as needed, using update data as input and providing updated software and firmware as output.

[1034] This ensures that your system always performs optimally.

[1035] (Application example 2)

[1036] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1037] The present invention aims to solve the problems of conventional golf swing analysis systems, such as limited returns when users try out equipment, feedback that does not take into account emotional state, and a lack of specific support for users to select appropriate equipment.

[1038] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a device means for filming the user's hitting motion, a machine learning means for analyzing the filmed hitting motion video, a visual and audio feedback means for providing the analysis results to the user, an emotion engine means for recognizing the user's emotional state and adjusting the feedback, and a means for recommending products for the user to try in a real-world store environment. This provides feedback according to the user's emotional state, enabling hitting motion analysis and appropriate product selection in a real store.

[1039] The term "user's hitting motion" refers to any swing or hitting motion performed with a golf club or other sporting equipment.

[1040] The term "photography device" refers to an electronic device or device, such as a camera or smartphone, that records a user's hitting motion as video.

[1041] "Machine learning means" refers to a technical means that uses artificial intelligence or machine learning algorithms to analyze and evaluate filmed footage of hitting movements.

[1042] "Visual and audio feedback means" refers to a display device or audio output device for providing the analysis results to the user visually or audio.

[1043] "Emotion engine" refers to a technical means for recognizing a user's emotional state by analyzing their facial expressions and tone of voice, and adjusting the feedback provided based on that state.

[1044] "Recommendation method" refers to the algorithms and systems used to select and recommend products for users to try out in physical stores.

[1045] "Display device" refers to a device that provides information visually to a user, such as smart glasses or a head-mounted display.

[1046] The system captures a user's hitting motion, analyzes it using machine learning, and provides the results through visual and audio feedback. By combining it with an emotion engine that recognizes the user's emotional state, the system can tailor the feedback based on the user's emotional state. Furthermore, the system has the ability to recommend products for the user to try in a brick-and-mortar store environment.

[1047] 1. Initial Setup

[1048] Users first download the corresponding app onto their smartphones, create an account, enter their profile information, and pair their smart glasses or other display device with their smartphone via Bluetooth. Once pairing is successful, the system will confirm the connection status and notify the user.

[1049] 2. Filming the hitting motion

[1050] The user places their smartphone in an appropriate position and films their batting motion. They tap the "Start Recording" button in the app to begin recording the video of their batting motion. The smartphone camera captures video of the user's batting motion in real time and temporarily saves the video data.

[1051] 3. Data transmission and AI analysis

[1052] The device sends the captured video of the batting motion to a server. The server then passes the video to a machine learning algorithm, which begins analysis. The machine learning algorithm analyzes each element of the batting motion, such as posture, timing, and speed, and identifies weaknesses and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[1053] 4. Providing Feedback

[1054] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through the display device and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is lowered too much at the top of your hitting motion," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[1055] 5. Feedback adjustment by emotion engine

[1056] The server uses an emotion engine to recognize the user's emotional state. Emotion recognition is performed by analyzing the user's facial expressions and voice tone. The recognized emotional state is used to adjust the content and method of feedback. For example, if the user looks tired, the feedback content can be changed to a more encouraging tone. If the user is concentrating, more detailed technical instructions can be provided.

[1057] Hardware and software used

[1058] Hardware: Smartphone camera, smart glasses, Bluetooth module

[1059] Software: Machine learning algorithms, emotion recognition engine, Bluetooth connectivity software

[1060] Specific examples

[1061] For example, imagine a scenario where a user is trying out golf clubs at a brick-and-mortar store. The user takes a video of their swing with their smartphone and sends it to a server. A machine learning algorithm identifies flaws in the swing, and an emotion engine recognizes the user's level of fatigue. Based on this, real-time feedback such as "Your swing angle isn't good. Take a short break to refresh yourself" is displayed on the smart glasses.

[1062] Example prompts for generative AI models

[1063] "Please point out areas for improvement when the golf swing angle is poor. Also, add encouraging comments if the user is tired."

[1064] The above is a detailed description of the embodiment of the present invention. This system is very useful for users in physical stores to analyze their hitting motion and select the most suitable equipment.

[1065] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1066] Step 1:

[1067] A user downloads the app to their smartphone and creates an account. They enter their profile information and pair their smart glasses or other display device with the smartphone via Bluetooth. A confirmation of successful pairing appears on the device.

[1068] Step 2:

[1069] The user places their smartphone in an appropriate position and records their hitting motion. When the user taps the "Start Recording" button in the app, the smartphone camera begins recording their hitting motion in real time. This video data is temporarily saved on the smartphone.

[1070] Step 3:

[1071] The device sends the captured video of the batting motion to the server. The input data is a video file of the batting motion. After receiving this, the server passes the video data to a machine learning algorithm and begins analysis. The output is the analysis results in JSON format data.

[1072] Step 4:

[1073] The server uses machine learning algorithms to analyze each element of the hitting motion, including posture, timing, and speed. The analysis results include swing flaws and areas for improvement. After data processing and calculations, the results are compiled into a report.

[1074] Step 5:

[1075] The server sends the analysis results back to the device, which then visually displays the received analysis results on the app's user interface. Additionally, the device provides visual feedback via a display device and specific advice via audio via Bluetooth. At this time, feedback based on the analysis results is output as text and audio files.

[1076] Step 6:

[1077] The server uses an emotion engine to recognize the user's emotional state. The input data is the user's facial expression and voice tone, which are analyzed by the emotion recognition algorithm. The output is tag information that indicates the user's emotional state.

[1078] Step 7:

[1079] The server adjusts the feedback content and method based on the user's recognized emotional state. For example, if the user looks tired, the feedback tone is changed to be more encouraging. If the user is concentrating, detailed technical instructions are added. The text and audio files with the adjusted feedback content and tone are output.

[1080] Step 8:

[1081] The server recommends products for the user to try in a physical store environment. The input data is the analysis results and the user's profile information, which are analyzed by the recommendation algorithm. The output is a list of optimal products. The device presents this list to the user within the app.

[1082] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1083] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1084] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1085] [Fourth embodiment]

[1086] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1087] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1088] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1089] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1090] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1091] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1092] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1093] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1094] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1095] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1096] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1097] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1098] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1099] The present invention is a system that captures a user's golf swing, analyzes it using artificial intelligence, and provides the results through visual and audio feedback. Using a smartphone and smart glasses, users can receive golf swing diagnosis and lessons anywhere, anytime. The following describes in detail the embodiments of the present invention.

[1100] 1. Initial Setup

[1101] First, users download a compatible golf app onto their smartphone. Next, they create an account and enter their profile information. Then, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the system confirms the connection status and notifies the user.

[1102] 2. Swing photography

[1103] The user places their smartphone in a suitable position and films their golf swing. They tap the "Start Recording" button in the app to begin recording the swing video. The smartphone camera captures the user's swing video in real time and temporarily saves the video data.

[1104] 3. Data transmission and AI analysis

[1105] The device sends the captured swing video to a server. The server then passes the received swing video to an AI algorithm, which begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, to identify swing flaws and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[1106] 4. Swing Feedback

[1107] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[1108] 5. Video lessons

[1109] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons.

[1110] 6. Monthly service and billing management

[1111] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that users always receive the latest lessons.

[1112] 7. Continued Use and Update Provision

[1113] The server collects user feedback and considers service improvements and feature additions based on that feedback. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally.

[1114] With the above configuration, the system of the present invention allows users to receive golf swing diagnosis and lessons regardless of location or time, thereby achieving effective training.

[1115] The processing flow will be explained below.

[1116] Step 1:

[1117] The user downloads the golf app on their smartphone and creates an account. The user completes the process by entering their profile information.

[1118] Step 2:

[1119] The user pairs the smart glasses with their smartphone via Bluetooth. The device notifies them that pairing was successful.

[1120] Step 3:

[1121] The user places the smartphone in the appropriate position and prepares to record their swing. The user then taps the "Start Recording" button in the app to begin recording.

[1122] Step 4:

[1123] The device captures the user's golf swing in real time using a smartphone camera and temporarily saves the captured swing video.

[1124] Step 5:

[1125] The device sends the captured swing video to the server, which receives the video and passes it to the AI ​​algorithm.

[1126] Step 6:

[1127] The server uses an AI algorithm to analyze each element of the swing (posture, timing, speed, etc.) and compiles the analysis results in a report format.

[1128] Step 7:

[1129] The server sends a report of the analysis results back to the device, which receives the analysis results.

[1130] Step 8:

[1131] The device displays the analysis results on the smartphone screen, where the user can check them.

[1132] Step 9:

[1133] The device visually feeds back the analysis results through the smart glasses, and the user receives the visual feedback.

[1134] Step 10:

[1135] The device provides voice feedback via Bluetooth, and the user receives voice advice.

[1136] Step 11:

[1137] The user selects "Watch Video Lessons" within the app and specifies the desired lesson content.

[1138] Step 12:

[1139] The device sends the appropriate video request to the server, which selects the corresponding lesson video in response to the request.

[1140] Step 13:

[1141] The server streams the lesson video to the device, and the device receives the lesson video.

[1142] Step 14:

[1143] The lesson video received by the device is played simultaneously on the smartphone and smart glasses, allowing the user to receive a visual lesson.

[1144] Step 15:

[1145] A user signs up for a monthly service. The user enters payment information and begins the subscription.

[1146] Step 16:

[1147] The server manages billing for the monthly service and periodically delivers new lesson content and updates to users.

[1148] Step 17:

[1149] The server collects user feedback, considers service improvements and feature additions, and provides firmware updates for the app and smart glasses as needed.

[1150] Example 1

[1151] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1152] Conventional golf swing analysis systems require users to spend a lot of time obtaining analysis results and often do not provide results in real time. Furthermore, they do not adequately integrate visual and audio feedback, and do not provide an effective training environment for users. Furthermore, there is a lack of systems that allow users to train anywhere, anytime using their own devices.

[1153] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1154] In this invention, the server includes a means for capturing images of the user's movements, an artificial intelligence means for analyzing the captured images of the movements, a visual and audio feedback means for providing the user with the analysis results, a terminal for the user, and a means for displaying the analysis results and providing advice using the terminal. This allows the user to obtain the analysis results in real time, enabling training regardless of location or time. Furthermore, by providing integrated visual and audio feedback, the user can train more effectively.

[1155] "User behavior" refers to a specific action or behavior generally undertaken by an individual.

[1156] "Photographing means" refers to a device or method for recording actions or movements using a camera or related photographic device.

[1157] "Motion footage" refers to video data of a captured movement.

[1158] "Artificial intelligence means for analysis" refers to AI technology that analyzes the characteristics and shortcomings of behavior based on collected data and extracts areas for improvement.

[1159] "Analysis results" refers to the data and reports obtained after analyzing motion footage.

[1160] "Means for providing visual and audio feedback to the user" refers to a method or device for conveying the analysis results to the user by visual information or audio.

[1161] "Terminal" refers to any electronic device used by a User to access the System, including, for example, a smartphone or tablet.

[1162] "Video display device" refers to any device for displaying images or video, including, for example, smart glasses and monitors.

[1163] "Means for providing advice" refers to a method or device for informing the user of improvements based on the analysis results.

[1164] The present invention is a system that allows a user to receive golf swing diagnosis and lessons regardless of location or time. The system includes a means for the user to film their movements, an artificial intelligence means for analyzing the filmed video of their movements, a visual and audio feedback means for providing the analysis results to the user, a terminal for use by the user, and a means for displaying the analysis results and providing advice using the terminal.

[1165] Initial Setup

[1166] First, the user downloads a compatible golf app onto their smartphone. In this example, the app name is "GolfSwingAnalyzer." Next, they create an account and enter their profile information. After that, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the device checks the connection status and notifies the user.

[1167] Swing shooting

[1168] The user places their smartphone in a suitable position and taps the "Start Recording" button in the app to record their golf swing. The device uses the smartphone camera to capture a video of the user's swing in real time and temporarily stores the video data. For example, the user can mount their smartphone on a tripod placed on the ground and adjust the screen so that the swing area fits within the screen.

[1169] Data transmission and AI analysis

[1170] The device sends the recorded swing video to a server. The server then passes the received swing video to an AI algorithm (e.g., TensorFlow) and begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, to identify swing flaws and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time. An example prompt is, "Analyze the user's swing video and generate specific improvement suggestions for shoulder movement."

[1171] Swing Feedback

[1172] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result might say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement might be, "To improve your shoulder movement, try practicing the following."

[1173] Video lessons

[1174] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons.

[1175] Monthly service and billing management

[1176] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that the user always receives the latest lessons.

[1177] Continued use and updates

[1178] The server collects user feedback and uses it to improve the service and add new features. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally.

[1179] In this way, the system of the present invention allows the user to receive golf swing diagnosis and lessons regardless of location or time, thereby achieving effective training.

[1180] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1181] Step 1:

[1182] The user downloads and installs the golf app "GolfSwingAnalyzer" onto their smartphone. At this time, the user enters search keywords into the smartphone's App Store or Google Play Store, and the app is found and installed based on those keywords.

[1183] Step 2:

[1184] A user launches the app and creates an account. Here, they enter their name, email address, and password. An account is created based on the information they entered. As an output, a user profile is generated and saved in the database. Specifically, the user enters the required information in each input field and taps the "Register" button.

[1185] Step 3:

[1186] The user pairs the smart glasses with their smartphone via Bluetooth. Input includes the smart glasses' device name and pairing code. The device detects the smart glasses based on this and performs pairing. If successful, the device checks the connection status and sends a notification to the user. Specifically, the user sets the smart glasses to connection mode and taps the "Device Pairing" button in the app.

[1187] Step 4:

[1188] The user places the smartphone in an appropriate position and taps the "Start Recording" button in the app to record their golf swing. The input includes the user's movement information (swing movement). The device uses the smartphone camera to record this movement and temporarily saves it as video data. Specific actions include the user fixing the smartphone to a tripod or other device and adjusting it so that the swing area fits on the screen.

[1189] Step 5:

[1190] The device sends the recorded swing video to the server. The input is the temporarily stored video data, which is sent via Wi-Fi or mobile data communication. The output is the generated video data stored on the server. The specific operation requires the user to tap the "Send Data" button.

[1191] Step 6:

[1192] The server passes the received swing video to an AI algorithm and begins analyzing the golf swing. The input is video data. The AI ​​algorithm (for example, TensorFlow) analyzes this and performs data calculations on swing posture, timing, speed, etc. to identify flaws and areas for improvement. The output is an analysis report. Specifically, the server passes the video data to the analysis module and executes the analysis process.

[1193] Step 7:

[1194] The server compiles the analysis results into a report format and sends it back to the device in real time. The input is the data analyzed by the AI. The output is an analysis report in JSON format that is generated and sent to the device. Specifically, the server formats the analysis data and sends it to the device.

[1195] Step 8:

[1196] The device displays the received analysis results on the smartphone screen. The input is an analysis report. The output is a graph or text display of the analysis results. Specifically, the app UI on the device performs processing to visually display the analysis results.

[1197] Step 9:

[1198] The device provides visual feedback to the smart glasses and specific audio advice via Bluetooth. The input is the analysis results. The output is visual and audio feedback. Specific operations include an overlay of the analysis results summary on the smart glasses and audio guidance.

[1199] Step 10:

[1200] The user selects "Watch a video lesson" within the app and selects a lesson video related to their swing. The input is the user's request for the lesson video they selected. The device receives the corresponding lesson video from the server and plays it simultaneously on the smartphone and smart glasses. The output is the playback of the lesson video. Specifically, the user taps the video play button, and the video plays synchronously on both devices.

[1201] Step 11:

[1202] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The input is the user's credit card information and registration information. The output is the completion of the monthly billing and the continuation of service use. Specifically, the user taps the "Sign up for monthly service" button and enters their credit card information.

[1203] Step 12:

[1204] The server monitors user usage and delivers new lesson content and updated information at appropriate times. The input is user usage data. The output is the provision of new lesson content. Specifically, the server periodically checks usage and delivers new content.

[1205] Step 13:

[1206] The server collects feedback from users and considers improving the service or adding new features based on that information. The input is user feedback information. The output is the development of new features and improvements to the service. Specifically, the user enters and submits their opinions and requests using the "Feedback Submission" form within the app.

[1207] Step 14:

[1208] The server provides firmware updates for the app and smart glasses as needed. The input is the latest version information. The output is the provision of updated firmware. Specifically, when a new version is released, the app automatically checks for updates and sends an update notification to the user. When the user taps the "Update Now" button, the update begins.

[1209] Through the above processing steps, the system enables users to receive golf swing diagnosis and lessons regardless of location or time, thereby achieving effective training.

[1210] (Application example 1)

[1211] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1212] Conventional motion analysis systems make it difficult for users to perform detailed analysis of their motions and conduct effective training regardless of location or time. Furthermore, they do not provide real-time feedback or simulations in virtual environments, limiting improvements based solely on analysis results. Furthermore, they lack the ability to learn and improve motivation by comparing data with other users.

[1213] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1214] In this invention, the server includes a means for filming the user's exercise movements, an artificial intelligence means for analyzing the filmed exercise video, a visual and audio feedback means for providing the user with the analysis results, a means for simulating exercise in a virtual environment, and a means for comparing with other users and providing lesson content. This allows the user to receive effective training regardless of location or time, and through real-time analysis and feedback, as well as simulation in a virtual environment and comparison with other users, the user can achieve exercise improvement with higher motivation.

[1215] "User" refers to an individual who utilizes the system to analyze and improve athletic performance.

[1216] "Athletic movements" refer to movements associated with specific sports or fitness, such as the golf swing.

[1217] "Means for taking photographs" refers to a function for recording the user's exercise movements using a device such as a smartphone or camera.

[1218] "Exercise video" refers to video data of a user's exercise movements that have been captured.

[1219] "Artificial intelligence means" refers to a system that uses AI algorithms to analyze exercise footage and extract and evaluate data about the user's movements.

[1220] "Analysis results" refers to information regarding the evaluation of exercise movements and areas for improvement generated by artificial intelligence means.

[1221] "Visual feedback means" refers to a function that provides visual information to the user through smart glasses or a smartphone screen.

[1222] "Audio feedback means" refers to a function that provides advice and information to the user by voice through a speaker or headphones.

[1223] "Virtual environment" refers to a computer-generated simulated environment.

[1224] "Movement simulation means" refers to a function that reproduces the user's movement within a virtual environment and provides a simulation based on the analysis results.

[1225] "Comparison means" refers to the functionality used to compare a user's athletic data with that of other users.

[1226] "Means for providing lesson content" refers to a function for providing users with educational content aimed at improving or learning exercise movements.

[1227] The present invention is a system that captures a user's exercise movements, analyzes the video using artificial intelligence, and provides the analysis results as visual and audio feedback. The realization of this system will be described in detail below.

[1228] First, users download a specific application onto their smartphone and create an account, then pair the smart glasses with the smartphone via Bluetooth, which enables the visual feedback mechanism.

[1229] When a user performs an exercise, the smartphone records the video. For example, when a user performs a golf swing, the user places the smartphone in an appropriate position, taps the "Start Recording" button, and records the swing video. At this time, the smartphone camera records the user's movements in real time and temporarily stores the video data.

[1230] The device then sends the captured video to a server, which then passes it to an AI algorithm to analyze the exercise. The server analyzes factors such as posture, timing, and speed to identify weaknesses and areas for improvement. The analysis results are compiled into a report and sent back to the device.

[1231] The device displays the analysis results on the smartphone screen, provides visual feedback through smart glasses, and provides specific audio advice via Bluetooth, such as "Your right shoulder is dropping too much at the top of your swing."

[1232] Users can also select "Simulation Mode" within the app to simulate athletic movements in a virtual environment, which provides a visual demonstration of how users can improve their movements in real time.

[1233] Furthermore, users can compare their data with that of other users to create more effective training plans, and can watch lesson content to learn specific training methods related to their movements.

[1234] This allows users to receive effective training regardless of location or time, and achieve improvements in their athletic performance through real-time analysis and feedback, as well as simulation in a virtual environment and comparison with other users.

[1235] As a concrete example, consider a user practicing their golf swing. In this case, the user wears smart glasses and films their swing with a smartphone. The captured video of their swing is analyzed by AI, and feedback such as "Your right shoulder is too low at the top of your swing" is displayed via the smart glasses and smartphone. After that, a simulation is performed on a virtual golf course, and further areas for improvement can be identified by comparing the data with other users' data.

[1236] Examples of prompts to input to a generative AI model include:

[1237] "Analyze the user's athletic movements and output improvements. The elements to analyze are posture, timing, and speed."

[1238] As described above, the present invention is a system that enables a user to effectively analyze and improve exercise movements regardless of location or time.

[1239] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1240] Step 1:

[1241] A user downloads a golf swing analysis application onto their smartphone and creates an account. They then enter their profile information and pair the smart glasses with their smartphone via Bluetooth. This information is sent to the server, which then creates an account and identifies the user's device.

[1242] Step 2:

[1243] The user performs an exercise (e.g., a golf swing) and captures it with the smartphone camera. The user taps the "Start Recording" button in the smartphone application to begin recording the exercise video. The captured video is temporarily saved in the smartphone's local storage. The input is the video of the user's movements, and the output is the saved video data.

[1244] Step 3:

[1245] The smartphone sends the saved video data to a server. The server receives this video data and passes it to an artificial intelligence tool to begin analysis. The input is the video data, and the output is the analysis results by the AI. Specifically, the video file is uploaded to the server via the internet, and then the data is input into the AI ​​model.

[1246] Step 4:

[1247] The server analyzes the received video data using an AI algorithm. Specifically, it processes and calculates data on each element of the movement, such as posture, timing, and speed, to identify shortcomings and areas for improvement. The input is the video data, and data processing involves extracting elements from each frame, followed by the actual analysis process. The output is a report of the analysis results.

[1248] Step 5:

[1249] The analysis results obtained from the server are sent back to the smartphone. The smartphone application receives these analysis results and provides visual and audio feedback to the user. The input is the analysis results, and the output is feedback to the user. Specifically, the application visualizes the analysis results and notifies the user in an easy-to-understand manner using text and audio.

[1250] Step 6:

[1251] The user selects "Simulation Mode" within the application and simulates their own athletic movements in a virtual environment. The device simulates the movement in real time and visually indicates areas for improvement. The input is the analysis results and current movement data, and the output is the simulation results. Specifically, a simulation begins within the app, highlighting areas for improvement.

[1252] Step 7:

[1253] Users compare their own exercise data with that of other users and watch lesson content. The application provides the comparison results and appropriate lesson videos. The input is the user's own and other users' exercise data, and the output is a comparison report and lesson videos. Specifically, the application retrieves other users' data from the database and performs a relative comparison.

[1254] In this way, this system supports effective training by analyzing the user's exercise movements in detail and suggesting areas for improvement.The following is an example of a prompt sentence input to the generative AI model.

[1255] "Analyze the user's athletic movements and output improvements. The elements to analyze are posture, timing, and speed."

[1256] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1257] The present invention is a system that films a user's golf swing, analyzes it using artificial intelligence, and provides the results through visual and audio feedback. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the feedback is adjusted based on the user's emotional state. The following describes in detail the embodiments of the present invention.

[1258] 1. Initial Setup

[1259] First, users download a compatible golf app onto their smartphone and create an account. Next, they enter their profile information. After that, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the system confirms the connection status and notifies the user.

[1260] 2. Swing photography

[1261] The user places their smartphone in a suitable position and films their golf swing. They tap the "Start Recording" button in the app to begin recording the swing video. The smartphone camera captures the user's swing video in real time and temporarily saves the video data.

[1262] 3. Data transmission and AI analysis

[1263] The device sends the captured swing video to a server. The server then passes the received swing video to an AI algorithm, which begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, to identify swing flaws and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[1264] 4. Swing Feedback

[1265] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[1266] 5. Feedback adjustment by emotion engine

[1267] The emotion engine recognizes the user's emotions. Emotion recognition is performed using facial expression analysis and voice tone analysis. The recognized emotional state is used to adjust the feedback content and method. For example, if the user looks tired, the feedback content can be changed to a more encouraging tone. Also, if the user is concentrating, more detailed technical instructions can be provided.

[1268] 6. Video lessons

[1269] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons. The content of the video is also customized based on the emotions recognized by the emotion engine.

[1270] 7. Monthly service and billing management

[1271] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that users always receive the latest lessons.

[1272] 8. Continued Use and Update Provision

[1273] The server collects user feedback and considers service improvements and feature additions based on that feedback. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally.

[1274] With the above configuration, the system of the present invention enables users to receive golf swing diagnosis and lessons regardless of location or time, achieving effective training. In addition, by combining it with an emotion engine, it provides feedback according to the user's emotional state, providing a more personalized training experience.

[1275] The processing flow will be explained below.

[1276] Step 1:

[1277] The user downloads the golf app on their smartphone and creates an account. The user enters the required profile information and completes the setup.

[1278] Step 2:

[1279] The user pairs the smart glasses with their smartphone via Bluetooth. The smartphone confirms the successful pairing and notifies the user.

[1280] Step 3:

[1281] The user positions the smartphone in the appropriate position and prepares to record their golf swing. The user taps the "Start Recording" button in the app to begin recording.

[1282] Step 4:

[1283] The device captures the user's golf swing in real time using a smartphone camera and temporarily stores the captured swing video.

[1284] Step 5:

[1285] The device sends the captured swing video to the server, which receives the video and passes it to the AI ​​algorithm.

[1286] Step 6:

[1287] The server uses an AI algorithm to analyze each element of the swing (posture, timing, speed, etc.) and compiles the analysis results in a report format.

[1288] Step 7:

[1289] The server sends a report of the analysis results back to the device, which receives the results and prepares them for display.

[1290] Step 8:

[1291] The device displays the analysis results on the smartphone screen, and the user checks them.

[1292] Step 9:

[1293] The device visually feeds back the analysis results through the smart glasses, which display the results.

[1294] Step 10:

[1295] The device provides voice feedback via Bluetooth, and the user receives voice advice.

[1296] Step 11:

[1297] The device activates an emotion engine and performs facial expression analysis and voice tone analysis to recognize the user's emotions.

[1298] Step 12:

[1299] The emotion engine identifies the user's emotional state and passes the results on to the rest of the system. For example, if the user is tired, they will receive the recognition result.

[1300] Step 13:

[1301] The device will adjust the feedback content based on the results of the emotion engine, for example adding an encouraging message if the user is tired.

[1302] Step 14:

[1303] The user selects "Watch Video Lessons" within the app and specifies the desired lesson content. The app prepares to retrieve the lesson video based on the request.

[1304] Step 15:

[1305] The device sends the appropriate video request to the server, which selects the appropriate lesson video in response to the request.

[1306] Step 16:

[1307] The server streams the lesson video to the device, which then plays the received video simultaneously on the smartphone and smart glasses.

[1308] Step 17:

[1309] Customize video content based on emotions recognized by the emotion engine, for example, providing detailed technical instructions if the user is distracted.

[1310] Step 18:

[1311] A user signs up for a monthly service, enters payment information, and begins their subscription.

[1312] Step 19:

[1313] The server manages billing for the monthly service, monitors user usage, and periodically delivers new lesson content and updates.

[1314] Step 20:

[1315] The server collects user feedback, considers service improvements and feature additions, and provides firmware updates for the app and smart glasses as needed.

[1316] Example 2

[1317] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1318] Conventional golf swing analysis systems have the ability to film a user's swing and provide the analysis results as visual and audio feedback, but they are unable to provide feedback that takes into account the user's emotional state. Furthermore, they lack the ability to provide real-time analysis and content based on the user's usage. Therefore, there is a need to provide users with a more effective and personalized training experience.

[1319] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1320] In this invention, the server includes a device that captures a user's actions, an artificial intelligence function that analyzes the captured video of the actions, a visual and audio feedback function that provides the user with the analysis results, an emotion recognition engine that recognizes the user's emotions and adjusts the feedback, and a content provision service. This makes it possible to analyze a user's actions in real time and provide personalized feedback according to the user's emotional state. It is also possible to deliver new content at an appropriate time based on the user's usage status.

[1321] The "device for capturing images of user actions" is a device for recording user actions in real time, and includes a smartphone camera, a camcorder, and the like.

[1322] "Artificial intelligence function that analyzes filmed video of movements" refers to artificial intelligence algorithms and software for analyzing filmed video of movements, and is a function that generates analysis results by evaluating the posture, timing, speed, etc. of the movement.

[1323] The "visual and audio feedback function for providing the analysis results to the user" is a function for presenting the analysis results to the user visually and audio-wise, and for providing information via a display or speaker.

[1324] The "emotion recognition engine that recognizes the user's emotions and adjusts feedback" is an engine that analyzes the user's facial expressions and tone of voice to recognize emotions, and appropriately adjusts the feedback content based on that information.

[1325] A "content provision service" is a service that delivers new learning and lesson content at appropriate times based on the user's usage status, and is managed and delivered on the server side.

[1326] A "head-mounted display" is a display device worn by a user on the head, which provides visual information.

[1327] "Real-time analysis" refers to the process of analyzing captured video instantly with almost no delay.

[1328] "Personalized feedback" refers to providing feedback optimized according to the characteristics and status of each individual user.

[1329] The present invention is a system that captures a user's actions, analyzes them using artificial intelligence, and provides the results through visual and audio feedback. Furthermore, by combining an emotion recognition engine that recognizes the user's emotions, the system also has the function of adjusting the feedback based on the user's emotional state. Specific embodiments for implementing the present invention are described in detail below.

[1330] 1. Initial Setup

[1331] First, users download a compatible golf app onto their smartphone and create an account. Next, they enter their profile information. After that, they pair the smart glasses with their smartphone via Bluetooth. Once pairing is successful, the device will confirm the connection status and notify the user.

[1332] 2. Swing photography

[1333] The user places their smartphone in a suitable position and films their golf swing. They tap the "Start Recording" button in the app to begin recording the swing video. The smartphone camera captures the user's swing video in real time and temporarily saves the video data.

[1334] 3. Data transmission and AI analysis

[1335] The device sends the captured swing video to a server. The server then passes the received swing video to the AI ​​algorithm "SwingAI," which begins analyzing the golf swing. The AI ​​algorithm analyzes each element of the swing, such as posture, timing, and speed, to identify flaws and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[1336] 4. Swing Feedback

[1337] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through smart glasses and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is dropping too much at the top of your swing," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[1338] 5. Feedback adjustment by emotion engine

[1339] The emotion recognition engine "EmotionAI" recognizes the user's emotions. Emotion recognition is performed by analyzing the user's facial expressions and vocal tone. The recognized emotional state is used to adjust the content and method of feedback. For example, if the user looks tired, the feedback content will be changed to a more encouraging tone. Also, if the user is concentrating, more detailed technical instructions will be provided.

[1340] 6. Video lessons

[1341] Users select "Watch Video Lessons" within the app and select a lesson video related to their swing. The device receives the relevant lesson video from the server and plays it simultaneously on their smartphone and smart glasses. This allows users to learn how to improve their swing while receiving visual lessons. The content of the video is also customized based on emotions recognized by the emotion recognition engine.

[1342] For example, if a user were to film their swing and use AI analysis, the prompt would look like this:

[1343] "Please provide me with feedback through the smart glasses so that I can improve my swing."

[1344] 7. Monthly service and billing management

[1345] When a user signs up for a monthly service, the system processes the monthly fee using a billing management method. The server monitors the user's usage and delivers new lesson content and updates at appropriate times, ensuring that users always receive the latest lessons.

[1346] 8. Continued Use and Update Provision

[1347] The server collects user feedback and considers service improvements and feature additions based on it. If necessary, it provides firmware updates for the app and smart glasses to keep the system performing optimally. For example, if the user is tired, it adjusts the feedback it provides to them in a more encouraging tone.

[1348] This system allows users to receive golf swing diagnosis and lessons regardless of location or time, enabling effective training. In addition, by combining it with an emotion recognition engine, it is possible to provide feedback according to the user's emotional state, providing a more personalized training experience.

[1349] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1350] Step 1:

[1351] Initial Setup

[1352] A user downloads a golf app onto their smartphone and creates an account by entering their username, email address, and password. This registers the account information in the app.

[1353] Next, users enter profile information (e.g., years of golf experience and handicap), which generates the basis for customized analysis and feedback.

[1354] Pair the smart glasses with the smartphone via Bluetooth, using the smart glasses' identification information and Bluetooth connection data as input, and notify the smartphone once pairing is successful.

[1355] Step 2:

[1356] Swing shooting

[1357] The user places the smartphone in an appropriate position and records the golf swing. The input is the smartphone camera position and shooting angle information.

[1358] When you tap the "Start Recording" button, the smartphone camera will start recording your swing video. The real-time video is saved temporarily during recording, and the data is saved after the recording ends. The output is a recorded swing video file.

[1359] Step 3:

[1360] Data transmission and AI analysis

[1361] The device sends the saved swing video to the server, using the video file path and the server URL as input.

[1362] The server passes the received swing video to the AI ​​algorithm "SwingAI" and begins analysis. The video data and user profile information are used as input. The AI ​​algorithm analyzes the swing posture, timing, speed, etc., and generates a report of the analysis results as output.

[1363] The analysis results are compiled in a report format and sent back to the terminal. An analysis report is sent to the terminal as output.

[1364] Step 4:

[1365] Swing Feedback

[1366] The terminal displays the received analysis results on the smartphone screen. It reads the analysis report as input and generates screen data to be displayed on the smartphone as output.

[1367] It also provides visual feedback through smart glasses and sends audio advice via Bluetooth. It uses text and audio analysis results as input. It provides visual and audio feedback as output. For example, it displays a message like, "Your right shoulder is dropping too low at the top of your swing."

[1368] Step 5:

[1369] Feedback adjustment by emotion engine

[1370] The emotion recognition engine "EmotionAI" uses the user's facial expression images and voice tone data as input to recognize the user's emotions.

[1371] The emotion engine analyzes these data and outputs the user's emotional state. The output is the user's current emotional state (e.g., tired, focused).

[1372] The system adjusts the feedback content and method based on the recognized emotional state. It uses the emotional state data and analysis result data as input and generates adjusted feedback as output. For example, if the user is tired, the system adjusts the feedback to be provided in an encouraging tone.

[1373] Step 6:

[1374] Video lessons

[1375] The user selects "Watch a video lesson" within the app and specifies the relevant lesson video. The user's selection information is used as input.

[1376] The device receives the relevant lesson video from the server and plays it on the smartphone or smart glasses. The URL or path of the video file is used as input. The lesson video is played as output.

[1377] The content of the video is also customized based on the emotions recognized by the emotion engine. Emotional state data and video data are used as input, and customized video content is provided as output.

[1378] Step 7:

[1379] Monthly service and billing management

[1380] When a user signs up for a monthly service, the user's registration information and billing information are used as input.

[1381] The server processes monthly charges and monitors usage using a billing management mechanism, generating a billing completion notice and usage report as outputs.

[1382] Delivering new lesson content and updates at the appropriate time, using usage data as input and generating new content delivery data as output.

[1383] Step 8:

[1384] Continued use and updates

[1385] The server collects user feedback and considers improving the service or adding new features based on it. The feedback data is used as input.

[1386] Provides app and smartglasses firmware updates as needed, using update data as input and providing updated software and firmware as output.

[1387] This ensures that your system always performs optimally.

[1388] (Application example 2)

[1389] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1390] The present invention aims to solve the problems of conventional golf swing analysis systems, such as limited returns when users try out equipment, feedback that does not take into account emotional state, and a lack of specific support for users to select appropriate equipment.

[1391] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a device means for filming the user's hitting motion, a machine learning means for analyzing the filmed hitting motion video, a visual and audio feedback means for providing the analysis results to the user, an emotion engine means for recognizing the user's emotional state and adjusting the feedback, and a means for recommending products for the user to try in a real-world store environment. This provides feedback according to the user's emotional state, enabling hitting motion analysis and appropriate product selection in a real store.

[1392] The term "user's hitting motion" refers to any swing or hitting motion performed with a golf club or other sporting equipment.

[1393] The term "photography device" refers to an electronic device or device, such as a camera or smartphone, that records a user's hitting motion as video.

[1394] "Machine learning means" refers to a technical means that uses artificial intelligence or machine learning algorithms to analyze and evaluate filmed footage of hitting movements.

[1395] "Visual and audio feedback means" refers to a display device or audio output device for providing the analysis results to the user visually or audio.

[1396] "Emotion engine" refers to a technical means for recognizing a user's emotional state by analyzing their facial expressions and tone of voice, and adjusting the feedback provided based on that state.

[1397] "Recommendation method" refers to the algorithms and systems used to select and recommend products for users to try out in physical stores.

[1398] "Display device" refers to a device that provides information visually to a user, such as smart glasses or a head-mounted display.

[1399] The system captures a user's hitting motion, analyzes it using machine learning, and provides the results through visual and audio feedback. By combining it with an emotion engine that recognizes the user's emotional state, the system can tailor the feedback based on the user's emotional state. Furthermore, the system has the ability to recommend products for the user to try in a brick-and-mortar store environment.

[1400] 1. Initial Setup

[1401] Users first download the corresponding app onto their smartphones, create an account, enter their profile information, and pair their smart glasses or other display device with their smartphone via Bluetooth. Once pairing is successful, the system will confirm the connection status and notify the user.

[1402] 2. Filming the hitting motion

[1403] The user places their smartphone in an appropriate position and films their batting motion. They tap the "Start Recording" button in the app to begin recording the video of their batting motion. The smartphone camera captures video of the user's batting motion in real time and temporarily saves the video data.

[1404] 3. Data transmission and AI analysis

[1405] The device sends the captured video of the batting motion to a server. The server then passes the video to a machine learning algorithm, which begins analysis. The machine learning algorithm analyzes each element of the batting motion, such as posture, timing, and speed, and identifies weaknesses and areas for improvement. The analysis results are compiled into a report and sent back to the device in real time.

[1406] 4. Providing Feedback

[1407] The device displays the received analysis results on the smartphone screen. It also provides visual feedback through the display device and specific audio advice via Bluetooth. For example, the analysis result may say, "Your right shoulder is lowered too much at the top of your hitting motion," and specific suggestions for improvement may be provided, such as, "To improve your shoulder movement, try practicing the following."

[1408] 5. Feedback adjustment by emotion engine

[1409] The server uses an emotion engine to recognize the user's emotional state. Emotion recognition is performed by analyzing the user's facial expressions and voice tone. The recognized emotional state is used to adjust the content and method of feedback. For example, if the user looks tired, the feedback content can be changed to a more encouraging tone. If the user is concentrating, more detailed technical instructions can be provided.

[1410] Hardware and software used

[1411] Hardware: Smartphone camera, smart glasses, Bluetooth module

[1412] Software: Machine learning algorithms, emotion recognition engine, Bluetooth connectivity software

[1413] Specific examples

[1414] For example, imagine a scenario where a user is trying out golf clubs at a brick-and-mortar store. The user takes a video of their swing with their smartphone and sends it to a server. A machine learning algorithm identifies flaws in the swing, and an emotion engine recognizes the user's level of fatigue. Based on this, real-time feedback such as "Your swing angle isn't good. Take a short break to refresh yourself" is displayed on the smart glasses.

[1415] Example prompts for generative AI models

[1416] "Please point out areas for improvement when the golf swing angle is poor. Also, add encouraging comments if the user is tired."

[1417] The above is a detailed description of the embodiment of the present invention. This system is very useful for users in physical stores to analyze their hitting motion and select the most suitable equipment.

[1418] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1419] Step 1:

[1420] A user downloads the app to their smartphone and creates an account. They enter their profile information and pair their smart glasses or other display device with the smartphone via Bluetooth. A confirmation of successful pairing appears on the device.

[1421] Step 2:

[1422] The user places their smartphone in an appropriate position and records their hitting motion. When the user taps the "Start Recording" button in the app, the smartphone camera begins recording their hitting motion in real time. This video data is temporarily saved on the smartphone.

[1423] Step 3:

[1424] The device sends the captured video of the batting motion to the server. The input data is a video file of the batting motion. After receiving this, the server passes the video data to a machine learning algorithm and begins analysis. The output is the analysis results in JSON format data.

[1425] Step 4:

[1426] The server uses machine learning algorithms to analyze each element of the hitting motion, including posture, timing, and speed. The analysis results include swing flaws and areas for improvement. After data processing and calculations, the results are compiled into a report.

[1427] Step 5:

[1428] The server sends the analysis results back to the device, which then visually displays the received analysis results on the app's user interface. Additionally, the device provides visual feedback via a display device and specific advice via audio via Bluetooth. At this time, feedback based on the analysis results is output as text and audio files.

[1429] Step 6:

[1430] The server uses an emotion engine to recognize the user's emotional state. The input data is the user's facial expression and voice tone, which are analyzed by the emotion recognition algorithm. The output is tag information that indicates the user's emotional state.

[1431] Step 7:

[1432] The server adjusts the feedback content and method based on the user's recognized emotional state. For example, if the user looks tired, the feedback tone is changed to be more encouraging. If the user is concentrating, detailed technical instructions are added. The text and audio files with the adjusted feedback content and tone are output.

[1433] Step 8:

[1434] The server recommends products for the user to try in a physical store environment. The input data is the analysis results and the user's profile information, which are analyzed by the recommendation algorithm. The output is a list of optimal products. The device presents this list to the user within the app.

[1435] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1436] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1437] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1438] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1439] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1440] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1441] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1442] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1443] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1444] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1445] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1446] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1447] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1448] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1449] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1450] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1451] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1452] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1453] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1454] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1455] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1456] The following is further disclosed regarding the above embodiment.

[1457] (Claim 1)

[1458] a means for capturing a user's golf swing;

[1459] An artificial intelligence means for analyzing the captured swing video;

[1460] visual and audio feedback means for providing the analysis results to the user;

[1461] A system including:

[1462] (Claim 2)

[1463] 10. The system of claim 1, wherein the artificial intelligence means analyzes the user's swing in real time.

[1464] (Claim 3)

[1465] 10. The system of claim 1, wherein the visual feedback means uses smart glasses.

[1466] (Claim 4)

[1467] 10. The system of claim 1, wherein audio feedback means is provided through Bluetooth.

[1468] (Claim 5)

[1469] A video lesson means for providing lesson videos;

[1470] A means to receive lesson videos via streaming from the server,

[1471] 10. The system of claim 1, comprising:

[1472] (Claim 6)

[1473] A billing management method for managing usage fees as a monthly service;

[1474] 10. The system of claim 1, comprising:

[1475] (Claim 7)

[1476] a means of collecting user feedback and improving the Service;

[1477] 10. The system of claim 1, comprising:

[1478] "Example 1"

[1479] (Claim 1)

[1480] means for capturing images of a user's actions;

[1481] An artificial intelligence means for analyzing the captured motion video;

[1482] visual and audio feedback means for providing the analysis results to the user;

[1483] a terminal for use by a user;

[1484] A means of displaying analysis results and providing advice using a terminal;

[1485] A system including:

[1486] (Claim 2)

[1487] 10. The system of claim 1, wherein the artificial intelligence means analyzes user behavior in real time.

[1488] (Claim 3)

[1489] 10. The system of claim 1, wherein the visual feedback means uses a video display device.

[1490] "Application Example 1"

[1491] (Claim 1)

[1492] A means for capturing images of a user's exercise movements;

[1493] An artificial intelligence means for analyzing the captured motion video;

[1494] visual and audio feedback means for providing the analysis results to the user;

[1495] A means for simulating movement in a virtual environment;

[1496] A means for comparing with other users and providing lesson content;

[1497] A system including:

[1498] (Claim 2)

[1499] 10. The system of claim 1, wherein the artificial intelligence means analyzes the user's movements in real time.

[1500] (Claim 3)

[1501] 10. The system of claim 1, wherein the visual feedback means uses smart glasses.

[1502] "Example 2: Combining Emotion Engines"

[1503] (Claim 1)

[1504] a device for capturing images of a user's actions;

[1505] AI function that analyzes the captured motion video,

[1506] Visual and audio feedback functions that provide analysis results to the user,

[1507] an emotion recognition engine that recognizes the user's emotions and adjusts the feedback;

[1508] Content provision services,

[1509] A system including:

[1510] (Claim 2)

[1511] 10. The system of claim 1, wherein the artificial intelligence function analyzes the user's actions in real time.

[1512] (Claim 3)

[1513] 10. The system of claim 1, wherein the visual feedback function uses a head-mounted display.

[1514] (Claim 4)

[1515] 2. The system according to claim 1, wherein the emotion recognition engine recognizes emotions by analyzing the user's facial expressions and voice.

[1516] (Claim 5)

[1517] 2. The system according to claim 1, wherein the content providing service delivers new content based on the user's usage.

[1518] "Application example 2 when combining emotion engines"

[1519] (Claim 1)

[1520] a device for capturing an image of a user's hitting motion;

[1521] A machine learning method for analyzing the captured hitting motion video;

[1522] visual and audio feedback means for providing the analysis results to the user;

[1523] an emotion engine that recognizes the user's emotional state and adjusts the feedback;

[1524] a means for recommending products for users to try in a real-world store environment;

[1525] A system including:

[1526] (Claim 2)

[1527] 2. The system of claim 1, wherein the machine learning means analyzes the user's hitting motion in real time.

[1528] (Claim 3)

[1529] 10. The system of claim 1, wherein the visual feedback means uses a display device. [Explanation of symbols]

[1530] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for capturing a user's golf swing; An artificial intelligence means for analyzing the captured swing video; visual and audio feedback means for providing the analysis results to the user; A system including:

2. 10. The system of claim 1, wherein the artificial intelligence means analyzes the user's swing in real time.

3. The system of claim 1 , wherein the visual feedback means uses smart glasses.

4. 10. The system of claim 1, wherein audio feedback means is provided through Bluetooth.

5. A video lesson means for providing lesson videos; A means to receive lesson videos via streaming from the server, The system of claim 1 , comprising:

6. A billing management method for managing usage fees as a monthly service; The system of claim 1 , comprising:

7. a means of collecting user feedback and improving the Service; The system of claim 1 , comprising:

Images