System

Abstract

A system is provided.SOLUTION: A system comprising: means for recording a swing motion of a user using a smart device; means for uploading data of the recorded swing motion to a cloud server; means for analyzing the data of the swing motion in the cloud server and generating an analysis result; means for specifying a task of the swing motion based on the analysis result and quantifying the task; and means for providing an improvement measure for the task.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Improving one's golf swing is an important challenge for many golfers. However, it is not easy to objectively evaluate one's own swing, identify specific issues, and then implement appropriate improvement measures. Hiring a professional coach is often not always realistic, especially for beginners and intermediate players. Furthermore, it is difficult to provide specific, quantified data for swing improvement and concrete solutions based on that data. Furthermore, there is little opportunity for interaction or information sharing with other golfers with the same goals, making it difficult to maintain motivation and absorb knowledge. A system that can solve these issues and efficiently improve one's golf swing is needed. [Means for solving the problem]

[0005] In this invention, a system is constructed that records a user's swing motion using a smart device and uploads the data to a cloud server, thereby analyzing the swing motion and providing detailed feedback for improvement. Specifically, the cloud server analyzes the recorded swing motion data and generates analysis results such as head speed, ball speed, and swing trajectory. Based on the analysis results, swing motion issues are identified and quantified, making the problems clear to the user in an easy-to-understand format. Specific improvement measures for the issues are also presented, helping the user efficiently practice for self-improvement. Furthermore, the ball trajectory is analyzed based on the recorded swing motion data, and the results are applied to more precisely evaluate swing motion issues. In addition, a community platform is provided where users can exchange information with other users who share the same goals, and an automatic inquiry response system using artificial intelligence is also established, providing an environment where users can quickly respond to their questions.

[0006] "Smart device" refers to a portable electronic device that allows a user to record video of a golf swing and process and transmit that data.

[0007] A "cloud server" is a remote computer system connected via the Internet that stores data, analyzes it, and generates feedback.

[0008] The term "swing motion" refers to a series of swinging movements using a golf club, including the body movements and the trajectory of the club during the process.

[0009] "Data analysis" refers to the process of extracting and evaluating various measurements (e.g., head speed, ball speed, swing trajectory) based on the swing video data received by the cloud server.

[0010] "Analysis results" refers to the collective term for various measurements and evaluations obtained by the cloud server from swing motion data.

[0011] "Quantifying issues" refers to the process of expressing the problems in a user's swing motion as specific numbers or indicators.

[0012] "Improvement measures" refer to specific instructions or suggestions for improving the user's swing motion based on the analysis results and quantified issues.

[0013] A "community platform" refers to an online space where users can share information and communicate with each other.

[0014] "Artificial intelligence" refers to computer systems that use techniques such as machine learning and natural language processing to generate automated responses to user inquiries.

[0015] "Ball trajectory analysis" refers to the process of measuring and evaluating the flight trajectory and characteristics of a golf ball based on recorded swing video data. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system that uses smart devices to record, analyze, and improve golf swings. The system quantifies the user's swing issues through video recording and data analysis on a cloud server, and provides specific improvement measures. It also includes a community platform where users can share information with other users, and an automated inquiry response function using artificial intelligence.

[0038] System Overview

[0039] 1. Record a video of your swing using a smart device

[0040] The user launches the smartphone app and takes a photo of their golf swing.

[0041] It provides a function to upload videos taken by the device to a cloud server.

[0042] 2. Data analysis using a cloud server

[0043] The server analyzes the video data it receives and extracts data such as swing trajectory, head speed, and ball speed.

[0044] The server identifies swing issues based on the analysis results and quantifies those issues.

[0045] 3. Providing remedial measures

[0046] The server provides the user with specific improvement measures based on the quantified issues.

[0047] The proposed improvements are displayed on the app in a way that is easy for users to understand.

[0048] 4. Providing a social community

[0049] The server provides a membership platform where users can share information and opinions.

[0050] 5. Automatic response to inquiries

[0051] The server uses the generative AI model to automatically generate answers to user inquiries.

[0052] To resolve any doubts, users can enter their questions within the app and receive a response.

[0053] Program processing

[0054] The program of this system performs the following processing.

[0055] 1. Video recording and uploading

[0056] The user launches the app and shoots a video of their swing. By tapping the recording button, a video of the golf swing is recorded.

[0057] Once the recording is complete, the device prompts the user to review the video and displays an "Upload" button, which the user taps to send the data to the cloud server.

[0058] 2. Data Analysis

[0059] The server analyzes the received video data and tracks the trajectory of the golf club for each frame.

[0060] The server extracts analytical data such as head speed, ball speed, and swing trajectory and stores this data in the user's profile.

[0061] 3. Quantifying issues and providing solutions

[0062] Based on the analysis results, the server quantifies the swing issues and identifies specific problems (e.g., slow head speed, improper swing trajectory).

[0063] The server provides users with specific advice and training methods to solve these problems.

[0064] 4. Use of social communities

[0065] Users have access to a membership platform where they can interact with other users and post and comment on specific issues or topics.

[0066] The device updates the community activity feed in real time, allowing users to see new posts and comments immediately.

[0067] 5. Automatic response to inquiries

[0068] The user enters a question in the app and taps the send button. For example, a question could be, "What should I do if my club head speed isn't increasing?"

[0069] The server uses a generative AI model to generate optimal answers to questions and responds to users in real time. The answers are specific and practical, and include methods that users can try immediately.

[0070] Specific examples

[0071] As a concrete example, consider the following scenario.

[0072] 1. Recording and analyzing swing videos

[0073] Users can record videos of their swings in their own backyard using a smartphone app and upload them to a cloud server. The videos clearly capture the full swing.

[0074] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph.

[0075] 2. Quantifying issues and providing solutions

[0076] Based on the analysis results, the server identifies the problem as an outside-in swing trajectory, and also points out that the head speed is slower than the average professional golfer.

[0077] The server will provide specific suggestions for improvement, such as, "To practice your inside-out swing, try the following drill."

[0078] 3. Sharing information with the community

[0079] A user accesses a community platform and posts a message such as, "I'm looking for advice on drills to improve my swing trajectory."

[0080] Other users respond by saying, "I had the same problem and this drill worked for me."

[0081] 4. Automatic response to inquiries

[0082] A user asks within the app, "What training methods can I use to increase my club head speed?"

[0083] The server uses the generative AI model to generate answers such as, "Specific training methods for increasing club head speed include training using weights and specific swing drills," and immediately provides them to the user.

[0084] In this way, the present invention provides a comprehensive system for golfers to efficiently improve their own swing and share information and questions with other users.

[0085] The processing flow will be explained below.

[0086] Step 1:

[0087] The user launches the smartphone app and taps the "swing video recording" button.

[0088] Step 2:

[0089] The device activates the smartphone camera and captures a picture of the user performing a golf swing.

[0090] Step 3:

[0091] After the device has finished recording, the user is prompted to check the video and, if there are no problems, tap the "Upload" button.

[0092] Step 4:

[0093] In order for the terminal to upload the captured video file to the cloud server, the terminal transmits the video data using the communication module.

[0094] Step 5:

[0095] The video data received by the server is temporarily stored.

[0096] Step 6:

[0097] The server invokes a video analysis algorithm and begins the analysis process to extract data such as swing trajectory, head speed, and ball speed.

[0098] Example: A server performs image analysis on each video frame to track the movement of a golf club.

[0099] Example: The server performs trajectory analysis to analyze the ball's flight pattern.

[0100] Step 7:

[0101] The server stores the analysis results as numerical data and associates them with the user's profile.

[0102] Step 8:

[0103] The server uses the quantified analysis results to identify issues with the user's swing and trajectory.

[0104] Example: If your clubhead speed is 90 mph, compare it to the pro average.

[0105] Example: If your swing path is "outside-in," it will generate feedback for general path corrections.

[0106] Step 9:

[0107] The server will provide the following information to the user along with the analysis results:

[0108] Numerical data on the problem (e.g., head speed, swing trajectory)

[0109] Advice for improvement (e.g., how to practice an inside-out swing)

[0110] Video links and text guides for recommended drills and practice techniques

[0111] Step 10:

[0112] The user taps the Community tab to access the membership platform.

[0113] Step 11:

[0114] The device displays the user's activity feed, updating the list of posts and comments in real time.

[0115] Step 12:

[0116] A user taps the "Create a new post" button and enters a question or comment.

[0117] Step 13:

[0118] The terminal transmits the input post content to the server.

[0119] Step 14:

[0120] The server saves the new post to a database and displays it to other users.

[0121] Step 15:

[0122] The user enters the question they want to ask the generated AI model and taps the "Send inquiry" button.

[0123] Step 16:

[0124] The terminal sends the entered question to the server.

[0125] Step 17:

[0126] The server calls the generative AI model and generates the best answer to the input question.

[0127] Example: The server responds to the question "What causes your swing to curve to the right?" with the answer "Your right hand may be too strong, so try to relax it."

[0128] Step 18:

[0129] The server sends the generated answer to the terminal for display to the user.

[0130] Example 1

[0131] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0132] Conventional golf swing analysis systems require expensive equipment and specialized knowledge, making them difficult for many amateur golfers to use. Furthermore, the time and cost required for individual lessons and coaching advice makes self-improvement of a golf swing extremely difficult. Furthermore, there is a lack of systems that provide specific improvement measures based on the results of swing analysis, leaving a need for a means for users to efficiently improve their skills.

[0133] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0134] In this invention, the server includes a means for recording a user's actions using an application launched by the user, a means for uploading the recorded action data to the server, a means for analyzing the action data on the server and generating analysis results, a means for identifying action issues based on the analysis results and quantifying the issues, and a means for providing solutions to the issues. This allows users to easily record their own actions and obtain detailed analysis and specific solutions for improvement via the server without specialized knowledge. Furthermore, by sharing information with other users, users can receive feedback and support within the community, thereby enabling efficient and continuous skill improvement.

[0135] A "user" is an entity that uses the system to record its own actions, obtain analysis results, and receive improvement measures.

[0136] An "application" is software that runs on a smart device and allows users to record actions, upload data, and display analysis results.

[0137] "Movement" is a physical movement, such as a golf swing, made by a user that is recorded by the system.

[0138] A "server" is a computer system that receives and analyzes action data uploaded by users.

[0139] "Data" refers to the video of the user's actions recorded and the quantified information resulting from that analysis.

[0140] "Analysis" refers to the process of tracking the characteristics of movements based on recorded movement data and deriving quantified results.

[0141] "Issues" refer to problems or areas for improvement in user behavior that have become apparent as a result of the analysis.

[0142] "Quantifying" means expressing the characteristics and challenges of an action as quantitative data.

[0143] "Improvement measures" refer to specific methods and advice on how the user should improve the issues identified in the analysis results.

[0144] A "Platform" is an online venue where users can share information and communicate with other users.

[0145] "Artificial intelligence" is a program that automatically generates responses to user inquiries.

[0146] This invention is a system that uses a smart device to record a user's golf swing and analyzes the data on a cloud server to quantify the user's swing issues and provide specific improvement measures. This system is implemented using a smart device application, a cloud server, analysis software, and an artificial intelligence model.

[0147] Hardware used

[0148] Smart devices: Smartphones and tablets are used. These devices must have a camera and internet connection.

[0149] Cloud server: A computer system where video data is uploaded and where analysis processing is performed.

[0150] Software used

[0151] Application: Software for smart devices that allows users to record their golf swings and upload the data to a server. Specifically, it is implemented as an application for iOS or Android.

[0152] Analysis software: To analyze golf swing videos, we use libraries such as OpenCV and TensorFlow, which allow us to perform frame analysis and feature extraction of movements.

[0153] Generative AI model: An artificial intelligence model that generates automated responses to user inquiries. This uses natural language processing (NLP) techniques such as GPT-3 and BERT.

[0154] Processing flow

[0155] 1. Video recording and uploading

[0156] The user starts the smartphone app and shoots a video of their swing. An easy-to-use interface is provided for video recording.

[0157] After shooting is complete, the device displays a video confirmation screen to the user, allowing them to edit the video or cut out unnecessary parts.

[0158] When the user checks the video and taps the upload button, the device sends the video data to the cloud server.

[0159] 2. Data analysis and result generation

[0160] The server temporarily stores the received video data in storage, then begins motion analysis for each video frame, tracking the movement of the club and the body.

[0161] The server quantifies and extracts motion data such as head speed, ball speed, and swing trajectory, and stores it in the user's profile.

[0162] 3. Quantifying issues and providing solutions

[0163] The server identifies the user's swing issues based on the analysis results and quantifies the specific problems.

[0164] Using a generative AI model, specific improvement measures for quantified issues are automatically generated, and these improvement measures are displayed in the app in a format that is easy for users to understand.

[0165] 4. Use of social communities

[0166] Users can use the in-app community feature to share information and opinions with other users, and can post and comment on specific topics.

[0167] The device updates the community feed in real time, allowing you to see new posts and comments immediately.

[0168] 5. Automatic response to inquiries

[0169] When a user enters a question in the app and taps the submit button, the server uses a generative AI model to generate the best answer.

[0170] Answers are provided to users in real time and include specific, practical content. For example, in response to a question like, "How can I train to increase my clubhead speed?", the answer might be, "There are weight training exercises and specific swing drills."

[0171] Specific examples

[0172] Examples of swing video recording and analysis

[0173] Users can record videos of their swings in their own backyard using a smart device, and then upload the videos to a cloud server.

[0174] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph.

[0175] Based on the analysis results, the server identifies the problem of an outside-in swing trajectory and identifies the head speed as slower than the average for professionals. The server then provides specific improvements to address these issues.

[0176] Examples from the community

[0177] When a user posts something like "I'm looking for advice on drills to improve my swing path," other users will respond with things like "I was having the same problem, and this drill worked for me."

[0178] Specific examples of automated response to inquiries

[0179] When a user asks, "What training methods can I use to increase my clubhead speed?", the server uses a generative AI model to respond, "Specific training methods for increasing clubhead speed include weight training and specific swing drills."

[0180] In this way, the present invention provides a comprehensive system for golfers to efficiently improve their swing and share information and questions with other users.

[0181] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0182] Step 1:

[0183] The user launches the smart device application and selects the "Swing Photo" option from the home screen.

[0184] Input: Launching applications, displaying the home screen

[0185] Output: Show "Swing Shooting" option

[0186] What it does: The app invokes the camera function and displays a message encouraging the user to prepare for their swing.

[0187] Step 2:

[0188] The user taps the recording button to record a video of their golf swing.

[0189] Input: Tap the recording button, video data of the swing

[0190] Output: Recorded swing video data

[0191] Specific operation: The device camera will start up and record the user's swing. When the recording is finished, a save button will be displayed.

[0192] Step 3:

[0193] Users can check the video on the video confirmation screen and edit or cut out unnecessary parts.

[0194] Input: Recorded swing video, user editing actions

[0195] Output: Edited swing video data

[0196] What it does: The app provides video playback and editing functionality and displays an interface for the user to finalize the footage.

[0197] Step 4:

[0198] The user taps the upload button to send the edited video data to the cloud server.

[0199] Input: Edited swing video data, tap the upload button

[0200] Output: Video data stored on a cloud server

[0201] Specific operation: The device sends data to the cloud server via the Internet. Once the upload is complete, a confirmation message will be displayed.

[0202] Step 5:

[0203] The video data received by the server is temporarily stored in storage.

[0204] Input: Uploaded video data

[0205] Output: Saved video data

[0206] Specific operation: The server receives the video data and stores it in the file system. This storage is temporary storage for fast access.

[0207] Step 6:

[0208] The server starts motion analysis for each video frame, tracking and analyzing the movement of the golf club and the body.

[0209] Input: Saved video data

[0210] Output: Analyzed motion data (head speed, ball speed, swing trajectory, etc.)

[0211] How it works: The server uses libraries such as OpenCV and TensorFlow to analyze the images of each frame, and then uses a motion detection algorithm to extract club and body movement data.

[0212] Step 7:

[0213] The server quantifies the extracted behavioral data and stores it in the user's profile.

[0214] Input: Parsed motion data

[0215] Output: Stored quantified operating data

[0216] Specific operation: The analysis results are processed as numerical data and added to the user's profile database.

[0217] Step 8:

[0218] The server identifies the user's swing issues based on the analysis results and quantifies the specific problems.

[0219] Input: digitized motion data

[0220] Output: Quantified issues

[0221] Specific operation: The server uses the generative AI model to extract issues using specific algorithms and quantitatively evaluate them.

[0222] Step 9:

[0223] The server uses the generated AI model to automatically generate specific improvement measures for the quantified issues.

[0224] Input: Quantified task

[0225] Output: Generated remediation measures

[0226] How it works: The generative AI model references past data and training methods to suggest improvements based on the user's challenges, which may include text and links to images or videos.

[0227] Step 10:

[0228] The improvement measures generated by the server are sent to the user's smart device and displayed on the app.

[0229] Input: Generated remediation measures

[0230] Output: Improvement measures displayed on the app

[0231] Specific behavior: The app displays the improvement measures obtained from the server in the interface so that the user can take action immediately.

[0232] Step 11:

[0233] Users access the Community tab and share information with other users.

[0234] Enter: Access to the Community tab

[0235] Output: Shared information and comments

[0236] What it does: The app provides a post creation feature that allows users to share their challenges and improvements with the community, making it easy to receive feedback and advice.

[0237] Step 12:

[0238] The user enters a question in the in-app inquiry tab and submits it.

[0239] Input: User question

[0240] Output: Query data sent to the server

[0241] Specific behavior: Provides an interface for the app to send entered questions to the server.

[0242] Step 13:

[0243] The server uses a generative AI model to generate optimal answers and provides them to the user in real time.

[0244] Input: Question data from the user

[0245] Output: The generated answer

[0246] Specific operation: The server uses the generative AI model to generate an appropriate answer based on the question and sends it in text format to the user's device.

[0247] Step 14:

[0248] The user references the answers provided and takes various actions to improve their swing.

[0249] Input: Generated Answer

[0250] Output: Actions taken

[0251] Specific Actions: Plan practice sessions or attempt specific training drills based on the answers the user generates.

[0252] (Application example 1)

[0253] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0254] Conventional golf swing analysis systems not only provide users with analysis results, but also make it difficult to share information with other users in real time or receive actual support at a golf club. Furthermore, there is a lack of advice on selecting appropriate golf equipment based on swing analysis results, leaving users lacking the information and tools to instantly improve their skills. This often means that users need time to improve their swing, and it is difficult to receive appropriate guidance and support during the process.

[0255] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0256] In this invention, the server includes means for recording a user's swing motion using a smart computing device, means for transmitting the recorded swing motion data to a remote server, means for analyzing the swing motion data in the remote server and generating analysis results, means for identifying problems in the swing motion based on the analysis results and quantifying the problems, means for providing solutions to the problems, means for sharing the analysis results with other users in the store in real time, and means for providing a function for making inquiries to store staff. This allows users to check the swing analysis results in real time in the store, share experiences and information with other users, and immediately receive appropriate advice from store staff.

[0257] A "smart computing device" is an electronic device that is portable by the user and has a high-resolution camera and internet connectivity that is capable of recording and transmitting swing motion.

[0258] "User" refers to a user who seeks to improve their golf swing using the system of the present invention.

[0259] The term "swing motion" refers to a series of physical movements that are made when using a golf club to hit a ball, including the trajectory of the golf club, head speed, input force, and the like.

[0260] "Remote Server" refers to a data center that is a data analysis and storage system accessible via the Internet and that analyzes the swing video sent by the User and provides the results.

[0261] The "analysis results" are technical evaluations and quantified data regarding the user's swing, generated by the server based on swing motion data.

[0262] "Quantification" refers to expressing various parameters of the analysis results as quantitative values, allowing users to concretely recognize their own performance.

[0263] "Improvement measures" refer to specific training methods and technical advice provided to address swing movement issues identified based on the analysis results.

[0264] "Real-time" refers to the near-instantaneous transmission of data, analysis, provision of results, and sharing of information.

[0265] "Information sharing" refers to users exchanging data such as analysis results, experiences, and opinions with other users and store staff.

[0266] The "inquiry function" refers to a system that allows users to input questions about analysis results and improvement measures and receive an automatic response or a reply from store staff on the spot.

[0267] This invention is a system for use in golf equipment stores that allows customers to receive real-time analysis and feedback on their swings. The system includes a smart computing device, a remote server, and a social community platform for sharing analysis results. It also provides query capabilities using generative AI models.

[0268] System Overview

[0269] 1. Video recording and uploading

[0270] Users use a smart computing device (such as a smartphone) to film their swing, and once filming is complete, the video data is automatically transmitted to a remote server.

[0271] 2. Data Analysis

[0272] The server analyzes the received swing video and extracts data such as the golf club trajectory, head speed, ball speed, etc. These analysis results are quantified and recorded in the user profile.

[0273] 3. Providing Feedback

[0274] The server quantifies the identified issues based on the analysis results and indicates specific areas for improvement to the user. The improvement measures are sent in real time to a smart computing device and displayed to the user.

[0275] 4. Social Community

[0276] Users can share analysis results with other customers in the store in real time, exchange information, and hold discussions and provide feedback on specific topics.

[0277] 5. Inquiry function

[0278] When a user inputs a question about improvement measures or analysis results into the server, the optimal answer is automatically generated using a generative AI model, and this answer is immediately provided to the user.

[0279] Hardware and Software Configuration

[0280] Smart computing devices: Smartphones and tablets equipped with high-resolution cameras and internet connectivity.

[0281] Remote server: Use cloud services such as Amazon Web Services (AWS) and Google Cloud Platform (GCP).

[0282] Data analysis software: Video data was analyzed using Python and OpenCV.

[0283] Generative AI model: Generates automatic responses to user inquiries using natural language processing.

[0284] Examples of concrete examples and prompts

[0285] Examples:

[0286] 1. The user takes a photo of their golf swing with their smartphone in the practice area inside the store.

[0287] 2. The captured video is automatically uploaded to a cloud server, where analysis begins.

[0288] 3. The server quantifies the analysis results (e.g., head speed, swing trajectory, etc.) and immediately sends them to your smartphone.

[0289] 4. The user checks the analysis results on the spot and continues practicing according to the provided improvement measures.

[0290] 5. When users ask questions about problems with their swing, the generative AI model provides specific answers. For example, in response to a question like, "What can I do if my club head speed isn't improving?", the model might respond with, "Training with weights and specific swing drills are effective."

[0291] Example prompt sentence:

[0292] "A user has uploaded a video of their swing taken in a golf shop. Please analyze this video and provide data on the following: 1. Head speed 2. Swing path 3. Ball speed. Also, based on this data, please provide specific advice on how to improve their swing."

[0293] In this way, the present invention provides a comprehensive support system for users to efficiently analyze and improve their swings and share information within a golf equipment store.

[0294] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0295] Step 1:

[0296] A user uses a smart computing device (such as a smartphone) to film their swing motion.

[0297] Input: A smartphone for users to record swing videos.

[0298] Data processing / calculation: Video is taken using a high-resolution camera.

[0299] Output: Recorded swing video file.

[0300] Step 2:

[0301] The device uploads the captured swing video to a cloud server.

[0302] Input: Recorded swing video file.

[0303] Data processing / calculation: Send the video file to the cloud server using an HTTP request.

[0304] Output: Video files stored on cloud server.

[0305] Step 3:

[0306] The server analyzes the received swing video and extracts swing motion data.

[0307] Input: Video files stored on a cloud server.

[0308] Data processing / calculation: Using Python and OpenCV, information such as golf club trajectory, head speed, and ball speed is extracted from video data.

[0309] Output: Swing motion analysis results (e.g. head speed, swing path, ball speed).

[0310] Step 4:

[0311] The server identifies issues in the swing motion based on the analysis results and quantifies those issues.

[0312] Input: Analysis results of swing motion.

[0313] Data processing / calculation: The analysis results are applied to an evaluation algorithm to quantitatively express the identified issues.

[0314] Output: Quantified swing motion issues (e.g., slow clubhead speed, inaccurate swing path).

[0315] Step 5:

[0316] The server generates solutions to the problems and provides them to the user.

[0317] Input: Quantified swing movement tasks.

[0318] Data processing / computation: Running algorithms to generate specific training or technical advice to address the problem.

[0319] Output: Specific improvement measures (e.g., training methods to increase clubhead speed, drills to improve swing path).

[0320] Step 6:

[0321] The device displays analysis results and improvement measures to the user in real time.

[0322] Input: Specific improvement measures.

[0323] Data processing / calculation: Rendering is performed to display the results and improvement measures on the smartphone UI.

[0324] Output: Analysis results and remediation measures displayed on the user's screen.

[0325] Step 7:

[0326] Users can share analysis results with other customers in the store and exchange information.

[0327] Input: Analysis results and remediation measures.

[0328] Data processing / computation: Use social community platforms to share data and post comments and feedback.

[0329] Output: Information exchanged with other users.

[0330] Step 8:

[0331] Users ask questions about analysis results and improvement measures, and the server automatically responds using the generated AI model.

[0332] Input: A question from the user (e.g., "What training methods can I use to increase my club head speed?").

[0333] Data processing / calculation: Input a prompt sentence into the generative AI model to generate the optimal answer.

[0334] Output: The automated response that is provided to the user.

[0335] Specific prompt examples:

[0336] "A user has uploaded a video of their swing taken in a golf shop. Please analyze this video and provide data on the following: 1. Head speed 2. Swing path 3. Ball speed. Also, based on this data, please provide specific advice on how to improve their swing."

[0337] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0338] This invention is a system that uses a smart device to record, analyze, and improve a golf swing, and also combines it with user emotion recognition. This system quantifies the user's swing issues through swing video recording and data analysis on a cloud server, and provides specific improvement measures. It also uses an emotion engine to grasp the user's psychological state and provides feedback based on that to support more effective swing improvement.

[0339] System Overview

[0340] 1. Record a video of your swing using a smart device

[0341] The user launches the smartphone app and takes a photo of their golf swing.

[0342] It provides a function to upload videos taken by the device to a cloud server.

[0343] 2. Data analysis using a cloud server

[0344] The server analyzes the video data it receives and extracts data such as swing trajectory, head speed, and ball speed.

[0345] The server identifies swing issues based on the analysis results and quantifies those issues.

[0346] 3. Providing remedial measures

[0347] The server provides the user with specific improvement measures based on the quantified issues.

[0348] The proposed improvements are displayed on the app in a way that is easy for users to understand.

[0349] 4. Emotion recognition

[0350] The device records the user's facial expressions and voice during the swing and uploads them to a cloud server.

[0351] The server uses an emotion engine to analyze emotions from the user's facial expressions and voice.

[0352] 5. Providing Feedback

[0353] Based on the emotion recognition results, the server generates improvement measures and advice that take into account the user's psychological state.

[0354] The server also incorporates the user's emotional data into the analysis results to provide more accurate feedback.

[0355] 6. Providing social communities

[0356] The server provides a membership platform where users can share information and opinions.

[0357] 7. Auto-response to inquiries

[0358] The server uses the generative AI model to automatically generate answers to user inquiries.

[0359] To resolve any doubts, users can enter their questions within the app and receive a response.

[0360] Program processing

[0361] The program of this system performs the following processing.

[0362] 1. Video recording and uploading

[0363] The user launches the app and shoots a video of their swing. By tapping the recording button, a video of the golf swing is recorded.

[0364] Once the recording is complete, the device prompts the user to review the video and displays an "Upload" button, which the user taps to send the data to the cloud server.

[0365] 2. Data Analysis

[0366] The server analyzes the received video data and tracks the trajectory of the golf club for each frame.

[0367] The server extracts analytical data such as head speed, ball speed, and swing trajectory and stores this data in the user's profile.

[0368] 3. Quantifying issues and providing solutions

[0369] Based on the analysis results, the server quantifies the swing issues and identifies specific problems (e.g., slow head speed, improper swing trajectory).

[0370] The server provides users with specific advice and training methods to solve these problems.

[0371] 4. Emotion Recognition and Feedback

[0372] When a user shoots a video of their swing, their facial expressions and voice are also recorded at the same time.

[0373] The device sends this facial expression and voice data to a cloud server.

[0374] The server uses an emotion engine to analyze the user's facial expressions and voice to determine their emotions. For example, if the user is nervous while taking a photo, the server can detect that emotion.

[0375] Based on the emotion recognition results, the server generates improvement measures and advice that take into account the user's psychological state. For example, it could provide advice such as "Try breathing techniques to help you swing more relaxed."

[0376] The server also incorporates the user's emotional data into the analysis results to provide more accurate feedback.

[0377] 5. Use of social communities

[0378] The user taps the Community tab to access the membership platform.

[0379] The device displays the user's activity feed, updating the list of posts and comments in real time.

[0380] A user taps the "Create a new post" button and enters a question or comment.

[0381] The terminal transmits the input post content to the server.

[0382] The server saves the new post to a database and displays it to other users.

[0383] 6. Automatic response to inquiries

[0384] The user enters the question they want to ask the generated AI model and taps the "Send inquiry" button.

[0385] The terminal sends the entered question to the server.

[0386] The server calls up the generative AI model and generates the optimal answer to the input question. For example, in response to the question, "What kind of training is effective for increasing club head speed?", it provides a specific training method.

[0387] The server sends the generated answer to the terminal for display to the user.

[0388] Specific examples

[0389] As a concrete example, consider the following scenario.

[0390] 1. Recording and analyzing swing videos

[0391] Users can record videos of their swings in their own backyard using a smartphone app and upload them to a cloud server. The videos include the full swing, as well as the user's facial expressions and voice.

[0392] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph. The emotion engine also recognizes that the user is tense during the swing.

[0393] 2. Quantifying issues and providing solutions

[0394] Based on the analysis results, the server identifies that the swing trajectory is outside-in as the problem, and points out that the head speed is slower than the average for professionals as an issue.

[0395] The server will provide specific improvement measures, such as "Try the following drill to practice your inside-out swing," and will also provide advice including "breathing techniques and mental training to relieve tension."

[0396] 3. Sharing information with the community

[0397] A user accesses a community platform and posts a message such as, "I'm looking for advice on drills to improve my swing trajectory."

[0398] Other users respond by saying, "I had the same problem and this drill worked for me."

[0399] 4. Automatic response to inquiries

[0400] A user asks within the app, "What kind of training is effective for increasing club head speed?"

[0401] The server uses the generative AI model to generate specific answers such as "Training with weights and specific swing drills are effective in increasing club head speed," and immediately provides them to the user.

[0402] In this way, the present invention provides a comprehensive system that allows golfers to efficiently improve their own swing, while also taking care of their psychological aspects, and allows them to share information and solve questions with other users.

[0403] The processing flow will be explained below.

[0404] Step 1:

[0405] The user launches the smartphone app and taps the "swing video recording" button.

[0406] Step 2:

[0407] The device activates the smartphone's camera and microphone to capture images and audio of the user performing a golf swing.

[0408] Step 3:

[0409] After the device has finished recording, the user is prompted to check the video and, if there are no problems, tap the "Upload" button.

[0410] Step 4:

[0411] The device transmits the data using a communication module to upload the captured video files and audio data to a cloud server.

[0412] Step 5:

[0413] The server temporarily stores the received video and audio data.

[0414] Step 6:

[0415] The server invokes a video analysis algorithm and begins the analysis process to extract data such as swing trajectory, head speed, and ball speed.

[0416] Example: A server performs image analysis on each video frame to track the movement of a golf club.

[0417] Example: The server performs trajectory analysis to analyze the ball's flight pattern.

[0418] Step 7:

[0419] The server uses an emotion engine to analyze the user's facial expressions in audio data and video to identify emotions.

[0420] Step 8:

[0421] The server stores the analysis results as numerical data and associates them with the user's profile.

[0422] Step 9:

[0423] The server uses the quantified analysis results to identify issues with the user's swing and trajectory.

[0424] Example: If your clubhead speed is 90 mph, compare it to the pro average.

[0425] Example: If your swing path is "outside-in," it will generate feedback for general path corrections.

[0426] Example: Identifying that a user is nervous based on facial and voice analysis.

[0427] Step 10:

[0428] The server will provide the following information to the user along with the analysis results:

[0429] Numerical data on the problem (e.g., head speed, swing trajectory)

[0430] Advice for improvement (e.g., how to practice an inside-out swing)

[0431] Emotion-based advice (e.g., breathing techniques for a relaxed swing)

[0432] Video links and text guides for recommended drills and practice techniques

[0433] Step 11:

[0434] The user taps the Community tab to access the membership platform.

[0435] Step 12:

[0436] The device displays the user's activity feed, updating the list of posts and comments in real time.

[0437] Step 13:

[0438] A user taps the "Create a new post" button and enters a question or comment.

[0439] Step 14:

[0440] The terminal transmits the input post content to the server.

[0441] Step 15:

[0442] The server saves the new post to a database and displays it to other users.

[0443] Step 16:

[0444] The user enters the question they want to ask the generated AI model and taps the "Send inquiry" button.

[0445] Step 17:

[0446] The terminal sends the entered question to the server.

[0447] Step 18:

[0448] The server calls the generative AI model and generates the best answer to the input question.

[0449] Example: The server responds to the question "What causes your swing to curve to the right?" with the answer "Your right hand may be too strong, so try to relax it."

[0450] Step 19:

[0451] The server sends the generated answer to the terminal for display to the user.

[0452] Example 2

[0453] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0454] Conventional golf swing analysis systems focus on analyzing technical issues in a user's swing and providing solutions for improvement, but do not consider psychological aspects. As a result, if a user feels nervous or stressed, this may not lead to actual performance improvement. Furthermore, smoother support is needed for sharing information with other users and resolving questions.

[0455] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0456] In this invention, the server includes a means for recording the user's facial expressions and voice during the swing, a means for uploading the recorded facial and voice data to a cloud server, and a means for analyzing the user's emotions using an emotion recognition engine in the cloud server. This enables feedback that takes the user's psychological state into consideration. The system also includes a means for identifying and quantifying issues in the swing based on the analysis results, and a means for automatically responding to inquiries from the user using a generative AI model, enabling comprehensive support for swing improvement from both technical and psychological aspects.

[0457] A "smart device" refers to an electronic device that can be carried by a user and that can connect to the Internet and run various applications. Examples include smartphones and tablets.

[0458] A "cloud server" refers to a remote server that can store, analyze, and communicate data over the Internet.

[0459] An "analysis algorithm" refers to a mathematically and logically defined method of defining rules and procedures for extracting, classifying, and evaluating useful information from specific data.

[0460] An "emotion recognition engine" refers to technology that analyzes non-verbal data such as a user's facial expressions and voice to identify their emotional state.

[0461] "Analysis results" refers to the specific numerical values ​​and evaluation results obtained after analyzing swing movements, facial expressions, and audio data.

[0462] "Means for quantifying problems" refers to methods and tools for quantitatively expressing problems in a user's swing motion from the analysis results.

[0463] The "means for providing improvement measures" refers to a method or system for proposing specific instruction content or training methods to the user for the identified problems in the swing motion.

[0464] A "generative AI model" refers to a mathematical model that uses artificial intelligence to automatically generate answers and advice to user inquiries.

[0465] A "community platform" refers to an online space where multiple users can share information, exchange opinions, ask questions, and answer questions.

[0466] This invention is a system that uses a smart device to record, analyze, and improve golf swings, and also combines it with user emotion recognition. This system starts when a user uses a smart device such as a smartphone to record a video of their golf swing and uploads the video data to a cloud server.

[0467] Hardware and software used:

[0468] Smart Device: An electronic device that is portable and has internet connectivity, such as a smartphone or tablet.

[0469] Cloud server: A remote server that stores, analyzes, and communicates data.

[0470] Analysis algorithm: Using image analysis libraries such as OpenCV, the swing trajectory, head speed, ball speed, etc. are analyzed for each frame.

[0471] Emotion recognition engine: Technology that analyzes emotions from a user's facial expressions and voice, such as Microsoft Azure's Emotion API.

[0472] Generative AI models: Use artificial intelligence models such as GPT-4 to automatically generate answers to user inquiries.

[0473] The specific operation steps of the system:

[0474] 1. Video recording and uploading

[0475] The user starts the smartphone app and records a swing video. They tap the record button to start recording and the end button to stop recording.

[0476] If the user checks the video and there are no problems, they tap the upload button to send the video to the cloud server.

[0477] 2. Data Analysis

[0478] The server receives the uploaded video data and uses an analysis algorithm to extract data such as swing trajectory, head speed, and ball speed for each frame.

[0479] The extracted data is stored in a profile for each user.

[0480] 3. Quantifying issues and providing solutions

[0481] The server then quantifies swing issues based on the analysis results. For example, if the head speed is less than 85 mph, it will be listed as a "low head speed" issue.

[0482] The server generates specific improvement measures for each quantified issue and displays them in the user's app, such as "training methods to improve club head speed."

[0483] 4. Emotion Recognition and Feedback

[0484] When a user shoots a video of their swing, their facial expressions and voice are also recorded at the same time.

[0485] The device transmits facial expression and voice data to a cloud server.

[0486] The server uses an emotion recognition engine to analyze the user's emotions, for example, detecting that the user is nervous during the shoot.

[0487] Based on the analysis results and emotional data, the server generates comments and improvement measures that take into account the user's psychological state and provides them to the user's smart device, such as "breathing techniques for relaxation."

[0488] 5. Use of social communities

[0489] Users access the platform by tapping the community tab within the smartphone app.

[0490] The device displays posts and comments in real time and provides an interface for users to post questions and opinions.

[0491] The server stores new posts and comments in a database, making them available for all users to see.

[0492] 6. Automatic response to inquiries

[0493] The user enters a question in the app and taps the "Send Inquiry" button.

[0494] The device sends the question to the server, which then uses a generative AI model (e.g., GPT-4) to generate the optimal answer.

[0495] The server displays the generated answer on the user's smart device.

[0496] Specific operation example

[0497] 1. Recording and analyzing swing videos

[0498] Users can record videos of their swings in their own backyard using their smartphones and upload them to a cloud server. The videos include the swing, the user's facial expressions, and their voice.

[0499] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph.

[0500] In addition, the emotion recognition engine detects whether the user is nervous during the swing.

[0501] 2. Quantifying issues and providing solutions

[0502] The server identifies the problem as slow head speed and swing trajectory, and then provides the user with specific improvement measures, such as drills to practice an inside-out swing and breathing techniques to relieve tension.

[0503] 3. Sharing information with the community

[0504] A user accesses a community platform and posts something like, "I'm looking for advice on improving my swing."

[0505] Other users will reply saying, "This drill was effective."

[0506] 4. Automatic response to inquiries

[0507] A user asks, "How can I train to increase my club head speed?"

[0508] The server uses the generative AI model to generate answers such as "Training with weights and specific swing drills are effective" and provides them to the user.

[0509] In this way, the present invention can comprehensively support the user's swing improvement and provide feedback that also includes psychological aspects.In addition, the information sharing function with other users and the automatic response function to inquiries create an efficient and effective training environment.

[0510] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0511] Step 1:

[0512] The user launches the smartphone app and records a video of their swing.

[0513] Input: Start of shooting by user operation

[0514] Data processing: Record a swing video using a smartphone camera

[0515] Output: Recorded swing video file

[0516] Specific behavior:

[0517] The user launches the app and switches to shooting mode.

[0518] The user taps the record button to begin recording the golf swing.

[0519] The user taps the end button to end the recording.

[0520] Step 2:

[0521] The device uploads the captured video to a cloud server.

[0522] Input: Recorded swing video file

[0523] Data processing: Transfer video data to the cloud server

[0524] Output: Video data on a cloud server

[0525] Specific behavior:

[0526] The device will prompt the user to review the video and display an "Upload" button.

[0527] When the user taps the upload button, the device uses its internet connection to send the video file to a cloud server.

[0528] Step 3:

[0529] The server analyzes the video data it receives and extracts data such as swing trajectory, head speed, and ball speed.

[0530] Input: Video data on a cloud server

[0531] Data processing: Perform video analysis using analysis algorithms such as OpenCV

[0532] Output: Analysis data such as swing trajectory, head speed, ball speed, etc.

[0533] Specific behavior:

[0534] The server analyzes the video data frame by frame.

[0535] The server extracts data such as swing trajectory, head speed, and ball speed and stores it in the user's profile.

[0536] Step 4:

[0537] The server quantifies swing issues based on the analysis results and generates specific improvement measures.

[0538] Input: Swing trajectory, head speed, ball speed data

[0539] Data processing: Applying algorithms to quantify swing problems using analytical data

[0540] Output: Quantified issues and concrete improvement measures

[0541] Specific behavior:

[0542] The server identifies swing issues based on the analysis results.

[0543] The server generates specific improvement measures and training methods based on the issues.

[0544] The server sends the generated advice to the user's app.

[0545] Step 5:

[0546] The device records the user's facial expressions and voice and uploads them to a cloud server.

[0547] Input: Photographed facial expressions and voice data

[0548] Data processing: Send facial expression and voice data to a cloud server

[0549] Output: Facial expression and voice data on a cloud server

[0550] Specific behavior:

[0551] When the device shoots a swing video, it also records the user's facial expressions and voice at the same time.

[0552] The device uploads facial expression and voice data to a cloud server.

[0553] Step 6:

[0554] The server uses an emotion recognition engine to analyze emotions from the user's facial expressions and voice.

[0555] Input: Facial expression and voice data on a cloud server

[0556] Data processing: Emotion analysis using an emotion recognition engine (e.g., Emotion API)

[0557] Output: User emotion data

[0558] Specific behavior:

[0559] The server inputs facial and voice data into an emotion recognition engine.

[0560] The server identifies the user's emotions (e.g., tension, relaxation, etc.).

[0561] Step 7:

[0562] The server reflects the emotional data in the analysis results and generates appropriate feedback that takes the user's psychological state into account.

[0563] Input: Analysis data and emotion data

[0564] Data processing: Adding emotional data to the analysis results and adjusting improvement measures and advice

[0565] Output: Feedback that takes into account the user's psychological state

[0566] Specific behavior:

[0567] If the server detects tension, it generates feedback that includes "breathing techniques to relax."

[0568] The server sends the generated feedback to the user's app.

[0569] Step 8:

[0570] Users access the community platform to share information and post questions.

[0571] Input: User-submitted content

[0572] Data processing: Save the posted content in a database and share it with other users

[0573] Output: Comments and replies by other users

[0574] Specific behavior:

[0575] A user taps the Community tab to access the platform.

[0576] The device displays the latest posts and comments and sends the user's new posts to the cloud server.

[0577] The server stores the posts in a database and makes them available for all users to view.

[0578] Step 9:

[0579] The user asks a question using the generative AI model, and the server generates an automated response.

[0580] Input: Question asked by the user (prompt text)

[0581] Data processing: Using a generative AI model (e.g., GPT-4) to generate optimal answers

[0582] Output: The generated answer

[0583] Specific behavior:

[0584] The user enters a question in the app and taps the "Send Inquiry" button.

[0585] The device sends the question to the cloud server.

[0586] The server uses a generative AI model to generate the best answer to the question.

[0587] The server generates the answer and sends it to the user, who displays it in the app.

[0588] In this way, a system is constructed that comprehensively supports the user in improving their swing by performing data processing and calculations based on the input data at each step and then using the resulting output to proceed to the next processing step.

[0589] (Application example 2)

[0590] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0591] Existing factory robot motion analysis systems lack the functionality to quantify motion efficiency and accuracy and provide specific improvement measures. Furthermore, they do not provide feedback that takes into account the psychological state of the worker or suggest improvements to the work environment, resulting in insufficient improvements to the performance of both the robot and the worker. This makes it difficult to create an efficient and stress-free work environment.

[0592] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recording the user's robot operations using a smart device, means for uploading the recorded operation data to a cloud server, means for analyzing the operation data and generating analysis results in the cloud server, means for identifying operation issues based on the analysis results and quantifying the issues, means for providing improvements to the issues, means for uploading emotional data recorded during the operations to the cloud server, means for analyzing the emotional data in the cloud server and identifying the user's psychological state, and means for providing feedback based on the psychological state. This makes it possible to scientifically improve the operational efficiency and accuracy of factory robots and provide feedback and improve the work environment that takes into account the psychological state of workers.

[0593] A "smart device" is an electronic device that can be carried and operated by a user, such as a smartphone or tablet.

[0594] A "cloud server" is a remote server for storing, managing, and analyzing data over a network.

[0595] "Analysis" is the process of evaluating recorded data using scientific methods and extracting the necessary information.

[0596] An "issue" refers to a problem or defect that needs to be resolved in a system or process.

[0597] "Quantification" refers to expressing the analysis results in concrete numerical values.

[0598] "Improvement measures" refer to specific policies and methods for solving identified issues.

[0599] "Emotion data" is voice and facial expression information recorded to assess the user's psychological state.

[0600] "Mental state" refers to the user's emotional and mental state.

[0601] "Feedback" refers to advice and suggestions for improvement based on analysis results and psychological state.

[0602] This invention is a comprehensive system that analyzes the behavior of factory robots and provides improvements to improve their efficiency based on the analysis. The system combines many components, including smart devices, cloud servers, analysis engines, emotion recognition engines, and generative AI models, to provide comprehensive support for factory robots and their operators.

[0603] Hardware and software used

[0604] Hardware: Smartphones, cloud servers

[0605] Software: Video analysis engine (e.g., OpenCV), emotion recognition engine (e.g., TensorFlow), AWS cloud services, EmotionAPI

[0606] Libraries / Tools: OpenCV, TensorFlow, AWS cloud services, EmotionAPI, generative AI models

[0607] System processing flow

[0608] 1. Video recording and uploading

[0609] The user launches a smartphone app and records the factory robot's operations.

[0610] After recording the video, the smartphone uploads the data to a cloud server, which also records the voices and facial expressions of the workers during the operation.

[0611] 2. Data Analysis

[0612] The cloud server analyzes the received video data and extracts data such as movement trajectory, speed, and angle. A video analysis engine (e.g., OpenCV) is used to evaluate the movement pattern and accuracy of the robot's movement for each frame.

[0613] As a result of the analysis, issues such as "uneven operation times" or "slow operation in certain processes" are quantified.

[0614] 3. Providing remedial measures

[0615] Based on the analysis results, the cloud server generates specific improvement measures to improve operational efficiency, such as providing advice on "reviewing operational paths" or "optimizing specific procedures."

[0616] 4. Emotion Recognition and Feedback

[0617] The cloud server analyzes the uploaded facial and voice data of the user and evaluates their psychological state, such as tension, fatigue, and stress level, using an emotion recognition engine (e.g., EmotionAPI).

[0618] Based on the analysis results, feedback is provided according to the psychological state, such as "adjust your work pace" or "take a break."

[0619] 5. Community and Auto-Responders

[0620] The cloud server provides a community platform where users can share information with other users. It also has an automatic response function to user questions using a generative AI model. For example, if a prompt sentence is entered as "What is the best way to shorten the robot's operating time?", the generative AI model will generate the optimal answer.

[0621] Specific examples

[0622] For example, if a user wants to analyze the packaging operation of a robot, the following procedure is performed.

[0623] Video recording: The user uses a smartphone app to record the robot's packaging operations. The user's voice and facial expressions are also recorded during the recording.

[0624] Data upload: The captured video is immediately uploaded to the cloud server.

[0625] Data analysis: The cloud server analyzes the video and quantifies issues such as "uneven speed of packaging operations" or "slow operations in certain parts."

[0626] Providing improvement measures: Based on the analysis results, improvement measures such as "correcting the movement path to make the speed uniform" and "introducing efficiency techniques" are provided.

[0627] Emotion recognition and feedback: The emotion recognition engine analyzes the user's tension and fatigue while working, and makes suggestions such as "improving the work environment" and "adjusting break times."

[0628] Query response: When a user types a question like, "How can I improve the robot's movement efficiency?", the generative AI model provides advice such as "reviewing the movement path" or "specific training methods."

[0629] Example prompt sentence:

[0630] "How can we make our robot's packaging operations more efficient?"

[0631] This will improve the operational efficiency and accuracy of factory robots, while also realizing a system that takes into account the psychological state of workers.

[0632] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0633] Step 1:

[0634] The user starts the smartphone app and records the factory robot's operations. The smartphone's camera function is used to start video recording, capturing the series of operations of the factory robot. The input is the captured video data, and the output is a video file saved on the smartphone.

[0635] Step 2:

[0636] The device uploads the captured video to the cloud server. When the user taps the upload button, the video file is sent to the cloud server via the Internet. The input is the video file on the smartphone, and the output is the video data stored on the cloud server.

[0637] Step 3:

[0638] The video data received by the server is analyzed using a video analysis engine (e.g., OpenCV). The movement trajectory, speed, angle, etc. of the factory robot are measured and extracted for each video frame. The input is the video data stored on the server, and the output is the analyzed movement data (e.g., numerical values ​​for movement trajectory, speed, angle, etc.).

[0639] Step 4:

[0640] The server identifies and quantifies operational issues based on the analysis results. It then verifies the analyzed data and identifies performance inconsistencies and errors. For example, it quantifies inconsistencies in operation times or delays in operations in specific processes. The input is the analyzed operational data, and the output is quantified data on the issues.

[0641] Step 5:

[0642] The server generates improvement measures for the identified issues. Using the generative AI model, it provides specific advice and training methods to improve operational efficiency. The input is quantified issue data, and the output is feedback data on the improvement measures.

[0643] Step 6:

[0644] The user records their facial expressions and voices while performing actions and uploads them to a cloud server. The input is the voice and facial expression data recorded by the smart device, and the output is the emotion data stored on the cloud server.

[0645] Step 7:

[0646] The server uses an emotion recognition engine (e.g., EmotionAPI) to analyze the uploaded emotion data. The input is the emotion data stored on the server, and the output is data about the user's psychological state (e.g., tension, fatigue, stress level).

[0647] Step 8:

[0648] The server provides feedback based on the user's psychological state. The user's psychological state is reflected in the analysis results, and specific advice for improving the work environment and mental care is generated. The input is psychological state data and analysis results, and the output is feedback data that takes the psychological state into account.

[0649] Step 9:

[0650] The server uses a generative AI model to automatically respond to user questions. When the user enters a prompt (e.g., "What is the best way to shorten the robot's operating time?"), the generative AI model generates the optimal answer and provides the response. The input is the user's prompt, and the output is the answer generated by the generative AI model.

[0651] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0652] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0653] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0654] [Second embodiment]

[0655] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0656] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0657] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0658] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0659] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0660] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0661] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0662] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0663] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0664] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0665] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0666] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0667] This invention is a system that uses smart devices to record, analyze, and improve golf swings. The system quantifies the user's swing issues through video recording and data analysis on a cloud server, and provides specific improvement measures. It also includes a community platform where users can share information with other users, and an automated inquiry response function using artificial intelligence.

[0668] System Overview

[0669] 1. Record a video of your swing using a smart device

[0670] The user launches the smartphone app and takes a photo of their golf swing.

[0671] It provides a function to upload videos taken by the device to a cloud server.

[0672] 2. Data analysis using a cloud server

[0673] The server analyzes the video data it receives and extracts data such as swing trajectory, head speed, and ball speed.

[0674] The server identifies swing issues based on the analysis results and quantifies those issues.

[0675] 3. Providing remedial measures

[0676] The server provides the user with specific improvement measures based on the quantified issues.

[0677] The proposed improvements are displayed on the app in a way that is easy for users to understand.

[0678] 4. Providing a social community

[0679] The server provides a membership platform where users can share information and opinions.

[0680] 5. Automatic response to inquiries

[0681] The server uses the generative AI model to automatically generate answers to user inquiries.

[0682] To resolve any doubts, users can enter their questions within the app and receive a response.

[0683] Program processing

[0684] The program of this system performs the following processing.

[0685] 1. Video recording and uploading

[0686] The user launches the app and shoots a video of their swing. By tapping the recording button, a video of the golf swing is recorded.

[0687] Once the recording is complete, the device prompts the user to review the video and displays an "Upload" button, which the user taps to send the data to the cloud server.

[0688] 2. Data Analysis

[0689] The server analyzes the received video data and tracks the trajectory of the golf club for each frame.

[0690] The server extracts analytical data such as head speed, ball speed, and swing trajectory and stores this data in the user's profile.

[0691] 3. Quantifying issues and providing solutions

[0692] Based on the analysis results, the server quantifies the swing issues and identifies specific problems (e.g., slow head speed, improper swing trajectory).

[0693] The server provides users with specific advice and training methods to solve these problems.

[0694] 4. Use of social communities

[0695] Users have access to a membership platform where they can interact with other users and post and comment on specific issues or topics.

[0696] The device updates the community activity feed in real time, allowing users to see new posts and comments immediately.

[0697] 5. Automatic response to inquiries

[0698] The user enters a question in the app and taps the send button. For example, a question could be, "What should I do if my club head speed isn't increasing?"

[0699] The server uses a generative AI model to generate optimal answers to questions and responds to users in real time. The answers are specific and practical, and include methods that users can try immediately.

[0700] Specific examples

[0701] As a concrete example, consider the following scenario.

[0702] 1. Recording and analyzing swing videos

[0703] Users can record videos of their swings in their own backyard using a smartphone app and upload them to a cloud server. The videos clearly capture the full swing.

[0704] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph.

[0705] 2. Quantifying issues and providing solutions

[0706] Based on the analysis results, the server identifies the problem as an outside-in swing trajectory, and also points out that the head speed is slower than the average professional golfer.

[0707] The server will provide specific suggestions for improvement, such as, "To practice your inside-out swing, try the following drill."

[0708] 3. Sharing information with the community

[0709] A user accesses a community platform and posts a message such as, "I'm looking for advice on drills to improve my swing trajectory."

[0710] Other users respond by saying, "I had the same problem and this drill worked for me."

[0711] 4. Automatic response to inquiries

[0712] A user asks within the app, "What training methods can I use to increase my club head speed?"

[0713] The server uses the generative AI model to generate answers such as, "Specific training methods for increasing club head speed include training using weights and specific swing drills," and immediately provides them to the user.

[0714] In this way, the present invention provides a comprehensive system for golfers to efficiently improve their own swing and share information and questions with other users.

[0715] The processing flow will be explained below.

[0716] Step 1:

[0717] The user launches the smartphone app and taps the "swing video recording" button.

[0718] Step 2:

[0719] The device activates the smartphone camera and captures a picture of the user performing a golf swing.

[0720] Step 3:

[0721] After the device has finished recording, the user is prompted to check the video and, if there are no problems, tap the "Upload" button.

[0722] Step 4:

[0723] In order for the terminal to upload the captured video file to the cloud server, the terminal transmits the video data using the communication module.

[0724] Step 5:

[0725] The video data received by the server is temporarily stored.

[0726] Step 6:

[0727] The server invokes a video analysis algorithm and begins the analysis process to extract data such as swing trajectory, head speed, and ball speed.

[0728] Example: A server performs image analysis on each video frame to track the movement of a golf club.

[0729] Example: The server performs trajectory analysis to analyze the ball's flight pattern.

[0730] Step 7:

[0731] The server stores the analysis results as numerical data and associates them with the user's profile.

[0732] Step 8:

[0733] The server uses the quantified analysis results to identify issues with the user's swing and trajectory.

[0734] Example: If your clubhead speed is 90 mph, compare it to the pro average.

[0735] Example: If your swing path is "outside-in," it will generate feedback for general path corrections.

[0736] Step 9:

[0737] The server will provide the following information to the user along with the analysis results:

[0738] Numerical data on the problem (e.g., head speed, swing trajectory)

[0739] Advice for improvement (e.g., how to practice an inside-out swing)

[0740] Video links and text guides for recommended drills and practice techniques

[0741] Step 10:

[0742] The user taps the Community tab to access the membership platform.

[0743] Step 11:

[0744] The device displays the user's activity feed, updating the list of posts and comments in real time.

[0745] Step 12:

[0746] A user taps the "Create a new post" button and enters a question or comment.

[0747] Step 13:

[0748] The terminal transmits the input post content to the server.

[0749] Step 14:

[0750] The server saves the new post to a database and displays it to other users.

[0751] Step 15:

[0752] The user enters the question they want to ask the generated AI model and taps the "Send inquiry" button.

[0753] Step 16:

[0754] The terminal sends the entered question to the server.

[0755] Step 17:

[0756] The server calls the generative AI model and generates the best answer to the input question.

[0757] Example: The server responds to the question "What causes your swing to curve to the right?" with the answer "Your right hand may be too strong, so try to relax it."

[0758] Step 18:

[0759] The server sends the generated answer to the terminal for display to the user.

[0760] Example 1

[0761] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0762] Conventional golf swing analysis systems require expensive equipment and specialized knowledge, making them difficult for many amateur golfers to use. Furthermore, the time and cost required for individual lessons and coaching advice makes self-improvement of a golf swing extremely difficult. Furthermore, there is a lack of systems that provide specific improvement measures based on the results of swing analysis, leaving a need for a means for users to efficiently improve their skills.

[0763] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0764] In this invention, the server includes a means for recording a user's actions using an application launched by the user, a means for uploading the recorded action data to the server, a means for analyzing the action data on the server and generating analysis results, a means for identifying action issues based on the analysis results and quantifying the issues, and a means for providing solutions to the issues. This allows users to easily record their own actions and obtain detailed analysis and specific solutions for improvement via the server without specialized knowledge. Furthermore, by sharing information with other users, users can receive feedback and support within the community, thereby enabling efficient and continuous skill improvement.

[0765] A "user" is an entity that uses the system to record its own actions, obtain analysis results, and receive improvement measures.

[0766] An "application" is software that runs on a smart device and allows users to record actions, upload data, and display analysis results.

[0767] "Movement" is a physical movement, such as a golf swing, made by a user that is recorded by the system.

[0768] A "server" is a computer system that receives and analyzes action data uploaded by users.

[0769] "Data" refers to the video of the user's actions recorded and the quantified information resulting from that analysis.

[0770] "Analysis" refers to the process of tracking the characteristics of movements based on recorded movement data and deriving quantified results.

[0771] "Issues" refer to problems or areas for improvement in user behavior that have become apparent as a result of the analysis.

[0772] "Quantifying" means expressing the characteristics and challenges of an action as quantitative data.

[0773] "Improvement measures" refer to specific methods and advice on how the user should improve the issues identified in the analysis results.

[0774] A "Platform" is an online venue where users can share information and communicate with other users.

[0775] "Artificial intelligence" is a program that automatically generates responses to user inquiries.

[0776] This invention is a system that uses a smart device to record a user's golf swing and analyzes the data on a cloud server to quantify the user's swing issues and provide specific improvement measures. This system is implemented using a smart device application, a cloud server, analysis software, and an artificial intelligence model.

[0777] Hardware used

[0778] Smart devices: Smartphones and tablets are used. These devices must have a camera and internet connection.

[0779] Cloud server: A computer system where video data is uploaded and where analysis processing is performed.

[0780] Software used

[0781] Application: Software for smart devices that allows users to record their golf swings and upload the data to a server. Specifically, it is implemented as an application for iOS or Android.

[0782] Analysis software: To analyze golf swing videos, we use libraries such as OpenCV and TensorFlow, which allow us to perform frame analysis and feature extraction of movements.

[0783] Generative AI model: An artificial intelligence model that generates automated responses to user inquiries. This uses natural language processing (NLP) techniques such as GPT-3 and BERT.

[0784] Processing flow

[0785] 1. Video recording and uploading

[0786] The user starts the smartphone app and shoots a video of their swing. An easy-to-use interface is provided for video recording.

[0787] After shooting is complete, the device displays a video confirmation screen to the user, allowing them to edit the video or cut out unnecessary parts.

[0788] When the user checks the video and taps the upload button, the device sends the video data to the cloud server.

[0789] 2. Data analysis and result generation

[0790] The server temporarily stores the received video data in storage, then begins motion analysis for each video frame, tracking the movement of the club and the body.

[0791] The server quantifies and extracts motion data such as head speed, ball speed, and swing trajectory, and stores it in the user's profile.

[0792] 3. Quantifying issues and providing solutions

[0793] The server identifies the user's swing issues based on the analysis results and quantifies the specific problems.

[0794] Using a generative AI model, specific improvement measures for quantified issues are automatically generated, and these improvement measures are displayed in the app in a format that is easy for users to understand.

[0795] 4. Use of social communities

[0796] Users can use the in-app community feature to share information and opinions with other users, and can post and comment on specific topics.

[0797] The device updates the community feed in real time, allowing you to see new posts and comments immediately.

[0798] 5. Automatic response to inquiries

[0799] When a user enters a question in the app and taps the submit button, the server uses a generative AI model to generate the best answer.

[0800] Answers are provided to users in real time and include specific, practical content. For example, in response to a question like, "How can I train to increase my clubhead speed?", the answer might be, "There are weight training exercises and specific swing drills."

[0801] Specific examples

[0802] Examples of swing video recording and analysis

[0803] Users can record videos of their swings in their own backyard using a smart device, and then upload the videos to a cloud server.

[0804] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph.

[0805] Based on the analysis results, the server identifies the problem of an outside-in swing trajectory and identifies the head speed as slower than the average for professionals. The server then provides specific improvements to address these issues.

[0806] Examples from the community

[0807] When a user posts something like "I'm looking for advice on drills to improve my swing path," other users will respond with things like "I was having the same problem, and this drill worked for me."

[0808] Specific examples of automated response to inquiries

[0809] When a user asks, "What training methods can I use to increase my clubhead speed?", the server uses a generative AI model to respond, "Specific training methods for increasing clubhead speed include weight training and specific swing drills."

[0810] In this way, the present invention provides a comprehensive system for golfers to efficiently improve their swing and share information and questions with other users.

[0811] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0812] Step 1:

[0813] The user launches the smart device application and selects the "Swing Photo" option from the home screen.

[0814] Input: Launching applications, displaying the home screen

[0815] Output: Show "Swing Shooting" option

[0816] What it does: The app invokes the camera function and displays a message encouraging the user to prepare for their swing.

[0817] Step 2:

[0818] The user taps the recording button to record a video of their golf swing.

[0819] Input: Tap the recording button, video data of the swing

[0820] Output: Recorded swing video data

[0821] Specific operation: The device camera will start up and record the user's swing. When the recording is finished, a save button will be displayed.

[0822] Step 3:

[0823] Users can check the video on the video confirmation screen and edit or cut out unnecessary parts.

[0824] Input: Recorded swing video, user editing actions

[0825] Output: Edited swing video data

[0826] What it does: The app provides video playback and editing functionality and displays an interface for the user to finalize the footage.

[0827] Step 4:

[0828] The user taps the upload button to send the edited video data to the cloud server.

[0829] Input: Edited swing video data, tap the upload button

[0830] Output: Video data stored on a cloud server

[0831] Specific operation: The device sends data to the cloud server via the Internet. Once the upload is complete, a confirmation message will be displayed.

[0832] Step 5:

[0833] The video data received by the server is temporarily stored in storage.

[0834] Input: Uploaded video data

[0835] Output: Saved video data

[0836] Specific operation: The server receives the video data and stores it in the file system. This storage is temporary storage for fast access.

[0837] Step 6:

[0838] The server starts motion analysis for each video frame, tracking and analyzing the movement of the golf club and the body.

[0839] Input: Saved video data

[0840] Output: Analyzed motion data (head speed, ball speed, swing trajectory, etc.)

[0841] How it works: The server uses libraries such as OpenCV and TensorFlow to analyze the images of each frame, and then uses a motion detection algorithm to extract club and body movement data.

[0842] Step 7:

[0843] The server quantifies the extracted behavioral data and stores it in the user's profile.

[0844] Input: Parsed motion data

[0845] Output: Stored quantified operating data

[0846] Specific operation: The analysis results are processed as numerical data and added to the user's profile database.

[0847] Step 8:

[0848] The server identifies the user's swing issues based on the analysis results and quantifies the specific problems.

[0849] Input: digitized motion data

[0850] Output: Quantified issues

[0851] Specific operation: The server uses the generative AI model to extract issues using specific algorithms and quantitatively evaluate them.

[0852] Step 9:

[0853] The server uses the generated AI model to automatically generate specific improvement measures for the quantified issues.

[0854] Input: Quantified task

[0855] Output: Generated remediation measures

[0856] How it works: The generative AI model references past data and training methods to suggest improvements based on the user's challenges, which may include text and links to images or videos.

[0857] Step 10:

[0858] The improvement measures generated by the server are sent to the user's smart device and displayed on the app.

[0859] Input: Generated remediation measures

[0860] Output: Improvement measures displayed on the app

[0861] Specific behavior: The app displays the improvement measures obtained from the server in the interface so that the user can take action immediately.

[0862] Step 11:

[0863] Users access the Community tab and share information with other users.

[0864] Enter: Access to the Community tab

[0865] Output: Shared information and comments

[0866] What it does: The app provides a post creation feature that allows users to share their challenges and improvements with the community, making it easy to receive feedback and advice.

[0867] Step 12:

[0868] The user enters a question in the in-app inquiry tab and submits it.

[0869] Input: User question

[0870] Output: Query data sent to the server

[0871] Specific behavior: Provides an interface for the app to send entered questions to the server.

[0872] Step 13:

[0873] The server uses a generative AI model to generate optimal answers and provides them to the user in real time.

[0874] Input: Question data from the user

[0875] Output: The generated answer

[0876] Specific operation: The server uses the generative AI model to generate an appropriate answer based on the question and sends it in text format to the user's device.

[0877] Step 14:

[0878] The user references the answers provided and takes various actions to improve their swing.

[0879] Input: Generated Answer

[0880] Output: Actions taken

[0881] Specific Actions: Plan practice sessions or attempt specific training drills based on the answers the user generates.

[0882] (Application example 1)

[0883] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0884] Conventional golf swing analysis systems not only provide users with analysis results, but also make it difficult to share information with other users in real time or receive actual support at a golf club. Furthermore, there is a lack of advice on selecting appropriate golf equipment based on swing analysis results, leaving users lacking the information and tools to instantly improve their skills. This often means that users need time to improve their swing, and it is difficult to receive appropriate guidance and support during the process.

[0885] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0886] In this invention, the server includes means for recording a user's swing motion using a smart computing device, means for transmitting the recorded swing motion data to a remote server, means for analyzing the swing motion data in the remote server and generating analysis results, means for identifying problems in the swing motion based on the analysis results and quantifying the problems, means for providing solutions to the problems, means for sharing the analysis results with other users in the store in real time, and means for providing a function for making inquiries to store staff. This allows users to check the swing analysis results in real time in the store, share experiences and information with other users, and immediately receive appropriate advice from store staff.

[0887] A "smart computing device" is an electronic device that is portable by the user and has a high-resolution camera and internet connectivity that is capable of recording and transmitting swing motion.

[0888] "User" refers to a user who seeks to improve their golf swing using the system of the present invention.

[0889] The term "swing motion" refers to a series of physical movements that are made when using a golf club to hit a ball, including the trajectory of the golf club, head speed, input force, and the like.

[0890] "Remote Server" refers to a data center that is a data analysis and storage system accessible via the Internet and that analyzes the swing video sent by the User and provides the results.

[0891] The "analysis results" are technical evaluations and quantified data regarding the user's swing, generated by the server based on swing motion data.

[0892] "Quantification" refers to expressing various parameters of the analysis results as quantitative values, allowing users to concretely recognize their own performance.

[0893] "Improvement measures" refer to specific training methods and technical advice provided to address swing movement issues identified based on the analysis results.

[0894] "Real-time" refers to the near-instantaneous transmission of data, analysis, provision of results, and sharing of information.

[0895] "Information sharing" refers to users exchanging data such as analysis results, experiences, and opinions with other users and store staff.

[0896] The "inquiry function" refers to a system that allows users to input questions about analysis results and improvement measures and receive an automatic response or a reply from store staff on the spot.

[0897] This invention is a system for use in golf equipment stores that allows customers to receive real-time analysis and feedback on their swings. The system includes a smart computing device, a remote server, and a social community platform for sharing analysis results. It also provides query capabilities using generative AI models.

[0898] System Overview

[0899] 1. Video recording and uploading

[0900] Users use a smart computing device (such as a smartphone) to film their swing, and once filming is complete, the video data is automatically transmitted to a remote server.

[0901] 2. Data Analysis

[0902] The server analyzes the received swing video and extracts data such as the golf club trajectory, head speed, ball speed, etc. These analysis results are quantified and recorded in the user profile.

[0903] 3. Providing Feedback

[0904] The server quantifies the identified issues based on the analysis results and indicates specific areas for improvement to the user. The improvement measures are sent in real time to a smart computing device and displayed to the user.

[0905] 4. Social Community

[0906] Users can share analysis results with other customers in the store in real time, exchange information, and hold discussions and provide feedback on specific topics.

[0907] 5. Inquiry function

[0908] When a user inputs a question about improvement measures or analysis results into the server, the optimal answer is automatically generated using a generative AI model, and this answer is immediately provided to the user.

[0909] Hardware and Software Configuration

[0910] Smart computing devices: Smartphones and tablets equipped with high-resolution cameras and internet connectivity.

[0911] Remote server: Use cloud services such as Amazon Web Services (AWS) and Google Cloud Platform (GCP).

[0912] Data analysis software: Video data was analyzed using Python and OpenCV.

[0913] Generative AI model: Generates automatic responses to user inquiries using natural language processing.

[0914] Examples of concrete examples and prompts

[0915] Examples:

[0916] 1. The user takes a photo of their golf swing with their smartphone in the practice area inside the store.

[0917] 2. The captured video is automatically uploaded to a cloud server, where analysis begins.

[0918] 3. The server quantifies the analysis results (e.g., head speed, swing trajectory, etc.) and immediately sends them to your smartphone.

[0919] 4. The user checks the analysis results on the spot and continues practicing according to the provided improvement measures.

[0920] 5. When users ask questions about problems with their swing, the generative AI model provides specific answers. For example, in response to a question like, "What can I do if my club head speed isn't improving?", the model might respond with, "Training with weights and specific swing drills are effective."

[0921] Example prompt sentence:

[0922] "A user has uploaded a video of their swing taken in a golf shop. Please analyze this video and provide data on the following: 1. Head speed 2. Swing path 3. Ball speed. Also, based on this data, please provide specific advice on how to improve their swing."

[0923] In this way, the present invention provides a comprehensive support system for users to efficiently analyze and improve their swings and share information within a golf equipment store.

[0924] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0925] Step 1:

[0926] A user uses a smart computing device (such as a smartphone) to film their swing motion.

[0927] Input: A smartphone for users to record swing videos.

[0928] Data processing / calculation: Video is taken using a high-resolution camera.

[0929] Output: Recorded swing video file.

[0930] Step 2:

[0931] The device uploads the captured swing video to a cloud server.

[0932] Input: Recorded swing video file.

[0933] Data processing / calculation: Send the video file to the cloud server using an HTTP request.

[0934] Output: Video files stored on cloud server.

[0935] Step 3:

[0936] The server analyzes the received swing video and extracts swing motion data.

[0937] Input: Video files stored on a cloud server.

[0938] Data processing / calculation: Using Python and OpenCV, information such as golf club trajectory, head speed, and ball speed is extracted from video data.

[0939] Output: Swing motion analysis results (e.g. head speed, swing path, ball speed).

[0940] Step 4:

[0941] The server identifies issues in the swing motion based on the analysis results and quantifies those issues.

[0942] Input: Analysis results of swing motion.

[0943] Data processing / calculation: The analysis results are applied to an evaluation algorithm to quantitatively express the identified issues.

[0944] Output: Quantified swing motion issues (e.g., slow clubhead speed, inaccurate swing path).

[0945] Step 5:

[0946] The server generates solutions to the problems and provides them to the user.

[0947] Input: Quantified swing movement tasks.

[0948] Data processing / computation: Running algorithms to generate specific training or technical advice to address the problem.

[0949] Output: Specific improvement measures (e.g., training methods to increase clubhead speed, drills to improve swing path).

[0950] Step 6:

[0951] The device displays analysis results and improvement measures to the user in real time.

[0952] Input: Specific improvement measures.

[0953] Data processing / calculation: Rendering is performed to display the results and improvement measures on the smartphone UI.

[0954] Output: Analysis results and remediation measures displayed on the user's screen.

[0955] Step 7:

[0956] Users can share analysis results with other customers in the store and exchange information.

[0957] Input: Analysis results and remediation measures.

[0958] Data processing / computation: Use social community platforms to share data and post comments and feedback.

[0959] Output: Information exchanged with other users.

[0960] Step 8:

[0961] Users ask questions about analysis results and improvement measures, and the server automatically responds using the generated AI model.

[0962] Input: A question from the user (e.g., "What training methods can I use to increase my club head speed?").

[0963] Data processing / calculation: Input a prompt sentence into the generative AI model to generate the optimal answer.

[0964] Output: The automated response that is provided to the user.

[0965] Specific prompt examples:

[0966] "A user has uploaded a video of their swing taken in a golf shop. Please analyze this video and provide data on the following: 1. Head speed 2. Swing path 3. Ball speed. Also, based on this data, please provide specific advice on how to improve their swing."

[0967] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0968] This invention is a system that uses a smart device to record, analyze, and improve a golf swing, and also combines it with user emotion recognition. This system quantifies the user's swing issues through swing video recording and data analysis on a cloud server, and provides specific improvement measures. It also uses an emotion engine to grasp the user's psychological state and provides feedback based on that to support more effective swing improvement.

[0969] System Overview

[0970] 1. Record a video of your swing using a smart device

[0971] The user launches the smartphone app and takes a photo of their golf swing.

[0972] It provides a function to upload videos taken by the device to a cloud server.

[0973] 2. Data analysis using a cloud server

[0974] The server analyzes the video data it receives and extracts data such as swing trajectory, head speed, and ball speed.

[0975] The server identifies swing issues based on the analysis results and quantifies those issues.

[0976] 3. Providing remedial measures

[0977] The server provides the user with specific improvement measures based on the quantified issues.

[0978] The proposed improvements are displayed on the app in a way that is easy for users to understand.

[0979] 4. Emotion recognition

[0980] The device records the user's facial expressions and voice during the swing and uploads them to a cloud server.

[0981] The server uses an emotion engine to analyze emotions from the user's facial expressions and voice.

[0982] 5. Providing Feedback

[0983] Based on the emotion recognition results, the server generates improvement measures and advice that take into account the user's psychological state.

[0984] The server also incorporates the user's emotional data into the analysis results to provide more accurate feedback.

[0985] 6. Providing social communities

[0986] The server provides a membership platform where users can share information and opinions.

[0987] 7. Auto-response to inquiries

[0988] The server uses the generative AI model to automatically generate answers to user inquiries.

[0989] To resolve any doubts, users can enter their questions within the app and receive a response.

[0990] Program processing

[0991] The program of this system performs the following processing.

[0992] 1. Video recording and uploading

[0993] The user launches the app and shoots a video of their swing. By tapping the recording button, a video of the golf swing is recorded.

[0994] Once the recording is complete, the device prompts the user to review the video and displays an "Upload" button, which the user taps to send the data to the cloud server.

[0995] 2. Data Analysis

[0996] The server analyzes the received video data and tracks the trajectory of the golf club for each frame.

[0997] The server extracts analytical data such as head speed, ball speed, and swing trajectory and stores this data in the user's profile.

[0998] 3. Quantifying issues and providing solutions

[0999] Based on the analysis results, the server quantifies the swing issues and identifies specific problems (e.g., slow head speed, improper swing trajectory).

[1000] The server provides users with specific advice and training methods to solve these problems.

[1001] 4. Emotion Recognition and Feedback

[1002] When a user shoots a video of their swing, their facial expressions and voice are also recorded at the same time.

[1003] The device sends this facial expression and voice data to a cloud server.

[1004] The server uses an emotion engine to analyze the user's facial expressions and voice to determine their emotions. For example, if the user is nervous while taking a photo, the server can detect that emotion.

[1005] Based on the emotion recognition results, the server generates improvement measures and advice that take into account the user's psychological state. For example, it could provide advice such as "Try breathing techniques to help you swing more relaxed."

[1006] The server also incorporates the user's emotional data into the analysis results to provide more accurate feedback.

[1007] 5. Use of social communities

[1008] The user taps the Community tab to access the membership platform.

[1009] The device displays the user's activity feed, updating the list of posts and comments in real time.

[1010] A user taps the "Create a new post" button and enters a question or comment.

[1011] The terminal transmits the input post content to the server.

[1012] The server saves the new post to a database and displays it to other users.

[1013] 6. Automatic response to inquiries

[1014] The user enters the question they want to ask the generated AI model and taps the "Send inquiry" button.

[1015] The terminal sends the entered question to the server.

[1016] The server calls up the generative AI model and generates the optimal answer to the input question. For example, in response to the question, "What kind of training is effective for increasing club head speed?", it provides a specific training method.

[1017] The server sends the generated answer to the terminal for display to the user.

[1018] Specific examples

[1019] As a concrete example, consider the following scenario.

[1020] 1. Recording and analyzing swing videos

[1021] Users can record videos of their swings in their own backyard using a smartphone app and upload them to a cloud server. The videos include the full swing, as well as the user's facial expressions and voice.

[1022] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph. The emotion engine also recognizes that the user is tense during the swing.

[1023] 2. Quantifying issues and providing solutions

[1024] Based on the analysis results, the server identifies that the swing trajectory is outside-in as the problem, and points out that the head speed is slower than the average for professionals as an issue.

[1025] The server will provide specific improvement measures, such as "Try the following drill to practice your inside-out swing," and will also provide advice including "breathing techniques and mental training to relieve tension."

[1026] 3. Sharing information with the community

[1027] A user accesses a community platform and posts a message such as, "I'm looking for advice on drills to improve my swing trajectory."

[1028] Other users respond by saying, "I had the same problem and this drill worked for me."

[1029] 4. Automatic response to inquiries

[1030] A user asks within the app, "What kind of training is effective for increasing club head speed?"

[1031] The server uses the generative AI model to generate specific answers such as "Training with weights and specific swing drills are effective in increasing club head speed," and immediately provides them to the user.

[1032] In this way, the present invention provides a comprehensive system that allows golfers to efficiently improve their own swing, while also taking care of their psychological aspects, and allows them to share information and solve questions with other users.

[1033] The processing flow will be explained below.

[1034] Step 1:

[1035] The user launches the smartphone app and taps the "swing video recording" button.

[1036] Step 2:

[1037] The device activates the smartphone's camera and microphone to capture images and audio of the user performing a golf swing.

[1038] Step 3:

[1039] After the device has finished recording, the user is prompted to check the video and, if there are no problems, tap the "Upload" button.

[1040] Step 4:

[1041] The device transmits the data using a communication module to upload the captured video files and audio data to a cloud server.

[1042] Step 5:

[1043] The server temporarily stores the received video and audio data.

[1044] Step 6:

[1045] The server invokes a video analysis algorithm and begins the analysis process to extract data such as swing trajectory, head speed, and ball speed.

[1046] Example: A server performs image analysis on each video frame to track the movement of a golf club.

[1047] Example: The server performs trajectory analysis to analyze the ball's flight pattern.

[1048] Step 7:

[1049] The server uses an emotion engine to analyze the user's facial expressions in audio data and video to identify emotions.

[1050] Step 8:

[1051] The server stores the analysis results as numerical data and associates them with the user's profile.

[1052] Step 9:

[1053] The server uses the quantified analysis results to identify issues with the user's swing and trajectory.

[1054] Example: If your clubhead speed is 90 mph, compare it to the pro average.

[1055] Example: If your swing path is "outside-in," it will generate feedback for general path corrections.

[1056] Example: Identifying that a user is nervous based on facial and voice analysis.

[1057] Step 10:

[1058] The server will provide the following information to the user along with the analysis results:

[1059] Numerical data on the problem (e.g., head speed, swing trajectory)

[1060] Advice for improvement (e.g., how to practice an inside-out swing)

[1061] Emotion-based advice (e.g., breathing techniques for a relaxed swing)

[1062] Video links and text guides for recommended drills and practice techniques

[1063] Step 11:

[1064] The user taps the Community tab to access the membership platform.

[1065] Step 12:

[1066] The device displays the user's activity feed, updating the list of posts and comments in real time.

[1067] Step 13:

[1068] A user taps the "Create a new post" button and enters a question or comment.

[1069] Step 14:

[1070] The terminal transmits the input post content to the server.

[1071] Step 15:

[1072] The server saves the new post to a database and displays it to other users.

[1073] Step 16:

[1074] The user enters the question they want to ask the generated AI model and taps the "Send inquiry" button.

[1075] Step 17:

[1076] The terminal sends the entered question to the server.

[1077] Step 18:

[1078] The server calls the generative AI model and generates the best answer to the input question.

[1079] Example: The server responds to the question "What causes your swing to curve to the right?" with the answer "Your right hand may be too strong, so try to relax it."

[1080] Step 19:

[1081] The server sends the generated answer to the terminal for display to the user.

[1082] Example 2

[1083] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1084] Conventional golf swing analysis systems focus on analyzing technical issues in a user's swing and providing solutions for improvement, but do not consider psychological aspects. As a result, if a user feels nervous or stressed, this may not lead to actual performance improvement. Furthermore, smoother support is needed for sharing information with other users and resolving questions.

[1085] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1086] In this invention, the server includes a means for recording the user's facial expressions and voice during the swing, a means for uploading the recorded facial and voice data to a cloud server, and a means for analyzing the user's emotions using an emotion recognition engine in the cloud server. This enables feedback that takes the user's psychological state into consideration. The system also includes a means for identifying and quantifying issues in the swing based on the analysis results, and a means for automatically responding to inquiries from the user using a generative AI model, enabling comprehensive support for swing improvement from both technical and psychological aspects.

[1087] A "smart device" refers to an electronic device that can be carried by a user and that can connect to the Internet and run various applications. Examples include smartphones and tablets.

[1088] A "cloud server" refers to a remote server that can store, analyze, and communicate data over the Internet.

[1089] An "analysis algorithm" refers to a mathematically and logically defined method of defining rules and procedures for extracting, classifying, and evaluating useful information from specific data.

[1090] An "emotion recognition engine" refers to technology that analyzes non-verbal data such as a user's facial expressions and voice to identify their emotional state.

[1091] "Analysis results" refers to the specific numerical values ​​and evaluation results obtained after analyzing swing movements, facial expressions, and audio data.

[1092] "Means for quantifying problems" refers to methods and tools for quantitatively expressing problems in a user's swing motion from the analysis results.

[1093] The "means for providing improvement measures" refers to a method or system for proposing specific instruction content or training methods to the user for the identified problems in the swing motion.

[1094] A "generative AI model" refers to a mathematical model that uses artificial intelligence to automatically generate answers and advice to user inquiries.

[1095] A "community platform" refers to an online space where multiple users can share information, exchange opinions, ask questions, and answer questions.

[1096] This invention is a system that uses a smart device to record, analyze, and improve golf swings, and also combines it with user emotion recognition. This system starts when a user uses a smart device such as a smartphone to record a video of their golf swing and uploads the video data to a cloud server.

[1097] Hardware and software used:

[1098] Smart Device: An electronic device that is portable and has internet connectivity, such as a smartphone or tablet.

[1099] Cloud server: A remote server that stores, analyzes, and communicates data.

[1100] Analysis algorithm: Using image analysis libraries such as OpenCV, the swing trajectory, head speed, ball speed, etc. are analyzed for each frame.

[1101] Emotion recognition engine: Technology that analyzes emotions from a user's facial expressions and voice, such as Microsoft Azure's Emotion API.

[1102] Generative AI models: Use artificial intelligence models such as GPT-4 to automatically generate answers to user inquiries.

[1103] The specific operation steps of the system:

[1104] 1. Video recording and uploading

[1105] The user starts the smartphone app and records a swing video. They tap the record button to start recording and the end button to stop recording.

[1106] If the user checks the video and there are no problems, they tap the upload button to send the video to the cloud server.

[1107] 2. Data Analysis

[1108] The server receives the uploaded video data and uses an analysis algorithm to extract data such as swing trajectory, head speed, and ball speed for each frame.

[1109] The extracted data is stored in a profile for each user.

[1110] 3. Quantifying issues and providing solutions

[1111] The server then quantifies swing issues based on the analysis results. For example, if the head speed is less than 85 mph, it will be listed as a "low head speed" issue.

[1112] The server generates specific improvement measures for each quantified issue and displays them in the user's app, such as "training methods to improve club head speed."

[1113] 4. Emotion Recognition and Feedback

[1114] When a user shoots a video of their swing, their facial expressions and voice are also recorded at the same time.

[1115] The device transmits facial expression and voice data to a cloud server.

[1116] The server uses an emotion recognition engine to analyze the user's emotions, for example, detecting that the user is nervous during the shoot.

[1117] Based on the analysis results and emotional data, the server generates comments and improvement measures that take into account the user's psychological state and provides them to the user's smart device, such as "breathing techniques for relaxation."

[1118] 5. Use of social communities

[1119] Users access the platform by tapping the community tab within the smartphone app.

[1120] The device displays posts and comments in real time and provides an interface for users to post questions and opinions.

[1121] The server stores new posts and comments in a database, making them available for all users to see.

[1122] 6. Automatic response to inquiries

[1123] The user enters a question in the app and taps the "Send Inquiry" button.

[1124] The device sends the question to the server, which then uses a generative AI model (e.g., GPT-4) to generate the optimal answer.

[1125] The server displays the generated answer on the user's smart device.

[1126] Specific operation example

[1127] 1. Recording and analyzing swing videos

[1128] Users can record videos of their swings in their own backyard using their smartphones and upload them to a cloud server. The videos include the swing, the user's facial expressions, and their voice.

[1129] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph.

[1130] In addition, the emotion recognition engine detects whether the user is nervous during the swing.

[1131] 2. Quantifying issues and providing solutions

[1132] The server identifies the problem as slow head speed and swing trajectory, and then provides the user with specific improvement measures, such as drills to practice an inside-out swing and breathing techniques to relieve tension.

[1133] 3. Sharing information with the community

[1134] A user accesses a community platform and posts something like, "I'm looking for advice on improving my swing."

[1135] Other users will reply saying, "This drill was effective."

[1136] 4. Automatic response to inquiries

[1137] A user asks, "How can I train to increase my club head speed?"

[1138] The server uses the generative AI model to generate answers such as "Training with weights and specific swing drills are effective" and provides them to the user.

[1139] In this way, the present invention can comprehensively support the user's swing improvement and provide feedback that also includes psychological aspects.In addition, the information sharing function with other users and the automatic response function to inquiries create an efficient and effective training environment.

[1140] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1141] Step 1:

[1142] The user launches the smartphone app and records a video of their swing.

[1143] Input: Start of shooting by user operation

[1144] Data processing: Record a swing video using a smartphone camera

[1145] Output: Recorded swing video file

[1146] Specific behavior:

[1147] The user launches the app and switches to shooting mode.

[1148] The user taps the record button to begin recording the golf swing.

[1149] The user taps the end button to end the recording.

[1150] Step 2:

[1151] The device uploads the captured video to a cloud server.

[1152] Input: Recorded swing video file

[1153] Data processing: Transfer video data to the cloud server

[1154] Output: Video data on a cloud server

[1155] Specific behavior:

[1156] The device will prompt the user to review the video and display an "Upload" button.

[1157] When the user taps the upload button, the device uses its internet connection to send the video file to a cloud server.

[1158] Step 3:

[1159] The server analyzes the video data it receives and extracts data such as swing trajectory, head speed, and ball speed.

[1160] Input: Video data on a cloud server

[1161] Data processing: Perform video analysis using analysis algorithms such as OpenCV

[1162] Output: Analysis data such as swing trajectory, head speed, ball speed, etc.

[1163] Specific behavior:

[1164] The server analyzes the video data frame by frame.

[1165] The server extracts data such as swing trajectory, head speed, and ball speed and stores it in the user's profile.

[1166] Step 4:

[1167] The server quantifies swing issues based on the analysis results and generates specific improvement measures.

[1168] Input: Swing trajectory, head speed, ball speed data

[1169] Data processing: Applying algorithms to quantify swing problems using analytical data

[1170] Output: Quantified issues and concrete improvement measures

[1171] Specific behavior:

[1172] The server identifies swing issues based on the analysis results.

[1173] The server generates specific improvement measures and training methods based on the issues.

[1174] The server sends the generated advice to the user's app.

[1175] Step 5:

[1176] The device records the user's facial expressions and voice and uploads them to a cloud server.

[1177] Input: Photographed facial expressions and voice data

[1178] Data processing: Send facial expression and voice data to a cloud server

[1179] Output: Facial expression and voice data on a cloud server

[1180] Specific behavior:

[1181] When the device shoots a swing video, it also records the user's facial expressions and voice at the same time.

[1182] The device uploads facial expression and voice data to a cloud server.

[1183] Step 6:

[1184] The server uses an emotion recognition engine to analyze emotions from the user's facial expressions and voice.

[1185] Input: Facial expression and voice data on a cloud server

[1186] Data processing: Emotion analysis using an emotion recognition engine (e.g., Emotion API)

[1187] Output: User emotion data

[1188] Specific behavior:

[1189] The server inputs facial and voice data into an emotion recognition engine.

[1190] The server identifies the user's emotions (e.g., tension, relaxation, etc.).

[1191] Step 7:

[1192] The server reflects the emotional data in the analysis results and generates appropriate feedback that takes the user's psychological state into account.

[1193] Input: Analysis data and emotion data

[1194] Data processing: Adding emotional data to the analysis results and adjusting improvement measures and advice

[1195] Output: Feedback that takes into account the user's psychological state

[1196] Specific behavior:

[1197] If the server detects tension, it generates feedback that includes "breathing techniques to relax."

[1198] The server sends the generated feedback to the user's app.

[1199] Step 8:

[1200] Users access the community platform to share information and post questions.

[1201] Input: User-submitted content

[1202] Data processing: Save the posted content in a database and share it with other users

[1203] Output: Comments and replies by other users

[1204] Specific behavior:

[1205] A user taps the Community tab to access the platform.

[1206] The device displays the latest posts and comments and sends the user's new posts to the cloud server.

[1207] The server stores the posts in a database and makes them available for all users to view.

[1208] Step 9:

[1209] The user asks a question using the generative AI model, and the server generates an automated response.

[1210] Input: Question asked by the user (prompt text)

[1211] Data processing: Using a generative AI model (e.g., GPT-4) to generate optimal answers

[1212] Output: The generated answer

[1213] Specific behavior:

[1214] The user enters a question in the app and taps the "Send Inquiry" button.

[1215] The device sends the question to the cloud server.

[1216] The server uses a generative AI model to generate the best answer to the question.

[1217] The server generates the answer and sends it to the user, who displays it in the app.

[1218] In this way, a system is constructed that comprehensively supports the user in improving their swing by performing data processing and calculations based on the input data at each step and then using the resulting output to proceed to the next processing step.

[1219] (Application example 2)

[1220] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1221] Existing factory robot motion analysis systems lack the functionality to quantify motion efficiency and accuracy and provide specific improvement measures. Furthermore, they do not provide feedback that takes into account the psychological state of the worker or suggest improvements to the work environment, resulting in insufficient improvements to the performance of both the robot and the worker. This makes it difficult to create an efficient and stress-free work environment.

[1222] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recording the user's robot operations using a smart device, means for uploading the recorded operation data to a cloud server, means for analyzing the operation data and generating analysis results in the cloud server, means for identifying operation issues based on the analysis results and quantifying the issues, means for providing improvements to the issues, means for uploading emotional data recorded during the operations to the cloud server, means for analyzing the emotional data in the cloud server and identifying the user's psychological state, and means for providing feedback based on the psychological state. This makes it possible to scientifically improve the operational efficiency and accuracy of factory robots and provide feedback and improve the work environment that takes into account the psychological state of workers.

[1223] A "smart device" is an electronic device that can be carried and operated by a user, such as a smartphone or tablet.

[1224] A "cloud server" is a remote server for storing, managing, and analyzing data over a network.

[1225] "Analysis" is the process of evaluating recorded data using scientific methods and extracting the necessary information.

[1226] An "issue" refers to a problem or defect that needs to be resolved in a system or process.

[1227] "Quantification" refers to expressing the analysis results in concrete numerical values.

[1228] "Improvement measures" refer to specific policies and methods for solving identified issues.

[1229] "Emotion data" is voice and facial expression information recorded to assess the user's psychological state.

[1230] "Mental state" refers to the user's emotional and mental state.

[1231] "Feedback" refers to advice and suggestions for improvement based on analysis results and psychological state.

[1232] This invention is a comprehensive system that analyzes the behavior of factory robots and provides improvements to improve their efficiency based on the analysis. The system combines many components, including smart devices, cloud servers, analysis engines, emotion recognition engines, and generative AI models, to provide comprehensive support for factory robots and their operators.

[1233] Hardware and software used

[1234] Hardware: Smartphones, cloud servers

[1235] Software: Video analysis engine (e.g., OpenCV), emotion recognition engine (e.g., TensorFlow), AWS cloud services, EmotionAPI

[1236] Libraries / Tools: OpenCV, TensorFlow, AWS cloud services, EmotionAPI, generative AI models

[1237] System processing flow

[1238] 1. Video recording and uploading

[1239] The user launches a smartphone app and records the factory robot's operations.

[1240] After recording the video, the smartphone uploads the data to a cloud server, which also records the voices and facial expressions of the workers during the operation.

[1241] 2. Data Analysis

[1242] The cloud server analyzes the received video data and extracts data such as movement trajectory, speed, and angle. A video analysis engine (e.g., OpenCV) is used to evaluate the movement pattern and accuracy of the robot's movement for each frame.

[1243] As a result of the analysis, issues such as "uneven operation times" or "slow operation in certain processes" are quantified.

[1244] 3. Providing remedial measures

[1245] Based on the analysis results, the cloud server generates specific improvement measures to improve operational efficiency, such as providing advice on "reviewing operational paths" or "optimizing specific procedures."

[1246] 4. Emotion Recognition and Feedback

[1247] The cloud server analyzes the uploaded facial and voice data of the user and evaluates their psychological state, such as tension, fatigue, and stress level, using an emotion recognition engine (e.g., EmotionAPI).

[1248] Based on the analysis results, feedback is provided according to the psychological state, such as "adjust your work pace" or "take a break."

[1249] 5. Community and Auto-Responders

[1250] The cloud server provides a community platform where users can share information with other users. It also has an automatic response function to user questions using a generative AI model. For example, if a prompt sentence is entered as "What is the best way to shorten the robot's operating time?", the generative AI model will generate the optimal answer.

[1251] Specific examples

[1252] For example, if a user wants to analyze the packaging operation of a robot, the following procedure is performed.

[1253] Video recording: The user uses a smartphone app to record the robot's packaging operations. The user's voice and facial expressions are also recorded during the recording.

[1254] Data upload: The captured video is immediately uploaded to the cloud server.

[1255] Data analysis: The cloud server analyzes the video and quantifies issues such as "uneven speed of packaging operations" or "slow operations in certain parts."

[1256] Providing improvement measures: Based on the analysis results, improvement measures such as "correcting the movement path to make the speed uniform" and "introducing efficiency techniques" are provided.

[1257] Emotion recognition and feedback: The emotion recognition engine analyzes the user's tension and fatigue while working, and makes suggestions such as "improving the work environment" and "adjusting break times."

[1258] Query response: When a user types a question like, "How can I improve the robot's movement efficiency?", the generative AI model provides advice such as "reviewing the movement path" or "specific training methods."

[1259] Example prompt sentence:

[1260] "How can we make our robot's packaging operations more efficient?"

[1261] This will improve the operational efficiency and accuracy of factory robots, while also realizing a system that takes into account the psychological state of workers.

[1262] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1263] Step 1:

[1264] The user starts the smartphone app and records the factory robot's operations. The smartphone's camera function is used to start video recording, capturing the series of operations of the factory robot. The input is the captured video data, and the output is a video file saved on the smartphone.

[1265] Step 2:

[1266] The device uploads the captured video to the cloud server. When the user taps the upload button, the video file is sent to the cloud server via the Internet. The input is the video file on the smartphone, and the output is the video data stored on the cloud server.

[1267] Step 3:

[1268] The video data received by the server is analyzed using a video analysis engine (e.g., OpenCV). The movement trajectory, speed, angle, etc. of the factory robot are measured and extracted for each video frame. The input is the video data stored on the server, and the output is the analyzed movement data (e.g., numerical values ​​for movement trajectory, speed, angle, etc.).

[1269] Step 4:

[1270] The server identifies and quantifies operational issues based on the analysis results. It then verifies the analyzed data and identifies performance inconsistencies and errors. For example, it quantifies inconsistencies in operation times or delays in operations in specific processes. The input is the analyzed operational data, and the output is quantified data on the issues.

[1271] Step 5:

[1272] The server generates improvement measures for the identified issues. Using the generative AI model, it provides specific advice and training methods to improve operational efficiency. The input is quantified issue data, and the output is feedback data on the improvement measures.

[1273] Step 6:

[1274] The user records their facial expressions and voices while performing actions and uploads them to a cloud server. The input is the voice and facial expression data recorded by the smart device, and the output is the emotion data stored on the cloud server.

[1275] Step 7:

[1276] The server uses an emotion recognition engine (e.g., EmotionAPI) to analyze the uploaded emotion data. The input is the emotion data stored on the server, and the output is data about the user's psychological state (e.g., tension, fatigue, stress level).

[1277] Step 8:

[1278] The server provides feedback based on the user's psychological state. The user's psychological state is reflected in the analysis results, and specific advice for improving the work environment and mental care is generated. The input is psychological state data and analysis results, and the output is feedback data that takes the psychological state into account.

[1279] Step 9:

[1280] The server uses a generative AI model to automatically respond to user questions. When the user enters a prompt (e.g., "What is the best way to shorten the robot's operating time?"), the generative AI model generates the optimal answer and provides the response. The input is the user's prompt, and the output is the answer generated by the generative AI model.

[1281] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1282] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1283] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1284] [Third embodiment]

[1285] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1286] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1287] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1288] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1289] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1290] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1291] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1292] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1293] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1294] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1295] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1296] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1297] This invention is a system that uses smart devices to record, analyze, and improve golf swings. The system quantifies the user's swing issues through video recording and data analysis on a cloud server, and provides specific improvement measures. It also includes a community platform where users can share information with other users, and an automated inquiry response function using artificial intelligence.

[1298] System Overview

[1299] 1. Record a video of your swing using a smart device

[1300] The user launches the smartphone app and takes a photo of their golf swing.

[1301] It provides a function to upload videos taken by the device to a cloud server.

[1302] 2. Data analysis using a cloud server

[1303] The server analyzes the video data it receives and extracts data such as swing trajectory, head speed, and ball speed.

[1304] The server identifies swing issues based on the analysis results and quantifies those issues.

[1305] 3. Providing remedial measures

[1306] The server provides the user with specific improvement measures based on the quantified issues.

[1307] The proposed improvements are displayed on the app in a way that is easy for users to understand.

[1308] 4. Providing a social community

[1309] The server provides a membership platform where users can share information and opinions.

[1310] 5. Automatic response to inquiries

[1311] The server uses the generative AI model to automatically generate answers to user inquiries.

[1312] To resolve any doubts, users can enter their questions within the app and receive a response.

[1313] Program processing

[1314] The program of this system performs the following processing.

[1315] 1. Video recording and uploading

[1316] The user launches the app and shoots a video of their swing. By tapping the recording button, a video of the golf swing is recorded.

[1317] Once the recording is complete, the device prompts the user to review the video and displays an "Upload" button, which the user taps to send the data to the cloud server.

[1318] 2. Data Analysis

[1319] The server analyzes the received video data and tracks the trajectory of the golf club for each frame.

[1320] The server extracts analytical data such as head speed, ball speed, and swing trajectory and stores this data in the user's profile.

[1321] 3. Quantifying issues and providing solutions

[1322] Based on the analysis results, the server quantifies the swing issues and identifies specific problems (e.g., slow head speed, improper swing trajectory).

[1323] The server provides users with specific advice and training methods to solve these problems.

[1324] 4. Use of social communities

[1325] Users have access to a membership platform where they can interact with other users and post and comment on specific issues or topics.

[1326] The device updates the community activity feed in real time, allowing users to see new posts and comments immediately.

[1327] 5. Automatic response to inquiries

[1328] The user enters a question in the app and taps the send button. For example, a question could be, "What should I do if my club head speed isn't increasing?"

[1329] The server uses a generative AI model to generate optimal answers to questions and responds to users in real time. The answers are specific and practical, and include methods that users can try immediately.

[1330] Specific examples

[1331] As a concrete example, consider the following scenario.

[1332] 1. Recording and analyzing swing videos

[1333] Users can record videos of their swings in their own backyard using a smartphone app and upload them to a cloud server. The videos clearly capture the full swing.

[1334] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph.

[1335] 2. Quantifying issues and providing solutions

[1336] Based on the analysis results, the server identifies the problem as an outside-in swing trajectory, and also points out that the head speed is slower than the average professional golfer.

[1337] The server will provide specific suggestions for improvement, such as, "To practice your inside-out swing, try the following drill."

[1338] 3. Sharing information with the community

[1339] A user accesses a community platform and posts a message such as, "I'm looking for advice on drills to improve my swing trajectory."

[1340] Other users respond by saying, "I had the same problem and this drill worked for me."

[1341] 4. Automatic response to inquiries

[1342] A user asks within the app, "What training methods can I use to increase my club head speed?"

[1343] The server uses the generative AI model to generate answers such as, "Specific training methods for increasing club head speed include training using weights and specific swing drills," and immediately provides them to the user.

[1344] In this way, the present invention provides a comprehensive system for golfers to efficiently improve their own swing and share information and questions with other users.

[1345] The processing flow will be explained below.

[1346] Step 1:

[1347] The user launches the smartphone app and taps the "swing video recording" button.

[1348] Step 2:

[1349] The device activates the smartphone camera and captures a picture of the user performing a golf swing.

[1350] Step 3:

[1351] After the device has finished recording, the user is prompted to check the video and, if there are no problems, tap the "Upload" button.

[1352] Step 4:

[1353] In order for the terminal to upload the captured video file to the cloud server, the terminal transmits the video data using the communication module.

[1354] Step 5:

[1355] The video data received by the server is temporarily stored.

[1356] Step 6:

[1357] The server invokes a video analysis algorithm and begins the analysis process to extract data such as swing trajectory, head speed, and ball speed.

[1358] Example: A server performs image analysis on each video frame to track the movement of a golf club.

[1359] Example: The server performs trajectory analysis to analyze the ball's flight pattern.

[1360] Step 7:

[1361] The server stores the analysis results as numerical data and associates them with the user's profile.

[1362] Step 8:

[1363] The server uses the quantified analysis results to identify issues with the user's swing and trajectory.

[1364] Example: If your clubhead speed is 90 mph, compare it to the pro average.

[1365] Example: If your swing path is "outside-in," it will generate feedback for general path corrections.

[1366] Step 9:

[1367] The server will provide the following information to the user along with the analysis results:

[1368] Numerical data on the problem (e.g., head speed, swing trajectory)

[1369] Advice for improvement (e.g., how to practice an inside-out swing)

[1370] Video links and text guides for recommended drills and practice techniques

[1371] Step 10:

[1372] The user taps the Community tab to access the membership platform.

[1373] Step 11:

[1374] The device displays the user's activity feed, updating the list of posts and comments in real time.

[1375] Step 12:

[1376] A user taps the "Create a new post" button and enters a question or comment.

[1377] Step 13:

[1378] The terminal transmits the input post content to the server.

[1379] Step 14:

[1380] The server saves the new post to a database and displays it to other users.

[1381] Step 15:

[1382] The user enters the question they want to ask the generated AI model and taps the "Send inquiry" button.

[1383] Step 16:

[1384] The terminal sends the entered question to the server.

[1385] Step 17:

[1386] The server calls the generative AI model and generates the best answer to the input question.

[1387] Example: The server responds to the question "What causes your swing to curve to the right?" with the answer "Your right hand may be too strong, so try to relax it."

[1388] Step 18:

[1389] The server sends the generated answer to the terminal for display to the user.

[1390] Example 1

[1391] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1392] Conventional golf swing analysis systems require expensive equipment and specialized knowledge, making them difficult for many amateur golfers to use. Furthermore, the time and cost required for individual lessons and coaching advice makes self-improvement of a golf swing extremely difficult. Furthermore, there is a lack of systems that provide specific improvement measures based on the results of swing analysis, leaving a need for a means for users to efficiently improve their skills.

[1393] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1394] In this invention, the server includes a means for recording a user's actions using an application launched by the user, a means for uploading the recorded action data to the server, a means for analyzing the action data on the server and generating analysis results, a means for identifying action issues based on the analysis results and quantifying the issues, and a means for providing solutions to the issues. This allows users to easily record their own actions and obtain detailed analysis and specific solutions for improvement via the server without specialized knowledge. Furthermore, by sharing information with other users, users can receive feedback and support within the community, thereby enabling efficient and continuous skill improvement.

[1395] A "user" is an entity that uses the system to record its own actions, obtain analysis results, and receive improvement measures.

[1396] An "application" is software that runs on a smart device and allows users to record actions, upload data, and display analysis results.

[1397] "Movement" is a physical movement, such as a golf swing, made by a user that is recorded by the system.

[1398] A "server" is a computer system that receives and analyzes action data uploaded by users.

[1399] "Data" refers to the video of the user's actions recorded and the quantified information resulting from that analysis.

[1400] "Analysis" refers to the process of tracking the characteristics of movements based on recorded movement data and deriving quantified results.

[1401] "Issues" refer to problems or areas for improvement in user behavior that have become apparent as a result of the analysis.

[1402] "Quantifying" means expressing the characteristics and challenges of an action as quantitative data.

[1403] "Improvement measures" refer to specific methods and advice on how the user should improve the issues identified in the analysis results.

[1404] A "Platform" is an online venue where users can share information and communicate with other users.

[1405] "Artificial intelligence" is a program that automatically generates responses to user inquiries.

[1406] This invention is a system that uses a smart device to record a user's golf swing and analyzes the data on a cloud server to quantify the user's swing issues and provide specific improvement measures. This system is implemented using a smart device application, a cloud server, analysis software, and an artificial intelligence model.

[1407] Hardware used

[1408] Smart devices: Smartphones and tablets are used. These devices must have a camera and internet connection.

[1409] Cloud server: A computer system where video data is uploaded and where analysis processing is performed.

[1410] Software used

[1411] Application: Software for smart devices that allows users to record their golf swings and upload the data to a server. Specifically, it is implemented as an application for iOS or Android.

[1412] Analysis software: To analyze golf swing videos, we use libraries such as OpenCV and TensorFlow, which allow us to perform frame analysis and feature extraction of movements.

[1413] Generative AI model: An artificial intelligence model that generates automated responses to user inquiries. This uses natural language processing (NLP) techniques such as GPT-3 and BERT.

[1414] Processing flow

[1415] 1. Video recording and uploading

[1416] The user starts the smartphone app and shoots a video of their swing. An easy-to-use interface is provided for video recording.

[1417] After shooting is complete, the device displays a video confirmation screen to the user, allowing them to edit the video or cut out unnecessary parts.

[1418] When the user checks the video and taps the upload button, the device sends the video data to the cloud server.

[1419] 2. Data analysis and result generation

[1420] The server temporarily stores the received video data in storage, then begins motion analysis for each video frame, tracking the movement of the club and the body.

[1421] The server quantifies and extracts motion data such as head speed, ball speed, and swing trajectory, and stores it in the user's profile.

[1422] 3. Quantifying issues and providing solutions

[1423] The server identifies the user's swing issues based on the analysis results and quantifies the specific problems.

[1424] Using a generative AI model, specific improvement measures for quantified issues are automatically generated, and these improvement measures are displayed in the app in a format that is easy for users to understand.

[1425] 4. Use of social communities

[1426] Users can use the in-app community feature to share information and opinions with other users, and can post and comment on specific topics.

[1427] The device updates the community feed in real time, allowing you to see new posts and comments immediately.

[1428] 5. Automatic response to inquiries

[1429] When a user enters a question in the app and taps the submit button, the server uses a generative AI model to generate the best answer.

[1430] Answers are provided to users in real time and include specific, practical content. For example, in response to a question like, "How can I train to increase my clubhead speed?", the answer might be, "There are weight training exercises and specific swing drills."

[1431] Specific examples

[1432] Examples of swing video recording and analysis

[1433] Users can record videos of their swings in their own backyard using a smart device, and then upload the videos to a cloud server.

[1434] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph.

[1435] Based on the analysis results, the server identifies the problem of an outside-in swing trajectory and identifies the head speed as slower than the average for professionals. The server then provides specific improvements to address these issues.

[1436] Examples from the community

[1437] When a user posts something like "I'm looking for advice on drills to improve my swing path," other users will respond with things like "I was having the same problem, and this drill worked for me."

[1438] Specific examples of automated response to inquiries

[1439] When a user asks, "What training methods can I use to increase my clubhead speed?", the server uses a generative AI model to respond, "Specific training methods for increasing clubhead speed include weight training and specific swing drills."

[1440] In this way, the present invention provides a comprehensive system for golfers to efficiently improve their swing and share information and questions with other users.

[1441] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1442] Step 1:

[1443] The user launches the smart device application and selects the "Swing Photo" option from the home screen.

[1444] Input: Launching applications, displaying the home screen

[1445] Output: Show "Swing Shooting" option

[1446] What it does: The app invokes the camera function and displays a message encouraging the user to prepare for their swing.

[1447] Step 2:

[1448] The user taps the recording button to record a video of their golf swing.

[1449] Input: Tap the recording button, video data of the swing

[1450] Output: Recorded swing video data

[1451] Specific operation: The device camera will start up and record the user's swing. When the recording is finished, a save button will be displayed.

[1452] Step 3:

[1453] Users can check the video on the video confirmation screen and edit or cut out unnecessary parts.

[1454] Input: Recorded swing video, user editing actions

[1455] Output: Edited swing video data

[1456] What it does: The app provides video playback and editing functionality and displays an interface for the user to finalize the footage.

[1457] Step 4:

[1458] The user taps the upload button to send the edited video data to the cloud server.

[1459] Input: Edited swing video data, tap the upload button

[1460] Output: Video data stored on a cloud server

[1461] Specific operation: The device sends data to the cloud server via the Internet. Once the upload is complete, a confirmation message will be displayed.

[1462] Step 5:

[1463] The video data received by the server is temporarily stored in storage.

[1464] Input: Uploaded video data

[1465] Output: Saved video data

[1466] Specific operation: The server receives the video data and stores it in the file system. This storage is temporary storage for fast access.

[1467] Step 6:

[1468] The server starts motion analysis for each video frame, tracking and analyzing the movement of the golf club and the body.

[1469] Input: Saved video data

[1470] Output: Analyzed motion data (head speed, ball speed, swing trajectory, etc.)

[1471] How it works: The server uses libraries such as OpenCV and TensorFlow to analyze the images of each frame, and then uses a motion detection algorithm to extract club and body movement data.

[1472] Step 7:

[1473] The server quantifies the extracted behavioral data and stores it in the user's profile.

[1474] Input: Parsed motion data

[1475] Output: Stored quantified operating data

[1476] Specific operation: The analysis results are processed as numerical data and added to the user's profile database.

[1477] Step 8:

[1478] The server identifies the user's swing issues based on the analysis results and quantifies the specific problems.

[1479] Input: digitized motion data

[1480] Output: Quantified issues

[1481] Specific operation: The server uses the generative AI model to extract issues using specific algorithms and quantitatively evaluate them.

[1482] Step 9:

[1483] The server uses the generated AI model to automatically generate specific improvement measures for the quantified issues.

[1484] Input: Quantified task

[1485] Output: Generated remediation measures

[1486] How it works: The generative AI model references past data and training methods to suggest improvements based on the user's challenges, which may include text and links to images or videos.

[1487] Step 10:

[1488] The improvement measures generated by the server are sent to the user's smart device and displayed on the app.

[1489] Input: Generated remediation measures

[1490] Output: Improvement measures displayed on the app

[1491] Specific behavior: The app displays the improvement measures obtained from the server in the interface so that the user can take action immediately.

[1492] Step 11:

[1493] Users access the Community tab and share information with other users.

[1494] Enter: Access to the Community tab

[1495] Output: Shared information and comments

[1496] What it does: The app provides a post creation feature that allows users to share their challenges and improvements with the community, making it easy to receive feedback and advice.

[1497] Step 12:

[1498] The user enters a question in the in-app inquiry tab and submits it.

[1499] Input: User question

[1500] Output: Query data sent to the server

[1501] Specific behavior: Provides an interface for the app to send entered questions to the server.

[1502] Step 13:

[1503] The server uses a generative AI model to generate optimal answers and provides them to the user in real time.

[1504] Input: Question data from the user

[1505] Output: The generated answer

[1506] Specific operation: The server uses the generative AI model to generate an appropriate answer based on the question and sends it in text format to the user's device.

[1507] Step 14:

[1508] The user references the answers provided and takes various actions to improve their swing.

[1509] Input: Generated Answer

[1510] Output: Actions taken

[1511] Specific Actions: Plan practice sessions or attempt specific training drills based on the answers the user generates.

[1512] (Application example 1)

[1513] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1514] Conventional golf swing analysis systems not only provide users with analysis results, but also make it difficult to share information with other users in real time or receive actual support at a golf club. Furthermore, there is a lack of advice on selecting appropriate golf equipment based on swing analysis results, leaving users lacking the information and tools to instantly improve their skills. This often means that users need time to improve their swing, and it is difficult to receive appropriate guidance and support during the process.

[1515] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1516] In this invention, the server includes means for recording a user's swing motion using a smart computing device, means for transmitting the recorded swing motion data to a remote server, means for analyzing the swing motion data in the remote server and generating analysis results, means for identifying problems in the swing motion based on the analysis results and quantifying the problems, means for providing solutions to the problems, means for sharing the analysis results with other users in the store in real time, and means for providing a function for making inquiries to store staff. This allows users to check the swing analysis results in real time in the store, share experiences and information with other users, and immediately receive appropriate advice from store staff.

[1517] A "smart computing device" is an electronic device that is portable by the user and has a high-resolution camera and internet connectivity that is capable of recording and transmitting swing motion.

[1518] "User" refers to a user who seeks to improve their golf swing using the system of the present invention.

[1519] The term "swing motion" refers to a series of physical movements that are made when using a golf club to hit a ball, including the trajectory of the golf club, head speed, input force, and the like.

[1520] "Remote Server" refers to a data center that is a data analysis and storage system accessible via the Internet and that analyzes the swing video sent by the User and provides the results.

[1521] The "analysis results" are technical evaluations and quantified data regarding the user's swing, generated by the server based on swing motion data.

[1522] "Quantification" refers to expressing various parameters of the analysis results as quantitative values, allowing users to concretely recognize their own performance.

[1523] "Improvement measures" refer to specific training methods and technical advice provided to address swing movement issues identified based on the analysis results.

[1524] "Real-time" refers to the near-instantaneous transmission of data, analysis, provision of results, and sharing of information.

[1525] "Information sharing" refers to users exchanging data such as analysis results, experiences, and opinions with other users and store staff.

[1526] The "inquiry function" refers to a system that allows users to input questions about analysis results and improvement measures and receive an automatic response or a reply from store staff on the spot.

[1527] This invention is a system for use in golf equipment stores that allows customers to receive real-time analysis and feedback on their swings. The system includes a smart computing device, a remote server, and a social community platform for sharing analysis results. It also provides query capabilities using generative AI models.

[1528] System Overview

[1529] 1. Video recording and uploading

[1530] Users use a smart computing device (such as a smartphone) to film their swing, and once filming is complete, the video data is automatically transmitted to a remote server.

[1531] 2. Data Analysis

[1532] The server analyzes the received swing video and extracts data such as the golf club trajectory, head speed, ball speed, etc. These analysis results are quantified and recorded in the user profile.

[1533] 3. Providing Feedback

[1534] The server quantifies the identified issues based on the analysis results and indicates specific areas for improvement to the user. The improvement measures are sent in real time to a smart computing device and displayed to the user.

[1535] 4. Social Community

[1536] Users can share analysis results with other customers in the store in real time, exchange information, and hold discussions and provide feedback on specific topics.

[1537] 5. Inquiry function

[1538] When a user inputs a question about improvement measures or analysis results into the server, the optimal answer is automatically generated using a generative AI model, and this answer is immediately provided to the user.

[1539] Hardware and Software Configuration

[1540] Smart computing devices: Smartphones and tablets equipped with high-resolution cameras and internet connectivity.

[1541] Remote server: Use cloud services such as Amazon Web Services (AWS) and Google Cloud Platform (GCP).

[1542] Data analysis software: Video data was analyzed using Python and OpenCV.

[1543] Generative AI model: Generates automatic responses to user inquiries using natural language processing.

[1544] Examples of concrete examples and prompts

[1545] Examples:

[1546] 1. The user takes a photo of their golf swing with their smartphone in the practice area inside the store.

[1547] 2. The captured video is automatically uploaded to a cloud server, where analysis begins.

[1548] 3. The server quantifies the analysis results (e.g., head speed, swing trajectory, etc.) and immediately sends them to your smartphone.

[1549] 4. The user checks the analysis results on the spot and continues practicing according to the provided improvement measures.

[1550] 5. When users ask questions about problems with their swing, the generative AI model provides specific answers. For example, in response to a question like, "What can I do if my club head speed isn't improving?", the model might respond with, "Training with weights and specific swing drills are effective."

[1551] Example prompt sentence:

[1552] "A user has uploaded a video of their swing taken in a golf shop. Please analyze this video and provide data on the following: 1. Head speed 2. Swing path 3. Ball speed. Also, based on this data, please provide specific advice on how to improve their swing."

[1553] In this way, the present invention provides a comprehensive support system for users to efficiently analyze and improve their swings and share information within a golf equipment store.

[1554] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1555] Step 1:

[1556] A user uses a smart computing device (such as a smartphone) to film their swing motion.

[1557] Input: A smartphone for users to record swing videos.

[1558] Data processing / calculation: Video is taken using a high-resolution camera.

[1559] Output: Recorded swing video file.

[1560] Step 2:

[1561] The device uploads the captured swing video to a cloud server.

[1562] Input: Recorded swing video file.

[1563] Data processing / calculation: Send the video file to the cloud server using an HTTP request.

[1564] Output: Video files stored on cloud server.

[1565] Step 3:

[1566] The server analyzes the received swing video and extracts swing motion data.

[1567] Input: Video files stored on a cloud server.

[1568] Data processing / calculation: Using Python and OpenCV, information such as golf club trajectory, head speed, and ball speed is extracted from video data.

[1569] Output: Swing motion analysis results (e.g. head speed, swing path, ball speed).

[1570] Step 4:

[1571] The server identifies issues in the swing motion based on the analysis results and quantifies those issues.

[1572] Input: Analysis results of swing motion.

[1573] Data processing / calculation: The analysis results are applied to an evaluation algorithm to quantitatively express the identified issues.

[1574] Output: Quantified swing motion issues (e.g., slow clubhead speed, inaccurate swing path).

[1575] Step 5:

[1576] The server generates solutions to the problems and provides them to the user.

[1577] Input: Quantified swing movement tasks.

[1578] Data processing / computation: Running algorithms to generate specific training or technical advice to address the problem.

[1579] Output: Specific improvement measures (e.g., training methods to increase clubhead speed, drills to improve swing path).

[1580] Step 6:

[1581] The device displays analysis results and improvement measures to the user in real time.

[1582] Input: Specific improvement measures.

[1583] Data processing / calculation: Rendering is performed to display the results and improvement measures on the smartphone UI.

[1584] Output: Analysis results and remediation measures displayed on the user's screen.

[1585] Step 7:

[1586] Users can share analysis results with other customers in the store and exchange information.

[1587] Input: Analysis results and remediation measures.

[1588] Data processing / computation: Use social community platforms to share data and post comments and feedback.

[1589] Output: Information exchanged with other users.

[1590] Step 8:

[1591] Users ask questions about analysis results and improvement measures, and the server automatically responds using the generated AI model.

[1592] Input: A question from the user (e.g., "What training methods can I use to increase my club head speed?").

[1593] Data processing / calculation: Input a prompt sentence into the generative AI model to generate the optimal answer.

[1594] Output: The automated response that is provided to the user.

[1595] Specific prompt examples:

[1596] "A user has uploaded a video of their swing taken in a golf shop. Please analyze this video and provide data on the following: 1. Head speed 2. Swing path 3. Ball speed. Also, based on this data, please provide specific advice on how to improve their swing."

[1597] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1598] This invention is a system that uses a smart device to record, analyze, and improve a golf swing, and also combines it with user emotion recognition. This system quantifies the user's swing issues through swing video recording and data analysis on a cloud server, and provides specific improvement measures. It also uses an emotion engine to grasp the user's psychological state and provides feedback based on that to support more effective swing improvement.

[1599] System Overview

[1600] 1. Record a video of your swing using a smart device

[1601] The user launches the smartphone app and takes a photo of their golf swing.

[1602] It provides a function to upload videos taken by the device to a cloud server.

[1603] 2. Data analysis using a cloud server

[1604] The server analyzes the video data it receives and extracts data such as swing trajectory, head speed, and ball speed.

[1605] The server identifies swing issues based on the analysis results and quantifies those issues.

[1606] 3. Providing remedial measures

[1607] The server provides the user with specific improvement measures based on the quantified issues.

[1608] The proposed improvements are displayed on the app in a way that is easy for users to understand.

[1609] 4. Emotion recognition

[1610] The device records the user's facial expressions and voice during the swing and uploads them to a cloud server.

[1611] The server uses an emotion engine to analyze emotions from the user's facial expressions and voice.

[1612] 5. Providing Feedback

[1613] Based on the emotion recognition results, the server generates improvement measures and advice that take into account the user's psychological state.

[1614] The server also incorporates the user's emotional data into the analysis results to provide more accurate feedback.

[1615] 6. Providing social communities

[1616] The server provides a membership platform where users can share information and opinions.

[1617] 7. Auto-response to inquiries

[1618] The server uses the generative AI model to automatically generate answers to user inquiries.

[1619] To resolve any doubts, users can enter their questions within the app and receive a response.

[1620] Program processing

[1621] The program of this system performs the following processing.

[1622] 1. Video recording and uploading

[1623] The user launches the app and shoots a video of their swing. By tapping the recording button, a video of the golf swing is recorded.

[1624] Once the recording is complete, the device prompts the user to review the video and displays an "Upload" button, which the user taps to send the data to the cloud server.

[1625] 2. Data Analysis

[1626] The server analyzes the received video data and tracks the trajectory of the golf club for each frame.

[1627] The server extracts analytical data such as head speed, ball speed, and swing trajectory and stores this data in the user's profile.

[1628] 3. Quantifying issues and providing solutions

[1629] Based on the analysis results, the server quantifies the swing issues and identifies specific problems (e.g., slow head speed, improper swing trajectory).

[1630] The server provides users with specific advice and training methods to solve these problems.

[1631] 4. Emotion Recognition and Feedback

[1632] When a user shoots a video of their swing, their facial expressions and voice are also recorded at the same time.

[1633] The device sends this facial expression and voice data to a cloud server.

[1634] The server uses an emotion engine to analyze the user's facial expressions and voice to determine their emotions. For example, if the user is nervous while taking a photo, the server can detect that emotion.

[1635] Based on the emotion recognition results, the server generates improvement measures and advice that take into account the user's psychological state. For example, it could provide advice such as "Try breathing techniques to help you swing more relaxed."

[1636] The server also incorporates the user's emotional data into the analysis results to provide more accurate feedback.

[1637] 5. Use of social communities

[1638] The user taps the Community tab to access the membership platform.

[1639] The device displays the user's activity feed, updating the list of posts and comments in real time.

[1640] A user taps the "Create a new post" button and enters a question or comment.

[1641] The terminal transmits the input post content to the server.

[1642] The server saves the new post to a database and displays it to other users.

[1643] 6. Automatic response to inquiries

[1644] The user enters the question they want to ask the generated AI model and taps the "Send inquiry" button.

[1645] The terminal sends the entered question to the server.

[1646] The server calls up the generative AI model and generates the optimal answer to the input question. For example, in response to the question, "What kind of training is effective for increasing club head speed?", it provides a specific training method.

[1647] The server sends the generated answer to the terminal for display to the user.

[1648] Specific examples

[1649] As a concrete example, consider the following scenario.

[1650] 1. Recording and analyzing swing videos

[1651] Users can record videos of their swings in their own backyard using a smartphone app and upload them to a cloud server. The videos include the full swing, as well as the user's facial expressions and voice.

[1652] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph. The emotion engine also recognizes that the user is tense during the swing.

[1653] 2. Quantifying issues and providing solutions

[1654] Based on the analysis results, the server identifies that the swing trajectory is outside-in as the problem, and points out that the head speed is slower than the average for professionals as an issue.

[1655] The server will provide specific improvement measures, such as "Try the following drill to practice your inside-out swing," and will also provide advice including "breathing techniques and mental training to relieve tension."

[1656] 3. Sharing information with the community

[1657] A user accesses a community platform and posts a message such as, "I'm looking for advice on drills to improve my swing trajectory."

[1658] Other users respond by saying, "I had the same problem and this drill worked for me."

[1659] 4. Automatic response to inquiries

[1660] A user asks within the app, "What kind of training is effective for increasing club head speed?"

[1661] The server uses the generative AI model to generate specific answers such as "Training with weights and specific swing drills are effective in increasing club head speed," and immediately provides them to the user.

[1662] In this way, the present invention provides a comprehensive system that allows golfers to efficiently improve their own swing, while also taking care of their psychological aspects, and allows them to share information and solve questions with other users.

[1663] The processing flow will be explained below.

[1664] Step 1:

[1665] The user launches the smartphone app and taps the "swing video recording" button.

[1666] Step 2:

[1667] The device activates the smartphone's camera and microphone to capture images and audio of the user performing a golf swing.

[1668] Step 3:

[1669] After the device has finished recording, the user is prompted to check the video and, if there are no problems, tap the "Upload" button.

[1670] Step 4:

[1671] The device transmits the data using a communication module to upload the captured video files and audio data to a cloud server.

[1672] Step 5:

[1673] The server temporarily stores the received video and audio data.

[1674] Step 6:

[1675] The server invokes a video analysis algorithm and begins the analysis process to extract data such as swing trajectory, head speed, and ball speed.

[1676] Example: A server performs image analysis on each video frame to track the movement of a golf club.

[1677] Example: The server performs trajectory analysis to analyze the ball's flight pattern.

[1678] Step 7:

[1679] The server uses an emotion engine to analyze the user's facial expressions in audio data and video to identify emotions.

[1680] Step 8:

[1681] The server stores the analysis results as numerical data and associates them with the user's profile.

[1682] Step 9:

[1683] The server uses the quantified analysis results to identify issues with the user's swing and trajectory.

[1684] Example: If your clubhead speed is 90 mph, compare it to the pro average.

[1685] Example: If your swing path is "outside-in," it will generate feedback for general path corrections.

[1686] Example: Identifying that a user is nervous based on facial and voice analysis.

[1687] Step 10:

[1688] The server will provide the following information to the user along with the analysis results:

[1689] Numerical data on the problem (e.g., head speed, swing trajectory)

[1690] Advice for improvement (e.g., how to practice an inside-out swing)

[1691] Emotion-based advice (e.g., breathing techniques for a relaxed swing)

[1692] Video links and text guides for recommended drills and practice techniques

[1693] Step 11:

[1694] The user taps the Community tab to access the membership platform.

[1695] Step 12:

[1696] The device displays the user's activity feed, updating the list of posts and comments in real time.

[1697] Step 13:

[1698] A user taps the "Create a new post" button and enters a question or comment.

[1699] Step 14:

[1700] The terminal transmits the input post content to the server.

[1701] Step 15:

[1702] The server saves the new post to a database and displays it to other users.

[1703] Step 16:

[1704] The user enters the question they want to ask the generated AI model and taps the "Send inquiry" button.

[1705] Step 17:

[1706] The terminal sends the entered question to the server.

[1707] Step 18:

[1708] The server calls the generative AI model and generates the best answer to the input question.

[1709] Example: The server responds to the question "What causes your swing to curve to the right?" with the answer "Your right hand may be too strong, so try to relax it."

[1710] Step 19:

[1711] The server sends the generated answer to the terminal for display to the user.

[1712] Example 2

[1713] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1714] Conventional golf swing analysis systems focus on analyzing technical issues in a user's swing and providing solutions for improvement, but do not consider psychological aspects. As a result, if a user feels nervous or stressed, this may not lead to actual performance improvement. Furthermore, smoother support is needed for sharing information with other users and resolving questions.

[1715] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1716] In this invention, the server includes a means for recording the user's facial expressions and voice during the swing, a means for uploading the recorded facial and voice data to a cloud server, and a means for analyzing the user's emotions using an emotion recognition engine in the cloud server. This enables feedback that takes the user's psychological state into consideration. The system also includes a means for identifying and quantifying issues in the swing based on the analysis results, and a means for automatically responding to inquiries from the user using a generative AI model, enabling comprehensive support for swing improvement from both technical and psychological aspects.

[1717] A "smart device" refers to an electronic device that can be carried by a user and that can connect to the Internet and run various applications. Examples include smartphones and tablets.

[1718] A "cloud server" refers to a remote server that can store, analyze, and communicate data over the Internet.

[1719] An "analysis algorithm" refers to a mathematically and logically defined method of defining rules and procedures for extracting, classifying, and evaluating useful information from specific data.

[1720] An "emotion recognition engine" refers to technology that analyzes non-verbal data such as a user's facial expressions and voice to identify their emotional state.

[1721] "Analysis results" refers to the specific numerical values ​​and evaluation results obtained after analyzing swing movements, facial expressions, and audio data.

[1722] "Means for quantifying problems" refers to methods and tools for quantitatively expressing problems in a user's swing motion from the analysis results.

[1723] The "means for providing improvement measures" refers to a method or system for proposing specific instruction content or training methods to the user for the identified problems in the swing motion.

[1724] A "generative AI model" refers to a mathematical model that uses artificial intelligence to automatically generate answers and advice to user inquiries.

[1725] A "community platform" refers to an online space where multiple users can share information, exchange opinions, ask questions, and answer questions.

[1726] This invention is a system that uses a smart device to record, analyze, and improve golf swings, and also combines it with user emotion recognition. This system starts when a user uses a smart device such as a smartphone to record a video of their golf swing and uploads the video data to a cloud server.

[1727] Hardware and software used:

[1728] Smart Device: An electronic device that is portable and has internet connectivity, such as a smartphone or tablet.

[1729] Cloud server: A remote server that stores, analyzes, and communicates data.

[1730] Analysis algorithm: Using image analysis libraries such as OpenCV, the swing trajectory, head speed, ball speed, etc. are analyzed for each frame.

[1731] Emotion recognition engine: Technology that analyzes emotions from a user's facial expressions and voice, such as Microsoft Azure's Emotion API.

[1732] Generative AI models: Use artificial intelligence models such as GPT-4 to automatically generate answers to user inquiries.

[1733] The specific operation steps of the system:

[1734] 1. Video recording and uploading

[1735] The user starts the smartphone app and records a swing video. They tap the record button to start recording and the end button to stop recording.

[1736] If the user checks the video and there are no problems, they tap the upload button to send the video to the cloud server.

[1737] 2. Data Analysis

[1738] The server receives the uploaded video data and uses an analysis algorithm to extract data such as swing trajectory, head speed, and ball speed for each frame.

[1739] The extracted data is stored in a profile for each user.

[1740] 3. Quantifying issues and providing solutions

[1741] The server then quantifies swing issues based on the analysis results. For example, if the head speed is less than 85 mph, it will be listed as a "low head speed" issue.

[1742] The server generates specific improvement measures for each quantified issue and displays them in the user's app, such as "training methods to improve club head speed."

[1743] 4. Emotion Recognition and Feedback

[1744] When a user shoots a video of their swing, their facial expressions and voice are also recorded at the same time.

[1745] The device transmits facial expression and voice data to a cloud server.

[1746] The server uses an emotion recognition engine to analyze the user's emotions, for example, detecting that the user is nervous during the shoot.

[1747] Based on the analysis results and emotional data, the server generates comments and improvement measures that take into account the user's psychological state and provides them to the user's smart device, such as "breathing techniques for relaxation."

[1748] 5. Use of social communities

[1749] Users access the platform by tapping the community tab within the smartphone app.

[1750] The device displays posts and comments in real time and provides an interface for users to post questions and opinions.

[1751] The server stores new posts and comments in a database, making them available for all users to see.

[1752] 6. Automatic response to inquiries

[1753] The user enters a question in the app and taps the "Send Inquiry" button.

[1754] The device sends the question to the server, which then uses a generative AI model (e.g., GPT-4) to generate the optimal answer.

[1755] The server displays the generated answer on the user's smart device.

[1756] Specific operation example

[1757] 1. Recording and analyzing swing videos

[1758] Users can record videos of their swings in their own backyard using their smartphones and upload them to a cloud server. The videos include the swing, the user's facial expressions, and their voice.

[1759] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph.

[1760] In addition, the emotion recognition engine detects whether the user is nervous during the swing.

[1761] 2. Quantifying issues and providing solutions

[1762] The server identifies the problem as slow head speed and swing trajectory, and then provides the user with specific improvement measures, such as drills to practice an inside-out swing and breathing techniques to relieve tension.

[1763] 3. Sharing information with the community

[1764] A user accesses a community platform and posts something like, "I'm looking for advice on improving my swing."

[1765] Other users will reply saying, "This drill was effective."

[1766] 4. Automatic response to inquiries

[1767] A user asks, "How can I train to increase my club head speed?"

[1768] The server uses the generative AI model to generate answers such as "Training with weights and specific swing drills are effective" and provides them to the user.

[1769] In this way, the present invention can comprehensively support the user's swing improvement and provide feedback that also includes psychological aspects.In addition, the information sharing function with other users and the automatic response function to inquiries create an efficient and effective training environment.

[1770] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1771] Step 1:

[1772] The user launches the smartphone app and records a video of their swing.

[1773] Input: Start of shooting by user operation

[1774] Data processing: Record a swing video using a smartphone camera

[1775] Output: Recorded swing video file

[1776] Specific behavior:

[1777] The user launches the app and switches to shooting mode.

[1778] The user taps the record button to begin recording the golf swing.

[1779] The user taps the end button to end the recording.

[1780] Step 2:

[1781] The device uploads the captured video to a cloud server.

[1782] Input: Recorded swing video file

[1783] Data processing: Transfer video data to the cloud server

[1784] Output: Video data on a cloud server

[1785] Specific behavior:

[1786] The device will prompt the user to review the video and display an "Upload" button.

[1787] When the user taps the upload button, the device uses its internet connection to send the video file to a cloud server.

[1788] Step 3:

[1789] The server analyzes the video data it receives and extracts data such as swing trajectory, head speed, and ball speed.

[1790] Input: Video data on a cloud server

[1791] Data processing: Perform video analysis using analysis algorithms such as OpenCV

[1792] Output: Analysis data such as swing trajectory, head speed, ball speed, etc.

[1793] Specific behavior:

[1794] The server analyzes the video data frame by frame.

[1795] The server extracts data such as swing trajectory, head speed, and ball speed and stores it in the user's profile.

[1796] Step 4:

[1797] The server quantifies swing issues based on the analysis results and generates specific improvement measures.

[1798] Input: Swing trajectory, head speed, ball speed data

[1799] Data processing: Applying algorithms to quantify swing problems using analytical data

[1800] Output: Quantified issues and concrete improvement measures

[1801] Specific behavior:

[1802] The server identifies swing issues based on the analysis results.

[1803] The server generates specific improvement measures and training methods based on the issues.

[1804] The server sends the generated advice to the user's app.

[1805] Step 5:

[1806] The device records the user's facial expressions and voice and uploads them to a cloud server.

[1807] Input: Photographed facial expressions and voice data

[1808] Data processing: Send facial expression and voice data to a cloud server

[1809] Output: Facial expression and voice data on a cloud server

[1810] Specific behavior:

[1811] When the device shoots a swing video, it also records the user's facial expressions and voice at the same time.

[1812] The device uploads facial expression and voice data to a cloud server.

[1813] Step 6:

[1814] The server uses an emotion recognition engine to analyze emotions from the user's facial expressions and voice.

[1815] Input: Facial expression and voice data on a cloud server

[1816] Data processing: Emotion analysis using an emotion recognition engine (e.g., Emotion API)

[1817] Output: User emotion data

[1818] Specific behavior:

[1819] The server inputs facial and voice data into an emotion recognition engine.

[1820] The server identifies the user's emotions (e.g., tension, relaxation, etc.).

[1821] Step 7:

[1822] The server reflects the emotional data in the analysis results and generates appropriate feedback that takes the user's psychological state into account.

[1823] Input: Analysis data and emotion data

[1824] Data processing: Adding emotional data to the analysis results and adjusting improvement measures and advice

[1825] Output: Feedback that takes into account the user's psychological state

[1826] Specific behavior:

[1827] If the server detects tension, it generates feedback that includes "breathing techniques to relax."

[1828] The server sends the generated feedback to the user's app.

[1829] Step 8:

[1830] Users access the community platform to share information and post questions.

[1831] Input: User-submitted content

[1832] Data processing: Save the posted content in a database and share it with other users

[1833] Output: Comments and replies by other users

[1834] Specific behavior:

[1835] A user taps the Community tab to access the platform.

[1836] The device displays the latest posts and comments and sends the user's new posts to the cloud server.

[1837] The server stores the posts in a database and makes them available for all users to view.

[1838] Step 9:

[1839] The user asks a question using the generative AI model, and the server generates an automated response.

[1840] Input: Question asked by the user (prompt text)

[1841] Data processing: Using a generative AI model (e.g., GPT-4) to generate optimal answers

[1842] Output: The generated answer

[1843] Specific behavior:

[1844] The user enters a question in the app and taps the "Send Inquiry" button.

[1845] The device sends the question to the cloud server.

[1846] The server uses a generative AI model to generate the best answer to the question.

[1847] The server generates the answer and sends it to the user, who displays it in the app.

[1848] In this way, a system is constructed that comprehensively supports the user in improving their swing by performing data processing and calculations based on the input data at each step and then using the resulting output to proceed to the next processing step.

[1849] (Application example 2)

[1850] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1851] Existing factory robot motion analysis systems lack the functionality to quantify motion efficiency and accuracy and provide specific improvement measures. Furthermore, they do not provide feedback that takes into account the psychological state of the worker or suggest improvements to the work environment, resulting in insufficient improvements to the performance of both the robot and the worker. This makes it difficult to create an efficient and stress-free work environment.

[1852] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recording the user's robot operations using a smart device, means for uploading the recorded operation data to a cloud server, means for analyzing the operation data and generating analysis results in the cloud server, means for identifying operation issues based on the analysis results and quantifying the issues, means for providing improvements to the issues, means for uploading emotional data recorded during the operations to the cloud server, means for analyzing the emotional data in the cloud server and identifying the user's psychological state, and means for providing feedback based on the psychological state. This makes it possible to scientifically improve the operational efficiency and accuracy of factory robots and provide feedback and improve the work environment that takes into account the psychological state of workers.

[1853] A "smart device" is an electronic device that can be carried and operated by a user, such as a smartphone or tablet.

[1854] A "cloud server" is a remote server for storing, managing, and analyzing data over a network.

[1855] "Analysis" is the process of evaluating recorded data using scientific methods and extracting the necessary information.

[1856] An "issue" refers to a problem or defect that needs to be resolved in a system or process.

[1857] "Quantification" refers to expressing the analysis results in concrete numerical values.

[1858] "Improvement measures" refer to specific policies and methods for solving identified issues.

[1859] "Emotion data" is voice and facial expression information recorded to assess the user's psychological state.

[1860] "Mental state" refers to the user's emotional and mental state.

[1861] "Feedback" refers to advice and suggestions for improvement based on analysis results and psychological state.

[1862] This invention is a comprehensive system that analyzes the behavior of factory robots and provides improvements to improve their efficiency based on the analysis. The system combines many components, including smart devices, cloud servers, analysis engines, emotion recognition engines, and generative AI models, to provide comprehensive support for factory robots and their operators.

[1863] Hardware and software used

[1864] Hardware: Smartphones, cloud servers

[1865] Software: Video analysis engine (e.g., OpenCV), emotion recognition engine (e.g., TensorFlow), AWS cloud services, EmotionAPI

[1866] Libraries / Tools: OpenCV, TensorFlow, AWS cloud services, EmotionAPI, generative AI models

[1867] System processing flow

[1868] 1. Video recording and uploading

[1869] The user launches a smartphone app and records the factory robot's operations.

[1870] After recording the video, the smartphone uploads the data to a cloud server, which also records the voices and facial expressions of the workers during the operation.

[1871] 2. Data Analysis

[1872] The cloud server analyzes the received video data and extracts data such as movement trajectory, speed, and angle. A video analysis engine (e.g., OpenCV) is used to evaluate the movement pattern and accuracy of the robot's movement for each frame.

[1873] As a result of the analysis, issues such as "uneven operation times" or "slow operation in certain processes" are quantified.

[1874] 3. Providing remedial measures

[1875] Based on the analysis results, the cloud server generates specific improvement measures to improve operational efficiency, such as providing advice on "reviewing operational paths" or "optimizing specific procedures."

[1876] 4. Emotion Recognition and Feedback

[1877] The cloud server analyzes the uploaded facial and voice data of the user and evaluates their psychological state, such as tension, fatigue, and stress level, using an emotion recognition engine (e.g., EmotionAPI).

[1878] Based on the analysis results, feedback is provided according to the psychological state, such as "adjust your work pace" or "take a break."

[1879] 5. Community and Auto-Responders

[1880] The cloud server provides a community platform where users can share information with other users. It also has an automatic response function to user questions using a generative AI model. For example, if a prompt sentence is entered as "What is the best way to shorten the robot's operating time?", the generative AI model will generate the optimal answer.

[1881] Specific examples

[1882] For example, if a user wants to analyze the packaging operation of a robot, the following procedure is performed.

[1883] Video recording: The user uses a smartphone app to record the robot's packaging operations. The user's voice and facial expressions are also recorded during the recording.

[1884] Data upload: The captured video is immediately uploaded to the cloud server.

[1885] Data analysis: The cloud server analyzes the video and quantifies issues such as "uneven speed of packaging operations" or "slow operations in certain parts."

[1886] Providing improvement measures: Based on the analysis results, improvement measures such as "correcting the movement path to make the speed uniform" and "introducing efficiency techniques" are provided.

[1887] Emotion recognition and feedback: The emotion recognition engine analyzes the user's tension and fatigue while working, and makes suggestions such as "improving the work environment" and "adjusting break times."

[1888] Query response: When a user types a question like, "How can I improve the robot's movement efficiency?", the generative AI model provides advice such as "reviewing the movement path" or "specific training methods."

[1889] Example prompt sentence:

[1890] "How can we make our robot's packaging operations more efficient?"

[1891] This will improve the operational efficiency and accuracy of factory robots, while also realizing a system that takes into account the psychological state of workers.

[1892] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1893] Step 1:

[1894] The user starts the smartphone app and records the factory robot's operations. The smartphone's camera function is used to start video recording, capturing the series of operations of the factory robot. The input is the captured video data, and the output is a video file saved on the smartphone.

[1895] Step 2:

[1896] The device uploads the captured video to the cloud server. When the user taps the upload button, the video file is sent to the cloud server via the Internet. The input is the video file on the smartphone, and the output is the video data stored on the cloud server.

[1897] Step 3:

[1898] The video data received by the server is analyzed using a video analysis engine (e.g., OpenCV). The movement trajectory, speed, angle, etc. of the factory robot are measured and extracted for each video frame. The input is the video data stored on the server, and the output is the analyzed movement data (e.g., numerical values ​​for movement trajectory, speed, angle, etc.).

[1899] Step 4:

[1900] The server identifies and quantifies operational issues based on the analysis results. It then verifies the analyzed data and identifies performance inconsistencies and errors. For example, it quantifies inconsistencies in operation times or delays in operations in specific processes. The input is the analyzed operational data, and the output is quantified data on the issues.

[1901] Step 5:

[1902] The server generates improvement measures for the identified issues. Using the generative AI model, it provides specific advice and training methods to improve operational efficiency. The input is quantified issue data, and the output is feedback data on the improvement measures.

[1903] Step 6:

[1904] The user records their facial expressions and voices while performing actions and uploads them to a cloud server. The input is the voice and facial expression data recorded by the smart device, and the output is the emotion data stored on the cloud server.

[1905] Step 7:

[1906] The server uses an emotion recognition engine (e.g., EmotionAPI) to analyze the uploaded emotion data. The input is the emotion data stored on the server, and the output is data about the user's psychological state (e.g., tension, fatigue, stress level).

[1907] Step 8:

[1908] The server provides feedback based on the user's psychological state. The user's psychological state is reflected in the analysis results, and specific advice for improving the work environment and mental care is generated. The input is psychological state data and analysis results, and the output is feedback data that takes the psychological state into account.

[1909] Step 9:

[1910] The server uses a generative AI model to automatically respond to user questions. When the user enters a prompt (e.g., "What is the best way to shorten the robot's operating time?"), the generative AI model generates the optimal answer and provides the response. The input is the user's prompt, and the output is the answer generated by the generative AI model.

[1911] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1912] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1913] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1914] [Fourth embodiment]

[1915] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1916] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1917] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1918] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1919] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1920] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1921] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1922] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1923] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1924] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1925] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1926] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1927] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1928] This invention is a system that uses smart devices to record, analyze, and improve golf swings. The system quantifies the user's swing issues through video recording and data analysis on a cloud server, and provides specific improvement measures. It also includes a community platform where users can share information with other users, and an automated inquiry response function using artificial intelligence.

[1929] System Overview

[1930] 1. Record a video of your swing using a smart device

[1931] The user launches the smartphone app and takes a photo of their golf swing.

[1932] It provides a function to upload videos taken by the device to a cloud server.

[1933] 2. Data analysis using a cloud server

[1934] The server analyzes the video data it receives and extracts data such as swing trajectory, head speed, and ball speed.

[1935] The server identifies swing issues based on the analysis results and quantifies those issues.

[1936] 3. Providing remedial measures

[1937] The server provides the user with specific improvement measures based on the quantified issues.

[1938] The proposed improvements are displayed on the app in a way that is easy for users to understand.

[1939] 4. Providing a social community

[1940] The server provides a membership platform where users can share information and opinions.

[1941] 5. Automatic response to inquiries

[1942] The server uses the generative AI model to automatically generate answers to user inquiries.

[1943] To resolve any doubts, users can enter their questions within the app and receive a response.

[1944] Program processing

[1945] The program of this system performs the following processing.

[1946] 1. Video recording and uploading

[1947] The user launches the app and shoots a video of their swing. By tapping the recording button, a video of the golf swing is recorded.

[1948] Once the recording is complete, the device prompts the user to review the video and displays an "Upload" button, which the user taps to send the data to the cloud server.

[1949] 2. Data Analysis

[1950] The server analyzes the received video data and tracks the trajectory of the golf club for each frame.

[1951] The server extracts analytical data such as head speed, ball speed, and swing trajectory and stores this data in the user's profile.

[1952] 3. Quantifying issues and providing solutions

[1953] Based on the analysis results, the server quantifies the swing issues and identifies specific problems (e.g., slow head speed, improper swing trajectory).

[1954] The server provides users with specific advice and training methods to solve these problems.

[1955] 4. Use of social communities

[1956] Users have access to a membership platform where they can interact with other users and post and comment on specific issues or topics.

[1957] The device updates the community activity feed in real time, allowing users to see new posts and comments immediately.

[1958] 5. Automatic response to inquiries

[1959] The user enters a question in the app and taps the send button. For example, a question could be, "What should I do if my club head speed isn't increasing?"

[1960] The server uses a generative AI model to generate optimal answers to questions and responds to users in real time. The answers are specific and practical, and include methods that users can try immediately.

[1961] Specific examples

[1962] As a concrete example, consider the following scenario.

[1963] 1. Recording and analyzing swing videos

[1964] Users can record videos of their swings in their own backyard using a smartphone app and upload them to a cloud server. The videos clearly capture the full swing.

[1965] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph.

[1966] 2. Quantifying issues and providing solutions

[1967] Based on the analysis results, the server identifies the problem as an outside-in swing trajectory, and also points out that the head speed is slower than the average professional golfer.

[1968] The server will provide specific suggestions for improvement, such as, "To practice your inside-out swing, try the following drill."

[1969] 3. Sharing information with the community

[1970] A user accesses a community platform and posts a message such as, "I'm looking for advice on drills to improve my swing trajectory."

[1971] Other users respond by saying, "I had the same problem and this drill worked for me."

[1972] 4. Automatic response to inquiries

[1973] A user asks within the app, "What training methods can I use to increase my club head speed?"

[1974] The server uses the generative AI model to generate answers such as, "Specific training methods for increasing club head speed include training using weights and specific swing drills," and immediately provides them to the user.

[1975] In this way, the present invention provides a comprehensive system for golfers to efficiently improve their own swing and share information and questions with other users.

[1976] The processing flow will be explained below.

[1977] Step 1:

[1978] The user launches the smartphone app and taps the "swing video recording" button.

[1979] Step 2:

[1980] The device activates the smartphone camera and captures a picture of the user performing a golf swing.

[1981] Step 3:

[1982] After the device has finished recording, the user is prompted to check the video and, if there are no problems, tap the "Upload" button.

[1983] Step 4:

[1984] In order for the terminal to upload the captured video file to the cloud server, the terminal transmits the video data using the communication module.

[1985] Step 5:

[1986] The video data received by the server is temporarily stored.

[1987] Step 6:

[1988] The server invokes a video analysis algorithm and begins the analysis process to extract data such as swing trajectory, head speed, and ball speed.

[1989] Example: A server performs image analysis on each video frame to track the movement of a golf club.

[1990] Example: The server performs trajectory analysis to analyze the ball's flight pattern.

[1991] Step 7:

[1992] The server stores the analysis results as numerical data and associates them with the user's profile.

[1993] Step 8:

[1994] The server uses the quantified analysis results to identify issues with the user's swing and trajectory.

[1995] Example: If your clubhead speed is 90 mph, compare it to the pro average.

[1996] Example: If your swing path is "outside-in," it will generate feedback for general path corrections.

[1997] Step 9:

[1998] The server will provide the following information to the user along with the analysis results:

[1999] Numerical data on the problem (e.g., head speed, swing trajectory)

[2000] Advice for improvement (e.g., how to practice an inside-out swing)

[2001] Video links and text guides for recommended drills and practice techniques

[2002] Step 10:

[2003] The user taps the Community tab to access the membership platform.

[2004] Step 11:

[2005] The device displays the user's activity feed, updating the list of posts and comments in real time.

[2006] Step 12:

[2007] A user taps the "Create a new post" button and enters a question or comment.

[2008] Step 13:

[2009] The terminal transmits the input post content to the server.

[2010] Step 14:

[2011] The server saves the new post to a database and displays it to other users.

[2012] Step 15:

[2013] The user enters the question they want to ask the generated AI model and taps the "Send inquiry" button.

[2014] Step 16:

[2015] The terminal sends the entered question to the server.

[2016] Step 17:

[2017] The server calls the generative AI model and generates the best answer to the input question.

[2018] Example: The server responds to the question "What causes your swing to curve to the right?" with the answer "Your right hand may be too strong, so try to relax it."

[2019] Step 18:

[2020] The server sends the generated answer to the terminal for display to the user.

[2021] Example 1

[2022] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2023] Conventional golf swing analysis systems require expensive equipment and specialized knowledge, making them difficult for many amateur golfers to use. Furthermore, the time and cost required for individual lessons and coaching advice makes self-improvement of a golf swing extremely difficult. Furthermore, there is a lack of systems that provide specific improvement measures based on the results of swing analysis, leaving a need for a means for users to efficiently improve their skills.

[2024] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2025] In this invention, the server includes a means for recording a user's actions using an application launched by the user, a means for uploading the recorded action data to the server, a means for analyzing the action data on the server and generating analysis results, a means for identifying action issues based on the analysis results and quantifying the issues, and a means for providing solutions to the issues. This allows users to easily record their own actions and obtain detailed analysis and specific solutions for improvement via the server without specialized knowledge. Furthermore, by sharing information with other users, users can receive feedback and support within the community, thereby enabling efficient and continuous skill improvement.

[2026] A "user" is an entity that uses the system to record its own actions, obtain analysis results, and receive improvement measures.

[2027] An "application" is software that runs on a smart device and allows users to record actions, upload data, and display analysis results.

[2028] "Movement" is a physical movement, such as a golf swing, made by a user that is recorded by the system.

[2029] A "server" is a computer system that receives and analyzes action data uploaded by users.

[2030] "Data" refers to the video of the user's actions recorded and the quantified information resulting from that analysis.

[2031] "Analysis" refers to the process of tracking the characteristics of movements based on recorded movement data and deriving quantified results.

[2032] "Issues" refer to problems or areas for improvement in user behavior that have become apparent as a result of the analysis.

[2033] "Quantifying" means expressing the characteristics and challenges of an action as quantitative data.

[2034] "Improvement measures" refer to specific methods and advice on how the user should improve the issues identified in the analysis results.

[2035] A "Platform" is an online venue where users can share information and communicate with other users.

[2036] "Artificial intelligence" is a program that automatically generates responses to user inquiries.

[2037] This invention is a system that uses a smart device to record a user's golf swing and analyzes the data on a cloud server to quantify the user's swing issues and provide specific improvement measures. This system is implemented using a smart device application, a cloud server, analysis software, and an artificial intelligence model.

[2038] Hardware used

[2039] Smart devices: Smartphones and tablets are used. These devices must have a camera and internet connection.

[2040] Cloud server: A computer system where video data is uploaded and where analysis processing is performed.

[2041] Software used

[2042] Application: Software for smart devices that allows users to record their golf swings and upload the data to a server. Specifically, it is implemented as an application for iOS or Android.

[2043] Analysis software: To analyze golf swing videos, we use libraries such as OpenCV and TensorFlow, which allow us to perform frame analysis and feature extraction of movements.

[2044] Generative AI model: An artificial intelligence model that generates automated responses to user inquiries. This uses natural language processing (NLP) techniques such as GPT-3 and BERT.

[2045] Processing flow

[2046] 1. Video recording and uploading

[2047] The user starts the smartphone app and shoots a video of their swing. An easy-to-use interface is provided for video recording.

[2048] After shooting is complete, the device displays a video confirmation screen to the user, allowing them to edit the video or cut out unnecessary parts.

[2049] When the user checks the video and taps the upload button, the device sends the video data to the cloud server.

[2050] 2. Data analysis and result generation

[2051] The server temporarily stores the received video data in storage, then begins motion analysis for each video frame, tracking the movement of the club and the body.

[2052] The server quantifies and extracts motion data such as head speed, ball speed, and swing trajectory, and stores it in the user's profile.

[2053] 3. Quantifying issues and providing solutions

[2054] The server identifies the user's swing issues based on the analysis results and quantifies the specific problems.

[2055] Using a generative AI model, specific improvement measures for quantified issues are automatically generated, and these improvement measures are displayed in the app in a format that is easy for users to understand.

[2056] 4. Use of social communities

[2057] Users can use the in-app community feature to share information and opinions with other users, and can post and comment on specific topics.

[2058] The device updates the community feed in real time, allowing you to see new posts and comments immediately.

[2059] 5. Automatic response to inquiries

[2060] When a user enters a question in the app and taps the submit button, the server uses a generative AI model to generate the best answer.

[2061] Answers are provided to users in real time and include specific, practical content. For example, in response to a question like, "How can I train to increase my clubhead speed?", the answer might be, "There are weight training exercises and specific swing drills."

[2062] Specific examples

[2063] Examples of swing video recording and analysis

[2064] Users can record videos of their swings in their own backyard using a smart device, and then upload the videos to a cloud server.

[2065] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph.

[2066] Based on the analysis results, the server identifies the problem of an outside-in swing trajectory and identifies the head speed as slower than the average for professionals. The server then provides specific improvements to address these issues.

[2067] Examples from the community

[2068] When a user posts something like "I'm looking for advice on drills to improve my swing path," other users will respond with things like "I was having the same problem, and this drill worked for me."

[2069] Specific examples of automated response to inquiries

[2070] When a user asks, "What training methods can I use to increase my clubhead speed?", the server uses a generative AI model to respond, "Specific training methods for increasing clubhead speed include weight training and specific swing drills."

[2071] In this way, the present invention provides a comprehensive system for golfers to efficiently improve their swing and share information and questions with other users.

[2072] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2073] Step 1:

[2074] The user launches the smart device application and selects the "Swing Photo" option from the home screen.

[2075] Input: Launching applications, displaying the home screen

[2076] Output: Show "Swing Shooting" option

[2077] What it does: The app invokes the camera function and displays a message encouraging the user to prepare for their swing.

[2078] Step 2:

[2079] The user taps the recording button to record a video of their golf swing.

[2080] Input: Tap the recording button, video data of the swing

[2081] Output: Recorded swing video data

[2082] Specific operation: The device camera will start up and record the user's swing. When the recording is finished, a save button will be displayed.

[2083] Step 3:

[2084] Users can check the video on the video confirmation screen and edit or cut out unnecessary parts.

[2085] Input: Recorded swing video, user editing actions

[2086] Output: Edited swing video data

[2087] What it does: The app provides video playback and editing functionality and displays an interface for the user to finalize the footage.

[2088] Step 4:

[2089] The user taps the upload button to send the edited video data to the cloud server.

[2090] Input: Edited swing video data, tap the upload button

[2091] Output: Video data stored on a cloud server

[2092] Specific operation: The device sends data to the cloud server via the Internet. Once the upload is complete, a confirmation message will be displayed.

[2093] Step 5:

[2094] The video data received by the server is temporarily stored in storage.

[2095] Input: Uploaded video data

[2096] Output: Saved video data

[2097] Specific operation: The server receives the video data and stores it in the file system. This storage is temporary storage for fast access.

[2098] Step 6:

[2099] The server starts motion analysis for each video frame, tracking and analyzing the movement of the golf club and the body.

[2100] Input: Saved video data

[2101] Output: Analyzed motion data (head speed, ball speed, swing trajectory, etc.)

[2102] How it works: The server uses libraries such as OpenCV and TensorFlow to analyze the images of each frame, and then uses a motion detection algorithm to extract club and body movement data.

[2103] Step 7:

[2104] The server quantifies the extracted behavioral data and stores it in the user's profile.

[2105] Input: Parsed motion data

[2106] Output: Stored quantified operating data

[2107] Specific operation: The analysis results are processed as numerical data and added to the user's profile database.

[2108] Step 8:

[2109] The server identifies the user's swing issues based on the analysis results and quantifies the specific problems.

[2110] Input: digitized motion data

[2111] Output: Quantified issues

[2112] Specific operation: The server uses the generative AI model to extract issues using specific algorithms and quantitatively evaluate them.

[2113] Step 9:

[2114] The server uses the generated AI model to automatically generate specific improvement measures for the quantified issues.

[2115] Input: Quantified task

[2116] Output: Generated remediation measures

[2117] How it works: The generative AI model references past data and training methods to suggest improvements based on the user's challenges, which may include text and links to images or videos.

[2118] Step 10:

[2119] The improvement measures generated by the server are sent to the user's smart device and displayed on the app.

[2120] Input: Generated remediation measures

[2121] Output: Improvement measures displayed on the app

[2122] Specific behavior: The app displays the improvement measures obtained from the server in the interface so that the user can take action immediately.

[2123] Step 11:

[2124] Users access the Community tab and share information with other users.

[2125] Enter: Access to the Community tab

[2126] Output: Shared information and comments

[2127] What it does: The app provides a post creation feature that allows users to share their challenges and improvements with the community, making it easy to receive feedback and advice.

[2128] Step 12:

[2129] The user enters a question in the in-app inquiry tab and submits it.

[2130] Input: User question

[2131] Output: Query data sent to the server

[2132] Specific behavior: Provides an interface for the app to send entered questions to the server.

[2133] Step 13:

[2134] The server uses a generative AI model to generate optimal answers and provides them to the user in real time.

[2135] Input: Question data from the user

[2136] Output: The generated answer

[2137] Specific operation: The server uses the generative AI model to generate an appropriate answer based on the question and sends it in text format to the user's device.

[2138] Step 14:

[2139] The user references the answers provided and takes various actions to improve their swing.

[2140] Input: Generated Answer

[2141] Output: Actions taken

[2142] Specific Actions: Plan practice sessions or attempt specific training drills based on the answers the user generates.

[2143] (Application example 1)

[2144] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2145] Conventional golf swing analysis systems not only provide users with analysis results, but also make it difficult to share information with other users in real time or receive actual support at a golf club. Furthermore, there is a lack of advice on selecting appropriate golf equipment based on swing analysis results, leaving users lacking the information and tools to instantly improve their skills. This often means that users need time to improve their swing, and it is difficult to receive appropriate guidance and support during the process.

[2146] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2147] In this invention, the server includes means for recording a user's swing motion using a smart computing device, means for transmitting the recorded swing motion data to a remote server, means for analyzing the swing motion data in the remote server and generating analysis results, means for identifying problems in the swing motion based on the analysis results and quantifying the problems, means for providing solutions to the problems, means for sharing the analysis results with other users in the store in real time, and means for providing a function for making inquiries to store staff. This allows users to check the swing analysis results in real time in the store, share experiences and information with other users, and immediately receive appropriate advice from store staff.

[2148] A "smart computing device" is an electronic device that is portable by the user and has a high-resolution camera and internet connectivity that is capable of recording and transmitting swing motion.

[2149] "User" refers to a user who seeks to improve their golf swing using the system of the present invention.

[2150] The term "swing motion" refers to a series of physical movements that are made when using a golf club to hit a ball, including the trajectory of the golf club, head speed, input force, and the like.

[2151] "Remote Server" refers to a data center that is a data analysis and storage system accessible via the Internet and that analyzes the swing video sent by the User and provides the results.

[2152] The "analysis results" are technical evaluations and quantified data regarding the user's swing, generated by the server based on swing motion data.

[2153] "Quantification" refers to expressing various parameters of the analysis results as quantitative values, allowing users to concretely recognize their own performance.

[2154] "Improvement measures" refer to specific training methods and technical advice provided to address swing movement issues identified based on the analysis results.

[2155] "Real-time" refers to the near-instantaneous transmission of data, analysis, provision of results, and sharing of information.

[2156] "Information sharing" refers to users exchanging data such as analysis results, experiences, and opinions with other users and store staff.

[2157] The "inquiry function" refers to a system that allows users to input questions about analysis results and improvement measures and receive an automatic response or a reply from store staff on the spot.

[2158] This invention is a system for use in golf equipment stores that allows customers to receive real-time analysis and feedback on their swings. The system includes a smart computing device, a remote server, and a social community platform for sharing analysis results. It also provides query capabilities using generative AI models.

[2159] System Overview

[2160] 1. Video recording and uploading

[2161] Users use a smart computing device (such as a smartphone) to film their swing, and once filming is complete, the video data is automatically transmitted to a remote server.

[2162] 2. Data Analysis

[2163] The server analyzes the received swing video and extracts data such as the golf club trajectory, head speed, ball speed, etc. These analysis results are quantified and recorded in the user profile.

[2164] 3. Providing Feedback

[2165] The server quantifies the identified issues based on the analysis results and indicates specific areas for improvement to the user. The improvement measures are sent in real time to a smart computing device and displayed to the user.

[2166] 4. Social Community

[2167] Users can share analysis results with other customers in the store in real time, exchange information, and hold discussions and provide feedback on specific topics.

[2168] 5. Inquiry function

[2169] When a user inputs a question about improvement measures or analysis results into the server, the optimal answer is automatically generated using a generative AI model, and this answer is immediately provided to the user.

[2170] Hardware and Software Configuration

[2171] Smart computing devices: Smartphones and tablets equipped with high-resolution cameras and internet connectivity.

[2172] Remote server: Use cloud services such as Amazon Web Services (AWS) and Google Cloud Platform (GCP).

[2173] Data analysis software: Video data was analyzed using Python and OpenCV.

[2174] Generative AI model: Generates automatic responses to user inquiries using natural language processing.

[2175] Examples of concrete examples and prompts

[2176] Examples:

[2177] 1. The user takes a photo of their golf swing with their smartphone in the practice area inside the store.

[2178] 2. The captured video is automatically uploaded to a cloud server, where analysis begins.

[2179] 3. The server quantifies the analysis results (e.g., head speed, swing trajectory, etc.) and immediately sends them to your smartphone.

[2180] 4. The user checks the analysis results on the spot and continues practicing according to the provided improvement measures.

[2181] 5. When users ask questions about problems with their swing, the generative AI model provides specific answers. For example, in response to a question like, "What can I do if my club head speed isn't improving?", the model might respond with, "Training with weights and specific swing drills are effective."

[2182] Example prompt sentence:

[2183] "A user has uploaded a video of their swing taken in a golf shop. Please analyze this video and provide data on the following: 1. Head speed 2. Swing path 3. Ball speed. Also, based on this data, please provide specific advice on how to improve their swing."

[2184] In this way, the present invention provides a comprehensive support system for users to efficiently analyze and improve their swings and share information within a golf equipment store.

[2185] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2186] Step 1:

[2187] A user uses a smart computing device (such as a smartphone) to film their swing motion.

[2188] Input: A smartphone for users to record swing videos.

[2189] Data processing / calculation: Video is taken using a high-resolution camera.

[2190] Output: Recorded swing video file.

[2191] Step 2:

[2192] The device uploads the captured swing video to a cloud server.

[2193] Input: Recorded swing video file.

[2194] Data processing / calculation: Send the video file to the cloud server using an HTTP request.

[2195] Output: Video files stored on cloud server.

[2196] Step 3:

[2197] The server analyzes the received swing video and extracts swing motion data.

[2198] Input: Video files stored on a cloud server.

[2199] Data processing / calculation: Using Python and OpenCV, information such as golf club trajectory, head speed, and ball speed is extracted from video data.

[2200] Output: Swing motion analysis results (e.g. head speed, swing path, ball speed).

[2201] Step 4:

[2202] The server identifies issues in the swing motion based on the analysis results and quantifies those issues.

[2203] Input: Analysis results of swing motion.

[2204] Data processing / calculation: The analysis results are applied to an evaluation algorithm to quantitatively express the identified issues.

[2205] Output: Quantified swing motion issues (e.g., slow clubhead speed, inaccurate swing path).

[2206] Step 5:

[2207] The server generates solutions to the problems and provides them to the user.

[2208] Input: Quantified swing movement tasks.

[2209] Data processing / computation: Running algorithms to generate specific training or technical advice to address the problem.

[2210] Output: Specific improvement measures (e.g., training methods to increase clubhead speed, drills to improve swing path).

[2211] Step 6:

[2212] The device displays analysis results and improvement measures to the user in real time.

[2213] Input: Specific improvement measures.

[2214] Data processing / calculation: Rendering is performed to display the results and improvement measures on the smartphone UI.

[2215] Output: Analysis results and remediation measures displayed on the user's screen.

[2216] Step 7:

[2217] Users can share analysis results with other customers in the store and exchange information.

[2218] Input: Analysis results and remediation measures.

[2219] Data processing / computation: Use social community platforms to share data and post comments and feedback.

[2220] Output: Information exchanged with other users.

[2221] Step 8:

[2222] Users ask questions about analysis results and improvement measures, and the server automatically responds using the generated AI model.

[2223] Input: A question from the user (e.g., "What training methods can I use to increase my club head speed?").

[2224] Data processing / calculation: Input a prompt sentence into the generative AI model to generate the optimal answer.

[2225] Output: The automated response that is provided to the user.

[2226] Specific prompt examples:

[2227] "A user has uploaded a video of their swing taken in a golf shop. Please analyze this video and provide data on the following: 1. Head speed 2. Swing path 3. Ball speed. Also, based on this data, please provide specific advice on how to improve their swing."

[2228] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2229] This invention is a system that uses a smart device to record, analyze, and improve a golf swing, and also combines it with user emotion recognition. This system quantifies the user's swing issues through swing video recording and data analysis on a cloud server, and provides specific improvement measures. It also uses an emotion engine to grasp the user's psychological state and provides feedback based on that to support more effective swing improvement.

[2230] System Overview

[2231] 1. Record a video of your swing using a smart device

[2232] The user launches the smartphone app and takes a photo of their golf swing.

[2233] It provides a function to upload videos taken by the device to a cloud server.

[2234] 2. Data analysis using a cloud server

[2235] The server analyzes the video data it receives and extracts data such as swing trajectory, head speed, and ball speed.

[2236] The server identifies swing issues based on the analysis results and quantifies those issues.

[2237] 3. Providing remedial measures

[2238] The server provides the user with specific improvement measures based on the quantified issues.

[2239] The proposed improvements are displayed on the app in a way that is easy for users to understand.

[2240] 4. Emotion recognition

[2241] The device records the user's facial expressions and voice during the swing and uploads them to a cloud server.

[2242] The server uses an emotion engine to analyze emotions from the user's facial expressions and voice.

[2243] 5. Providing Feedback

[2244] Based on the emotion recognition results, the server generates improvement measures and advice that take into account the user's psychological state.

[2245] The server also incorporates the user's emotional data into the analysis results to provide more accurate feedback.

[2246] 6. Providing social communities

[2247] The server provides a membership platform where users can share information and opinions.

[2248] 7. Auto-response to inquiries

[2249] The server uses the generative AI model to automatically generate answers to user inquiries.

[2250] To resolve any doubts, users can enter their questions within the app and receive a response.

[2251] Program processing

[2252] The program of this system performs the following processing.

[2253] 1. Video recording and uploading

[2254] The user launches the app and shoots a video of their swing. By tapping the recording button, a video of the golf swing is recorded.

[2255] Once the recording is complete, the device prompts the user to review the video and displays an "Upload" button, which the user taps to send the data to the cloud server.

[2256] 2. Data Analysis

[2257] The server analyzes the received video data and tracks the trajectory of the golf club for each frame.

[2258] The server extracts analytical data such as head speed, ball speed, and swing trajectory and stores this data in the user's profile.

[2259] 3. Quantifying issues and providing solutions

[2260] Based on the analysis results, the server quantifies the swing issues and identifies specific problems (e.g., slow head speed, improper swing trajectory).

[2261] The server provides users with specific advice and training methods to solve these problems.

[2262] 4. Emotion Recognition and Feedback

[2263] When a user shoots a video of their swing, their facial expressions and voice are also recorded at the same time.

[2264] The device sends this facial expression and voice data to a cloud server.

[2265] The server uses an emotion engine to analyze the user's facial expressions and voice to determine their emotions. For example, if the user is nervous while taking a photo, the server can detect that emotion.

[2266] Based on the emotion recognition results, the server generates improvement measures and advice that take into account the user's psychological state. For example, it could provide advice such as "Try breathing techniques to help you swing more relaxed."

[2267] The server also incorporates the user's emotional data into the analysis results to provide more accurate feedback.

[2268] 5. Use of social communities

[2269] The user taps the Community tab to access the membership platform.

[2270] The device displays the user's activity feed, updating the list of posts and comments in real time.

[2271] A user taps the "Create a new post" button and enters a question or comment.

[2272] The terminal transmits the input post content to the server.

[2273] The server saves the new post to a database and displays it to other users.

[2274] 6. Automatic response to inquiries

[2275] The user enters the question they want to ask the generated AI model and taps the "Send inquiry" button.

[2276] The terminal sends the entered question to the server.

[2277] The server calls up the generative AI model and generates the optimal answer to the input question. For example, in response to the question, "What kind of training is effective for increasing club head speed?", it provides a specific training method.

[2278] The server sends the generated answer to the terminal for display to the user.

[2279] Specific examples

[2280] As a concrete example, consider the following scenario.

[2281] 1. Recording and analyzing swing videos

[2282] Users can record videos of their swings in their own backyard using a smartphone app and upload them to a cloud server. The videos include the full swing, as well as the user's facial expressions and voice.

[2283] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph. The emotion engine also recognizes that the user is tense during the swing.

[2284] 2. Quantifying issues and providing solutions

[2285] Based on the analysis results, the server identifies that the swing trajectory is outside-in as the problem, and points out that the head speed is slower than the average for professionals as an issue.

[2286] The server will provide specific improvement measures, such as "Try the following drill to practice your inside-out swing," and will also provide advice including "breathing techniques and mental training to relieve tension."

[2287] 3. Sharing information with the community

[2288] A user accesses a community platform and posts a message such as, "I'm looking for advice on drills to improve my swing trajectory."

[2289] Other users respond by saying, "I had the same problem and this drill worked for me."

[2290] 4. Automatic response to inquiries

[2291] A user asks within the app, "What kind of training is effective for increasing club head speed?"

[2292] The server uses the generative AI model to generate specific answers such as "Training with weights and specific swing drills are effective in increasing club head speed," and immediately provides them to the user.

[2293] In this way, the present invention provides a comprehensive system that allows golfers to efficiently improve their own swing, while also taking care of their psychological aspects, and allows them to share information and solve questions with other users.

[2294] The processing flow will be explained below.

[2295] Step 1:

[2296] The user launches the smartphone app and taps the "swing video recording" button.

[2297] Step 2:

[2298] The device activates the smartphone's camera and microphone to capture images and audio of the user performing a golf swing.

[2299] Step 3:

[2300] After the device has finished recording, the user is prompted to check the video and, if there are no problems, tap the "Upload" button.

[2301] Step 4:

[2302] The device transmits the data using a communication module to upload the captured video files and audio data to a cloud server.

[2303] Step 5:

[2304] The server temporarily stores the received video and audio data.

[2305] Step 6:

[2306] The server invokes a video analysis algorithm and begins the analysis process to extract data such as swing trajectory, head speed, and ball speed.

[2307] Example: A server performs image analysis on each video frame to track the movement of a golf club.

[2308] Example: The server performs trajectory analysis to analyze the ball's flight pattern.

[2309] Step 7:

[2310] The server uses an emotion engine to analyze the user's facial expressions in audio data and video to identify emotions.

[2311] Step 8:

[2312] The server stores the analysis results as numerical data and associates them with the user's profile.

[2313] Step 9:

[2314] The server uses the quantified analysis results to identify issues with the user's swing and trajectory.

[2315] Example: If your clubhead speed is 90 mph, compare it to the pro average.

[2316] Example: If your swing path is "outside-in," it will generate feedback for general path corrections.

[2317] Example: Identifying that a user is nervous based on facial and voice analysis.

[2318] Step 10:

[2319] The server will provide the following information to the user along with the analysis results:

[2320] Numerical data on the problem (e.g., head speed, swing trajectory)

[2321] Advice for improvement (e.g., how to practice an inside-out swing)

[2322] Emotion-based advice (e.g., breathing techniques for a relaxed swing)

[2323] Video links and text guides for recommended drills and practice techniques

[2324] Step 11:

[2325] The user taps the Community tab to access the membership platform.

[2326] Step 12:

[2327] The device displays the user's activity feed, updating the list of posts and comments in real time.

[2328] Step 13:

[2329] A user taps the "Create a new post" button and enters a question or comment.

[2330] Step 14:

[2331] The terminal transmits the input post content to the server.

[2332] Step 15:

[2333] The server saves the new post to a database and displays it to other users.

[2334] Step 16:

[2335] The user enters the question they want to ask the generated AI model and taps the "Send inquiry" button.

[2336] Step 17:

[2337] The terminal sends the entered question to the server.

[2338] Step 18:

[2339] The server calls the generative AI model and generates the best answer to the input question.

[2340] Example: The server responds to the question "What causes your swing to curve to the right?" with the answer "Your right hand may be too strong, so try to relax it."

[2341] Step 19:

[2342] The server sends the generated answer to the terminal for display to the user.

[2343] Example 2

[2344] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2345] Conventional golf swing analysis systems focus on analyzing technical issues in a user's swing and providing solutions for improvement, but do not consider psychological aspects. As a result, if a user feels nervous or stressed, this may not lead to actual performance improvement. Furthermore, smoother support is needed for sharing information with other users and resolving questions.

[2346] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2347] In this invention, the server includes a means for recording the user's facial expressions and voice during the swing, a means for uploading the recorded facial and voice data to a cloud server, and a means for analyzing the user's emotions using an emotion recognition engine in the cloud server. This enables feedback that takes the user's psychological state into consideration. The system also includes a means for identifying and quantifying issues in the swing based on the analysis results, and a means for automatically responding to inquiries from the user using a generative AI model, enabling comprehensive support for swing improvement from both technical and psychological aspects.

[2348] A "smart device" refers to an electronic device that can be carried by a user and that can connect to the Internet and run various applications. Examples include smartphones and tablets.

[2349] A "cloud server" refers to a remote server that can store, analyze, and communicate data over the Internet.

[2350] An "analysis algorithm" refers to a mathematically and logically defined method of defining rules and procedures for extracting, classifying, and evaluating useful information from specific data.

[2351] An "emotion recognition engine" refers to technology that analyzes non-verbal data such as a user's facial expressions and voice to identify their emotional state.

[2352] "Analysis results" refers to the specific numerical values ​​and evaluation results obtained after analyzing swing movements, facial expressions, and audio data.

[2353] "Means for quantifying problems" refers to methods and tools for quantitatively expressing problems in a user's swing motion from the analysis results.

[2354] The "means for providing improvement measures" refers to a method or system for proposing specific instruction content or training methods to the user for the identified problems in the swing motion.

[2355] A "generative AI model" refers to a mathematical model that uses artificial intelligence to automatically generate answers and advice to user inquiries.

[2356] A "community platform" refers to an online space where multiple users can share information, exchange opinions, ask questions, and answer questions.

[2357] This invention is a system that uses a smart device to record, analyze, and improve golf swings, and also combines it with user emotion recognition. The system starts when a user uses a smart device such as a smartphone to record a video of their golf swing and uploads the video data to a cloud server.

[2358] Hardware and software used:

[2359] Smart Device: An electronic device that is portable and has internet connectivity, such as a smartphone or tablet.

[2360] Cloud server: A remote server that stores, analyzes, and communicates data.

[2361] Analysis algorithm: Using image analysis libraries such as OpenCV, the swing trajectory, head speed, ball speed, etc. are analyzed for each frame.

[2362] Emotion recognition engine: Technology that analyzes emotions from a user's facial expressions and voice, such as Microsoft Azure's Emotion API.

[2363] Generative AI models: Use artificial intelligence models such as GPT-4 to automatically generate answers to user inquiries.

[2364] The specific operation steps of the system:

[2365] 1. Video recording and uploading

[2366] The user starts the smartphone app and records a swing video. They tap the record button to start recording and the end button to stop recording.

[2367] If the user checks the video and there are no problems, they tap the upload button to send the video to the cloud server.

[2368] 2. Data Analysis

[2369] The server receives the uploaded video data and uses an analysis algorithm to extract data such as swing trajectory, head speed, and ball speed for each frame.

[2370] The extracted data is stored in a profile for each user.

[2371] 3. Quantifying issues and providing solutions

[2372] The server then quantifies swing issues based on the analysis results. For example, if the head speed is less than 85 mph, it will be listed as a "low head speed" issue.

[2373] The server generates specific improvement measures for each quantified issue and displays them in the user's app, such as "training methods to improve club head speed."

[2374] 4. Emotion Recognition and Feedback

[2375] When a user shoots a video of their swing, their facial expressions and voice are also recorded at the same time.

[2376] The device transmits facial expression and voice data to a cloud server.

[2377] The server uses an emotion recognition engine to analyze the user's emotions, for example, detecting that the user is nervous during the shoot.

[2378] Based on the analysis results and emotional data, the server generates comments and improvement measures that take into account the user's psychological state and provides them to the user's smart device, such as "breathing techniques for relaxation."

[2379] 5. Use of social communities

[2380] Users access the platform by tapping the community tab within the smartphone app.

[2381] The device displays posts and comments in real time and provides an interface for users to post questions and opinions.

[2382] The server stores new posts and comments in a database, making them available for all users to see.

[2383] 6. Automatic response to inquiries

[2384] The user enters a question in the app and taps the "Send Inquiry" button.

[2385] The device sends the question to the server, which then uses a generative AI model (e.g., GPT-4) to generate the optimal answer.

[2386] The server displays the generated answer on the user's smart device.

[2387] Specific operation example

[2388] 1. Recording and analyzing swing videos

[2389] Users can record videos of their swings in their own backyard using their smartphones and upload them to a cloud server. The videos include the swing, the user's facial expressions, and their voice.

[2390] The server analyzes the video and determines the head speed is 85 mph, the swing path is outside-in, and the ball speed is 120 mph.

[2391] In addition, the emotion recognition engine detects whether the user is nervous during the swing.

[2392] 2. Quantifying issues and providing solutions

[2393] The server identifies the problem as slow head speed and swing trajectory, and then provides the user with specific improvement measures, such as drills to practice an inside-out swing and breathing techniques to relieve tension.

[2394] 3. Sharing information with the community

[2395] A user accesses a community platform and posts something like, "I'm looking for advice on improving my swing."

[2396] Other users will reply saying, "This drill was effective."

[2397] 4. Automatic response to inquiries

[2398] A user asks, "How can I train to increase my club head speed?"

[2399] The server uses the generative AI model to generate answers such as "Training with weights and specific swing drills are effective" and provides them to the user.

[2400] In this way, the present invention can comprehensively support the user's swing improvement and provide feedback that also includes psychological aspects.In addition, the information sharing function with other users and the automatic response function to inquiries create an efficient and effective training environment.

[2401] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2402] Step 1:

[2403] The user launches the smartphone app and records a video of their swing.

[2404] Input: Start of shooting by user operation

[2405] Data processing: Record a swing video using a smartphone camera

[2406] Output: Recorded swing video file

[2407] Specific behavior:

[2408] The user launches the app and switches to shooting mode.

[2409] The user taps the record button to begin recording the golf swing.

[2410] The user taps the end button to end the recording.

[2411] Step 2:

[2412] The device uploads the captured video to a cloud server.

[2413] Input: Recorded swing video file

[2414] Data processing: Transfer video data to the cloud server

[2415] Output: Video data on a cloud server

[2416] Specific behavior:

[2417] The device will prompt the user to review the video and display an "Upload" button.

[2418] When the user taps the upload button, the device uses its internet connection to send the video file to a cloud server.

[2419] Step 3:

[2420] The server analyzes the video data it receives and extracts data such as swing trajectory, head speed, and ball speed.

[2421] Input: Video data on a cloud server

[2422] Data processing: Perform video analysis using analysis algorithms such as OpenCV

[2423] Output: Analysis data such as swing trajectory, head speed, ball speed, etc.

[2424] Specific behavior:

[2425] The server analyzes the video data frame by frame.

[2426] The server extracts data such as swing trajectory, head speed, and ball speed and stores it in the user's profile.

[2427] Step 4:

[2428] The server quantifies swing issues based on the analysis results and generates specific improvement measures.

[2429] Input: Swing trajectory, head speed, ball speed data

[2430] Data processing: Applying algorithms to quantify swing problems using analytical data

[2431] Output: Quantified issues and concrete improvement measures

[2432] Specific behavior:

[2433] The server identifies swing issues based on the analysis results.

[2434] The server generates specific improvement measures and training methods based on the issues.

[2435] The server sends the generated advice to the user's app.

[2436] Step 5:

[2437] The device records the user's facial expressions and voice and uploads them to a cloud server.

[2438] Input: Photographed facial expressions and voice data

[2439] Data processing: Send facial expression and voice data to a cloud server

[2440] Output: Facial expression and voice data on a cloud server

[2441] Specific behavior:

[2442] When the device shoots a swing video, it also records the user's facial expressions and voice at the same time.

[2443] The device uploads facial expression and voice data to a cloud server.

[2444] Step 6:

[2445] The server uses an emotion recognition engine to analyze emotions from the user's facial expressions and voice.

[2446] Input: Facial expression and voice data on a cloud server

[2447] Data processing: Emotion analysis using an emotion recognition engine (e.g., Emotion API)

[2448] Output: User emotion data

[2449] Specific behavior:

[2450] The server inputs facial and voice data into an emotion recognition engine.

[2451] The server identifies the user's emotions (e.g., tension, relaxation, etc.).

[2452] Step 7:

[2453] The server reflects the emotional data in the analysis results and generates appropriate feedback that takes the user's psychological state into account.

[2454] Input: Analysis data and emotion data

[2455] Data processing: Adding emotional data to the analysis results and adjusting improvement measures and advice

[2456] Output: Feedback that takes into account the user's psychological state

[2457] Specific behavior:

[2458] If the server detects tension, it generates feedback that includes "breathing techniques to relax."

[2459] The server sends the generated feedback to the user's app.

[2460] Step 8:

[2461] Users access the community platform to share information and post questions.

[2462] Input: User-submitted content

[2463] Data processing: Save the posted content in a database and share it with other users

[2464] Output: Comments and replies by other users

[2465] Specific behavior:

[2466] A user taps the Community tab to access the platform.

[2467] The device displays the latest posts and comments and sends the user's new posts to the cloud server.

[2468] The server stores the posts in a database and makes them available for all users to view.

[2469] Step 9:

[2470] The user asks a question using the generative AI model, and the server generates an automated response.

[2471] Input: Question asked by the user (prompt text)

[2472] Data processing: Using a generative AI model (e.g., GPT-4) to generate optimal answers

[2473] Output: The generated answer

[2474] Specific behavior:

[2475] The user enters a question in the app and taps the "Send Inquiry" button.

[2476] The device sends the question to the cloud server.

[2477] The server uses a generative AI model to generate the best answer to the question.

[2478] The server generates the answer and sends it to the user, who displays it in the app.

[2479] In this way, a system is constructed that comprehensively supports the user in improving their swing by performing data processing and calculations based on the input data at each step and then using the resulting output to proceed to the next processing step.

[2480] (Application example 2)

[2481] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2482] Existing factory robot motion analysis systems lack the functionality to quantify motion efficiency and accuracy and provide specific improvement measures. Furthermore, they do not provide feedback that takes into account the psychological state of the worker or suggest improvements to the work environment, resulting in insufficient improvements to the performance of both the robot and the worker. This makes it difficult to create an efficient and stress-free work environment.

[2483] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recording the user's robot operations using a smart device, means for uploading the recorded operation data to a cloud server, means for analyzing the operation data and generating analysis results in the cloud server, means for identifying operation issues based on the analysis results and quantifying the issues, means for providing improvements to the issues, means for uploading emotional data recorded during the operations to the cloud server, means for analyzing the emotional data in the cloud server and identifying the user's psychological state, and means for providing feedback based on the psychological state. This makes it possible to scientifically improve the operational efficiency and accuracy of factory robots and provide feedback and improve the work environment that takes into account the psychologi...

Claims

1. A means for recording a user's swing motion using a smart device; A means for uploading the recorded swing motion data to a cloud server; A means for analyzing swing motion data in a cloud server and generating an analysis result; A means for identifying problems in swing movements based on the analysis results and quantifying those problems; A means of providing remedial measures for the issues; A system including:

2. means for analyzing the trajectory of the ball based on the recorded swing motion data; A means for identifying problems in the analyzed swing motion based on the trajectory analysis results; The system of claim 1 further comprising:

3. means for providing a community platform where users can share information with other users; A means for automatically responding to inquiries from users using artificial intelligence; The system of claim 1 further comprising:

Images