Image analysis-based golf swing analysis device and control method thereof

Abstract

The present disclosure relates to an image analysis-based golf swing analysis device. The image analysis-based golf swing analysis device may: acquire video captured of a user's golf swing through a camera; compare each of neighboring frames in the image; extract a plurality of first images for respective regions corresponding to differences between the frames on the basis of the results of the comparisons; generate an analysis result for the golf swing on the basis of the plurality of extracted first images; generate a second image by combining the plurality of extracted first images; and output the generated second image and the analysis result.

Description

Image analysis-based golf swing analysis device and control method thereof

[0001] The present disclosure relates to a golf swing analysis device, and more specifically, to a device that photographs a golf swing scene using a camera and analyzes the golf swing through the same.

[0002] Recently, not only professional golfers but also ordinary people are increasingly using golf lessons to improve their skills, and are investing time in separate practice outside of golf lessons.

[0003] To receive feedback on your golf swing during these personal practice sessions, various devices and methods are being proposed, including golf training machines, golf simulators, golf swing guides, golf swing correction devices, weight transfer sensors, and putting practice mats. Among these, golf swing analysis devices help both beginners and experts practice and master accurate swings to improve their skills.

[0004] These conventional golf swing analysis devices use multiple cameras, such as halogen lights, attached to the ceiling to capture the user's golf swing and provide analysis results. However, due to design issues, they can only be used in certain spaces and are significantly less portable.

[0005] Accordingly, a golf swing analysis device that can be implemented in a form that users can easily carry is needed, but such technology is not currently available.

[0006] The purpose of the embodiment disclosed in the present disclosure is to provide a golf swing analysis device based on image analysis.

[0007] In addition, the embodiment disclosed in the present disclosure aims to provide a golf swing analysis device that can capture a video of a user's golf swing using one camera and generate a composite image that can represent the user's golf swing trajectory by comparing image frames within the video.

[0008] In addition, the embodiment disclosed in the present disclosure aims to provide a golf swing analysis device capable of generating prediction results for the quality and distance of a golf swing by analyzing multiple frames of a user's golf video.

[0009] The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0010] In order to solve the above-described problem, an image analysis-based golf swing analysis device according to one embodiment of the present disclosure comprises: a camera; a processor for obtaining an image of a user's golf swing captured through the camera, comparing neighboring frames within the image, extracting a plurality of first images for respective areas corresponding to differences between the frames based on the comparison results, combining the plurality of extracted first images to generate a second image, generating an analysis result for the golf swing based on the generated second image, and controlling the generated analysis result to be output, wherein the analysis result includes a prediction result for a ball quality and a distance due to the golf swing.

[0011] In addition, a method for controlling a golf swing analysis device based on image analysis according to an embodiment of the present disclosure for solving the above-described problem is a method performed by a computing device, comprising the steps of: obtaining an image of a user's golf swing captured by a camera; comparing neighboring frames within the image; extracting a plurality of first images for respective areas corresponding to differences between the frames based on the results of the comparison; generating a second image by combining the plurality of extracted first images; generating an analysis result for the golf swing based on the generated second image; and outputting the generated analysis result.

[0012] In addition, a computer program stored in a computer-readable recording medium for executing a method for implementing the present disclosure may be further provided.

[0013] In addition, a computer-readable recording medium recording a computer program for executing a method for implementing the present disclosure may be further provided.

[0014] According to the above-described problem solving means of the present disclosure, an effect of providing a golf swing analysis device is provided.

[0015] In addition, according to the aforementioned problem solving means of the present disclosure, a composite image capable of representing the user's golf swing trajectory can be generated by taking a video of the user's golf using one camera and comparing image frames within the video.

[0016] In addition, the aforementioned problem solving means of the present disclosure provides an effect of generating prediction results for the quality and distance of a golf swing by analyzing multiple frames of a user's golf video.

[0017] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0018] FIG. 1 is a schematic diagram of an image analysis-based golf swing analysis system according to an embodiment of the present disclosure.

[0019] FIG. 2 is a block diagram of a golf swing analysis device based on image analysis according to an embodiment of the present disclosure.

[0020] FIG. 3 is a flowchart of a control method of an image analysis-based golf swing analysis device according to an embodiment of the present disclosure.

[0021] FIG. 4 is a diagram illustrating an operation process of a golf swing analysis device based on image analysis according to an embodiment of the present disclosure.

[0022] Figure 5 is a flowchart illustrating the process of recognizing the first frame in a video of a golf swing and performing homography.

[0023] Figures 6 and 7 are diagrams illustrating a second image that is a composite of multiple first images extracted from differences between frames within a captured video.

[0024] Figure 8 is a drawing illustrating the exclusion of parts judged as noise in Figure 6.

[0025] Figure 9 is a drawing illustrating the exclusion of parts judged as noise in Figure 7.

[0026] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and any content that is common in the technical field to which this disclosure pertains or that overlaps between embodiments is omitted. The terms "part, module, element, block" used in the specification may be implemented in software or hardware, and depending on the embodiments, multiple "parts, modules, elements, blocks" may be implemented as a single component, or a single "part, module, element, block" may include multiple components.

[0027] Throughout the specification, when a part is said to be "connected" to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection via a wireless communication network.

[0028] Additionally, when a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0029] Throughout the specification, when we say that an element is "on" another element, this includes not only cases where the element is in contact with the other element, but also cases where another element exists between the two elements.

[0030] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0031] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0032] The identification codes for each step are used for convenience of explanation and do not describe the order of each step. Each step may be performed in a different order than specified unless the context clearly indicates a specific order.

[0033] The operating principle and embodiments of the present disclosure are described below with reference to the attached drawings.

[0034] In this specification, the term "device according to the present disclosure" includes various devices capable of performing computational processing and providing results to a user. For example, the device according to the present disclosure may include a computer, a server device, and a portable terminal, or may be in the form of any one of them.

[0035] Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

[0036] The above server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0037] The above portable terminal may include, for example, all kinds of handheld-based wireless communication devices such as PCS, GSM, PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, smart phones, etc., as well as wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMD).

[0038] The artificial intelligence-related functions according to the present disclosure are operated via a processor and memory. The processor may be comprised of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a DSP (Digital Signal Processor), a graphics-only processor such as a GPU or a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU. One or more processors control the processing of input data according to predefined operating rules or artificial intelligence models stored in memory. Alternatively, if one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0039] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the basic artificial intelligence model is learned by a learning algorithm using a plurality of learning data, thereby creating the predefined operation rules or artificial intelligence models set to perform a desired characteristic (or purpose). This learning may be performed on the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0040] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weights, and performs neural network operations through operations between the operation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), and examples thereof include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q-network.

[0041] According to an exemplary embodiment of the present disclosure, a processor can implement artificial intelligence. Artificial intelligence refers to a machine learning method based on an artificial neural network that mimics human neurons (biological neurons) to enable machines to learn. Artificial intelligence methodologies can be categorized into supervised learning, where input and output data are provided together as training data, thereby determining the solution (output data) to a problem (input data); unsupervised learning, where only input data is provided without output data, so that the solution (output data) to a problem (input data) is not determined; and reinforcement learning, where a reward is provided from an external environment each time an action is taken in the current state, and learning proceeds in a direction that maximizes this reward. Furthermore, artificial intelligence methodologies can be categorized by the architecture of the learning model. The architectures of widely used deep learning technologies can be categorized into convolutional neural networks, recurrent neural networks, transformers, and generative adversarial networks.

[0042] The device may include an artificial intelligence model. The artificial intelligence model may be a single artificial intelligence model or may be implemented as multiple artificial intelligence models. The artificial intelligence model may be composed of a neural network (or artificial neural network) and may include statistical learning algorithms that mimic biological neurons in machine learning and cognitive science. A neural network may refer to a model in general that has problem-solving capabilities by changing the binding strength of synapses through learning, formed by artificial neurons (nodes) that form a network by combining synapses. The neurons of the neural network may include a combination of weights or biases. The neural network may include one or more layers composed of one or more neurons or nodes. For example, the device may include an input layer, a hidden layer, and an output layer. The neural network constituting the device can infer a desired outcome from an arbitrary input by changing the weights of the neurons through learning.

[0043] The processor can create a neural network, train (or learn) a neural network, perform a calculation based on received input data, generate an information signal based on the calculation result, or retrain the neural network. The models of the neural network can include various types of models such as CNN, R-CNN, RPN, RNN, S-DNN, S-SDNN, Deconvolution Network, DBN, RBM, Fully Convolutional Network, LSTM Network, Classification Network, etc., such as GoogleNet, AlexNet, VGG Network, etc., but are not limited thereto. The processor can include one or more processors for performing calculations according to the models of the neural network. For example, the neural network can include a deep neural network.

[0044] Neural networks include CNN, RNN, perceptron, multilayer perceptron, Feed Forward (FF), Radial Basis Network (RBF), Deep Feed Forward (DFF), Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), Auto Encoder (AE), Variational Auto Encoder (VAE), Denoising Auto Encoder (DAE), Sparse Auto Encoder (SAE), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), GAN (Generative Adversarial Network), LSM (Liquid State Machine), ELM (Extreme Learning) Machine), ESN (Echo State Network), DRN (Deep Residual Network), DNC (Differentiable Network) It will be understood by those skilled in the art that the neural network may include any neural network, including but not limited to a Neural Computer (NN), a Neural Turning Machine (NTM), a Capsule Network (CN), a Kohonen Network (KN), and an Attention Network (AN).

[0045] According to an exemplary embodiment of the present disclosure, the processor may be configured to perform a process for generating a CNN (Convolution Neural Network) such as GoogleNet, AlexNet, VGG Network, Region with Convolution Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restrcted Boltzman Machine (RBM), Fully Convolutional Network, Long Short-Term Memory (LSTM) Network, Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT, SP-BERT, MRC / QA for natural language processing, Text Analysis, Dialog System, GPT-3, GPT-4, Visual Analytics for vision processing, Visual Understanding, Video Synthesis, ResNet for data intelligence, Anomaly Detection, Prediction, Time-Series Forecasting, Various artificial intelligence structures and algorithms, including optimization, recommendation, and data creation, can be utilized, but are not limited thereto. Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0046] FIG. 1 is a schematic diagram of an image analysis-based golf swing analysis system (10) according to an embodiment of the present disclosure.

[0047] In the embodiment of the present disclosure, the electronic device (100) refers to a golf swing analysis device, and may be configured in a form similar to glasses as shown in FIG. 1, or in a form similar to a hat. Various electronic devices (100) may be implemented as long as they are portable and easy for the user to film the user's golf swing.

[0048] Referring to FIG. 1, when a user wears a golf swing analysis device and makes a golf swing, the camera (140) of the golf swing analysis device captures the user's golf swing and generates analysis results for the golf swing through image frames included in the captured video.

[0049] In addition, the golf swing analysis device can output analysis results. At this time, the golf swing analysis device can output analysis results through an output module (170) included within the device, or can output analysis results through an external device connected to the golf swing analysis device.

[0050] For example, if the golf swing analysis device is configured as glasses, the golf swing analysis device may be configured with an output module (170) such as a transparent display, such that the glass included in the glasses outputs the analysis results through the glasses so that the user can check them.

[0051] Alternatively, the golf swing analysis device may transmit the analysis results to the user's terminal (e.g., smartphone, tablet PC, etc.) or wearable device (e.g., smart watch, etc.) so that the analysis results are output to the terminal or wearable device.

[0052] FIG. 2 is a block diagram of a golf swing analysis device based on image analysis according to an embodiment of the present disclosure.

[0053] Referring to FIG. 2, an electronic device (100) for image analysis-based golf swing analysis according to an embodiment of the present disclosure includes a processor (110), a communication module (120), a memory (130), a camera (140), an extraction module (150), an analysis module (160), and an output module (170).

[0054] However, in some embodiments, the golf swing analysis device may include fewer or more components than those illustrated in FIG. 2.

[0055] The processor (110) may be implemented as a memory (130) that stores data regarding an algorithm for controlling the operation of components within the device or a program that reproduces the algorithm, and at least one processor (110) that performs the aforementioned operation using the data stored in the memory (130). In this case, the memory (130) and the processor (110) may each be implemented as separate chips. Alternatively, the memory (130) and the processor (110) may be implemented as a single chip.

[0056] In addition, the processor (110) can control any one or a combination of the components described above to implement various embodiments according to the present disclosure described in the drawings below on the device.

[0057] In addition to operations related to the above-described application, the processor (110) can typically control the overall operation of the device. The processor (110) can process signals, data, information, etc. input or output through the components described above, or run application programs stored in the memory (130), thereby providing or processing appropriate information or functions to the user.

[0058] In addition, the processor (110) may control at least some of the components of the device to drive an application program stored in the memory (130). Furthermore, the processor (110) may operate at least two or more of the components included in the device in combination to drive the application program.

[0059] The processor (110) may be implemented as one or more. Hereinafter, even if the processor (110) is expressed as singular, it may be considered as plural. The processor (110) may control the configurations of the @@@ device. The processor (110) may refer to a data processing device built into hardware that has a physically structured circuit to perform a function expressed by a code or command included in a program. As such, the processor (110) is an example of a data processing device built into hardware, and may encompass processing devices such as a microprocessor, a central processing unit (CPU), a processor (110) core, a multiprocessor, an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA), but the scope of the present invention is not limited thereto. The processor (110) may separately include a learning processor (110) for performing artificial intelligence operations, or may include a learning processor (110) on its own.

[0060] In various embodiments, the processor (110) may include one or more of a central processing unit (CPU), an application processor (AP), or a communication processor (CP). At least a portion of the processor (110) may be hardware, access memory (130), and perform functions related to instructions stored in the memory (130).

[0061] The communication module (120) may include one or more modules that connect the electronic device (100) to one or more networks.

[0062] The communication module (120) may include one or more components that enable communication with an external device, and may include, for example, at least one of a broadcast reception module, a wired communication module, a wireless communication module, a short-range communication module, and a location information module.

[0063] The wired communication module may include various wired communication modules such as a Local Area Network (LAN) module, a Wide Area Network (WAN) module, or a Value Added Network (VAN) module, as well as various cable communication modules such as a Universal Serial Bus (USB), a High Definition Multimedia Interface (HDMI), a Digital Visual Interface (DVI), RS-232 (recommended standard 232), power line communication, or plain old telephone service (POTS).

[0064] The wireless communication module may include a wireless communication module that supports various wireless communication methods such as GSM (global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), UMTS (universal mobile telecommunications system), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, and 6G, in addition to a WiFi module and a Wireless Broadband module.

[0065] The wireless communication module may include a wireless communication interface including an antenna and a transmitter for transmitting communication signals. In addition, the wireless communication module (120) may further include a signal conversion module that modulates a digital control signal output from the processor (110) through the wireless communication interface into an analog wireless signal under the control of the processor (110).

[0066] The short-range communication module is for short-range communication, and can support short-range communication using at least one of Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0067] The communication module (120) may also use the name of the communication interface.

[0068] The communication interface can establish communication between the electronic device (100) and an external device. For example, the communication interface can communicate with the external device via wireless communication (e.g., Wi-Fi (Wireless Fidelity), Bluetooth, NFC (Near Field Communication), MST (Magnetic Stripe Transmission), etc.) or wired communication.

[0069] The memory (130) can store data supporting various functions of the device. The memory (130) can store a plurality of application programs (or applications) running on the device, data for the operation of the device, and commands. At least some of these application programs may exist for the basic functions of the device. Meanwhile, the application programs can be stored in the memory (130), installed on the device, and driven to perform operations (or functions) by the processor (110).

[0070] The memory (130) can store data supporting various functions of the device and programs for the operation of the processor (110), input / output data (e.g., music files, still images, moving images, etc.) can be stored, and a plurality of application programs (or applications) run on the device, data for the operation of the device, and commands can be stored. At least some of these application programs can be downloaded from an external server via wireless communication.

[0071] Such memory may include at least one type of storage medium among flash memory type, hard disk type, SSD (Solid State Disk type), SDD (Silicon Disk Drive type), multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, and optical disk. In addition, the memory (130) may be a database that is separate from the device but is connected by wire or wirelessly.

[0072] The memory (130) may be electrically connected to the processor (110) and may store at least one code executed by the processor (110). The memory (130) may collectively refer to various types of storage devices. The memory (130) may store information necessary for performing operations using artificial intelligence, machine learning, and artificial neural networks.

[0073] The memory (130) can store various learning models. The learning models stored in the memory (130) can infer result values ​​for new input data other than learning data, and the inferred values ​​can be used as a basis for judgment to perform a certain action. The learning models stored in the memory (130) can perform learning based on label information, and various backpropagation algorithms can be applied so that the loss function has a target value to increase the accuracy of learning.

[0074] Additionally, the memory (130) may have multiple processes for the electronic device (100).

[0075] The camera (140) processes image frames, such as still images or moving images, obtained by the image sensor in shooting mode. The processed image frames can be displayed on the display unit or stored in the memory (130).

[0076] The camera (140) can capture still images and videos. For example, when the electronic device (100) is mounted on the HMD device, the camera (140) can capture images of at least the front area of ​​the HMD device. In one embodiment, the camera (140) can be activated after a specified time has elapsed since the electronic device (100) is mounted on the HMD device and the HMD device is started to operate. In various embodiments, the camera (140) may be activated from the time the electronic device (100) is mounted on the HMD device. Alternatively, the camera (140) may be activated from the time the user puts on the HMD device.

[0077] The camera (140) module may include components for photographing an object, such as an image sensor and an image processor (110).

[0078] In various embodiments, the camera (140) may include, for example, at least one depth camera (e.g., a Time Of Flight (TOF) method or a structured light method) and a color camera (e.g., an RGB camera (140)). In addition, the camera (140) may further include at least one sensor (e.g., a proximity sensor), a light source (e.g., an LED array), etc., in relation to performing a function. In various embodiments, the at least one sensor may be configured as a separate module from the camera (140) and may perform sensing of at least a front area of ​​the HMD device. For example, a sensor (e.g., a proximity sensor) module may perform sensing of an object by emitting infrared light (or transmitting ultrasonic waves) to the front area of ​​the HMD device and receiving infrared light (or receiving ultrasonic waves) reflected from an object. In this case, the camera (140) may be activated from the time at which at least one object is sensed by the sensor module.

[0079] The processor (110) can perform calculations or data processing related to control and communication of at least one other component of the electronic device (100). For example, the processor (110) can receive captured image data from the camera (140) and, based on the captured image data, detect an object existing within the shooting range of the camera (140).

[0080] In one embodiment, the processor (110) may calculate or detect the quantity of at least one detected object, the size of the objects, the distance between the objects and the HMD device (or a user wearing the HMD device), and the movement of the objects. The processor (110) may control the operation of the display based on the calculated or detected information.

[0081] The extraction module (150) can compare multiple images and extract image areas corresponding to differences. To this end, the extraction module (150) can include an image comparison algorithm.

[0082] The analysis module (160) can analyze an image to generate analysis results for a golf swing, and the analysis results can include prediction results for the quality and distance of the golf swing. In this case, the image to be analyzed includes the trajectory (path due to the swing motion) of the golf club for the golf swing.

[0083] The output module (170) is for generating output related to visual, auditory, or tactile sensations, and may include at least one of a display unit, an audio output module (170), a haptic module, and an optical output module (170). The display unit may be formed as a layer structure with a touch sensor or formed as an integral part, thereby implementing a touch screen. This touch screen may function as a user input unit that provides an input interface between the device and a user, and at the same time, provide an output interface between the device and the user.

[0084] The display unit displays (outputs) information processed by this device. For example, the display unit may display execution screen information of an application program (e.g., an application) running on this device, or UI (User Interface) or GUI (Graphical User Interface) information based on such execution screen information.

[0085] The display can display various contents (e.g., text, images, videos, icons, symbols, etc.). For example, the display can display an image corresponding to at least one image data included in the application program. In various embodiments, when the electronic device (100) adopts VR mode, the display can separate one image into two images corresponding to the user's left and right eyes and display them. In various embodiments, the display can include a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, a microelectromechanical systems (MEMS) display, or an electronic paper display.

[0086] The audio output module (170) can output audio data received through the communication module (120) or stored in the memory (130), or output an audio signal related to a function performed in the device. The audio output module (170) can include a receiver, a speaker, a buzzer, etc.

[0087] The interface unit serves as a passageway for various types of external devices connected to the device. The interface unit may include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory (130) card port, a port for connecting a device equipped with an identification module (SIM), an audio I / O (Input / Output) port, a video I / O (Input / Output) port, and an earphone port. The device may perform appropriate control related to the external device connected to the interface unit.

[0088] The input / output interface receives commands or data from the user or other external devices.

[0089] It can serve as an interface that can transmit data to other components of the electronic device (100). In addition, the input / output interface can output commands or data received from other components of the electronic device (100) to a user or other external device.

[0090] FIG. 3 is a flowchart of a control method of an image analysis-based golf swing analysis device according to an embodiment of the present disclosure.

[0091] FIG. 4 is a diagram illustrating an operation process of a golf swing analysis device based on image analysis according to an embodiment of the present disclosure.

[0092] FIGS. 5 to 9 are various exemplary drawings for explaining an image analysis-based golf swing analysis device according to an embodiment of the present disclosure.

[0093] Referring to FIGS. 3 to 9, a golf swing analysis device based on image analysis, a control method of the device, and a program according to an embodiment of the present disclosure will be described in detail.

[0094] The processor (110) controls the camera (140) to obtain a video of the user's golf swing. (S100)

[0095] In one embodiment, the electronic device (100) may include automatic shooting and automatic recognition functions.

[0096] The processor (110) can start shooting when the user's swing preparation posture is recognized if the automatic shooting function is turned on, and end shooting when the user's swing is recognized as completed.

[0097] In one embodiment, the electronic device (100) can continuously capture images when the automatic capture function is turned on, recognize images corresponding to golf swings from the captured images, store images for each golf swing in the memory (130), and then analyze each image to generate analysis results.

[0098] Figure 5 is a flowchart illustrating the process of recognizing the first frame in a video of a golf swing and performing homography.

[0099] Referring to FIG. 5, the step (S100) of obtaining a video of a golf swing may further include the following steps.

[0100] The processor (110) recognizes the first frame in the video that can be analyzed for a golf swing. (S110)

[0101] The processor (110) recognizes a reference object excluding the image of the club head in the first frame. (S120)

[0102] The processor (110) performs homography on frames based on a reference object. (S130)

[0103] For example, a video of a golf swing is acquired, and although the video contains 20 image frames, image frames 1 through 6 may be image frames that are not necessary for analysis of the golf swing.

[0104] That is, the processor (110) recognizes the first image frame that is deemed necessary for analysis of a golf swing within the acquired image, thereby having the effect of excluding previous unnecessary image frames.

[0105] In the same manner, the processor (110) can recognize the last image frame within the acquired image that is deemed necessary for analysis of the golf swing.

[0106] In addition, the processor (110) can recognize at least one reference object for performing homography in an area excluding the image of the club head in the first frame, and at this time, the reference object may be for performing homography of a plurality of frames.

[0107] The processor (110) can recognize the user's feet as a reference object in the first frame.

[0108] However, it is not limited to this, and any object other than the club head, such as a golf tee, pin, grass, or dust, can be applied as a reference object depending on the location where the user makes a golf swing.

[0109] In one embodiment, the processor (110) may perform a background fitting (homography) process on a plurality of frames included in an image.

[0110] In addition, the processor (110) compares frames in which homography has been completed and extracts a first image corresponding to the difference, thereby achieving the effect of more accurately identifying the difference.

[0111] The processor (110) compares neighboring frames within the image acquired from S100. (S200)

[0112] The processor (110) extracts multiple first images for each area corresponding to the difference based on the comparison result of S200. (S300)

[0113] The processor (110) generates analysis results for a golf swing based on a plurality of first images extracted from S300. (S400)

[0114] The processor (110) combines multiple first images extracted from S300 to generate a second image. (S400)

[0115] The processor (110) combines multiple first images extracted from S300 to generate a second image. (S500)

[0116] The processor (110) controls the second image and analysis results to be output through the output module (170). (S600)

[0117] In one embodiment, the processor (110) can compare each frame within the image acquired from S100 with the frame at the point immediately preceding that frame.

[0118] More specifically, it is assumed that the captured video of the S100 includes a total of 10 image frames, from the first frame to the tenth frame, and that the first frame is the image frame in which the golf swing is first captured, and the tenth frame is the last image frame captured in chronological order.

[0119] The processor (110) compares the second frame with the first frame, compares the third frame with the second frame, sequentially performs the same process, and finally compares the tenth frame with the ninth frame.

[0120] And, based on the results of the comparison as above, the processor (110) extracts multiple first images for each area corresponding to the difference between neighboring frames.

[0121] For example, the processor (110) can compare the second frame and the first frame, and extract an image area corresponding to a difference between the second frame and the first frame as a result of the comparison as the first image.

[0122] And, the same process is performed for the third to tenth frames to extract multiple first images.

[0123] The memory (130) may store an algorithm for comparing images.

[0124] In the embodiment of the present disclosure, the purpose of the electronic device (100) comparing neighboring frames and extracting an image corresponding to a difference is to extract an image of a club head for a golf swing. Accordingly, the first image extracted as a difference may correspond to a golf club head.

[0125] The processor (110) can extract a first image corresponding to the difference between frames based on the comparison result of S200 and the user's club head image.

[0126] In one embodiment, the processor (110) may request the user to take a picture of a club head, store an image of the club head taken through the camera (140) in the memory (130), and use it in the first image extraction process.

[0127] And, the second image generated by combining multiple first images may be a silhouette trajectory of a golf club head.

[0128] As described above, the processor (110) can recognize the first frame required for analyzing a golf swing from a video of the golf swing. Furthermore, the processor (110) can combine the extracted first images into the first frame to generate a second image.

[0129] In one embodiment, the processor (110) may display an object recognized as a club head and a plurality of first images in the first frame so that they are distinct from the background. Specifically, the processor (110) may control the display of an object recognized as a club head and a plurality of first images in the first frame in a color that is distinct from the background.

[0130] Figures 6 and 7 are diagrams illustrating a second image that is a composite of multiple first images extracted from differences between frames within a captured video.

[0131] Figure 8 is a drawing illustrating the exclusion of parts judged as noise in Figure 6.

[0132] Figure 9 is a drawing illustrating the exclusion of parts judged as noise in Figure 7.

[0133] Referring to FIGS. 6 and 7, a second image, which is a result of performing the processes S100 to S400, is illustrated.

[0134] FIG. 6 illustrates a second image (600), which is a merged image of a club head (610) extracted from the first frame, a first image (620) extracted from the second frame, a first image (630) extracted from the third frame, a first image (640) extracted from the fourth frame, an image (650) extracted from the fifth frame, and a first image (660) extracted from the last frame.

[0135] FIG. 7 illustrates a second image (700), in which a plurality of first images (710, 720, 730, 740, 750) extracted according to the process are merged.

[0136] In one embodiment, the processor (110) may remove a portion of a plurality of first images that is determined not to be a club head by determining it as noise, and may combine the first images from which the noise has been removed to generate a second image.

[0137] And, the second image generated by removing noise in this way is as shown in Figs. 8 and 9.

[0138] At this time, the processor (110) may remove noise from each first image, or may remove noise from the second image after combining the first images to create a second image.

[0139] In one embodiment, the processor (110) may recognize the user's foot as a reference object and generate position information of a virtual golf ball for the user's golf swing on a second image based on the position of the recognized reference object.

[0140] In addition, the processor (110) can generate an analysis result for a golf swing based on the location information of the virtual golf ball and a plurality of first images extracted from S300.

[0141] The processor (110) can perform a simulation of a virtual golf ball by a golf swing based on the position information of the virtual golf ball and the swing trajectory of the club head of the plurality of first images extracted from S300, and generate an analysis result based on the simulation performance result.

[0142] In one embodiment, the processor (110) may determine a frames per second setting for capturing a golf swing based on the user's golf swing speed.

[0143] In one embodiment, the processor (110) may set the number of frames per second for video recording to be relatively high when the user's golf swing speed is faster than average, and conversely, the number of frames per second may be set to be relatively low when the user's golf swing speed is slower than average.

[0144] For example, the processor (110) can set the number of frames per second for video recording to be higher when the user's golf swing speed is faster, and conversely, the number of frames per second can be set to be lower when the user's golf swing speed is somewhat slower.

[0145] The second image generated by combining multiple first images extracted from S300 is movement visualization data of the club head, and the processor (110) outputs the second image together with the analysis result, thereby providing the analysis result of the user's swing and visualizing the movement path of the club head due to the golf swing.

[0146] The method according to one embodiment of the present disclosure described above can be implemented as a program (or application) and stored in a medium to be executed in combination with a hardware server.

[0147] The above-described program may include codes coded in a computer language, such as C, C++, JAVA, or machine language, that can be read by the processor (CPU) of the computer through the device interface of the computer, so that the computer reads the program and executes the methods implemented as a program. Such codes may include functional codes related to functions that define functions necessary for executing the methods, and may include control codes related to execution procedures necessary for the processor of the computer to execute the functions according to a predetermined procedure. In addition, such codes may further include memory reference-related codes regarding which location (address address) of the internal or external memory of the computer should reference additional information or media necessary for the processor of the computer to execute the functions. In addition, if the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module (120) of the computer, what information or media to send and receive during communication, etc.

[0148] The above storage medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program can be stored in various recording media on various servers that the computer can access or in various recording media on the user's computer. In addition, the medium can be distributed across network-connected computer systems, so that computer-readable code can be stored in a distributed manner.

[0149] The steps of a method or algorithm described in connection with the embodiments of the present disclosure may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in a random access memory (RAM), a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, a CD-ROM, or any other form of computer-readable recording medium well known in the art to which the present disclosure pertains.

[0150] While the embodiments of the present disclosure have been described above with reference to the attached drawings, those skilled in the art will appreciate that the present disclosure can be implemented in other specific forms without altering the technical spirit or essential features thereof. Therefore, the embodiments described above should be understood to be illustrative in all respects and not restrictive.

Claims

1. Camera; Obtain a video of the user's golf swing through the above camera, Compare each neighboring frame within the above video, Based on the results of the above comparison, multiple first images are extracted for each area corresponding to the difference between the frames, Based on the plurality of first images extracted above, an analysis result for the golf swing is generated, Generating a second image by combining the plurality of first images extracted above, A processor is included that controls the second image generated above and the analysis result to be output, The above analysis results include prediction results for the ball quality and distance by the golf swing. Electronic devices.

2. In paragraph 1, The above processor, Recognize the first frame in the above video that allows analysis of the golf swing, Recognize at least one reference object for performing homography in an area excluding the image of the club head in the first frame above, A process characterized in that the homography is performed on the frames in the image based on the recognized reference object, and the frames for which the homography is completed are compared with each other. Electronic devices.

3. In paragraph 2, The above processor, characterized in that the user's foot is recognized as the reference object in the first frame. Electronic devices.

4. In paragraph 3, The above processor, Recognize the user's feet as the reference object, Generate location information of a virtual golf ball for the golf swing on the second image based on the location of the recognized reference object, Characterized in that it generates an analysis result for the golf swing based on the generated location information and the generated second image. Electronic devices.

5. In paragraph 2, The above processor, The first frame is characterized in that the plurality of extracted first images are combined, and the object recognized as the club head in the first frame and the plurality of extracted first images are displayed so as to be distinguished from the background. Electronic devices.

6. In paragraph 2, The above processor, In the plurality of first images extracted above, a part that is judged not to be the club head is judged to be noise and removed, Characterized in that the second image is generated by combining the first image from which the noise has been removed. Electronic devices.

7. In paragraph 2, The above processor, Characterized in that, in the second image generated above, a part determined not to be the club head is removed by determining it as noise. Electronic devices.

8. In paragraph 1, The above processor, A method characterized in that a first image corresponding to the difference between the frames is extracted based on the results of the comparison and the shape of the user's club head. Electronic devices.

9. In a method for controlling an electronic device performed by a processor, A step of acquiring a video of a user's golf swing through a camera; A step of comparing each neighboring frame within the above image; A step of extracting multiple first images for each area corresponding to the difference between the frames based on the results of the comparison; A step of generating an analysis result for the golf swing based on the plurality of first images extracted above; A step of generating a second image by combining the plurality of extracted first images; and A step of outputting the second image and analysis results generated above, Method of controlling an electronic device.

10. A computer-readable recording medium having recorded thereon a computer program for performing the method of Article 9.

Images