Method and apparatus for predicting ball position

Abstract

Disclosed are a method and an apparatus for predicting a ball position. The apparatus for predicting a ball position according to one embodiment comprises: a memory in which a program for performing a ball position prediction method is stored; and a control unit including one or more processors and operated by executing the program stored in the memory, wherein the control unit extracts an image feature vector from hole image data using a first artificial intelligence model and inputs a total vector based on the extracted image feature vector, environmental condition data, and user data to a second artificial intelligence model different from the first artificial intelligence model, thereby obtaining a predicted ball position.

Description

Ball position prediction method and ball position prediction device

[0001] The embodiments disclosed in this specification relate to a ball position prediction method and a ball position prediction device, and more specifically, to a method and device for predicting the expected landing position (expected ball drop point) of a ball intended to be struck by a user from the user's current ball position using hole image data based on artificial intelligence technology before the user strikes the ball.

[0002] As living standards improve, golf, once considered a luxury sport, is becoming increasingly popular. Consequently, the number of people looking to learn golf is growing, and the demand for participation in golf tournaments is also rising.

[0003] A golf game is played at a golf course (including ancillary facilities) equipped with a golf course. The golf game may be played in an offline space equipped with an actual field, or on a field in an online virtual space implemented through virtual simulation.

[0004] Golf courses can be designed in various forms depending on the course. When playing on courses with diverse characteristics, it is crucial to establish a course strategy that suits oneself. However, users who lack or have insufficient experience with golf courses find it difficult to formulate a suitable strategy because they are unaware of their own skill level. If the ball is simply sent toward the hole or the flagstick (pin), unnecessary scores accumulate, resulting in a complete failure to manage scores. Therefore, there is a growing need to establish and provide appropriate golf course strategies to users; to achieve this, it is necessary to accurately predict the landing point of the ball when it is struck.

[0005] Meanwhile, artificial intelligence technology is currently being developed. However, in the field of golf, AI technology is limited to analyzing a user's swing posture and is not being utilized to calculate the ball's predicted landing point. Furthermore, to satisfy the need to provide relevant information to golfers in real time, data must be delivered within a limited timeframe using limited computing resources; however, publicly available AI models present the problem of requiring significant computing resources and taking a long time to process.

[0006] Korean Registered Patent No. 10-2664956, a prior art document, describes an invention relating to a virtual golf device and a system for predicting the result of hitting a golf ball. The prior art document discloses only the provision of a target point to be used at the time of impact before the user hits the ball, and it does not disclose a technology for predicting the landing position of the ball using an artificial intelligence model.

[0007] Meanwhile, the aforementioned background technology is technical information that the inventor possessed for the derivation of the present invention or acquired during the process of deriving the present invention, and it cannot be considered as prior art disclosed to the general public prior to the filing of the present invention.

[0008] The embodiments disclosed in this specification are intended to present a ball position prediction method and a ball position prediction device.

[0009] The embodiments disclosed in this specification are intended to predict the expected position of the ball that the user intends to hit from the user's current ball position before the user hits the ball during a golf game.

[0010] The embodiments disclosed in this specification are intended to enable striking a target point according to the individual user's ability level, current location, and situation.

[0011] The embodiments disclosed in this specification aim to predict the position of a ball using an artificial intelligence model. In particular, the purpose is to effectively predict the position of a ball while considering limited time and computing resources.

[0012] The embodiments disclosed in this specification aim to predict the position of the ball in real time as well as dynamically.

[0013] Other objects and advantages of the present invention may be understood from the following description and will be more clearly understood by one embodiment. Furthermore, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims.

[0014] According to one embodiment, as a technical means for achieving the technical problem described above, a device for predicting a ball position on a golf course comprises: a memory storing a program for performing a ball position prediction method; and a control unit that operates by executing a program stored in the memory. The control unit can obtain a predicted ball position by using a first artificial intelligence model to extract an image feature vector from hole image data, and inputting a total vector based on the extracted image feature vector, environmental condition data, and user data into a second artificial intelligence model different from the first artificial intelligence model.

[0015] According to another embodiment, a method for predicting a ball position on a golf course, performed by a ball position prediction device, may include the step of extracting an image feature vector from hole image data using a first artificial intelligence model, and the step of obtaining a predicted ball position by inputting a total vector based on the extracted image feature vector, environmental condition data, and user data into a second artificial intelligence model different from the first artificial intelligence model.

[0016] According to another embodiment, the recording medium is a computer-readable recording medium on which a program for performing the ball position prediction method is recorded.

[0017] According to another embodiment, the computer program is performed by a ball position prediction device and is a computer program stored in a recording medium to perform the ball position prediction method.

[0018] According to any one of the aforementioned means for solving the problem, a ball position prediction method and a ball position prediction device can be presented.

[0019] According to any one of the aforementioned means for solving the problem, during a golf game, the expected position of the ball that the user intends to hit can be predicted from the user's current ball position before the user hits the ball. Based on the expected position of the ball, course guide information can be provided, and the user's score can be managed dynamically.

[0020] According to any one of the aforementioned means for solving the problem, automatic aiming correction is possible to enable hitting a target point based on the individual user's ability, current location, and situation.

[0021] According to any one of the aforementioned means for solving the problem, the position of the ball can be predicted using an artificial intelligence model. In particular, since the position of the ball can be predicted effectively while considering limited time and computing resources, it is possible to accurately predict the expected landing position of the ball in a short period of time, and also to perform ball prediction even on low-spec systems. Furthermore, information regarding the predicted position of the ball can be obtained in real time, and course guidance information can be provided quickly based on the predicted position of the ball.

[0022] According to any one of the aforementioned means for solving the problem, the ball's position can be inferred quickly by optimizing resource consumption and inference speed. In other words, various services based on ball position prediction (e.g., course guide information provision services) can be provided with minimal resources and time.

[0023] According to any one of the aforementioned means for solving the problem, compatibility and consistency can be improved by extracting hole images of various sizes into a consistent image feature vector.

[0024] In addition, according to any one of the aforementioned means for solving the problem, the difference between the expected position of the ball and the target position based on user data regarding the user's ability, environmental condition data, and hole image data can be calculated, and an artificial intelligence model can be used to dynamically output the direction of impact of the ball so that the expected position of the ball reaches the target position.

[0025] Furthermore, according to any one of the aforementioned means for solving the problem, the ball's position can be predicted not only in real time but also dynamically. Therefore, course guide information can be provided dynamically based on the user's individual abilities and situation.

[0026] The effects obtainable from the disclosed embodiments are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the disclosed embodiments belong from the description below.

[0027] The attached drawings illustrative of preferred embodiments disclosed in this specification serve to further enhance understanding of the technical concept disclosed in this specification, along with specific details for implementing the invention; therefore, the contents disclosed in this specification should not be interpreted as being limited only to the matters described in such drawings.

[0028] FIG. 1 is a block diagram illustrating a ball position prediction device according to one embodiment.

[0029] FIGS. 2 and 3 are drawings illustrating the overall operation of a ball position prediction device according to one embodiment.

[0030] FIGS. 4 and 5 are exemplary drawings for explaining a ball position prediction device according to one embodiment, illustrating an example of user data.

[0031] FIG. 6 is an illustrative diagram for explaining a ball position prediction device according to one embodiment, and is a diagram illustrating an example of environmental condition data.

[0032] FIGS. 7 and FIGS. 8 are drawings illustrating an example of a ball position prediction device outputting a ball position prediction device according to one embodiment.

[0033] FIG. 9 is a flowchart illustrating a ball position prediction method according to one embodiment.

[0034] Various embodiments are described in detail below with reference to the attached drawings. The embodiments described below may be implemented in various different forms. In order to explain the features of the embodiments more clearly, detailed descriptions of matters widely known to those skilled in the art to which the following embodiments belong have been omitted. Additionally, parts of the drawings unrelated to the description of the embodiments have been omitted, and similar parts throughout the specification have been given similar reference numerals.

[0035] Throughout the specification, when a configuration is described as being "connected" to another configuration, this includes not only cases where they are "directly connected," but also cases where they are "connected with another configuration in between." Furthermore, when a configuration is described as "including" another configuration, this means that, unless specifically stated otherwise, it does not exclude other configurations but may include additional configurations.

[0036] Terms including ordinal numbers such as 'first' or 'second' used herein may be used to describe various components, but said components should not be limited by said terms. Such terms are used solely for the purpose of distinguishing one component from other components.

[0037] First, the terms used in this specification will be explained.

[0038] A golf course is a golf course designed to put the ball into the hole cup with the fewest strokes. A golf course is usually designed with 18 holes, and playing 18 holes is called one round, and the game is played in several rounds. Here, a golf course can include an offline golf course equipped with actual fields or a golf course in an online virtual space implemented through a virtual golf simulation (e.g., screen golf).

[0039] A stroke is the act of striking a golf ball with a club. The number of strokes performed from the tee shot to putting, until the ball is placed into the hole, is accumulated and processed as a score. The user with the lowest score on each hole or the entire course wins the golf match. The standard number of strokes designated for a specific hole is Par, and achievements include Birdie (one stroke less than Par), Eagle (two strokes less than Par), Bogey (one stroke more than Par), Double Bogey (two strokes more than Par), and Hole-in-One (putting the ball into the hole in a single shot with a tee shot).

[0040] Environmental condition data is data related to the current position or movement of the ball and includes variable elements that directly affect score fluctuations. For example, it is ball-related data that affects score fluctuations at each hole, such as the ball's position, the pin's position, target information (e.g., IP information), wind information, and terrain elevation information. At least some of the data constituting this environmental condition data can be set before the user's round or before entering the relevant hole.

[0041] The ball's expected location refers to the point where the ball reaches its final position when the user strikes it. For example, the ball's expected location may be the point where the ball lands upon impact, or the final point where the ball stops moving while rolling. This ball's expected location may be within the hole, but is not limited to this and may also be located outside the hole.

[0042] Hole image data is data regarding images of holes constituting a golf course, and may be two-dimensional or three-dimensional image data. It may be images of maps used when providing course guide information to users. Such hole image data may, for example, be images based on video or satellite maps of holes constituting a golf course, or may be images provided to users playing golf in a virtual golf simulation system (screen golf). Hole image data may, for example, be obtained from a database or through an image sensor. Hole image data may include at least a portion of the hole, and, for example, the entire hole may be included in the form of a map.

[0043] Course guide information is course strategy information for managing the user's score before the user hits the ball. For example, course guide information may include the ball's expected trajectory calculated based on the ball's predicted position, the hitting direction, the predicted score, or a combination thereof. The ball's expected trajectory is the path connecting the ball's current position and its predicted position; the hitting direction is directional information intended to send the ball to a target position considering the user's ability; and the predicted score is the score based on the probability predicted before the hit. For example, the course guide information can be displayed as a specific message including whether a shot was missed, shot direction guidance, and predicted position probabilities.

[0044] The ball position prediction device according to the present embodiment can predict the expected position of the ball that a user intends to hit based on an artificial intelligence model. Additionally, the ball position prediction device can additionally provide course guide information suitable for the user based on the expected position of the ball.

[0045] The embodiments will be described in detail below with reference to the attached drawings.

[0046] FIG. 1 is a block diagram illustrating a ball position prediction device according to one embodiment. FIG. 2 and 3 are diagrams illustrating the overall operation of a ball position prediction device according to one embodiment.

[0047] The ball position prediction device (100) can predict the expected position of the ball that the user intends to hit from the user's current ball position before the user hits the ball using a golf club during a golf game. Additionally, it can perform the action of providing course guide information suitable for the user based on the expected position of the ball.

[0048] The ball position prediction device (100) can interact with a user account through a user terminal and communicate with an administrator server or golf simulation server that operates a golf course to receive golf services.

[0049] According to one embodiment, the ball position prediction device (100) may be implemented in the form of a device, and according to another embodiment, it may also be implemented as a server-client system. In this case, the device may be implemented as a computer, portable terminal, television, wearable device, etc., which can connect to a remote server via a network or connect to other devices and servers. Here, the computer includes, for example, a notebook, desktop, or laptop equipped with a web browser, and the portable terminal may include, for example, any type of handheld-based wireless communication device that ensures portability and mobility, such as PCS (Personal Communication System), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), GSM (Global System for Mobile communications), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet), smartphone, and Mobile WiMAX (Mobile Worldwide Interoperability for Microwave Access). Additionally, the television may include IPTV (Internet Protocol Television), Internet TV, terrestrial TV, cable TV, etc.Furthermore, the wearable device is a type of information processing device that can be worn directly on the human body, such as a watch, glasses, accessories, clothing, or shoes, and can connect to a server at a remote location or to another terminal via a network, either directly or through another information processing device. For example, the ball position prediction device (100) may be implemented as a device installed in a screen golf course.

[0050] According to another embodiment, the ball position prediction device (100) may be implemented as a server, for example, as a computer capable of communicating with other devices, or as a cloud computing server.

[0051] Referring to FIG. 1, a ball position prediction device (100) according to one embodiment may include an input / output unit (110), a control unit (120), a communication unit (130), and a memory (140).

[0052] The input / output unit (110) may include an input unit for receiving input from a user and an output unit for displaying information such as the result of performing an operation or the status of the ball position prediction device (100). For example, the input / output unit (110) may include an operation panel for receiving user input and a display panel for displaying a screen. According to an embodiment, the input / output unit (110) may receive data and output the result of processing it.

[0053] Specifically, the input unit may include devices capable of receiving various forms of user input, such as a keyboard, physical buttons, a touch screen, a camera, or a microphone. Additionally, the output unit may include a display panel or a speaker. However, the input / output unit (110) is not limited thereto and may include a configuration that supports various inputs and outputs.

[0054] According to an embodiment, a camera constituting the input / output unit (110) can acquire an image by photographing a golfer or a golf course, etc. Then, as the image is analyzed by the control unit (120), the information generated can be output by the input / output unit (110) through a display panel, etc. Additionally, according to an embodiment, the input / output unit (110) can provide a user interface that outputs the expected position of the ball. Additionally, the input / output unit (110) can provide a user interface that outputs course guide information. Additionally, according to an embodiment, the input / output unit (110) can acquire user data (or user vector), environmental condition data (or environmental vector), and hole image data (or image feature vector). Additionally, according to an embodiment, the input / output unit (110) can output the expected position of the ball and output the probability of the expected position of the ball. Additionally, according to an embodiment, the input / output unit (110) can output course guide information based on the expected position of the ball.

[0055] The control unit (120) controls the overall operation of the ball position prediction device (100) and may include a processor such as a CPU, GPU, etc. The control unit (120) may control other components included in the ball position prediction device (100) for performing the ball position prediction method.

[0056] For example, the control unit (120) may execute a program stored in memory (140), read a file stored in memory (140), or store a new file in memory (140). Additionally, for example, the control unit (120) may include at least one processor and execute a program for performing a ball position prediction method.

[0057] The communication unit (130) can perform wired or wireless communication with another device or network. To this end, the communication unit (130) may include a communication module that supports at least one of various wired or wireless communication methods. For example, the communication module may be implemented in the form of a chipset.

[0058] According to the embodiment, the communication unit (130) can receive user data (or user vector), environment condition data (or environment vector), and hole image data (or image feature vector) from an external database, device, or server, and can transmit the received data to the memory (140) and the control unit (120).

[0059] The wireless communication supported by the communication unit (130) may be, for example, generational mobile communication, Wi-Fi (Wireless Fidelity), Wi-Fi Direct, Bluetooth, Ultra Wide Band (UWB), or Near Field Communication (NFC). Additionally, the wired communication supported by the communication unit (130) may be, for example, USB or HDMI (High Definition Multimedia Interface).

[0060] Various types of data, such as files, applications, and programs, can be installed and stored in the memory (140). The control unit (120) may access and use the data stored in the memory (140) or store new data in the memory (140). Additionally, the control unit (120) may execute a program installed in the memory (140). Referring to FIG. 1, a program for performing a ball position prediction method may be stored in the memory (140). According to an embodiment, the memory (140) may store various artificial intelligence models, and the memory (140) may also store user data (or user vector), environmental condition data (or environmental vector), hole image data (or image feature vector), total vector, etc.

[0061] According to the embodiment, the control unit (120) can predict the ball position when the user hits the ball from the current ball position using a ball position prediction model.

[0062] Referring to FIG. 2, the control unit (120) can predict the expected position of the ball when the user hits the ball from the user's current ball position through an artificial intelligence-based ball position prediction model (200).

[0063] In the process of predicting the expected position of the ball that the user intends to hit, the control unit (120) can calculate the expected position of the ball through an artificial intelligence-based ball position prediction model based on the user's user data, environmental condition data including the user's current ball position, and hole image data. That is, as illustrated in FIG. 2, when the control unit (120) inputs hole image data (21), user data (22), and environmental condition data (23) into the ball position prediction model (200), it can obtain the expected ball position (25) as an output. At this time, if the expected ball position (25) is predicted based on a two-dimensional hole image, it may be a two-dimensional coordinate value (e.g., (x,y)), and if it is predicted based on a three-dimensional hole image, it may be a three-dimensional coordinate (e.g., (x,y,z)).

[0064] More specifically, referring to FIG. 3, the ball position prediction model (200) may include an image processing model (310), a decoder (320), and a language processing model (330).

[0065] According to the embodiment, the control unit (120) can extract an image feature vector (V) from hole image data (21) using an image processing model (310).

[0066] As illustrated in FIG. 3, hole image data (21) can be input into an image processing model (310) to output an image feature vector (V). At this time, the image processing model (310) is an artificial intelligence model trained to output an image feature vector (V) when hole image data (21) is input. It may be implemented, for example, as a CNN, ViT (Vision Transformer), EfficientNet, ResNet series, MobileNet series, MobileNetV3-Small, etc., but is not limited thereto; any artificial intelligence model capable of extracting features from an image is sufficient. The image processing model (310) can be trained to extract features regarding the characteristics and conditions of the hole from the hole image data, thereby enabling spatial recognition. When hole image data is input, the trained image processing model (310) outputs an image feature vector. The hole image data input into the image processing model (310) may include images of each of multiple holes, and may include images of each hole included in each golf course playable by the user. In this case, a playable golf course refers to, for example, an offline golf course that a user can visit and play, or a virtual golf course registered in screen golf.

[0067] In this regard, MobileNetV3-Small may include a feature extraction layer, which can convert the original image into a multidimensional feature vector. Additionally, MobileNetV3-Small may include a linear layer (FC layer), which is a layer that generates a vector that expands the extracted feature vectors and may be composed of, for example, one or more dense layers.

[0068] When the image processing model (310) is implemented with MobileNetV3-Small, the linear layer at the end can be configured with a value of 1280, which is larger than that of the feature extraction layer. As shown in the feature structure (302) illustrated in FIG. 3, the output value of the linear layer can be represented by a total of 1280 feature vectors. That is, the output dimension of the linear layer can be configured to be larger than the output dimension of the feature extraction layer (size of the feature vector (number of channels)). The linear layer is a layer for generating high-dimensional feature vectors from the feature vector generated in the feature extraction layer, and by configuring the linear layer with a value larger than that of the feature extraction layer, the features can be diversified (for example, it can ensure the ability to distinguish 400 CC x 18 hole images). In addition, in the image processing model (310), the feature extraction layer can be minimized and the linear layer at the end can be maximized. For example, the output dimension of the feature extraction layer can be minimized and the output dimension of the linear layer can be maximized. Since the role of the image processing model (310) is simply to distinguish the feature values ​​of a large number of hole image data, there is no problem even if the linear layer and the feature extraction layer are asymmetric.

[0069] As described above, the image processing model (310) can be trained to receive a plurality of hole image data (21) as input and output an image feature vector (F) having a feature structure (302).

[0070] Accordingly, according to one embodiment, when the control unit (120) intends to provide the user with the ball's expected position, it may input the hole image data (21) of the hole where the ball is currently located into an image processing model (310) to obtain an image feature vector (F) and input the image feature vector (F) into a decoder (320). According to another embodiment, when the control unit (120) intends to provide the user with the ball's expected position, it may input the image feature vector (F) corresponding to the hole where the ball is currently located into the decoder (320), and in this case, the image feature vector (F) may be obtained using a past image processing model (310).

[0071] In this regard, according to the embodiment disclosed in this specification, before the total vector is input to the language processing model (330), the image processing model (310) converts the hole image data into an image feature vector, thereby allowing the view location prediction model (200) to operate with minimal resources. For example, since each CC contains 18 holes, only about 1,280 image feature vectors need to be generated to distinguish a total of 7,200 holes. However, if hole image data is used without an image processing model, the language processing model (330) must process at least 265,600 vectors, so a lot of resources are required for vector processing.

[0072] According to the embodiment, the control unit (120) can obtain an environment vector from environment condition data and obtain a user vector from user data.

[0073] The control unit (120) can extract a user vector (U) based on user data (22). For example, as illustrated in FIG. 3, the user data (22) may include data such as the user's 'driver power' or 'iron-200m front-back deviation' according to driver, iron, approach, etc., and the control unit (120) can vectorize the user data.

[0074] In this regard, FIGS. 4 and 5 are exemplary drawings for explaining a ball position prediction device according to one embodiment, illustrating an example of user data.

[0075] In this regard, user data is data regarding a user who intends to hit a ball for which an expected position is to be calculated, and may be data regarding the user's golf ability. Such user data may be determined based on accumulated values ​​of the user's golf ability (e.g., driver distance, accuracy, green hit rate, short game, average number of putts, ball speed, ball flight quality, hit accuracy, proximity to hole, left-right deviation, front-back deviation, fairway landing rate, etc.) based on one or more rounds played by the user in the past, and may be determined, for example, as an average value or a median value of values ​​accumulated over a predetermined period. User data may be, for example, numerical values ​​of items regarding the user's golf ability when playing golf, such as the user's handicap, driver accuracy, power, and green hit rate, along with user identification information (user ID) for identifying the user.

[0076] The control unit (120) can acquire user data, for example, from a device located at a screen golf facility or from a database where user data is stored. For example, a golf simulation device may include a sensor and acquire physical information about the user's swing through the sensor. The control unit (120) can extract a portion of the user data and utilize it as common data or specific data (by situation).

[0077] Referring to FIG. 4, the control unit (120) vectorizes user data and converts it into a user vector. For example, the user vector, which is vectorized user data, is a vector representation of user data including user ID, handicap, driver wave, etc.

[0078] Referring to FIG. 5, user data may include numbers and items, and may include numerical values ​​or calculation formulas in units of specific games, rounds, or holes.

[0079] For example, user data can be specified as user ID, handicap, driver power, driver accuracy, green hit rate, short game (shots near the green), putting, driver average distance, driver ball speed, driver solid hit rate, driver average ball flight quality, iron average proximity to hole rate, iron average left-right deviation, iron average front-back deviation, iron 50m proximity to hole rate, iron 50m left-right deviation, iron 50m front-back deviation, iron 100m proximity to hole rate, iron 100m left-right deviation, iron 100m front-back deviation, iron 150m proximity to hole rate, iron 150m left-right deviation, iron 150m front-back deviation, iron 200m proximity to hole rate, iron 200m left-right deviation, iron 200m front-back deviation, etc.

[0080] For example, the handicap can be specified by green speed, wind speed, tee position, player difficulty, swing plate, concede, tour mode, hardness, course difficulty, green difficulty (slope rating), system correction value, etc.

[0081] For example, the short game can be specified by whether or not it is on the green (GIR), par, strokes, or whether or not it enters a green bunker.

[0082] Returning to FIG. 3, the control unit (120) can extract an environment vector (A) based on environment condition data (23). For example, the control unit (120) can generate an environment vector (A) by vectorizing the environment condition data (23).

[0083] In this regard, FIG. 6 is an exemplary diagram illustrating a ball position prediction device according to one embodiment, and is a diagram illustrating an example of environmental condition data.

[0084] The control unit (120) can obtain environmental condition data, for example, from a device located at a screen golf course (e.g., a golf simulation device) or from a database where environmental condition data is stored. For example, when a user is playing a golf game, information regarding the pin location or wind direction / wind speed of the hole where the user is located can be obtained, and information regarding the location of the ball the user intends to hit, the height difference between the ball's location and the pin can be obtained.

[0085] Referring to FIG. 6, the control unit (120) vectorizes environmental condition data and converts it into an environment vector. For example, the environment vector, which is vectorized user data, is a vector representing environmental condition data including the current ball position, the current ball position and the difference in elevation of the pin.

[0086] Referring to Fig. 6, environmental condition data may include numbers and items, and specific numerical values, etc.

[0087] For example, environmental condition data can be specified as the current ball position, the elevation difference between the current ball position and the pin, the target position, the pin position, aiming information, wind direction, and wind speed.

[0088] Returning to Fig. 3, the control unit (120) can obtain a total vector in which an image feature vector (F), a user vector (U), and an environment vector (A) are connected using a decoder.

[0089] The decoder (320) according to the embodiment can generate a total vector (V) based on an image feature vector (F), a user vector (U), and an environment vector (A). At this time, the decoder (320) is not an artificial intelligence model implemented as a learning parameter. Additionally, when training the ball position prediction model (200), the decoder (320) may be a model that integrates data as a parameter not used for training.

[0090] The decoder (320) can merge the image feature vector (F), user vector (U), and environment vector (A) into a single data. For example, the decoder (320) can generate a total vector (V), which is a single data, by concatenating the image feature vector (F), user vector (U), and environment vector (A). At this time, the total vector (V) can be configured in a consistent data format to improve the accuracy of learning and inference of the language processing model (330) to be described later.

[0091] According to an embodiment, the decoder (320) can obtain a user vector (U) or an environment vector (A) extracted by the control unit (120). According to another embodiment, the decoder (320) can extract a user vector (U) from user data and extract an environment vector (A) from environment condition data.

[0092] According to an embodiment, the decoder (320) can normalize the image feature vector (F), user data, and environment condition data. According to another embodiment, the decoder (320) can normalize the image feature vector (F), user vector (U), and environment vector (A). When normalizing, the image feature vector (F) is used as a raw value without passing through an additional activation function, as it is sufficient to distinguish a large number of hole images.

[0093] And by concatenating the normalized vectors, the decoder (320) can generate a total vector (V).

[0094] The control unit (120) can input the acquired total vector (V) into the language processing model (330). That is, the control unit (120) can obtain the predicted ball position by inputting the total vector based on image feature vectors, environmental condition data, and user data into the language processing model (330). Due to the total vector, the language processing model (330) can be made lightweight and can perform inference quickly with fewer resources. This improves the performance of the service provided based on the predicted ball position.

[0095] According to the embodiment, the language processing model (330) can be trained to output the predicted position of the ball when the total vector (V) is input. At this time, the language processing model (330) is an artificial intelligence model trained to output coordinates on the hole image data when the vector is input, and can be implemented, for example, as an RNN, BERT, T5, Transformer, etc., but is not limited thereto; any artificial intelligence model capable of processing text input values ​​and providing output as a natural language processing model is sufficient.

[0096] At this time, a language processing model (330) based on the transformer can be implemented. That is, the language processing model (330) is an optimized version of a conventional transformer model. A conventional transformer model can be composed of a positional encoding, an encoder, and a decoder, and can additionally be composed of a skip-linear layer. The positional encoding of the transformer injects positional information regarding the order of tokens, the encoder of the transformer represents each token within the input sequence so that it understands the context by referring to other tokens, and the decoder of the transformer predicts the next word by referring to the output and input sentence up to this point. At this time, the language processing model (330) disclosed in this specification specifically uses only the encoder of the transformer without using the decoder that constitutes the conventional transformer. Since regression is performed to calculate the expected position of the ball based on the user's strike (golf shot), a complex transformer decoder is not used, and only a linear layer is used. In this regard, for example, as a vector output from 400 CCs, and since each CC contains 18 holes, a total of 1280 image feature vectors (F) can be generated to distinguish a total of 7200 holes. Additionally, the user vector (U) extracted from the user data can be generated as 784 by adding 3 zero paddings to avoid prime numbers and reduce the d_model value while lowering the chunking value of the SkipLinear layer described later. Also, the environment vector (A) can have a length of 6. Accordingly, a total vector (V) with a total length of 2070 can be generated. The linear layer of the language processing model (330) can be composed of 2070, which corresponds to the length of the total vector. In score prediction, the linear layer is composed of 784.

[0097] In addition, since the type and form of data used by the language processing model (330) for the task is simple, the encoder in the language processing model (330) can use only 2 layers instead of all 4 to 6 layers used in the encoder of the known transformer model.

[0098] Additionally, the skip-linear layer is a custom module for converting numerical data into a form suitable for a transformer model, and can perform a role similar to that of an embedding layer. The skip-linear layer can increase computational efficiency and model diversity by utilizing only selected connections instead of using all connections of the fully connected layer. These skip-linear values ​​are typically processed by chunking the original data by expanding it by 2 to 4 times for diversity, but for faster inference speeds, this specification uses only the length of the original data, which is the total feature. Since the input data is used as a fixed column when obtaining the ball's predicted position, there is no need to expand it significantly, and this is sufficient for training the language processing model (330).

[0099] In addition, the control unit (120) can minimize the value of d_model (dimension of the model or vector size of the model) in the language processing model (330) and allocate the batch size during inference to the total number of golf courses (CC). That is, the language processing model (330) can set the batch size to the total number of CCs that a user can play. In this regard, CCs typically include 18 holes, and in screen golf or offline golf, users play golf games in CCs. By minimizing the value of d_model in the language processing model (330) and allocating the batch size to the total number of CCs, real-time inference can be enabled. For example, in a typical transformer model, d_model has a value of 256,512, but according to the embodiment disclosed in this specification, d_model of the language processing model (330) can be set to a value of 16, and the batch can be allocated to 400, which is the total number of CCs, thereby increasing the inference speed. In fact, when predicting scores based on predicted locations, a total of 1600 CC inferences for 4 users shows an inference time of less than 1 second on the GPU.

[0100] Based on the language processing model (330) created by customizing the Transformer model in this way, the expected location of the view can be predicted more accurately and quickly. In addition, the size of the language processing model (330) can be efficiently reduced.

[0101] When the total vector (V) input from the decoder (320) is input, the language processing model (330) outputs the expected position (25) of the ball. If hole image data is input directly into the language processing model (330), the processing speed becomes slow due to the large amount of data processing based on the image size and requires a lot of resources. According to the embodiment disclosed in this specification for real-time processing, hole image data is processed in the image processing model (310), and accordingly, the language processing model (330) can use, for example, 1280 fixed image feature vectors.

[0102] As described above, the ball position prediction model (200) may include an image processing model (310), a decoder (320), and a language processing model (330). That is, the ball position prediction model (200) is composed of an image processing model (310) for processing hole image data, a decoder (320) that converts and processes various data including hole image data so that it can be input to the language processing model (330), and a language processing model (330) that outputs the predicted position of the ball based on the total features input from the decoder (320). The ball position prediction model (200) implemented as a single model in this way can be implemented as a multi-model that processes images and text simultaneously. Through this, the predicted position of the ball can be predicted quickly with minimal resources, and various services can be provided based on the predicted ball position.

[0103] According to an embodiment, the ball position prediction device (100) can provide course guide information for managing a user's score based on the predicted position of the ball. That is, the ball position prediction device can dynamically provide course guide information in real time that includes the user's current skill level at a specific hole, predicted ball trajectory information suitable for the position, hitting direction, predicted score, or information combined therefrom.

[0104] For example, by utilizing the predicted ball position, it is possible to dynamically guide the user on which direction to aim to reach the target position (e.g., the center of the fairway). Specifically, if the user's shot has a trajectory that slices to the right, the ball position prediction device (100) can predict the next landing position of the ball based on this. At this time, the degree of deviation between the predicted position and the target point can be calculated, and based on this, it is possible to guide the user on which direction to aim to reach the target point even with a slice trajectory, or to perform automatic direction correction. For example, the angle between the line segment connecting the ball's current position and the ball's predicted position, and the line segment connecting the ball's current position and the target point, can be used to guide the user on how many degrees the ball's predicted position deviates from the target point. In this case, the direction of the target point and the aiming direction can be displayed together through the user interface. The aiming direction can be a direction that directs the ball toward the target point, for example, by the angle between the line segment connecting the ball's current position and the ball's predicted position, and the line segment connecting the ball's current position and the target point. The target point can be pre-set, and multiple coordinate values ​​may be configured. Depending on the case, the target point may have various meanings or be referred to as an Intersection Point (IP), intermediate target, Landing Zone, hole cup point, etc.

[0105] According to an embodiment, the ball position prediction device (100) can provide a course management service suitable for the user's current situation by predicting the probability-based expected position of the ball based on user data, which is information regarding the user's golf ability.

[0106] According to an embodiment, the ball position prediction device (100) can predict and guide the optimal expected path that can be taken from the current location to the user's ability level. At this time, the expected path of the ball is a line segment connecting the ball's starting point and the destination point (expected ball position), which can be displayed as a straight line, a curve, etc., and if necessary, it can be highlighted using an arrow, a dotted line, a color, etc. Additionally, the expected path of the ball can be created by connecting the ball's starting point and the destination point while considering the terrain from the ball's starting point to the destination point, and for example, the expected path of the ball can be generated by considering the way the ball moves to the green by following the slope of the terrain or avoiding obstruction areas.

[0107] According to the embodiment, the ball position prediction device (100) does not provide a path from the current location to the hole all at once, but rather guides the optimal next location that can be reached from the current location based on the user's ability level. Because the next location is predicted in a gradual manner, it is possible to respond flexibly even if a miss shot occurs.

[0108] The ball position prediction device (100) can predict the direction in which a shot should be hit according to the user so that the ball falls toward the target direction, and can automatically correct the aiming based on this. The direction of impact can be indicated by an arrow, an angle, etc., and can be indicated by an azimuth, an elevation angle, etc.

[0109] The ball predicted position output from the ball position prediction device (200) can be used as an input value for another artificial intelligence model. For example, the ball predicted position can be input into a course guide providing model that includes a hitting direction correction model, a score prediction model, or a model combined thereof. The hitting direction correction model may include a hitting direction correction model, a score prediction model, or a model combined thereof. Additionally, the score prediction model can predict how much of an expected score will occur on the corresponding hole before the hit. At this time, the hitting direction correction model, the score prediction model, or the model combined thereof may apply network models such as an Artificial Neural Network (ANN), a Deep Neural Network (DNN), a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), an Autoencoder, a feature point extraction model, or a Decoder, but is not limited thereto, and can be implemented as various network models capable of learning to obtain information such as an expected trajectory, hitting direction, expected score, or a combination thereof based on the ball's predicted position.

[0110] According to an embodiment, the ball position prediction device (100) can predict the probability of the expected position of the ball that the user intends to hit. The ball position prediction device can predict the expected position of the ball with one or more probabilities for each hole.

[0111] According to an embodiment, the ball position prediction device (100) can calculate the user's expected score (including probability) using a score prediction model for the golf course currently being played by the user.

[0112] According to an embodiment, the ball position prediction device (100) can calculate the user's predicted score for the entire golf course and provide the probability as a reference value. Alternatively, the control unit (120) can calculate the user's predicted score for each hole constituting the entire golf course and calculate the predicted score for the entire golf course based on the user's predicted score for each hole.

[0113] According to an embodiment, the ball position prediction device (100) may calculate a set of ball position predictions or a set of predicted scores having multiple probabilities for each hole, and may sort and output the set of ball position predictions or the set of predicted scores having multiple probabilities. For example, in the process of providing course guide information for managing a user's score, the ball position prediction device may provide the probability of achieving a score in the corresponding hole in real time when following the ball's expected path of travel or the hitting direction provided as guide information. For example, the ball position prediction device may provide a predicted score per hole when hitting according to the recommended hitting direction, and may provide one or more predicted scores including the probability of achieving individual scores in real time.

[0114] In this regard, FIGS. 7 and 8 are drawings illustrating an example of a ball position prediction device outputting a ball position prediction device according to one embodiment.

[0115] The ball position prediction device (100) may provide a user interface that provides ball position prediction and course guide information. The visualized information provided through the user interface may include information visualized in various forms by changing the color, size, font, display position, border, adding symbols, adding highlights, animation effects, graph representation, shape representation, or a combination thereof.

[0116] As illustrated in FIG. 7, with reference to reference numeral 710, the ball position prediction device can output the user's current ball position (710) to the hole image data. With reference to reference numeral 720, the ball position prediction model (200) can output the expected ball position on the hole image. The expected ball position may include coordinates, a range, a probability, etc.

[0117] Likewise, as illustrated in FIG. 8, with reference to reference numeral 810, the ball position prediction device (100) can output the user's current ball position to the hole image data. With reference to reference numeral 820, the ball position prediction model (200) can predict the expected position of the ball based on the user's user data, environmental condition data including the user's current ball position, and the hole image data, and output the expected position of the ball along with the target position in the golf course area of ​​the hole image data. With reference to reference numeral 830, the ball position prediction device (100) can output the ball's striking direction (aiming direction) to cause the expected position of the ball to reach the target position. That is, the ball position prediction device (100) can output a striking direction to send the ball to the target position based on the target position in the golf course area, and can also output the expected position of the ball when striking according to the proposed striking direction.

[0118] FIG. 9 is a flowchart illustrating a ball position prediction method according to one embodiment.

[0119] The ball position prediction method according to the embodiment illustrated in FIG. 9 includes steps processed in a time-series manner in the ball position prediction device illustrated in FIG. 1 to FIG. 8. Even if some details regarding the ball position prediction method are omitted, the details described above regarding the ball position prediction device may be applied to the ball position prediction method according to the embodiment illustrated in FIG. 9, and even if some details regarding the ball position prediction device are omitted, the details described regarding the ball position prediction method may be applied to the ball position prediction device.

[0120] The ball position prediction device (100) can train an image processing model (310) and a language processing model (330), respectively, before or while inferring the ball's predicted position.

[0121] Referring to FIG. 9, the ball position prediction device (100) can extract image feature vectors from hole image data using a first artificial intelligence model (S910). At this time, the hole image data may include one or more hole images, or may include images of each of a plurality of holes. Also, the first artificial intelligence model refers to an image processing model (310). The first artificial intelligence model can be trained to extract image feature vectors when hole image data is input prior to inference.

[0122] According to the embodiment, the ball position prediction device (100) can collect user data, environmental condition data, hole image data, or a combination thereof prior to S910.

[0123] The ball position prediction device (100) can acquire an image feature vector based on hole image data of the hole where the current user is located and playing a golf game. In this regard, according to one embodiment, when the ball position prediction device (100) intends to calculate the expected position of the ball in the hole where the current user is located and playing a golf game, it may cause the image processing model (310) to extract an image feature vector based on the hole image data of the hole where the current user is located and playing a golf game in real time. According to another embodiment, the ball position prediction device (100) may cause the image processing model (310) to extract and store an image feature vector for each of a plurality of hole images before calculating the expected position of the ball or before receiving a request for the expected position of the ball, and when there is a request for the expected position of the ball, it may use an image feature vector corresponding to the hole where the current user is located and playing a golf game.

[0124] The ball position prediction device (100) can use the image feature vector obtained in step S910 to generate a total vector together with environmental condition data and user data, and can obtain the ball predicted position by inputting the total vector into a second artificial intelligence model.

[0125] At this time, the ball position prediction device (100) can extract an environment vector from environment condition data including the 'current ball position (e.g., coordinate values ​​at the hole)' at the hole where the user's golf game is currently being played, and can extract a user vector from user data of the user currently playing the golf game. Then, a total vector can be obtained by connecting the image feature vector, the environment vector, and the user vector.

[0126] At this time, the ball position prediction device (100) may store user vectors regarding user data corresponding to each user in advance, and may use the stored user vectors when generating a total vector. Additionally, the ball position prediction device (100) may store environmental condition vectors regarding environmental condition data corresponding to each hole in advance, and when generating a total vector, may use the updated environmental condition vectors by updating only the information related to the current ball position (e.g., current ball position, difference in elevation between the current ball position and the pin) among the environmental condition vectors.

[0127] The ball position prediction device (100) can obtain a predicted ball position by inputting a total vector based on image feature vectors, environmental condition data, and user data into a second artificial intelligence model that is different from the first artificial intelligence model (S920). At this time, the second artificial intelligence model refers to a language processing model.

[0128] For example, a ball position prediction device (100) can obtain a ball position prediction device (100) by inputting a total vector based on an image feature vector of hole image data on a golf course currently played by a user, an environment condition data (or environment vector) including 'current ball position' on the hole, and user data (or user vector) of the user playing golf into a second artificial intelligence model to obtain a ball position prediction device on a golf course currently played by a user. At this time, the image feature vector may be generated in real time by a first artificial intelligence model based on hole image data on a golf course currently played by a user, or may have been generated in the past by a first artificial intelligence model based on hole image data on a golf course.

[0129] The ball position prediction device (100) can guide the user to the expected position of the ball by outputting the expected position of the acquired ball. At this time, when outputting the expected position of the ball, the probability of the probabilistically possible range of movement of the ball may be output according to user information.

[0130] After step S920, the ball position prediction device (100) can provide course guide information for managing the user's score based on the predicted position of the ball.

[0131] For example, the ball position prediction device (100) can display and provide the ball's expected path of travel calculated based on the ball's expected position on the hole image data.

[0132] In addition, for example, the ball position prediction device (100) can provide a ball striking direction that causes the ball's predicted position to reach the target position.

[0133] In addition, for example, the ball position prediction device (100) can calculate an expected score with one or more probabilities corresponding to the hole and provide an expected score with one or more probabilities.

[0134] In addition, for example, the ball position prediction device (100) can calculate a set of predicted scores with multiple probabilities for each hole and provide the set of predicted scores sorted.

[0135] In addition, for example, when the current ball position prediction device (100) updates the current ball position to a different position, it can recalculate the expected position of the ball due to the user's current strike and dynamically update the course guide information accordingly.

[0136] In the embodiments above, the term 'part' refers to a software or hardware component such as an FPGA (field programmable gate array) or an ASIC, and the 'part' performs certain roles. However, the meaning of 'part' is not limited to software or hardware. The 'part' may be configured to reside in an addressable storage medium or may be configured to run one or more processors. Accordingly, as an example, the 'part' includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables.

[0137] The functions provided within the components and 'parts' can be combined into a smaller number of components and 'parts' or separated from additional components and 'parts'.

[0138] In addition, the components and '~parts' may be implemented to play one or more CPUs, GPUs within the device or secure multimedia card.

[0139] Meanwhile, the ball position prediction method according to one embodiment described herein may also be implemented in the form of a computer-readable medium that stores instructions and data executable by a computer. In this case, the instructions and data may be stored in the form of program code, and when executed by a processor, may generate a specific program module to perform a specific operation. Furthermore, the computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, as well as removable and non-removable media. Additionally, the computer-readable medium may be a computer recording medium, which may include both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. For example, the computer recording medium may be a magnetic storage medium such as an HDD and an SSD, an optical recording medium such as a CD, DVD, or Blu-ray disc, or a memory included in a server accessible via a network.

[0140] Additionally, a ball position prediction method according to one embodiment described herein may be implemented as a computer program (or computer program product) comprising instructions executable by a computer. The computer program includes programmable machine instructions processed by a processor and may be implemented in a high-level programming language, an object-oriented programming language, assembly language, or machine language, etc. Additionally, the computer program may be recorded on a tangible computer-readable recording medium (e.g., memory, hard disk, magnetic / optical medium, or SSD (Solid-State Drive), etc.).

[0141] Accordingly, a ball position prediction method according to one embodiment described herein may be implemented by executing a computer program as described above by a computing device. The computing device may include at least some of a processor, memory, a storage device, a high-speed interface connected to the memory and a high-speed expansion port, and a low-speed interface connected to a low-speed bus and a storage device. Each of these components may be connected to one another using various buses and may be mounted on a common motherboard or mounted in other suitable ways.

[0142] Here, the processor can process instructions within the computing device, such as instructions stored in memory or storage devices to display graphic information for providing a Graphic User Interface (GUI) on external input and output devices, such as a display connected to a high-speed interface. In another embodiment, a plurality of processors and / or a plurality of buses may be utilized together with a plurality of memories and memory types. Additionally, the processor may be implemented as a chipset comprising chips including a plurality of independent analog and / or digital processors.

[0143] In addition, memory stores information within a computing device. For example, memory may consist of volatile memory units or a set thereof. As another example, memory may consist of non-volatile memory units or a set thereof. Furthermore, memory may be other forms of computer-readable media, such as magnetic or optical discs.

[0144] And the storage device can provide a large amount of storage space to the computing device. The storage device may be a computer-readable medium or a configuration containing such a medium, and may include, for example, devices or other configurations within a Storage Area Network (SAN), and may be a floppy disk device, a hard disk device, an optical disk device, or a tape device, flash memory, or other similar semiconductor memory device or device array.

[0145] The embodiments described above are for illustrative purposes only, and those skilled in the art will understand that the embodiments described above can be easily modified into other specific forms without altering the technical concept or essential features of the embodiments described above. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0146] The scope of protection sought through this specification is defined by the claims set forth below rather than by the detailed description above, and should be interpreted to include all modifications or variations derived from the meaning and scope of the claims and the concept of equivalents.

Claims

1. As a device for predicting the position of a ball on a golf course, Memory storing a program for performing a ball position prediction method; and It includes one or more processors and a control unit that operates by executing a program stored in memory, and The above control unit is, A ball position prediction device that obtains a predicted ball position by using a first artificial intelligence model to extract an image feature vector from hole image data, and inputting a total vector based on the extracted image feature vector, environmental condition data, and user data into a second artificial intelligence model different from the first artificial intelligence model.

2. In Paragraph 1, The above control unit is, A ball position prediction device that obtains an environment vector from the above-mentioned environment condition data and a user vector from user data.

3. In Paragraph 2, The above control unit is, A ball position prediction device that obtains a total vector by concatenating the image feature vector, the environment vector, and the user vector.

4. In Paragraph 1, The above control unit is, A ball position prediction device that provides course guide information for managing the user's score based on the above-mentioned ball position.

5. In Paragraph 1, A ball position prediction device comprising the hole image data including one or more images of each hole.

6. In Paragraph 1, A ball position prediction device wherein the first artificial intelligence model is composed of a feature extraction layer and a linear layer, wherein the output dimension of the linear layer is configured to be greater than the output dimension of the feature extraction layer.

7. In Paragraph 1, The above second artificial intelligence model is a ball position prediction device in which the batch size is set based on the total number of golf courses playable by the user.

8. A method for predicting the position of a ball on a golf course, performed by a ball position prediction device, A step of extracting image feature vectors from hole image data using a first artificial intelligence model; and A ball position prediction method comprising the step of obtaining a ball predicted position by inputting a total vector based on extracted image feature vectors, environmental condition data, and user data into a second artificial intelligence model different from the first artificial intelligence model.

9. In Paragraph 8, The step of obtaining the above-mentioned ball predicted position is, A ball position prediction method further comprising the step of obtaining an environment vector from the above-mentioned environment condition data and a user vector from the above-mentioned user data.

10. In Paragraph 9, The step of obtaining the above-mentioned ball predicted position is, A ball position prediction method further comprising the step of obtaining a total vector by concatenating the image feature vector, the environment vector, and the user vector.

11. In Paragraph 8, The above method is, A ball position prediction method further comprising the step of providing course guide information for managing the user's score based on the ball position prediction above.

Images