Method for displaying player information in sports image

Abstract

A method for displaying player information in a sports image is provided. The method comprises the steps of: acquiring a captured image of a sports event in a target time period; acquiring sensor-based location information in the target time period, the location information being acquired for each of players participating in the sports event; determining corresponding locations for each of the players in the captured image, the locations corresponding to the sensor-based location information for each of the players; and displaying information on each of the players at each of the corresponding locations.

Description

How to display player information in exercise videos

[0001] This disclosure relates generally to motion images and additional information, and more specifically, but not exclusively, to techniques for displaying player information in motion images.

[0002] The explosive growth of the sports industry and the advancement of sports science are steadily increasing the importance of sports analytics. Amidst this trend, Electronic Performance Tracking Systems (EPTS) are increasingly being introduced, particularly in major sports like soccer, to track players during matches and training. In EPTS, the position and movement of players serve as crucial baseline data for providing a variety of additional information. Therefore, various efforts are being made to facilitate the acquisition of tracking information on players' positions and improve its accuracy.

[0003] In this regard, methods for determining an object's location based on the characteristics of signals measured by sensors corresponding to the object, such as the Global Positioning System (GPS), have been widely utilized. Recently, with the advancement of image analysis technology, methods for calculating an object's location from images containing the object captured by a camera have also been utilized. However, image-based and sensor-based location determination methods each have different limitations, and various studies are being conducted to improve these limitations.

[0004] Meanwhile, sports analysis and information provision have evolved into two broad categories: video analysis, which relies on traditional video footage, and wearable analytics, which relies on information captured through sensors like wearable devices. However, recent efforts are increasingly focused on integrating video information about sporting events, event information about incidents occurring during the event, and tracking information collected and analyzed based on the movement of sports players. In other words, demand for displaying player information alongside sporting events is steadily increasing.

[0005] The following provides a summary of specific embodiments disclosed in this disclosure. It should be understood that the aspects presented in the following summary are intended merely to provide a brief overview of specific embodiments and are not intended to limit the scope of this disclosure. Therefore, it should be noted in advance that this disclosure may include various aspects not presented below.

[0006] The present invention and the inventive concepts disclosed herein provide methods, devices, and computer-readable storage media for displaying player information in motion images.

[0007] The objective of this invention is to provide a method for easily displaying player information in a video recording without incurring high costs for video analysis by obtaining sensor-based location information for each player participating in a sports event and displaying at least some of the obtained location information and additional information based on analysis at corresponding locations in a video recording related to the sports event.

[0008] The object of the present invention is to provide a device capable of easily displaying player information on a filmed video without incurring high costs for video analysis by obtaining sensor-based location information for each player participating in a sports event and displaying at least some of the obtained location information and additional information based on analysis at a corresponding location in a filmed video related to the sports event.

[0009] However, the problem to be solved in this paper is not limited to this, and can be expanded in various ways without departing from the scope and idea of ​​this paper.

[0010] Embodiments include methods, devices, and computer-readable storage media for displaying player information in a motion video.

[0011] According to one embodiment, a method for displaying player information in a motion video, performed by a computing device, is provided. The method may include: acquiring a video of a motion event in a target time period; acquiring sensor-based location information of the target time period for each of the players participating in the motion event; determining a corresponding location within the video corresponding to the sensor-based location information for each of the players; and displaying information for each of the players at each of the corresponding locations.

[0012] According to another embodiment, a device for displaying player information in a motion picture is provided. The device includes a processor; and a memory; wherein the processor is configured to: acquire a captured image for a motion picture event of a target time period; acquire sensor-based location information of the target time period for each of the players participating in the motion picture event; determine a corresponding location within the captured image corresponding to the sensor-based location information for each of the players; and display information for each of the players at each of the corresponding locations.

[0013] According to another embodiment, a computer-readable storage medium is provided comprising instructions executable by a processor. The instructions are for displaying player information in a motion picture, and are configured to be executed by the processor to cause the processor to: acquire a captured image of a motion event in a target time period; acquire sensor-based location information of the target time period for each of the players participating in the motion event; determine a respective corresponding location within the captured image corresponding to the sensor-based location information for each of the players; and display information for each of the players at each of the corresponding locations.

[0014] The above exemplary embodiments and other exemplary embodiments will be explained or clarified by the detailed description set forth below of exemplary embodiments to be read in connection with the accompanying drawings.

[0015] The disclosed technology may have the following effects. However, this does not mean that a particular embodiment must include all or only the following effects, and thus the scope of the disclosed technology should not be construed as being limited thereby.

[0016] According to an embodiment of the present disclosure, it is possible to provide a method for acquiring sensor-based location information for each player participating in an exercise event, and displaying at least some of the acquired location information and additional information based on analysis at a corresponding location in a captured image associated with the exercise event.

[0017] Therefore, while preventing the high overall costs required to obtain location information through image analysis, it is possible to display location information and / or additional information about the corresponding player at the exact corresponding location of the captured image with greater simplicity and lower computational complexity.

[0018] The above description of the invention is not intended to be an exhaustive list of all aspects of the invention. It should be understood that the invention encompasses all methods, devices, and systems capable of being implemented from all appropriate combinations of the various aspects disclosed in the detailed description and claims below, as well as those summarized above.

[0019] Figure 1 is an example of image-based location information acquisition.

[0020] Figure 2 is an example of sensor-based location information acquisition.

[0021] Figure 3 illustrates an example of a state including a missing object.

[0022] Figure 4 illustrates an example of a state including a misdetected object.

[0023] Figure 5 shows object matching between frames in an image containing multiple objects.

[0024] Figure 6 shows a comparison between the GPS method and the OTS method.

[0025] FIG. 7 illustrates an exemplary system in which a method for displaying player information according to one embodiment of the present disclosure can be performed.

[0026] FIG. 8 is a block diagram of a sensor device that can be used to acquire sensor-based location information according to one embodiment of the present disclosure.

[0027] Figure 9 is a block diagram of a server according to one embodiment of the present disclosure.

[0028] FIG. 10 is a conceptual diagram of a method for displaying player information in a motion video according to one embodiment of the present disclosure.

[0029] FIG. 11 is an example diagram showing a state in which player information is displayed in an exercise video according to one embodiment of the present disclosure.

[0030] Figure 12 is a drawing for detailed explanation of player information displayed in Figure 11.

[0031] FIG. 13 is a schematic flowchart of a method for displaying player information in a motion video according to one embodiment of the present disclosure.

[0032] Figure 14 is a detailed flowchart for the corresponding position determination step of Figure 13.

[0033] Figure 15 is a second detailed flowchart for the corresponding position determination step of Figure 13.

[0034] FIG. 16 is an example diagram of a display screen of a captured image and sensor-based location information according to one embodiment of the present disclosure.

[0035] Figure 17 is an example of a screen for inputting corners of a stadium by a user.

[0036] Figure 18 is a diagram illustrating the conversion between the image coordinate system (Pixel Coordination) and the stadium coordinate system (Pitch Coordination).

[0037] Figure 19 is an example of a location input request screen for the first player.

[0038] Figure 20 is an example of a position input screen for the first player by a user.

[0039] Figure 21 is an example of a position input screen for a second player by a user.

[0040] Figure 22 is an example of a location input screen for a third player by a user.

[0041] Figure 23 is an example of the position input results for multiple players.

[0042] Figure 24 is a diagram illustrating the conversion between the image coordinate system (Pixel Coordination) and the global coordinate system (Global Coordination).

[0043] Figure 25 is an example of a procedure for receiving a batch change value for each corresponding position of multiple players from a user.

[0044] Figure 26 shows the information display status according to the batch change result of Figure 25.

[0045] Figure 27 is an example of a procedure for receiving individual change values ​​for the corresponding positions of a specific player among multiple players from a user.

[0046] Figure 28 shows the information display status according to the individual change results of Figure 27.

[0047] Figure 29 is a block diagram showing a GNSS module configuration for time synchronization between captured images and sensor-based location information.

[0048] FIG. 30 is a block diagram showing an exemplary configuration of a computing system in which a method according to one embodiment of the present disclosure can be performed.

[0049] The present invention can be modified in various ways and has various embodiments, and specific embodiments are illustrated in the drawings and described in detail.

[0050] However, this is not intended to limit the present invention to a specific embodiment, but should be understood to include all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention.

[0051] While terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a "second component," and similarly, a second component may also be referred to as a "first component." The term "and / or" includes any combination of multiple related items described herein or any item among multiple related items described herein.

[0052] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0053] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0054] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0055] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. In order to facilitate an overall understanding in describing the present invention, identical reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0056] Since the embodiments described in this document are intended to clearly explain the idea of ​​the present invention to a person having ordinary skill in the art to which the present invention pertains, the present invention is not limited to the embodiments described in this document, and the scope of the present invention should be interpreted to include modified or altered examples that do not depart from the idea of ​​the present invention.

[0057] The terms used in this document are selected from commonly used terms in the technical field to which the present invention pertains. However, their meanings may vary depending on the intentions of those skilled in the art, customs, or the emergence of new technologies. However, if a specific term is defined and used with an arbitrary meaning, the meaning of that term will be described separately. Therefore, the terms used in this document should be interpreted based on their actual meaning and the overall content of this document, rather than simply their names.

[0058] The drawings attached to this document are intended to facilitate the explanation of the present invention, and the shapes depicted in the drawings may be exaggerated or abbreviated as necessary to help the understanding of the present invention, and therefore the present invention is not limited by the drawings.

[0059] In this document, if it is determined that a specific description of the composition or function of a public notice related to the present invention may obscure the gist of the present invention, a detailed description thereof will be omitted as necessary.

[0060]

[0061] A method for displaying player information in a motion image according to one aspect of the present disclosure may be performed by a computing device, and the method may include: obtaining a captured image for a motion event in a target time period; obtaining sensor-based location information for each of the players participating in the motion event in the target time period; determining a corresponding location within the captured image corresponding to the sensor-based location information for each of the players; and displaying information for each of the players at each of the corresponding locations.

[0062] According to one aspect, the sensor-based location information may be location information obtained based on a global navigation satellite system (GNSS).

[0063] According to one aspect, the player information may include at least one of: personal information of the player; sensor-based location information of the player; speed of the player; and trajectory of the player during a predetermined time interval.

[0064] According to one aspect, the step of determining each corresponding position may include: displaying at least a portion of the captured image and the sensor-based position information; receiving information from a user about positions of corners of the stadium within the captured image; receiving information from the user about positions of a first player within the captured image; and receiving information from the user about positions of a second player within the captured image.

[0065] According to one aspect, the first player and the second player may be players with the greatest distance between themselves among the plurality of players included in the filmed video.

[0066] According to one aspect, the step of receiving information about the position of the first player within the filmed video and the step of receiving information about the position of the second player within the filmed video may be performed for different frames among a plurality of frames included in the filmed video.

[0067] According to one aspect, the information about the position of the first player within the filmed image and the information about the position of the second player within the filmed image may be position information in different frames of a single player.

[0068] According to one aspect, the step of receiving information about a location of the first player within the captured image may be performed in response to a determination that the sensor-based location information for the first player is acquired in RTK-GPS mode.

[0069] According to one aspect, the step of determining each corresponding position may further include, after the step of receiving information from the user about the position of the second player within the captured image, the step of calculating the corresponding position of each of the plurality of players based on the information about the positions of the corners of the stadium, the first player, and the second player within the captured image; and the step of receiving, from the user, a batch change value for the corresponding position of each of the plurality of players that has been calculated.

[0070] According to one aspect, information about the positions of the corners of the stadium within the captured image can be used for conversion between an image coordinate system (Pixel Coordination) representing the position within the captured image and a stadium coordinate system (Pitch Coordination) representing the position within the stadium where the athletic event is performed.

[0071] According to one aspect, information about the position of the first player within the captured image and information about the position of the second player within the captured image can be used for conversion between a global coordinate system (Global Coordination) used by the sensor-based position information and an image coordinate system (Pixel Coordination) indicating the position within the captured image.

[0072] According to one aspect, the step of determining each corresponding position may perform a conversion between a global coordinate system (Global Coordination) used by the sensor-based position information and a pitch coordinate system (Pitch Coordination) indicating a position within the stadium where an athletic event is performed, based on information about the positions of the corners of the stadium within the captured image, information about the positions of the first player within the captured image, and information about the positions of the second player within the captured image.

[0073] According to one aspect, the step of displaying the captured image and at least a portion of the sensor-based location information may be configured to display a relative positional relationship of each sensor-based location information with respect to a representative value of a plurality of sensor-based location information; identification information of a sensor device corresponding to each sensor-based location information; and a location input request message for the first player or the second player.

[0074] According to one aspect, the first player and the second player can be determined based on the distance between a plurality of sensor-based positional information.

[0075] According to one aspect, after the step of receiving input of the above-mentioned batch change value from the user, the method may further include a step of receiving input of a change value for a corresponding position of one of the plurality of players from the user.

[0076] According to one aspect, the captured image is captured by a camera having a first GNSS module, and the sensor-based location information can be determined based on a second GNSS module of the same type as the first GNSS module.

[0077] According to one aspect, the step of determining the corresponding position may be configured to match each frame of the captured image with the acquisition time of the sensor-based position information based on time information of the first GNSS module and the second GNSS module.

[0078] A device for displaying player information in a motion image according to one aspect of the present disclosure includes a processor; and a memory; wherein the processor may be configured to: acquire a captured image for a motion event in a target time period; acquire sensor-based location information for each of the players participating in the motion event in the target time period; determine a corresponding location within the captured image corresponding to the sensor-based location information for each of the players; and display information for each of the players at each of the corresponding locations.

[0079] A computer-readable storage medium comprising instructions executable by a processor according to one aspect of the present disclosure, wherein the instructions are for displaying player information in a motion image, and are configured to be executed by the processor to cause the processor to: obtain a captured image for a motion event in a target time period; obtain sensor-based location information for each of the players participating in the motion event in the target time period; determine a respective corresponding location within the captured image corresponding to the sensor-based location information for each of the players; and display information for each of the players at each of the corresponding locations.

[0080]

[0081] Obtain location information

[0082] As previously discussed, the explosive growth of the sports industry and the advancement of sports science are steadily increasing the importance of sports analytics. Amidst this trend, Electronic Performance Tracking Systems (EPTS) are increasingly being introduced, particularly in major sports like soccer, to track players during matches and training. In EPTS, the location and movements of players serve as crucial baseline data for providing a variety of additional information. Therefore, various efforts are ongoing to facilitate the acquisition and increase the accuracy of tracking information on players' locations.

[0083]

[0084] More specifically, data science has become a crucial tool in the sports industry, as it has in many other industries. The data primarily utilized in sports can be divided into event data and tracking data. Event data may include information about ball-related events that may occur during a sporting event, while tracking data may include information collected about the positions of individual players over a specific time scale.

[0085] In dynamic team sports like soccer, basketball, or ice hockey, player tracking data can provide rich information, such as player interactions and off-the-ball movements that may be overlooked in event data. In soccer, for example, such player tracking data can be used for a variety of applications, including formation and role estimation, spatial control analysis, playing style identification, and even false predictions.

[0086]

[0087] In recent years, different types of tracking systems have been proposed and successfully adopted for soccer matches to obtain such tracking data, such as the Global Positioning System (GPS), the Local Positioning System (LPS), or the Optical Tracking System (OTS) based on multiple cameras.

[0088] GPS has the advantage of being low cost and easy to install compared to other methods, but it is known to be more sensitive to the measurement environment, such as weather and stadium conditions. On the other hand, OTS can provide more accurate tracking information when equipped with a sufficient number of high-definition cameras positioned to surround the stadium from different angles. However, it is not easy to install OTS equipment in all stadiums, and it is often difficult to install OTS equipment in training stadiums or away stadiums that are not home stadiums. Moreover, when trying to implement OTS with a small number of cameras, low-quality raw data is obtained and a manual modification process is required, making it difficult to secure reliable tracking data.

[0089]

[0090] In relation to this, Fig. 1 is an example of image-based location information acquisition. As illustrated in Fig. 1, a positioning method that calculates the location of a player from an image captured by a camera (3) can be utilized.

[0091] In the image-based positioning method, the player to be tracked must be recognized within the video to accurately calculate the player's location.

[0092] For example, for a player located in the second area (2) of Fig. 1, the position of the player to be recognized and tracked can be calculated.

[0093] However, for a player located in the first area (1) of Fig. 1, an occlusion event may occur, making it difficult to recognize the player. Specifically, since multiple players are densely located in the first area (1), occlusion may occur, in which the player to be tracked in the image is occluded by another player. Therefore, the position of the tracked player occluded by another player may not be accurately acquired.

[0094] In other words, the positioning method using video may not be able to respond to occlusion situations, such as when the player to be tracked is obscured by another player within the video.

[0095]

[0096] Figure 2 is an example of sensor-based position information acquisition. As illustrated in Figure 2, a positioning method that calculates the player's location using sensor-based positioning devices, such as a GPS module, may be utilized.

[0097] Specifically, the positioning method using a GPS module calculates the location of the athlete to be tracked based on signals transmitted from satellites (4a, 4b, 4c, 4d). However, signals transmitted from satellites can be significantly affected by structures surrounding the athlete to be tracked.

[0098] For example, GPS signals transmitted from some satellites (4b, 4c) of FIG. 2 can be transmitted into the stadium without being affected by structures surrounding the player to be tracked. However, GPS signals transmitted from some satellites (4a, 4d) of FIG. 2 may be affected by structures surrounding the player to be tracked and may not reach the stadium. In this case, if the GPS signals transmitted from some satellites (4a, 4d) are affected by structures surrounding the player, the position of the player calculated from the GPS signals may have an error.

[0099]

[0100] Meanwhile, object tracking requires object detection and identification. That is, tracking a specific object may involve obtaining information about the object's successive locations over a time interval of a predetermined length and determining its trajectory. For example, in team sports, object tracking is often performed on multiple objects rather than a single object. Therefore, after multiple objects are detected, identifying which object each object corresponds to is essential for object tracking.

[0101]

[0102] In this regard, for example, in the case of sensor-based location information acquisition such as GPS or LPS, a separate identification process may not be required. Multiple sensors for acquiring location information may each correspond to a specific object among multiple objects, and each sensor device may be configured to have a device ID. Accordingly, location information measured by a specific sensor can be identified as location information for a specific object using the device ID without a separate identification process.

[0103]

[0104] On the other hand, in acquiring image-based location information such as OTS, when multiple objects are detected in each frame of a plurality of frames constituting a video, it is required to identify which object corresponds to the object detected in the next frame. That is, for example, when the first to tenth objects exist as tracking targets, and when ten objects are detected in the first frame and ten objects are detected in the second frame, it must be determined which object in the first frame corresponds to which object in the second frame. It is possible to secure tracking data for the trajectory of the object by matching time series information for a time interval having a predetermined time length.

[0105] To this end, the process of acquiring tracking data using image-based location information may include object detection for each frame and object matching between frames. However, errors may occur in both the object detection and object matching processes.

[0106]

[0107] First, in the object detection process, a false negative error may occur, in which an object that should be detected is not detected, or a false positive error, in which an incorrect object is detected.

[0108] Fig. 3 exemplarily illustrates a false negative error state including a missing object. As illustrated in Fig. 3, objects located in detection areas (6a, 6b), such as bounding boxes (B-boxes) within an image acquired by a camera, may be detected normally, but three objects located in detection area (7) may experience mutual occlusion, resulting in a problem in which at least one of the three objects is not detected. That is, some of the objects to be tracked may not be detected in the corresponding frame.

[0109] FIG. 4 exemplarily illustrates a false positive error condition including an incorrectly detected object. As illustrated in FIG. 4 , objects located in detection areas (6a, 6b, 6c, 6d) may be normally detected objects. However, objects located in detection area (8) may be, for example, referees, and since they are not objects to be tracked, they may be detected even though they should not be detected, which may cause problems. For example, in some frames, 11 objects may be detected instead of the required 10, or a situation may occur where detection of a specific object is missed and at the same time, a situation may occur where 10 objects, including 9 objects to be tracked and 1 object not to be tracked, are detected.

[0110]

[0111] Errors in the object detection process can further increase the likelihood of errors in the object matching process. To obtain continuous positional information for a specific object, for example, matching between detected objects in consecutive frames can be performed. Various algorithms can be used to match detected objects across frames. For example, the Hungarian algorithm, which minimizes matching costs, such as the sum of the distance differences between objects, can be applied.

[0112] FIG. 5 illustrates object matching between frames in an image containing multiple objects. Depending on the arrangement of the detected objects, such as when the distance between specific objects is very close, an error may occur in which different objects in each frame are matched, and the possibility of error is particularly high when an error has already occurred during the object detection step. As illustrated in FIG. 5, the object matching procedure is examined while passing through multiple frames, including the previous frame (pf), the current frame (cf), and the next frame (nf). The three objects on the right side of the frame, which are not assigned separate signs, can be matched as detected objects frame by frame without difficulty. However, an error that may occur in the frame-to-frame matching of the first object (11) and the second object (12) is examined with reference to FIG. 5. The first object (11) may be detected as an object (11a) in the previous frame and may also be detected as an object (11c) in the next frame, but a situation may occur in which detection is missed due to an error in the object detection procedure, even though it should have been detected as an object (11b) in the current frame. The second object (12) may be detected as object (12a) in the previous frame, as object (12b) in the current frame, and as object (12c) in the next frame. When examining object matching between frames, when performing object matching between the previous frame and the current frame, it may be determined that object (12a) and object (12b) match each other, and object (11a) does not have a matching object. When performing object matching between the current frame and the next frame, it may be a problem whether object (12b) should be matched with object (11c) or with object (12c). If it is matched with object (11c), an error occurs in which the first object and the second object are incorrectly matched.Even if the distance between multiple objects is relatively close and there is continuous movement, proper object matching may be performed, but as disclosed in FIG. 5, if the detected position for the first object (11) instantaneously moves from the position of the object (11a) to the position of the object (11c), proper inter-frame object matching may be difficult to expect.

[0113] That is, when tracking objects using image-based location information such as OTS, problems may arise in object detection, and problems may also arise in frame-by-frame matching between detected objects. To solve this problem, a method has been proposed to utilize images from multiple angles by installing multiple cameras at different locations surrounding the objects. However, not only is the installation cost high, but it is also difficult to secure appropriate locations for installing such multiple cameras within the stadium.

[0114]

[0115] As discussed above, object tracking using image-based location information such as OTS and sensor-based location information such as GPS may have different advantages and disadvantages. Figure 6 shows a comparison between the GPS method and the OTS method.

[0116] As illustrated in Figure 6, in terms of object tracking, GPS can track specific objects simply by collecting time-series location data from specific sensors, utilizing the device ID of the sensor corresponding to each object, without a separate identification procedure. On the other hand, OTS must keep in mind the possibility of errors occurring in the object detection and frame-to-frame matching procedures.

[0117] However, GPS generally has an error range of about 600 to 3,500 mm in terms of position accuracy. On the other hand, OTS can obtain information about the location of an object with high accuracy, with an error range of 100 to 350 mm.

[0118] The accuracy of displacement information such as speed may be higher in GPS than in OTS. Due to the factors of Doppler displacement measurement using signals from satellites, the measurement error (E) of displacement-based information such as speed measured by GPS is higher than that of OTS, unlike the accuracy of determining the position at a specific point in time. rGPS ) is the measurement error (E) in the OTS method rOTS ) appears much less frequently than in the previous example.

[0119] When it comes to ball detection in sports, the On-Screen Shot (OTS) method offers advantages. OTS can detect ball location by processing captured images, eliminating the need for specific sensors or devices. However, detecting ball location via GPS requires inserting a GPS sensor into the ball. This, especially in professional sports, where the outcome of a game is crucial, can be a significant challenge, as inserting a sensor into the ball can potentially impact the outcome.

[0120] In terms of action event recognition (AE) in sports, the OTS method offers advantages. GPS-based methods rely on acquiring information about an object's location, making it difficult to recognize specific actions, such as passes or shots. In contrast, OTS methods rely on video analysis, making it possible to detect specific action events, such as player shots or passes, by establishing appropriate video processing procedures and analysis algorithms.

[0121] From a legal rights perspective, the OTS method may have some advantages. In the case of GPS, data measured through it may belong to the owner of the GPS sensor. For example, in the case of sports data, the rights to GPS-based data collected about a specific team's players belong to the team. While analyzing the performances of competing teams can be just as important as analyzing one's own team, obtaining GPS data on players from other teams can be challenging. On the other hand, videos for OTS can be made publicly available through broadcasts, such as sports broadcasts. Therefore, tracking data on players or games from other teams can be obtained and utilized by analyzing broadcasted footage.

[0122]

[0123] As discussed above, object tracking using image-based location information and sensor-based location information have distinct advantages and disadvantages. However, from an economic perspective, OTS requires multiple high-performance cameras to address occlusion issues, as well as a high-capacity data storage medium to store and utilize video data for image analysis. Using commercial cloud systems for this purpose incurs significant cloud service fees, considering the large volume of video data. Furthermore, even if a separate private server is installed at the stadium, the server construction costs are high. Furthermore, since the server is only installed at the home stadium, it cannot analyze away games or training videos when used for sports events.

[0124] Regarding this, in the case of GPS, the shortcomings in all aspects except measurement accuracy are judged to be tolerable considering the advantages, and in terms of measurement accuracy, for example, if a solution such as RTK is introduced, it is possible to reduce the error range to about 10 to 30 mm. The disadvantage of introducing such a high-accuracy RTK solution is that it requires a considerable amount of money, but in the case of the RTK method, it is quite feasible to customize the method to achieve a reasonable level of accuracy while also achieving a reasonable level of equipment cost.

[0125]

[0126] The embodiments of the present disclosure take such aspects into consideration, and according to the embodiments of the present disclosure, location information is acquired based on a sensor for each player participating in an exercise event, and at least some of the acquired location information and additional information based on analysis are displayed at a corresponding location in a captured video associated with the exercise event, thereby enabling player information to be easily displayed in the captured video without incurring high overall costs for image analysis.

[0127] That is, by introducing a sensor-based location information acquisition method with improved accuracy, such as RTK-GPS, the accuracy of sensor-based location information can be improved to a reasonable level, and by displaying the sensor-based location information thus acquired at an appropriate location in the motion video, tracking data such as the player's location, speed, and trajectory can be integrated and provided in the motion video without incurring the cost required for image-based location determination.

[0128]

[0129] In this document, for the convenience of explanation below, RTK-GPS may be disclosed as an example of a sensor-based position information acquisition procedure with improved accuracy, but it should be noted that this is only an example and that the sensor-based position information according to embodiments of the present document is not limited thereto.

[0130]

[0131] Terminology

[0132] Certain technical terms may be used in this document, and the following provides a definition of the terms used in this document to establish definitional support for the terms used in this document.

[0133] The following is a summary of preferred definitions of some terms used in this document. The definitions provided below are provided for illustrative purposes only and are not intended to be exhaustive or limiting.

[0134]

[0135] The term "image-based location information" may refer to information about the location of an object determined using an image containing the object. The image-based location information may include information about the location of at least one object determined from a captured image of one or more objects. For example, the image-based location information may include information about the location of each player determined by analyzing video of a team sporting event involving multiple players, but is not limited thereto. The image-based location information should be interpreted in a comprehensive sense, including location information obtained from captured images of any type of object.

[0136] The term "sensor-based location information" may refer to information about the location of an object determined using signals from sensors corresponding to the object. The sensor-based location information may include information about the location or displacement of at least one object determined based on signals from sensors corresponding to each of one or more objects. For example, it may include, but is not limited to, location information acquired through positioning solutions such as a Global Navigation Satellite System (GNSS) such as GPS or a Local Positioning System (LPS), and should be interpreted in a comprehensive sense including location information of an object acquired through signals from sensors corresponding to a specific object.

[0137] For example, an acceleration-related signal can be measured from an inertial measurement unit (IMU) corresponding to an object, and by integrating such acceleration measurement values ​​to obtain information on velocity, and then integrating this again to obtain displacement, a form of measuring the movement path of the corresponding object can be implemented. It should be interpreted that information on the displacement of an object like this can also be included in sensor-based location information.

[0138]

[0139] System

[0140] FIG. 7 illustrates an exemplary system in which a player information display method according to an embodiment of the present disclosure can be performed. Hereinafter, a player information display system (1000) according to an embodiment of the present disclosure will be described with reference to FIG. 7. However, it should be understood that the player information display procedure according to the present disclosure is not limited to being performed only by the system configuration of FIG. 7, and that any hardware configuration and combination thereof for acquiring sensor-based position information and displaying it at a corresponding position of a motion image can be employed to implement the player information display method according to embodiments of the present disclosure.

[0141]

[0142] As illustrated in FIG. 7, the system (1000) may include a sensing platform (1100) and a server (1500). In one aspect, the system (1000) may further include a terminal (1700).

[0143] The system (1000) can detect information about an object, for example, a player (10), during a target time period through a sensing platform (1100), and determine information about the player from the detected information through a server (1500). In addition, information about the trajectory of the determined object can be displayed through a terminal (1700). In FIG. 7, the server (1500) and the terminal (1700) are illustrated as separate entities as an example, but it should be noted that the embodiments according to the present disclosure are not limited thereto. For example, a tablet PC in which the server (1500) and the terminal (1700) are implemented as an integral unit can be used to provide information about the player in real time and also to display the player information together with a motion video in real time. In addition, for example, although not shown in FIG. 7, a relay device such as a live hub for receiving data to collect information from the sensing platform (1100) may be further provided in advance of the server (1500), so that information from a plurality of devices provided in the sensing platform is received by the relay device, and such information may be configured to be transmitted again to the server (1500) or the terminal (1700).

[0144]

[0145] Below, exemplary components of an object tracking system according to an embodiment of the present disclosure are described.

[0146]

[0147] Sensing Platform

[0148] The sensing platform (1100) can detect various information about an object (10), such as a player, for example. For example, the sensing platform (1100) can perform positioning of the object (10) or detect movement of the object (10).

[0149]

[0150] For example, the sensing platform (1100) can detect kinematic information about an object (10). Kinematic information is information about a location, posture, or movement of the object (10), and the kinematic information may include at least one of locational information, orientational information, and movement information, and the movement information may include at least one of velocity, acceleration, jerk, angular velocity, angular acceleration, angular jerk, magnitude thereof (e.g., in the case of velocity, speed), and direction thereof (e.g., in the case of velocity, direction of movement).

[0151] The result of the positioning performed on the object (10) by the sensing platform (1100) may be provided as the location information of the object, or the location of the object may be determined by processing at least one of the various kinematic information described above or a combination thereof, and provided as the location information of the object.

[0152]

[0153] The sensing platform (1100) may be provided in various forms for sensing the various types of information described above. For example, the sensing platform (1100) may be implemented as a sensor-based platform that utilizes sensor devices (1300) that are provided to correspond to each object (10) and provide signals regarding measurement results.

[0154]

[0155] Sensor-based platform

[0156] Below, we describe a sensor-based sensing platform.

[0157] A sensor-based sensing platform may include a sensor device (1300). The sensing platform may obtain activity information about an object (10) corresponding to the sensor device (1300) using a sensor mounted on the sensor device (1300). According to one aspect, the sensor device (1300) may be implemented in the form of an attachable device attached to each object (10), but is not limited thereto and may include any form of device configured to perform sensing on a corresponding object.

[0158] A sensor device (1300) according to an example may be attached to an object (10) and may be used to detect activity information of the object (10). For example, the sensor device (1300) may include a Global Positioning System (GPS) sensor and may be used to determine the position of the object (10) to which the sensor device (1300) is attached.

[0159] One or more sensor devices (1300) may be attached to one object (10). In particular, when it is difficult for a sensing platform to obtain all the information it wants to obtain through the sensor device (1300) with a single sensor device (1300), it may be necessary to attach multiple sensor devices (1300) to one object (10). For example, one attachable device including a GPS sensor and an IMU (Inertial Measurement Unit) sensor and another attachable device including a heart rate sensor may be attached to the torso and wrist of the object (10), respectively, and the attachable device attached to the torso may be configured to sense the position and movement of the object (10).

[0160]

[0161] For example, a sensor-based sensing platform can obtain kinematic information using a sensor device (1300).

[0162] Below, we describe several examples of sensor-based sensing platforms acquiring kinematic information.

[0163]

[0164] The sensing platform can obtain location information using the positioning module of the sensor device (1300).

[0165] For example, the sensor device (1300) includes a global positioning module (in other words, a satellite positioning module or a GNSS (Global Navigation Satellite System) module), and the sensing platform can measure the position of the object (10) by performing global positioning using the global positioning module. Specifically, the sensing platform can perform global positioning of the object (10) by having the GNSS module receive satellite signals from navigation satellites (20) and derive a global position (e.g., latitude and longitude) from the received satellite signals using a triangulation technique. Meanwhile, the sensing platform can additionally include a base station for performing RTK (Real-Time Kinematic) for more accurate positioning. The sensing platform can use the global position as activity information as it is, but can also process the global position into a local position defined by the pitch coordination system for the stadium and use this as activity information. Here, the stadium coordinate system may be a two-dimensional planar coordinate system with the longitudinal and transverse directions of the stadium as axes and a point on the stadium or its periphery (e.g., one of the corners or the center of the stadium) as the origin. It should be noted in advance that all location-related information processed below may be processed without limitation according to the stadium coordinate system.

[0166] For another example, the sensor device (1300) includes a local positioning module, and the sensing platform can measure the location of the object (10) by performing local positioning using a local positioning sensor network that includes the sensor device (1300). The local positioning sensor network includes a tag node that is attached to the object that is the target of positioning and moves, and an anchor node (30) that is fixedly installed in the positioning area, and can perform positioning using a local positioning system (LPS) signal that is transmitted and received between the tag node and the anchor node. The sensing platform can perform local positioning of the object (10) using the results of transmission and reception of LPS signals between the sensor device (1300) that includes a local positioning module and is attached to the object (10) that is the target of positioning and operates as a tag node, and the anchor node (30) that is fixedly installed in the playground or its surroundings.

[0167]

[0168] The sensing platform can obtain movement information and / or direction information using the motion sensing module of the sensor device (1300).

[0169] For example, the sensor device (1300) includes an IMU sensor (or an Attitude and Heading Reference System (AHRS) sensor), and the sensing platform can obtain movement information and / or direction information of the object (10) or a body part of the object (10) using the acceleration, angular velocity, and azimuth detected by the IMU sensor. When the sensor device (1300) is attached to the torso of the object (10), movement information according to the positional movement of the object (10) can be obtained, and when the sensor device (1300) is attached to the foot or leg of the object (10), movement information of the arm or leg of the object (10) can be obtained.

[0170]

[0171] Meanwhile, the sensing platform can produce other kinematic information using the measured kinematic information. For example, the sensing platform can output the speed based on the global position continuously measured by the GPS module and its sampling interval. Since the production of kinematic information is possible through simple mathematical operations such as vector operations or calculus on the time axis, a detailed description thereof will be omitted. Here, the production of kinematic information can be performed internally in the sensor device (1300) or in a server (1500) that receives information from the sensor device (1300). In the sensor device (1300), the sensing module can perform the production directly or the controller of the sensor device (1300) can perform the production based on the detection result of the sensing module.

[0172]

[0173] sensor devices

[0174] FIG. 8 is a block diagram of a sensor device that can be used to acquire sensor-based location information according to one embodiment of the present disclosure.

[0175] As illustrated in FIG. 8, a sensor device (1300) according to one embodiment of the present disclosure may include a sensing module (1310), a communication module (1320), a controller (1330), a memory (1340), and a battery (1350).

[0176]

[0177] The sensing module (1310) can detect various signals to obtain activity information. The sensing module may be a positioning module or a motion sensing module.

[0178]

[0179] The positioning module may be a satellite positioning module (1311) or a local positioning module (1313). The satellite positioning module (1311) may perform positioning using a navigation satellite system, i.e., GNSS. Here, the GNSS may include a global positioning system (GPS), GLONASS, BeiDou, Galileo, etc. Specifically, the global positioning module may include a satellite antenna that receives satellite signals and a positioning processor that performs positioning using the received satellite signals. For example, the satellite positioning module may be a GPS module that includes a GPS antenna that receives GPS signals and a GPS processor that performs positioning using the GPS signals. In this case, the sensor device (1300) may operate as a GPS receiver, receive GPS signals from satellites, and obtain latitude, longitude, altitude, speed, azimuth, diluition of precision (DOF), and time therefrom.

[0180] The local positioning module (1313) can perform positioning in collaboration with a sensor network. A sensor device (1300) including a local positioning module can operate as a tag node of a local positioning sensor network to transmit and receive LPS signals with surrounding anchor nodes, and positioning can be performed using the results of transmitting and receiving the LPS signals. For example, the sensor device (1300) can transmit LPS signals as a transmitter of a sensor network for local positioning and anchor nodes can receive the LPS signals, or anchor nodes can transmit LPS signals and the sensor device (1300) can receive LPS signals as a receiver of a sensor network for local positioning. At this time, the LPS signal can be periodically broadcast according to a communication method such as ultra-wideband (UWB) communication, Bluetooth, Wi-Fi, or RFID, and a time or device identifier can be embedded in the LPS signal. Position estimation can be performed based on the measurement results according to the transmission and reception of LPS signals, and can use the Received Signal Strength (RSS) technique, the Time-of-Arrival (ToA) technique, the Time Difference-of-Arrival (TDoA) technique, the Angle-of-Arrival (AOA) technique, triangulation technique, hyperbolic triangulation technique, etc. Position estimation can be performed by the master of the sensor network, and any one of the tag nodes, the anchor node, or the server can function as the master of the sensor network. If the sensing platform additionally includes an RTK base station, the base station can also be equipped with the position estimation function of the master.

[0181]

[0182] The motion sensing module (1315) may be referred to as a kinematic sensing module and can detect movement and / or posture. Here, the motion sensing module (1315) may include an IMU module or an AHRS module, and the IMU module and the AHRS module include sensors including an accelerometer and a gyroscope and optionally a magnetometer, and can measure movement and posture from the detection results of the sensors.

[0183]

[0184] Meanwhile, the above-described sensing modules do not necessarily have to be all equipped in one sensor device (1300), and multiple sensing modules of the same type may be equipped in one sensor device (1300), and the sensing modules equipped in each sensor device (1300) may be different. For example, the sensing platform may include two different sensor devices (1300), one of which includes a GPS sensor and an IMU (Inertial Measurement Unit) sensor to obtain location information and movement information about the object (10), and one of which includes one attached to the torso of the object (10) and the other attached to the wrist of the object (10).

[0185]

[0186] The sensor device (1300) can transmit and receive data with an external device such as a server (1500) or another sensor device (1300) through a communication module (1320). The communication module (1320) can perform wired communication and / or wireless communication. The sensor device (1300) can transmit and receive data with an external device using a wireless communication module that performs wireless communication of various standards such as a mobile communication network (e.g., LTE, 5G, etc.), Wi-Fi, Bluetooth, Zigbee, or other standards. For example, the wireless communication module (1320) can transmit information acquired by the sensor device (1300) using the sensing module (1310) to the server (1500) in real time so that the system can monitor the object (10) using activity information, or, when multiple sensor devices (1300) are attached to one object (10), data can be transmitted and received between the sensor devices (1300). In addition, the sensor device (1300) can transmit and receive data with an external device using a wired communication module that performs universal serial bus (USB) communication or wired local area network (LAN) communication. For example, the wired communication module can collectively transmit data collected by the sensor device (1300) to the server (1500) or a docking station that performs charging and / or data management of the sensor device (1300).

[0187]

[0188] The controller (1330) can control the overall operation of the sensor device (1300). The controller (1330) can be implemented as a hardware configuration, a software configuration, or a combination thereof. From a hardware perspective, the controller (1330) can be provided in various forms capable of performing operations or data processing, including electronic circuits, integrated circuits (ICs), microchips, and processors. In addition, since the physical configuration of the controller (1330) is not necessarily limited to a single physical entity, the controller (1330) can be provided as a single processor that comprehensively processes all processing of the sensor device (1300), a plurality of processors that each perform different functions, or can be provided in a form combined with some of the other components of the sensor device (1300). For example, the controller (1330) may be provided in a form that includes a GPS processor that processes GPS signals to perform positioning, an IMU processor that performs various operations using the results detected by the sensor of the IMU module, and a main processor that controls the operation and overall operation of the sensor device (1300).

[0189]

[0190] The memory (1340) can store various data related to operations in the sensor device (1300). For example, the memory (1340) can store firmware that manages the operation of the sensor device (1300) or detection results of sensors of the sensor device (1300). The memory (1340) can be provided as various volatile or non-volatile memories.

[0191] The battery (1350) can provide power necessary to drive the operation of the sensor device (1300). The battery (1350) can be provided as a built-in or detachable type, and can be charged by receiving power from an external power source.

[0192]

[0193] Exercise video acquisition platform

[0194] Below, a motion image acquisition platform is described. The motion image acquisition platform may include, for example, a camera (1200).

[0195] A camera (1200) may be positioned on or around a sports field to capture images of objects (10) moving within the field or sports field, for example, for sports analysis applications. The camera (1200) may be installed semi-permanently (e.g., installed on an auxiliary facility of the sports field), temporarily installed (installed on a movable pole), or carried by a cameraman. The sports video captured by the camera (1200) may be a tactical view, a broadcast view, or a player-focused view. The tactical view is an image generally used for sports tactical analysis, and may be an image captured so that most of the objects (10) are captured in the image for team tactical analysis, and the horizontal axis of the tactical view may correspond to the longitudinal direction or the width direction of the field. The broadcast view is a video mainly used for sports broadcasting and can be captured with a smaller angle of view than the tactical view, and the player-focused view is a video captured along a specific object (10) and can be mainly used to analyze the individual capabilities of the object (10). In addition, the camera (1200) may not necessarily have a fixed field of view, and the field of view direction adjustment and zoom adjustment may be performed manually or automatically.

[0196] According to one aspect of the present disclosure, the motion image acquisition platform may include one or more cameras. The multiple cameras may be provided in a multi-camera configuration capable of capturing panoramic images from a single spot, distributed across multiple spots, or a combination of the two.

[0197] The exercise video acquisition platform can be configured to acquire various types of exercise videos through a camera (1200).

[0198]

[0199] Meanwhile, according to one aspect, image-based location information may be obtained separately from sensor-based location information. For example, the sensing platform may analyze an image captured by a camera (1200) to obtain activity information therefrom. For example, the sensing platform may perform object recognition on an object (10) in an image, extract the position (e.g., pixel coordinates) of the object (10) within the image, and project the position within the image onto the ground using the pose information of the camera (1200), thereby obtaining location information on the object (10). Here, a deep learning algorithm may be used for image processing such as object detection. Specifically, for object detection (e.g., object (10) detection), various deep learning algorithms for object detection ranging from Region-based Convolutional Neural Network (R-CNN) to You-Only-Look-Once (YOLO) can be used, and for coordinate transformation, top-view transformation using camera parameters (e.g., installation location, shooting posture, etc.) can be used. For example, the server (1500) obtains a sports image from a camera (1200) positioned to capture a tactical view, obtains a bounding box for a sports object such as a ball or an object (10) in the sports image through an artificial neural network model that performs object detection, determines a representative pixel (e.g., the lower center of the bounding box) for the sports object considering the bounding box, and performs top-view transformation using a coordinate transformation metric between a pixel coordinate system and a reference coordinate system in the sports image generated considering the camera parameters for the determined representative pixel, thereby obtaining position data for the sports object. Here, object detection using artificial neural networks can be replaced with object segmentation.Accordingly, the server (1500) can obtain location data for a sports object in a sports video captured by the camera (1200) through video analysis.

[0200]

[0201] As mentioned in the above description, the sensing platform (1100) may optionally further include a base station for RTK correction and an anchor node for building a local positioning sensor network as needed.

[0202] In addition, some of the various operations performed in the sensing platform (1100) may be processed in the processor or controller mounted on the sensor device (1300), but others may be processed in the server (1500). For example, object detection for an image may be processed by the server (1500), location calculation using the results of LPS signal transmission and reception between each node in a local positioning sensor network may be processed by the server (1500), and various processing related to image analysis may be processed by the server (1500). Therefore, at this time, it may also be understood that the server (1500) is included in the sensing platform (1100).

[0203]

[0204] Server

[0205] The server (1500) may be configured to receive location information from the sensing platform (1100) or to collaborate with the sensing platform (1100) to obtain location information and use the information to track the trajectory of an object.

[0206] The server (1500) may be provided as a local server located near the playground and / or a web server connected via the web. In addition, the server (1500) does not necessarily have to be implemented as a single entity. For example, the web server may be implemented as including a main server for processing operations and a data server for storing various data. Meanwhile, the local server may be provided as an independently operated device that performs only the original function of the server, but may also be provided as a device having a composite function combined with other components of the exercise load information provision system. For example, the server may be provided in the form of a docking station, which is a container for accommodating, storing, and managing the sensor device (1300). Here, the docking station may have an internal space for accommodating multiple sensor devices (1300), including a docking unit for accommodating the sensor device (1300), and may store the sensor device (1300) when it is not in use. Furthermore, the docking station can provide various convenient functions necessary for using the sensor device (1300) in addition to simply storing the sensor device (1300). For example, the docking station can perform functions such as charging the sensor device (1300), displaying the battery status, updating firmware, and collecting data.

[0207] Additionally, as described above, the server (1500) may be implemented as an integral part of the terminal (1700), and may be a mobile computing device such as a tablet PC, for example.

[0208]

[0209] Figure 9 is a block diagram of a server according to one embodiment of the present disclosure.

[0210] The server (1500) may include a communication module (1510), a controller (1520), and memory (1530).

[0211] The communication module (1510) can transmit and receive data between the server (1500) and other components of the object tracking system or external devices. For example, the server (1500) can collect data from a sensor device (1300), receive images from a camera (1200), or transmit various types of information to a terminal (1700) via the web through the communication module (1510).

[0212] The controller (1520) can control the overall operation of the server (1500). Like the controller of the sensor device (1300), the controller (1520) of the server can be implemented as a hardware configuration, a software configuration, or a combination thereof, and from a hardware perspective, the controller can be provided in various forms capable of performing operations or data processing, including electronic circuits, integrated circuits (ICs), microchips, and processors, and its physical configuration is not necessarily limited to a single physical entity. Meanwhile, specific operations performed by the server (1500) will be described below, and it is to be understood in advance that methods or operations according to the embodiments of the present disclosure described below can be performed by the controller (1520) of the server, unless otherwise mentioned.

[0213] However, as examined in this description, the embodiments according to this description are not necessarily limited to being performed by the controller (1520) of the server, and it should be understood that the embodiments according to this description may be performed by an operable processor, such as the controller (1330) of the sensor device (1300), or at least some of the procedures may be performed by the server, and at least some of the procedures may be performed by another processor.

[0214]

[0215] Terminal

[0216] A terminal (or terminal device) (1700) may function as a user interface that provides various data or information collected or produced by the system to a user, or receives user input from a user. For example, the terminal (1700) may receive input from a user, such as a corner of a stadium or a location within a specific player's motion image, and transmit the input information to a server (1500) so that the corresponding location within the motion image of each of a plurality of players can be determined. In addition, the terminal may provide integrated content to the user by displaying motion images and information about the plurality of players based on the determined corresponding locations. The terminal (1700) may be a smart device such as a smart phone or tablet, a personal computer such as a laptop or desktop, or any other electronic device having an input interface for receiving user input and an output interface such as a display.

[0217]

[0218] Display player information for exercise videos

[0219] Below, a procedure for displaying player information in a motion video according to an embodiment of the present disclosure is described. In the following description, the player information display procedure may be exemplarily described as being performed by the aforementioned system (1000). However, this is merely for convenience of explanation and is not limited to the procedures being performed by the system.

[0220]

[0221] FIG. 10 is a conceptual diagram of a method for displaying player information in a motion video according to one embodiment of the present disclosure. The method for displaying player information in a motion video according to embodiments of the present disclosure may include a sensor-based location information acquisition procedure (step 1010) and an integrated display procedure for motion information (step 1020).

[0222] More specifically, but not exclusively, as illustrated in FIG. 10, a method for displaying player information in a motion image according to embodiments of the present disclosure may include a procedure of obtaining location information for each of a plurality of players based on a sensor device such as, for example, a GPS (step 1020). Based on the obtained location information, the player's location as well as various additional information that can be determined based on the location information, such as, for example, speed or trajectory, may be determined. Thereafter, in a video information integration display step (step 1020), the determined additional information may be integrated into corresponding locations in the motion image of each player, so that a motion image with integrated player information may be provided to a user.

[0223] FIG. 11 is an exemplary diagram showing a state in which player information is displayed in a motion video according to one embodiment of the present disclosure, and FIG. 12 is a diagram for a detailed explanation of the player information displayed in FIG. 11. As illustrated in FIG. 11, the motion video may include multiple players, and information about each player may be displayed at an appropriate location corresponding to each player. According to embodiments of the present disclosure, the player information displayed in the motion video may be determined based on sensor-based location information acquired using a sensor.

[0224] As exemplarily illustrated in FIG. 12, according to one aspect of the present disclosure, player information (100) that can be displayed in a motion video can include, for example, at least one of the personal information (111, 113) of the corresponding player, sensor-based position information (140) of the corresponding player, speed (120) of the corresponding player, and trajectory (130) of the player during a predetermined time interval.

[0225] As illustrated in FIG. 12, the player's personal information may include, but is not limited to, additional information such as the player's jersey number or name (111) or position (113).

[0226] The player's speed (120) may be the player's speed determined during a predetermined time interval prior to the time at which player information is displayed in the motion video or a predetermined time interval. Such speed may be calculated based on the player's position information determined by the sensor device, or may be speed information directly acquired and transmitted by the sensor device.

[0227] A player's trajectory (130) may be a time series of information about the positions the player existed in during a predetermined time interval. As exemplarily illustrated in FIG. 12, the player's trajectory may be displayed in a motion video in the form of, for example, a parabola or a curve. The time duration of the position information displayed as the trajectory may be changed, for example, by inputting a user-defined value.

[0228] According to one aspect, the sensor-based position information (140) of the player may be displayed together with the motion video. As exemplarily illustrated in FIG. 12, the sensor-based position information (140) may be displayed, for example, as coordinate values. Such coordinate values ​​may be coordinate values ​​according to the stadium coordinate system, or may be coordinate values ​​according to the global coordinate system or the image coordinate system, and are not limited to coordinate values ​​according to a specific coordinate system. Furthermore, according to another aspect, information about the player's position may be displayed in the form of a figure indicating a corresponding position, such as a circle, at the bottom of the player in the video, without displaying separate specific coordinate values.

[0229]

[0230] FIG. 13 is a schematic flowchart of a method for displaying player information in a motion image according to one embodiment of the present disclosure. Hereinafter, with reference to FIG. 13, a method for displaying player information in a motion image according to one embodiment of the present disclosure will be described in more detail.

[0231] As an example, the method according to the embodiments of the present disclosure may be performed by a computing device. Hereinafter, for convenience of explanation, the methods according to the embodiments of the present disclosure may be described as being performed by a computing device. Here, the computing device may be, for example, a server (1500) or a terminal (1700), but is not limited thereto, and should be understood to include at least one of any computing device capable of performing calculations, including a processor.

[0232] Referring to FIG. 13, a method for displaying player information in a motion video according to one embodiment of the present disclosure may include a step of obtaining a captured video for a motion event in a target time period (step 1310), a step of obtaining sensor-based location information for each player (step 1320), a step of determining a corresponding location within the captured video for the sensor-based location information (step 1330), and a step of displaying information of each player at the corresponding location (step 1340).

[0233] More specifically, but not exclusively, as illustrated in FIG. 13, the computing device may first acquire a captured image of an exercise event in a target time interval (step 1310). Here, the target time interval may be determined, for example, from the start time to the end time of a regular match when the exercise event is a regular match, or from a preset training start time to the end time of a training schedule when the exercise event is a training schedule. Alternatively, a time interval having an arbitrarily set start point and end point may be set as the target time interval for acquiring an exercise image.

[0234] The captured video of the exercise event can be acquired by the camera (1200) as exemplarily described above in this description. The camera (1200) can be a single camera or multiple cameras, and when multiple cameras are utilized, stitching and alignment between the multiple images, as well as distortion compensation procedures, can be performed. The computing device can acquire the captured video of the exercise event by receiving the video of the exercise event captured by the camera (1200) through a wireless or wired communication means, for example.

[0235]

[0236] Referring again to FIG. 13, the computing device can obtain sensor-based location information for a target time period for each player participating in the exercise event (step 1320). Here, according to one aspect, the sensor-based location information may be location information obtained based on a Global Navigation Satellite System (GNSS), but is not limited thereto.

[0237] As described above as an example in this disclosure, according to one aspect, sensor-based location information for each player participating in an exercise event during a target time period can be acquired, for example, by a sensing platform (1100), more specifically, a sensor-based platform. An attachable device, such as a sensor device (1300), can be attached to each of a plurality of players, and sensor-based location information for each player can be acquired based on this.

[0238]

[0239] Referring again to FIG. 13, the computing device can determine each corresponding position within the captured video corresponding to the acquired sensor-based position information for each player (step 1330). That is, within the captured video for the exercise event, it can be determined where information about a particular player should be displayed. To ensure that information about a particular player is displayed at a location corresponding to that player's position within the video, a matching between the information about each player and the respective player's position within the video is required.

[0240] According to one embodiment of the present disclosure, a procedure for determining the corresponding position of each player within a motion video may be performed by receiving certain information from a user of a computing device and performing calculations based on the input information. For example, the user may directly input where the corners of the field are located within the motion video, and the corresponding positions of at least some of the players at least from some of the viewpoints of the plurality of players may be directly input from the user. Based on such input information, the corresponding positions within the motion video for each of the remaining players at the remaining viewpoints may be determined by the computing device.

[0241]

[0242] FIG. 14 is a detailed flowchart of the corresponding position determination step of FIG. 13. Referring to FIG. 14, the corresponding position determination procedure within the motion images of each player will be described in a non-limiting but more specific manner. As illustrated in FIG. 14, for corresponding position determination, the computing device may first display at least a portion of the captured image and sensor-based position information (step 1331).

[0243] In relation to this, FIG. 16 is an exemplary diagram of a display screen of a captured image and sensor-based location information according to one embodiment of the present disclosure. As illustrated in FIG. 16, a display provided in a computing device or a user terminal connected to the computing device may include a motion image display area (1610), a sensor-based location information display area (1620), and an input item selection area (1630).

[0244] The motion image display area (1610) may display at least a portion of previously captured video footage of a sporting event. The motion image displayed may be, for example, a tactical view video, but is not limited thereto. Motion images of a planar view of the stadium may also be displayed, and motion images of any other view may be displayed.

[0245] The sensor-based location information display area (1620) may display sensor-based location information for at least some of the previously acquired plurality of players. As exemplarily illustrated in FIG. 16, the sensor-based location information may display, for example, an identification number of a corresponding sensor device and acquired location information. More specifically, but not limited to, the displayed sensor-based location information may include at least one of a relative positional relationship of each sensor-based location information with respect to a representative value of each of the plurality of sensor-based location information, or identification information of a sensor device corresponding to each of the sensor-based location information. That is, the plurality of sensor devices may be, for example, GPS sensors, and may acquire location values ​​according to each global coordinate system. In this way, the representative value (e.g., average value) of the coordinate values ​​according to the global coordinate system of the plurality of sensor devices secured is set to be located at the center of the sensor-based location information display area (1620), and according to the relative positional relationship of the position values ​​according to the global coordinate system of each sensor device with respect to the representative value, the identification information (e.g., 60009, 60002, etc.) of each sensor device and the mark indicating the measured position value can be displayed at an appropriate position of the sensor-based location information display area (1620). Accordingly, as illustrated in FIG. 16, the user can recognize which mark among the marks indicating the identification numbers of the plurality of sensor devices and the corresponding position values ​​displayed in the sensor-based location information display area (1620) corresponds to which of the marks indicating the position values ​​corresponding thereto, with respect to the plurality of players present in the exercise image displayed in the exercise image display area (1610). Accordingly, the user can determine whether the position measurement value according to the sensor device has a certain identification information for a particular player displayed in the exercise image at a certain point in time or a certain frame as illustrated in FIG. 16, for example.

[0246] According to one aspect, the sensor-based position information display area (1620) may display a position input request message for, for example, a first player or a second player. As described later in the present disclosure, information about a position within an image for a specific player may be input by a user, and whether or not information about a position within an image for a specific player is to be input may be displayed in the sensor-based position information display area (1620). In this regard, for example, as illustrated in FIG. 19, in order to receive input from a user about a position within an image of a player corresponding to a measurement value of sensor device identification number 60001, a position input request message (1660) for a specific player may be displayed in such a way that the identification number 60001 is highlighted in the sensor-based position information display area (1620). However, it should be noted that the form of the position input request message is not limited to such an emphasis display method, and any means or form for instructing the user to input a position within an image of a specific player may be conveyed to the user.

[0247]

[0248] Meanwhile, an interface for selecting the type of information that the user wishes to input may be displayed in the input item selection area (1630). As illustrated in FIG. 16, for example, in a step for receiving information on the location of a corner of a stadium within an exercise video from the user, an interface for selecting which corner of the stadium to input the location for may be based on a point (1640) may be displayed. For example, in the case of wanting to input location information for the lower left corner (LEFT BOTTOM) within the video, the user may position the point (1640) in the "LEFT BOTTOM" area and click to enter an input mode for the location of the lower left corner of the stadium within the video. As exemplarily illustrated in FIG. 19, in a mode for inputting information on a position within a motion image of a specific player, a user may enter a mode for inputting a position within a motion image of a player corresponding to identification number 60001 by clicking on the identification number 60001 information input interface located in the input item selection area (1630) in response to a request message (1660) for inputting a position of a corresponding player for identification number 60001.

[0249]

[0250] Referring again to FIG. 14, the corresponding position determination step is further described. For example, as illustrated in FIG. 16, after displaying at least a portion of the captured image and sensor-based position information (step 1331), the computing device may receive information from the user regarding the position of the corners of the stadium within the captured image (step 1332).

[0251] FIG. 17 is an example of a screen for inputting a stadium corner by a user. As illustrated in FIG. 17, in order to input a location within a captured image of stadium corners, the user can select which corner of the stadium to input a location within the captured image within the input item selection area (1630). As illustrated in FIG. 17, for example, in order to input a location within the captured image of the lower left (LEFT BOTTOM) corner, the user can click the LEFT BOTTOM portion in the input item selection area (1630), and then use the point (1640) to click the lower left corner of the stadium within the exercise image displayed in the exercise image display area (1610). Accordingly, the user can input the location of the lower left corner of the stadium within the exercise image to the computing device. In one aspect, for selecting a specific point within the exercise image, for example, an image area within a predetermined range indicated by the point (1640) may be enlarged and displayed.

[0252] Sequentially, when position input is performed for different corners of the stadium within the motion image, information about the boundary (1650) of the stadium within the image can be secured. In this way, based on the information about the boundary of the stadium or at least one of a plurality of corners of the stadium, conversion between the image coordinate system (Pixel Coordination) and the stadium coordinate system (Pitch Coordination) can be performed. More specifically, according to one aspect, information about the positions of the corners of the stadium within the captured image can be used for conversion between the image coordinate system (Pixel Coordination) indicating the position within the captured image and the stadium coordinate system (Pitch Coordination) indicating the position within the stadium where the motion event is performed. That is, the image coordinate system can be a value indicating the position of a specific pixel within the image, and for example, when the resolution of the image is 1080 * 768, the position of a specific pixel can be expressed as a position value based on the image coordinate system, such as, for example, (872, 111). Additionally, when the size of the stadium where the exercise event is performed is, for example, 80 m * 45 m, the location of a specific player within the stadium can be expressed as a location value based on the stadium coordinate system, for example, (45, 8). In each coordinate system, the origin can be selected as, for example, the upper left corner, but is not limited thereto, and the coordinate system can be set by selecting any corner as the origin.

[0253] Fig. 18 is a diagram illustrating a conversion between an image coordinate system (Pixel Coordination) and a stadium coordinate system (Pitch Coordination). As illustrated in Fig. 18, for example, in the image coordinate system (1810), a plurality of corners may have position values ​​of upper left (100, 100), lower left (70, 620), upper right (980, 100), and lower right (1010, 620), and for example, in the stadium coordinate system (1820), a plurality of corners corresponding thereto may have position values ​​of upper left (0, 0), lower left (0, 75), upper right (100, 0), and lower right (100, 75), but it should be noted that the present invention is not limited thereto.

[0254] Since position values ​​according to the stadium coordinate system and position values ​​according to the image coordinate system can be secured for multiple corners, conversion between the position values ​​according to the stadium coordinate system and the position values ​​according to the image coordinate system can be performed by operation. For example, a conversion matrix for conversion between the two can be set, and conversion between the image coordinate system and the stadium coordinate system at a specific point can be performed based on interpolation or extrapolation for multiple corners.

[0255] According to one aspect, the sports video can be displayed by converting it into a view (e.g., a top-view) different from the view obtained through image processing (e.g., a tactical view). Furthermore, the view can be converted and displayed from a point desired by the user based on a user's control input. For example, the region of interest may change depending on the progress of the sporting event that is the subject of the game video, or the region of interest may change depending on the purpose of analyzing the game video. Based on the user's viewpoint conversion input, the game video can be provided by performing image processing in a form in which the user views the game video from a point desired by the user. Here, for example, the top-view can display the positions of one or more players based on the stadium coordinate system, and can be set to serve as a reference for conversion between different views. However, the reference image is not limited to an image based on the stadium coordinate system, and any other coordinate system, including a spherical coordinate system in which the stadium is expressed at a predetermined distance and angle from a virtual center, can be used as the reference coordinate system.

[0256]

[0257] Referring again to FIG. 14, the computing device may receive information from the user regarding the location of the first player within the captured image (step 1333).

[0258] In relation to this, FIG. 19 is an example of a location input request screen for a first player. As illustrated in FIG. 19, a motion image display area (1610) may display a motion image in a specific frame, such as frame 576, for example. Here, a location information input request message (1660) for the first player, for example, may be displayed in a sensor-based location information display area (1620). In the example of FIG. 19, the first player may be a player corresponding to a sensor device having an identification number of, for example, 60001.

[0259] The user may, in response to a request message (1660) for inputting location information for the first player, select to input location information for the first player (identification number 60001) in the input item selection area (1630). For example, the user may enter the location input mode for the first player by clicking in the area indicating the first player input using the point (1640).

[0260] Fig. 20 is an example of a location input screen for a first player by a user. As illustrated in Fig. 20, the user can input a location within the motion image of the first player into the computing device by selecting and clicking a location within the motion image of the player corresponding to the identification number 60001 based on a point (1640). Here, according to one aspect, the input of a location for a specific player may be set to click the center of the foot of the corresponding player, but is not limited thereto. For example, in the player location input mode, a guidance message instructing to click the center of the foot of the corresponding player may be configured to be output.

[0261]

[0262] Referring again to FIG. 14, the computing device may receive information from the user regarding the location of the second player within the captured image (step 1334).

[0263] In relation to this, FIG. 21 is an example of a position input screen for a second player by a user.

[0264] In a similar manner to the position input request screen for the first player described above, the motion image display area (1610) may display motion images from a specific frame, such as frame 1221, for example. Here, a position information input request message (2160) for the second player, for example, may be displayed in the sensor-based position information display area (1620). In the example of FIG. 21, the second player may be a player corresponding to a sensor device having an identification number of, for example, 60014.

[0265] The user may, in response to a request message (2160) for inputting location information for a second player, select to input location information for the second player (identification number 60014) in the input item selection area (1630). For example, the user may enter the second player's location input mode by clicking in the area indicating second player input using point (1640).

[0266] As illustrated in FIG. 21, a user may input a position within a motion image of a second player into a computing device by selecting and clicking a position within a motion image of a player corresponding to identification number 60014 based on point (1640). Here, according to one aspect, the input of a position for a specific player may be set to click the center of the bottom of the foot of the corresponding player, but is not limited thereto. For example, in the player position input mode, a guidance message instructing to click the center of the bottom of the foot of the corresponding player may be configured to be output.

[0267]

[0268] Meanwhile, Fig. 22 is an example of a location input screen for a third player by a user.

[0269] In a similar manner to the position input request screen for the first or second player described above, the motion image display area (1610) may display motion images from a specific frame, such as, for example, frame 2778. Here, a position information input request message (2260) for, for example, a third player may be displayed in the sensor-based position information display area (1620). In the example of FIG. 22, the third player may be a player corresponding to a sensor device having an identification number of, for example, 60014.

[0270] The user may, in response to a request message (2260) for inputting location information for a third player, select to input location information for the third player (identification number 60014) in the input item selection area (1630). For example, the user may enter the location input mode for the third player by clicking in the area indicating third player input using the point (1640).

[0271] As illustrated in FIG. 22, the user can input a location within a third player's motion image into the computing device by selecting and clicking a location within the motion image of the player corresponding to the identification number 60014 based on the point (1640). Here, according to one aspect, the input of a location for a specific player may be set to click the center of the bottom of the player's foot, but is not limited thereto. For example, in the player location input mode, a guidance message instructing to click the center of the bottom of the foot of the corresponding player may be configured to be output.

[0272]

[0273] Figure 23 is an example of the position input results for multiple players. As illustrated in Figure 23, the positions of the first player, the second player, and the third player within the image can be stored together with the sensor-based position information at the corresponding time obtained by the sensor device corresponding to the corresponding player.

[0274] According to one aspect, at least some of the first player, the second player, or the third player may be players with the largest distance between them among the plurality of players included in the captured video. For example, the positions of the plurality of players at at least one of the plurality of viewpoints in the captured video may be determined as the first player, the second player, or the third player, and the players selected in this way may be determined as the players with the largest distance between them, and may be determined by, but is not limited to, a convex hull algorithm, more specifically, by way of example only. By determining players who are as far apart as possible as the reference players in this way, the accuracy for determining the positions of the remaining players in the video whose positions have not been input by the user in the motion video may be improved. In this regard, this improvement in accuracy may be due to the higher accuracy of position determination by interpolation compared to extrapolation.

[0275]

[0276] According to one aspect, the procedure of inputting information about a position in a captured image for at least some of the first player, the second player, or the third player may be performed for different frames among a plurality of frames included in the captured image. That is, for example, as illustrated in FIG. 21, the position in the input image for the first player may be input at frame 576, as illustrated in FIG. 22, the position in the input image for the second player may be input at frame 1221, and as illustrated in FIG. 23, the position in the input image for the third player may be input at frame 2778.

[0277] In addition, according to one aspect, the information about the position of at least some of the first player, the second player, or the third player within the captured video may be position information in different frames of a single player. For example, as illustrated in FIGS. 21 and 22, both the second player and the third player may be configured to input positions within the motion video of sensor-based position information acquired by the sensor device of identification number 60014.

[0278] According to one embodiment of the present disclosure, inputting a position in a motion image of a first player, a second player, or a third player may be, rather than obtaining a position in a motion image for a sensor device having a specific identification number, obtaining a pair of specific sensor-based position information at a specific point in time and a position in the motion image corresponding to the sensor-based position information. Accordingly, regardless of whether positions in the image are input from a single or multiple frames or whether positions in the image corresponding to a single or multiple sensor devices are input, a plurality of pairs (sensor-based position information, position information in the image) are obtained, and a reference player may be determined such that the plurality of possible position information pairs are positioned as far apart from each other as possible, as described above. In one aspect, such determination of a reference player may be configured to be automatically calculated by a computing device so as to prompt a user to input a position in the image of a specific player in a specific frame, or may be configured so as to prompt a user to select players that are as far apart from each other as possible and input positions in the image accordingly.

[0279]

[0280] Meanwhile, according to one embodiment of the present disclosure, the determination of the reference player may be performed by considering the accuracy of the corresponding sensor-based location information. More specifically, but not limitingly, the computing device may be configured to receive information regarding the location within the captured image of at least one of the first, second, or third players in response to a determination that the sensor-based location information for at least one of the first, second, or third players was acquired in RTK-GPS mode.

[0281] As described above, according to one embodiment of the present disclosure, an improved sensor-based positioning technique or facility compared to general GPS may be applied to improve the accuracy of sensor-based position information, for example, RTK-GPS may be used. Even when an RTK-GPS facility is applied, a situation may arise where positioning is not performed in RTK-GPS mode at a specific time or location. Whether or not positioning was performed based on RTK may be included in the GPS signal as information about the positioning mode, for example, and, for example, as illustrated in FIG. 19, information (1611) indicating that a specific sensor device, for example, a sensor device with identification number 60001, performed positioning in RTK-GPS mode at the corresponding frame time may be displayed as a mark such as R. According to one aspect of the present disclosure, by inputting a location within a motion image corresponding only to sensor-based positioning information measured in RTK mode in this way, the accuracy for determining the locations within the motion images of the remaining players who have not received the location information may be improved.

[0282]

[0283] According to one embodiment of the present disclosure, information about the position of the first player within the captured video and information about the position of the second player within the captured video can be used for conversion between a global coordinate system (Global Coordination) used by sensor-based position information and an image coordinate system (Pixel Coordination) representing the position within the captured video. In this regard, FIG. 24 is an explanatory diagram for conversion between an image coordinate system (Pixel Coordination) and a global coordinate system (Global Coordination).

[0284] More specifically, but not exclusively, for example, as illustrated in FIG. 24, information about a position within a captured image of a first player (2415), information about a position within a captured image of a second player (2413), or information about a position within a captured image of a third player (2411) may be paired with information about corresponding sensor-based position information. Conversion between the image coordinate system (2410) and the global coordinate system (2420) may be performed based on pairs of position information within the image and sensor-based position information. For example, based on the information about the position in the captured image of the first player (2415), the information about the position in the captured image of the second player (2413), or the information about the position in the captured image of the third player (2411) and the corresponding sensor-based position information value, the corresponding position in the captured image (2417) can be determined for the first position (2427) among the sensor-based position information.

[0285] That is, since position values ​​within multiple images and corresponding position values ​​in the global coordinate system can be secured, transformation of the position values ​​according to the global coordinate system and the position values ​​according to the image coordinate system can be performed by operation. For example, a transformation matrix for transformation between the two can be set, and transformation between the global coordinate system and the image coordinate system of a specific point can be performed based on interpolation or extrapolation for multiple edges.

[0286]

[0287] According to one aspect of the present disclosure, a computing device can perform a conversion between a global coordinate system used by sensor-based position information and a pitch coordinate system representing a position within a stadium where an athletic event is performed, based on at least some of information about positions within a captured video of corners of a stadium, information about positions within a captured video of a first player, information about positions within a captured video of a second player, or information about positions within a captured video of a third player. As discussed above, for example, conversion between the image coordinate system and the pitch coordinate system is possible based on information about corners of a stadium, and conversion between the global coordinate system and the image coordinate system is possible based on pairs of positions within a video of a plurality of players - positions according to the global coordinate system. Therefore, conversion between specific coordinate systems among the stadium coordinate system - the image coordinate system - the global coordinate system can be performed as needed.

[0288]

[0289] Figure 15 is a second detailed flowchart for the corresponding position determination step of Figure 13.

[0290] As illustrated in FIG. 15, according to one aspect of the present disclosure, in order to determine corresponding positions within a captured image for sensor-based position information (step 1330 of FIG. 13), the computing device may, after receiving information about positions of a first player and a second player within a captured image from a user (steps 1333 to 1334 of FIG. 14), calculate corresponding positions of each of a plurality of players based on information about positions within the captured image of the first player and the second player, the corners of the field, and the corners of the field (step 1335). That is, corresponding positions of information of a plurality of players included in the acquired sensor-based position information within a motion image may be calculated. Such corresponding positions may be determined for each point in time or frame, for each piece of sensor-based position information.

[0291] The computing device can display information about each player at each corresponding position calculated in this manner. Based on this, a supplementary procedure for the corresponding position can be initiated.

[0292] As illustrated in FIG. 15, according to one embodiment of the present disclosure, the computing device may be configured to receive, from a user, input of a batch change value for each of the calculated corresponding positions of a plurality of players (step 1336). In this regard, FIG. 25 is an exemplary diagram of a procedure for receiving, from a user, input of a batch change value for each of the corresponding positions of a plurality of players, and FIG. 26 illustrates an information display state according to the batch change result of FIG. 25.

[0293] As illustrated in FIG. 25, even in the case of GPS or RTK-GPS, for example, an error may occur between the player's position in the image and the displayed player information, so as to have a certain tendency overall. Such an error may be caused, for example, by an inherent bias of the positioning mechanism according to GPS. On the other hand, a separate RTK device may be provided to implement RTK-GPS, and if the setting for the position of the RTK antenna itself includes an error, the measured sensor-based position information may include an error with a certain tendency overall.

[0294] According to one embodiment of the present disclosure, the computing device can collectively change the corresponding positions of each of the calculated plurality of players, as illustrated by arrows in FIG. 25. For example, by receiving a collective change value for the corresponding positions of each of the plurality of players from a user, the corresponding positions can be moved as a whole to correspond to the received value. FIG. 25 exemplarily illustrates receiving an input for moving the corresponding positions of all player information to the left on the screen. Accordingly, as illustrated in FIG. 26, player information can be displayed at a more accurate position within the motion image.

[0295]

[0296] Meanwhile, as illustrated in FIG. 15, according to one embodiment of the present disclosure, the computing device may be configured to receive, from the user, a change value for a corresponding position of any one of a plurality of players, after the step of receiving a batch change value from the user (step 1337). In this regard, FIG. 27 is an exemplary diagram of a procedure for receiving, from the user, an individual change value for a corresponding position of a specific player among a plurality of players, and FIG. 28 illustrates an information display state according to the individual change result of FIG. 27.

[0297] As illustrated in FIG. 27, information about a specific player (e.g., a player corresponding to sensor device identification number 60002) may be displayed in an incorrect location compared to information about other players within a motion video. This may be caused, for example, by the measurement environment at the time of sensor-based positioning, but is not limited thereto. For the corresponding locations of such individual players, the computing device may be configured to receive an individual location change value from the user and change the corresponding location. For example, as illustrated in FIG. 27, an input may be received from the user to move the corresponding location within the video for identification number 60002 in the 7 o'clock direction, thereby changing the corresponding location. Accordingly, player information may be displayed at a more accurate location, as illustrated in FIG. 28.

[0298]

[0299] Referring again to FIG. 13, the computing device can display information about each player at each corresponding location (step 1340). That is, corresponding locations within the motion image corresponding to the plurality of sensor-based location information described above can be determined, and information about each player can be displayed at the determined locations. Accordingly, a motion image integrating player information, such as that exemplified in FIG. 11, can be provided to the user.

[0300]

[0301] Meanwhile, when it comes to improving the accuracy of corresponding positions within exercise videos, synchronization issues between the captured video and sensor-based position information can be considered. Even if the sensor-based position information is measured with high accuracy, if the timing of the sensor-based position information and the corresponding frame of the exercise video do not match correctly, errors will inevitably occur between the player's actual position within the exercise video and the corresponding position within the video based on the sensor-based position information.

[0302] FIG. 29 is a block diagram illustrating a GNSS module configuration for time synchronization between captured images and sensor-based location information. As illustrated in FIG. 29, according to one embodiment of the present disclosure, captured images of a sports event are captured by a camera (2910) having a first GNSS module (2911), and sensor-based location information can be determined using a positioning device (2920) including a second GNSS module (2921) of the same type as the first GNSS module. That is, the camera (2910) for acquiring captured images of a sports event and the positioning device (2920) for acquiring sensor-based location information can be equipped with GNSS modules of the same type. Such GNSS modules of the same type can include timestamps of the same type, and can provide information about UTC time, for example. Accordingly, the acquisition time can be recorded based on UTC time for each frame of the image acquired through the camera (2910). It goes without saying that the acquisition time can also be recorded based on UTC time for sensor-based location information.

[0303] According to one aspect, the computing device (2930) can be configured to match the acquisition time of each frame of the captured image with the sensor-based position information based on the time information of the first GNSS module (2911) and the second GNSS module (2921). Accordingly, temporal synchronization between each frame of the motion image and the sensor-based position information can be achieved, and positional consistency between the image and the sensor-based position information can be improved.

[0304]

[0305] FIG. 30 is a block diagram showing an exemplary configuration of a computing system in which a method according to one embodiment of the present disclosure can be performed.

[0306] Referring to FIG. 30, the computing system (800) may include flash storage (3010), a processor (3020), RAM (3030), an input / output device (3040), and a power supply (3050). In addition, the flash storage (3010) may include a memory device (3011) and a memory controller (3012). Meanwhile, although not shown in FIG. 8, the computing system (3000) may further include ports for communicating with a video card, a sound card, a memory card, a USB device, or the like, or for communicating with other electronic devices.

[0307] The computing system (3000) may be implemented as a personal computer or a portable electronic device such as a laptop computer, a mobile phone, a personal digital assistant (PDA), or a camera.

[0308] The processor (3020) can perform specific calculations or tasks. Depending on the embodiment, the processor (3020) can be a microprocessor, a central processing unit (CPU). The processor (3020) can communicate with the RAM (3030), the input / output device (3040), and the flash storage (3010) via a bus (3060), such as an address bus, a control bus, and a data bus.

[0309] According to one embodiment, the processor (3020) may also be connected to an expansion bus, such as a Peripheral Component Interconnect (PCI) bus.

[0310] RAM (3030) can store data required for the operation of the computing system (3000). For example, any type of random access memory including DRAM, mobile DRAM, SRAM, PRAM, FRAM, MRAM, and RRAM can be used as RAM (3030).

[0311] The input / output device (3040) may include input means such as a keyboard, keypad, mouse, etc. and output means such as a printer, display, etc. The power supply (3050) may supply an operating voltage necessary for the operation of the computing system (3000).

[0312]

[0313] The method according to the present invention described above can be implemented as computer-readable code on a computer-readable recording medium. Computer-readable recording media include all types of recording media that store data that can be deciphered by a computer system. Examples include read-only memory (ROM), random access memory (RAM), magnetic tape, magnetic disks, flash memory, and optical data storage devices. Furthermore, the computer-readable recording medium can be distributed across computer systems connected to a computer communications network, and stored and executed as readable code in a distributed manner.

[0314] Although the present invention has been described with reference to the drawings and embodiments, it does not mean that the scope of protection of the present invention is limited by the drawings or embodiments, and it will be understood that a person skilled in the art can modify and change the present invention in various ways without departing from the spirit and scope of the present invention as described in the following claims.

[0315] Specifically, the described features may be implemented within digital electronic circuitry, or within computer hardware, firmware, or combinations thereof. The features may be implemented, for example, in a computer program product embodied within a storage device within a machine-readable storage device for execution by a programmable processor. And the features may be implemented by a programmable processor executing a program of instructions for performing the functions of the described embodiments by operating on input data and generating output. The described features may be implemented within one or more computer programs executable on a programmable system comprising at least one programmable processor, at least one input device, and at least one output device coupled to receive data and instructions from a data storage system, and to transmit data and instructions to the data storage system. A computer program comprises a set of instructions that can be used directly or indirectly within a computer to perform a particular operation for a given result. A computer program may be written in any programming language, including compiled or interpreted languages, and may be used in any form, including as a module, component, subroutine, or other unit suitable for use in another computing environment, or as a standalone program.

[0316] Suitable processors for executing the program of instructions include, for example, both general-purpose and special-purpose microprocessors, and either a single processor or multiple processors of another type of computer. Also suitable storage devices for implementing the computer program instructions and data implementing the described features include, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic devices such as internal hard disks and removable disks, magneto-optical disks, and all forms of non-volatile memory including CD-ROM and DVD-ROM disks. The processor and memory may be integrated within or added to application-specific integrated circuits (ASICs).

[0317] Although the present invention described above is described based on a series of functional blocks, it is not limited to the above-described embodiments and the attached drawings, and it will be apparent to those skilled in the art to which the present invention pertains that various substitutions, modifications, and changes are possible within a scope that does not depart from the technical spirit of the present invention.

[0318] The combination of the above-described embodiments is not limited to the above-described embodiments, and various combinations may be provided in addition to the above-described embodiments depending on implementation and / or needs.

[0319] In the above-described embodiments, the methods are described based on a flowchart as a series of steps or blocks. However, the present invention is not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps described above. Furthermore, those skilled in the art will understand that the steps depicted in the flowchart are not exclusive, and other steps may be included, or one or more steps in the flowchart may be deleted without affecting the scope of the present invention.

[0320] The above-described embodiments include examples of various aspects. While not all possible combinations to illustrate the various aspects can be described, those skilled in the art will recognize that other combinations are possible. Accordingly, the present invention is intended to encompass all other alterations, modifications, and variations within the scope of the following claims.

Claims

1. A method for displaying player information in a motion video performed by a computing device, A step of acquiring video footage for a movement event in a target time interval; A step of acquiring sensor-based location information of the target time period for each player who participated in the above exercise event; A step of determining each corresponding position in the captured image corresponding to the sensor-based position information for each of the above players; and A step of displaying information about each player at each of the corresponding positions; including; A method for displaying player information in a sports video.

2. In paragraph 1, The above sensor-based location information is: Location information obtained based on the Global Navigation Satellite System (GNSS). A method for displaying player information in a sports video.

3. In paragraph 1, The above player information is, Personal information of the above player; Sensor-based location information of the above player; The speed of the above player; a trajectory of the player during a predetermined time interval; comprising at least one of: A method for displaying player information in a sports video.

4. In paragraph 1, The step of determining each of the above corresponding positions is: A step of displaying at least a portion of the above-described captured image and the above-described sensor-based location information; A step of receiving information from a user about the positions of the corners of the stadium within the above-described captured image; A step of receiving information about the position of the first player within the above-described shooting image from the user; and A step of receiving information about the position of the second player within the above-described shooting image from the user; comprising; A method for displaying player information in a sports video.

5. In paragraph 4, The above first and second players, Among the multiple players included in the above video footage, the players with the largest distance between themselves are: A method for displaying player information in a sports video.

6. In paragraph 4, The step of receiving information about the position of the first player in the above-mentioned filmed video and the step of receiving information about the position of the second player in the above-mentioned filmed video are: It is performed on different frames among multiple frames included in the above shooting video. A method for displaying player information in a sports video.

7. In paragraph 6, Information about the position of the first player in the above-mentioned filmed video and information about the position of the second player in the above-mentioned filmed video, Position information of a single player in different frames, A method for displaying player information in a sports video.

8. In paragraph 4, The step of receiving information about the position of the first player within the above-mentioned shooting video is as follows: In response to a determination that sensor-based position information for the first player is acquired in RTK-GPS mode, A method for displaying player information in a sports video.

9. In paragraph 4, The step of determining each of the above corresponding positions is: After the step of receiving information about the location of the second player within the above-mentioned shooting video from the user, A step of calculating the corresponding positions of each of the plurality of players based on information about the positions of the corners of the above stadium, the first player and the second player within the above-described captured images; and A step of receiving a batch change value for each corresponding position of each of the calculated plurality of players from the user; further comprising; A method for displaying player information in a sports video.

10. In paragraph 4, Information about the locations of the above stadium corners within the above-mentioned filmed images, Used for conversion between the image coordinate system (Pixel Coordination) indicating the position within the above-mentioned shooting video and the stadium coordinate system (Pitch Coordination) indicating the position within the stadium where the exercise event is performed. A method for displaying player information in a sports video.

11. In paragraph 4, Information about the position of the first player in the above-mentioned filmed video and information about the position of the second player in the above-mentioned filmed video, Used for conversion between the global coordinate system (Global Coordination) used by the above sensor-based position information and the image coordinate system (Pixel Coordination) indicating the position within the captured image. A method for displaying player information in a sports video.

12. In paragraph 4, The step of determining each of the above corresponding positions is: Based on information about the positions of the corners of the stadium within the captured image, information about the positions of the first player within the captured image, and information about the positions of the second player within the captured image, performing a conversion between a global coordinate system used by the sensor-based position information and a pitch coordinate system representing a position within the stadium where an athletic event is performed. A method for displaying player information in a sports video.

13. In paragraph 4, The step of displaying at least a portion of the above-described captured image and the above-described sensor-based location information is: Relative positional relationship of each sensor-based location information to the representative value of multiple sensor-based location information; Identification information of the sensor device corresponding to each sensor-based location information; and configured to display a location input request message for the first player or the second player; A method for displaying player information in a sports video.

14. In paragraph 5, The above first and second players, Determined based on the distance between multiple sensor-based location information, A method for displaying player information in a sports video.

15. In paragraph 9, After the step of receiving the above batch change values ​​from the user, A step of receiving a change value for the corresponding position of one of a plurality of players from a user; further comprising: A method for displaying player information in a sports video.

16. A device for displaying player information on a motion video, the device comprising: a processor; and a memory; The above processor, Acquire video footage of a motion event over a target time interval; Obtain sensor-based location information for the target time period for each player who participated in the above exercise event; Determining each corresponding position in the captured image corresponding to the sensor-based position information for each of the above players; and configured to display information about each player at each of the corresponding locations above; A device for displaying player information in a sports video.

17. A computer-readable storage medium containing instructions executable by a processor, the instructions being for displaying player information in a motion video, and being executed by the processor to cause the processor to: Acquire video footage of a motion event over a target time interval; Obtain sensor-based location information for the target time period for each player who participated in the above exercise event; Determining each corresponding position in the captured image corresponding to the sensor-based position information for each of the above players; and configured to display information about each player at each corresponding location above; Computer readable storage medium.

Images