Method for predicting location of sports ball, electronic apparatus and system performing the same

Abstract

An electronic device is disclosed. The electronic device includes a communication interface, a memory, and a processor. The processor acquires a plurality of player position datasets corresponding to a plurality of time points within a sports session, generates a plurality of team position datasets from the plurality of player position datasets, acquires a plurality of ball position data corresponding to a plurality of time points, and acquires information regarding ball ownership of a plurality of teams using a neural network model of the recurrent neural network family that receives the plurality of team position datasets and the plurality of ball position data in segment units corresponding to a predetermined time length.

Description

Technology Field

[0001] The present disclosure generally relates to predicting the position of a sports ball, and more specifically to predicting the position of a sports ball using the position of a player. Background Technology

[0002] In the modern sports industry, there is a growing demand for 'Electronic Performance Tracking Systems (EPTS),' which allow for the objective evaluation of athletes using machines. EPTS enables the acquisition of information regarding a player's position or movement, which in turn allows for sports analysis by calculating various indicators such as the player's performance or condition.

[0003] Recently, there have been attempts to acquire the position of a sports ball for use in determining indicators. Conventional ball position detection is generally performed based on video; specifically, the method primarily involves detecting the ball using object detection algorithms such as deep learning from a video feed obtained by filming the field where the sport is being played, and then projecting the ball onto the field to calculate its coordinates. However, the aforementioned conventional method has limitations, such as low detection accuracy—a disadvantage of small object detection techniques—due to issues like the ball moving very fast or occlusion caused by players, where the area occupied by the ball is relatively very small compared to the image size when the field is large. Furthermore, the large amount of computation makes it difficult to ensure real-time performance. The problem to be solved

[0004] A summary of specific embodiments disclosed herein is provided. It should be understood that the embodiments presented in the following summary are merely intended to provide a brief summary of specific embodiments and are not intended to limit the scope of this description. Accordingly, it should be noted in advance that this description may include various embodiments not presented below.

[0005] The concept of the invention disclosed herein and below provides methods, devices, systems, and computer-readable storage media for predicting the position of a sports ball.

[0006] The technical problem that the present invention aims to solve is to provide a method for predicting the position of a ball based on the position of a player participating in a sports session.

[0007] Another technical problem that the present invention aims to solve is to provide a method for obtaining team ball possession information based on the position of the ball.

[0008] Another technical problem that the present invention aims to solve is to provide a method for obtaining team ball possession information based on the position of the ball.

[0009] Another technical problem that the present invention aims to solve is to provide a method for identifying the state of a ball related to ownership and correcting the position data of the ball based on the state of the ball.

[0010] However, the problem intended to be solved by this description is not limited thereto and may be expanded in various ways within the scope and spirit of this description. means of solving the problem

[0011] The embodiments include methods, devices, systems, and computer-readable storage media for predicting the position of a sports ball.

[0012] According to one embodiment, an electronic device for predicting ball ownership comprises: a memory for storing at least one instruction; and a processor; wherein the processor, by executing the at least one instruction, acquires a plurality of player position datasets corresponding to a plurality of time points within a sports session—each of which includes a plurality of position values ​​assigned to a plurality of player identifiers at the corresponding time point—generates a plurality of team position datasets from the plurality of player position datasets—the plurality of team position datasets includes a plurality of first team position datasets generated from position values ​​assigned to a player identifier corresponding to a first team and reflecting the placement status of the first team's players in the stadium, and a plurality of second team position datasets generated from position values ​​assigned to a player identifier corresponding to a second team and reflecting the placement status of the second team's players in the stadium—and acquires a plurality of ball position data corresponding to the plurality of time points—each of which includes a ball position value at the corresponding time point—and uses a neural network model of the recurrent neural network family that receives the plurality of team position datasets and the plurality of ball position data as input in segment units corresponding to a predetermined time length to derive from the plurality of team position datasets and the plurality of ball position data the plurality of teams An electronic device for obtaining information regarding ball ownership may be provided.

[0013] According to another embodiment, an electronic device for predicting ball ownership may be provided, comprising: a communication interface; a memory for storing at least one instruction; and a processor; wherein the processor, by executing the at least one instruction, acquires a plurality of player position datasets corresponding to a plurality of time points within a sports session—each of which includes a plurality of position values ​​assigned to a plurality of player identifiers at a corresponding time point—and acquires a plurality of ball position data corresponding to the plurality of time points—each of which includes a ball position value at a corresponding time point—and acquires information regarding ball ownership of the plurality of players from the plurality of player position datasets and the plurality of ball position data using a neural network model of the recurrent neural network family that receives the plurality of player position datasets and the plurality of ball position data in segment units corresponding to a predetermined time length.

[0014] According to another embodiment, a method for predicting ball possession comprises: obtaining a plurality of player position datasets corresponding to a plurality of time points within a sports session—each of which includes a plurality of position values ​​assigned to a plurality of player identifiers at a corresponding time point; generating a plurality of team position datasets from the plurality of player position datasets—the plurality of team position datasets including a plurality of first team position datasets generated from position values ​​assigned to a player identifier corresponding to a first team and reflecting the placement status of the players of the first team in the stadium, and a plurality of second team position datasets generated from position values ​​assigned to a player identifier corresponding to a second team and reflecting the placement status of the players of the second team in the stadium; and obtaining a plurality of ball position data corresponding to the plurality of time points—each of which includes a ball position value at a corresponding time point. A method may be provided comprising the step of obtaining information regarding ball ownership of the plurality of teams from the plurality of team location datasets and the plurality of ball location data using a neural network model of the recurrent neural network family that receives the plurality of team location datasets and the plurality of ball location data in segment units corresponding to a predetermined time length.

[0015] The exemplary embodiments above and other exemplary embodiments will be explained or clarified by the detailed description that follows regarding exemplary embodiments to be read in conjunction with the accompanying drawings. Effects of the invention

[0016] The disclosed technology may have the following effects. However, this does not mean that a specific embodiment must include all of the following effects or only the following effects; therefore, the scope of the rights of the disclosed technology should not be understood as being limited by this.

[0017] According to an embodiment of the present invention, information on a team's ball possession can be obtained based on the position of the ball. According to an embodiment, the state of the ball related to possession can be identified, and ball position data can be corrected based on the state of the ball.

[0018] The foregoing description of the invention does not constitute a complete list of all aspects of the invention. The invention should be understood to include all methods, apparatuses, and systems that are feasible from all appropriate combinations of the various aspects disclosed in the following detailed description and claims, as well as the matters summarized above. Furthermore, effects that may be obtained or predicted by the embodiments of the present disclosure are to be disclosed directly or implicitly in the detailed description of the embodiments of the present disclosure. For example, various effects predicted according to the embodiments of the present disclosure will be disclosed in the detailed description that follows. Brief explanation of the drawing

[0019] Aspects, features, and advantages of specific embodiments of the present disclosure will become more apparent from the following description with reference to the accompanying drawings. FIG. 1 is a drawing for explaining a method for obtaining a ball position according to one embodiment of the present disclosure. FIG. 2 is a drawing for explaining a method of obtaining the position of a ball according to one embodiment of the present disclosure. FIG. 3 is a diagram showing a player location dataset according to one embodiment of the present disclosure. FIG. 4 is a diagram showing a team location dataset according to one embodiment of the present disclosure. FIG. 5 is a drawing showing team location information according to another embodiment of the present disclosure. FIG. 6 is a diagram illustrating a method for processing a team location dataset according to one embodiment of the present disclosure. FIG. 7 is a diagram illustrating the structure of a first neural network model according to one embodiment of the present disclosure. FIG. 8 is a diagram illustrating a learning method for a first neural network model according to one embodiment of the present disclosure. FIG. 9 is a drawing for explaining a method for obtaining a ball position according to one embodiment of the present disclosure. FIG. 10 is a drawing for explaining the structure of a second neural network model according to one embodiment of the present disclosure. FIG. 11 is a drawing for explaining a learning method of a second neural network model according to one embodiment of the present disclosure. FIG. 12 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure. FIG. 13 is a drawing for explaining a method of obtaining ball ownership information of a team according to one embodiment of the present disclosure. FIG. 14 shows input data of a third neural network model according to one embodiment of the present disclosure. FIG. 15 is a diagram illustrating the structure of a third neural network model according to one embodiment of the present disclosure. FIG. 16 is a diagram illustrating a learning method for a third neural network model according to one embodiment of the present disclosure. FIG. 17 is a flowchart for explaining a method for determining ball ownership information of a team according to one embodiment of the present disclosure. FIG. 18 is a drawing for explaining a method for identifying a ball-owning team according to one embodiment of the present disclosure. FIG. 19 is a drawing for explaining a method for identifying a ball-owning player according to one embodiment of the present disclosure. FIG. 20 is a drawing for explaining a method for identifying a ball-owning player according to one embodiment of the present disclosure. FIG. 21 is a drawing for explaining a method for identifying a ball-owning player according to one embodiment of the present disclosure. FIG. 22 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure. FIG. 23 is a flowchart illustrating a method for acquiring ball position data according to one embodiment of the present disclosure. FIG. 24 is a graph showing the velocity information and acceleration information of a ball according to one embodiment of the present disclosure. FIG. 25 is a drawing for explaining a method for determining the state of a ball related to ownership according to one embodiment of the present disclosure. FIG. 26 is a diagram illustrating a method for acquiring a ball position based on the state of the ball related to possession. FIG. 27 is a graph showing ball speed information according to one embodiment of the present disclosure. FIG. 28 is a flowchart illustrating a method for obtaining a ball position according to one embodiment of the present disclosure. FIG. 29 is a drawing for explaining a method for setting a region of interest according to one embodiment of the present disclosure. FIG. 30 is a drawing for explaining a method for setting a region of interest according to one embodiment of the present disclosure. FIG. 31 is a drawing for explaining a ball identification method according to one embodiment of the present disclosure. FIG. 32 is a drawing for explaining a method of obtaining second position data of a ball according to one embodiment of the present disclosure. FIG. 33 is a block diagram showing the configuration of an electronic device according to one embodiment of the present disclosure. Specific details for implementing the invention

[0020] The terms used in this specification will be briefly explained, and the present disclosure will be described in detail.

[0021] The terms used in the embodiments of this disclosure have been selected to be as widely used and general as possible, taking into account their functions within this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant explanatory section of this disclosure. Therefore, terms used in this disclosure should be defined not merely by their names, but based on their meanings and the overall content of this disclosure.

[0022] The embodiments of the present disclosure are subject to various modifications and may have various embodiments; therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of specific embodiments, and it should be understood that the scope includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the disclosed art. In describing the embodiments, detailed descriptions of related prior art are omitted if it is determined that such detailed descriptions may obscure the essence.

[0023] Terms such as "first," "second," etc., may be used to describe various components, but components should not be limited by these terms. Terms are used solely for the purpose of distinguishing one component from another.

[0024] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "consisting of" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0025] Embodiments of the present disclosure are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present disclosure in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0026] According to an exemplary embodiment of the present disclosure, an electronic device for predicting the position of a sports ball comprises: a communication interface including at least one communication circuit; and a memory for storing at least one instruction. An electronic device may be provided comprising: a processor; wherein the processor, by executing at least one instruction, acquires a plurality of player position datasets corresponding to a plurality of time points within a sports session, wherein each of the plurality of player position datasets includes a plurality of position values ​​assigned to a plurality of player identifiers at a corresponding time point, and generates a plurality of team position datasets from the plurality of player position datasets, wherein the plurality of team position datasets include a plurality of first team position datasets generated from position values ​​assigned to a player identifier corresponding to a first team and reflecting the placement status of the players of the first team in the stadium, and a plurality of second team position datasets generated from position values ​​assigned to a player identifier corresponding to a second team and reflecting the placement status of the players of the second team in the stadium, and acquires the position of a sports ball from the plurality of team position datasets using a first neural network model of a recurrent neural network series that receives the plurality of team position datasets in segment units corresponding to a predetermined time length.

[0027] The plurality of players may include a plurality of field players and a plurality of goalkeepers. The processor may receive a plurality of location information obtained by a tracking device installed on clothing worn by the plurality of field players from an external device via the communication interface, and obtain the plurality of location values ​​based on the plurality of location information.

[0028] The processor can obtain the positions of the multiple goalkeepers based on the multiple position values ​​by using a second neural network model trained to receive the position values ​​of the multiple field players and output the position value of the goalkeeper.

[0029] The first neural network model receives as input to the first neural network model a first segment of the plurality of team location datasets corresponding to a first time interval and a second segment of the plurality of team location datasets corresponding to a second time interval following the first time interval, and the first time interval and the second time interval may overlap in at least some time intervals.

[0030] The processor can acquire a video feed for the sports session and analyze the video feed to acquire a plurality of player location datasets.

[0031] The above first neural network model may include a first neural network that receives a plurality of player position datasets as input and outputs a plurality of first feature vectors corresponding to the plurality of time points, a second neural network that outputs a plurality of second feature vectors corresponding to the plurality of time points based on the plurality of first feature vectors, and a third neural network that outputs the position of a ball based on the plurality of second feature vectors.

[0032] The first neural network is a convolutional neural network, and the second neural network may be a bidirectional Long Short-Term Memory (LSTM) network including a forward network and a backward network.

[0033] The above LSTM network can obtain the second feature vector by performing computational processing on the third feature vector output by the forward network and the fourth feature vector output by the reverse network.

[0034] The above operation processing may include at least one of concatenation, summation, or averaging.

[0035] According to an exemplary embodiment of the present disclosure, a method for predicting the position of a sports ball may be provided, comprising: a step of obtaining a plurality of player position datasets corresponding to a plurality of time points within a sports session—each of which includes a plurality of position values ​​assigned to a plurality of player identifiers at a corresponding time point—; a step of generating a plurality of team position datasets from the plurality of player position datasets—the plurality of team position datasets including a plurality of first team position datasets generated from position values ​​assigned to a player identifier corresponding to a first team and reflecting the placement status of the players of the first team in the stadium, and a plurality of second team position datasets generated from position values ​​assigned to a player identifier corresponding to a second team and reflecting the placement status of the players of the second team in the stadium—and a step of obtaining the position of the sports ball from the plurality of team position datasets using a first neural network model of a recurrent neural network family that receives the plurality of team position datasets in segment units corresponding to a predetermined time length.

[0036] The plurality of players may include a plurality of field players and a plurality of goalkeepers. The step of acquiring the plurality of player location dataset may include receiving a plurality of location information acquired by a tracking device installed on clothing worn by the plurality of field players from an external device, and acquiring the plurality of location values ​​based on the plurality of location information.

[0037] The step of acquiring the plurality of player position datasets may include the step of acquiring the positions of the plurality of goalkeepers based on the plurality of position values ​​by using a second neural network model trained to receive position values ​​of the plurality of field players and output a position value of the goalkeeper.

[0038] According to an exemplary embodiment of the present disclosure, an electronic device for predicting ball ownership comprises: a memory for storing at least one instruction; and a processor; wherein the processor, by executing the at least one instruction, acquires a plurality of player position datasets corresponding to a plurality of time points within a sports session—each of which includes a plurality of position values ​​assigned to a plurality of player identifiers at the corresponding time point—generates a plurality of team position datasets from the plurality of player position datasets—the plurality of team position datasets includes a plurality of first team position datasets generated from position values ​​assigned to a player identifier corresponding to a first team and reflecting the placement status of the first team's players in the stadium, and a plurality of second team position datasets generated from position values ​​assigned to a player identifier corresponding to a second team and reflecting the placement status of the second team's players in the stadium—and acquires a plurality of ball position data corresponding to the plurality of time points—each of which includes a ball position value at the corresponding time point—and uses a neural network model of the recurrent neural network family that receives the plurality of team position datasets and the plurality of ball position data as input in segment units corresponding to a predetermined time length to derive from the plurality of team position datasets and the plurality of ball position data the plurality of teams An electronic device for obtaining information regarding ball ownership may be provided.

[0039] The information regarding ball ownership includes the probability that each of the plurality of teams possesses the ball, and the processor can identify the team with the highest probability among the plurality of teams as the team possessing the ball.

[0040] The processor can acquire a plurality of distances between each of the plurality of players and the ball at a corresponding point in time based on the plurality of team position datasets and the plurality of ball position data, and identify the player who belongs to the team possessing the ball and corresponds to the smallest distance among the plurality of distances as the player possessing the ball.

[0041] The processor can acquire a plurality of distances between each of the plurality of players and the ball at a corresponding point in time based on the plurality of team location datasets and the plurality of ball location data, acquire a plurality of modified distances between each of the plurality of players and the ball based on information regarding the ball ownership of the plurality of teams and the plurality of distances, and identify a player belonging to the team that owns the ball and corresponding to the smallest modified distance among the plurality of modified distances as the player who owns the ball.

[0042] The processor can identify a touch state of the ball—the touch state being a first state in which the ball is touched by at least one of the plurality of players and a second state in which the ball is not touched by the plurality of players—based on the plurality of team location datasets and the plurality of ball location data, and can identify the team possessing the ball based on the touch state of the ball and information regarding the ball ownership of the plurality of teams.

[0043] The processor can identify the touch state of the ball in the first time interval as the first state if there is a point in time within the first time interval where the angle according to the trajectory of the ball is smaller than the preset angle, and can identify the touch state of the ball in the first time interval as the second state if there is no point in time within the first time interval where the angle according to the trajectory of the ball is smaller than the preset angle.

[0044] The processor can identify the touch state of the ball as the first state in the first time interval if there exists a point in time within the first time interval where the distance between at least one of the plurality of players and the ball is smaller than the preset distance, and identify the touch state of the ball as the second state in the first time interval if there does not exist a point in time within the first time interval where the distance between at least one of the plurality of players and the ball is smaller than the preset distance.

[0045] The processor can identify the team possessing the ball at the first time point based on information regarding the ball ownership of the plurality of teams corresponding to the first time point if the touch state of the ball at the first time point is the first state, and if the touch state of the ball at the first time point is the second state, it can identify the team possessing the ball at the first time point based on information regarding the ball ownership of the plurality of teams corresponding to the second time point preceding the first time point, and information regarding the ball ownership of the plurality of teams corresponding to the third time point following the first time point.

[0046] The processor may identify the team identified as possessing the ball at the second time point as the team possessing the ball at the first time point if the team identified as possessing the ball at the second time point and the team identified as possessing the ball at the third time point are the same, and may identify the team identified as possessing the ball at the time point closer to the first time point among the second time point and the third time point as the team possessing the ball at the first time point if the team identified as possessing the ball at the second time point and the team identified as possessing the ball at the third time point are different.

[0047] The above-mentioned preset angle may be 45 degrees.

[0048] According to an exemplary embodiment of the present disclosure, an electronic device for predicting ball ownership may be provided, comprising: a communication interface; a memory for storing at least one instruction; and a processor; wherein the processor, by executing the at least one instruction, acquires a plurality of player position datasets corresponding to a plurality of time points within a sports session—each of which includes a plurality of position values ​​assigned to a plurality of player identifiers at a corresponding time point—and acquires a plurality of ball position data corresponding to the plurality of time points—each of which includes a ball position value at a corresponding time point—and acquires information regarding ball ownership of the plurality of players from the plurality of player position datasets and the plurality of ball position data using a neural network model of the recurrent neural network family that receives the plurality of player position datasets and the plurality of ball position data in segment units corresponding to a predetermined time length.

[0049] According to an exemplary embodiment of the present disclosure, a method for predicting ball possession comprises: obtaining a plurality of player position datasets corresponding to a plurality of time points within a sports session—each of which includes a plurality of position values ​​assigned to a plurality of player identifiers at a corresponding time point; generating a plurality of team position datasets from the plurality of player position datasets—the plurality of which include a plurality of first team position datasets generated from position values ​​assigned to a player identifier corresponding to a first team and reflecting the placement status of the players of the first team in the field, and a plurality of second team position datasets generated from position values ​​assigned to a player identifier corresponding to a second team and reflecting the placement status of the players of the second team in the field—and obtaining a plurality of ball position data corresponding to the plurality of time points—each of which includes a ball position value at a corresponding time point. A method may be provided comprising the step of obtaining information regarding ball ownership of the plurality of teams from the plurality of team location datasets and the plurality of ball location data using a neural network model of the recurrent neural network family that receives the plurality of team location datasets and the plurality of ball location data in segment units corresponding to a predetermined time length.

[0050] According to an exemplary embodiment of the present disclosure, an electronic device for acquiring the position of a sports ball comprises: a memory for storing at least one instruction; An electronic device may be provided comprising: a processor; wherein the processor, by executing the at least one instruction, obtains a plurality of predicted ball position data corresponding to a plurality of time points within a sports session—each of which the plurality of predicted ball position data includes a ball position value at the corresponding time point—and obtains velocity information and acceleration information of the ball based on the plurality of predicted ball position data, and identifies the state of the ball related to possession based on the velocity information and the acceleration information—the state of the ball includes a possessed state in which the ball is possessed by at least one of a plurality of players participating in the sports and a free state in which the ball is not possessed by any of the plurality of players—a plurality of player position datasets corresponding to the plurality of time points—each of which includes a position value of each of the plurality of players at the corresponding time point—and obtains a plurality of ball position data corresponding to the plurality of time points—each of which includes a ball position value at the corresponding time point.

[0051] If the state of the ball at a first point in time within the sports session is the possession state, the processor can obtain the position value of the player possessing the ball at the first point in time as the position value of the ball corresponding to the first point in time.

[0052] If the state of the ball at a first time point within the sports session is the free state, the processor can obtain a first position value of the ball corresponding to a first time point based on a second position value of the ball corresponding to a second time point preceding the first time point and a third position value of the ball corresponding to a third time point following the first time point.

[0053] At the second and third points in time above, the state of the ball may be the state of ownership.

[0054] The processor can obtain the first position value of the ball based on the second position value of the ball, the third position value of the ball, and the speed information of the ball between the second time point and the third time point.

[0055] The processor identifies a first acceleration of the ball at a first time point in which the rate of change of the ball's acceleration is 0, and a second acceleration of the ball at a second time point in which the rate of change of the ball's acceleration is 0 and follows the first time point, and if the first acceleration is greater than 0 and the second acceleration is less than 0 and there is no time point in which the rate of change of the ball's acceleration becomes 0 between the first time point and the second time point, the state of the ball between the first time point and the second time point can be identified as the free state.

[0056] The processor identifies a first acceleration of the ball at a first point in time when the rate of change of the ball's acceleration is 0, and a second acceleration of the ball at a second point in time when the rate of change of the ball's acceleration is 0 and follows the first point in time, and if the first acceleration is less than 0 and the second acceleration is greater than 0 and there is no point in time between the first point in time and the second point in time when the rate of change of the ball's acceleration becomes 0, the state of the ball between the first point in time and the second point in time can be identified as the possession state.

[0057] The processor identifies a first acceleration of the ball at a first time point in which the rate of change of the ball's acceleration is 0, and a second acceleration of the ball at a second time point in which the rate of change of the ball's acceleration is 0 and follows the first time point, and if the first acceleration is greater than 0, the second acceleration is less than 0, the rate of change of the ball's acceleration is 0, and the acceleration of the ball is greater than 0 at a third time point existing between the first time point and the second time point, the state of the ball between the first time point and the third time point is the possessed state and the state of the ball between the third time point and the second time point is the free state.

[0058] The processor identifies a first acceleration of the ball at a first time point in which the rate of change of the ball's acceleration is 0, and a second acceleration of the ball at a second time point in which the rate of change of the ball's acceleration is 0 and follows the first time point, and if the first acceleration is less than 0, the second acceleration is greater than 0, the rate of change of the ball's acceleration is 0, and the acceleration of the ball is less than 0 at a third time point existing between the first time point and the second time point, the state of the ball between the first time point and the third time point is the free state and the state of the ball between the third time point and the second time point is the possessed state.

[0059] According to an exemplary embodiment of the present disclosure, a method for obtaining the position of a sports ball may be provided, comprising: obtaining a plurality of predicted ball position data corresponding to a plurality of time points within a sports session, wherein each of the plurality of predicted ball position data includes a position value of the ball at the corresponding time point; obtaining velocity information and acceleration information of the ball based on the plurality of predicted ball position data; identifying a state of the ball related to possession based on the velocity information and the acceleration information, wherein the state of the ball includes a possessed state in which the ball is possessed by at least one of a plurality of players participating in the sports and a free state in which the ball is not possessed by any of the plurality of players; and obtaining a plurality of player position datasets corresponding to the plurality of time points, wherein each of the plurality of player position datasets includes a position value of each of the plurality of players at the corresponding time point, and a plurality of ball position data corresponding to the plurality of time points, wherein each of the plurality of ball position data includes a position value of the ball at the corresponding time point.

[0060] The step of acquiring the plurality of player position datasets may include, if the state of the ball at a first time point within the sports session is the possession state, acquiring the position value of the player possessing the ball at the first time point as the position value of the ball corresponding to the first time point.

[0061] The step of acquiring the plurality of player position datasets may include, if the state of the ball at a first time point within the sports session is the free state, acquiring a first position value of the ball corresponding to the first time point based on a second position value of the ball corresponding to a second time point preceding the first time point and a third position value of the ball corresponding to a third time point following the first time point.

[0062] The step of obtaining the first position value of the ball may include the step of obtaining the first position value of the ball based on the second position value of the ball, the third position value of the ball, and speed information of the ball between the second time point and the third time point.

[0063] According to an exemplary embodiment of the present disclosure, an electronic device for acquiring the position of a sports ball comprises: a memory for storing at least one instruction; and a processor; wherein the processor, by executing the at least one instruction, acquires position data of a plurality of players participating in a sports session, acquires first position data of the ball based on the position data of the plurality of players, sets a region of interest in a video feed of the sports session based on the first position data of the ball, extracts an image segment corresponding to the region of interest from the video feed, detects the ball from the image segment using a neural network model trained to detect an object included in the image, and acquires second position data of the ball based on the position of a pixel corresponding to the ball, the position of the camera that acquired the video feed, and the attitude of the camera.

[0064] The processor can acquire a first coordinate within the video feed corresponding to the first position data of the ball, and set an area extended by a preset range centered on the first coordinate as the region of interest.

[0065] The processor can acquire the distance between the camera and the ball based on the first position data of the ball and the position of the camera, and identify the preset range based on the distance.

[0066] The larger the above distance, the smaller the above preset range may be.

[0067] The processor can obtain the image segment by cropping the region of interest in the video feed.

[0068] The processor can generate a team position dataset reflecting the placement status of the plurality of players in the stadium from the position data of the plurality of players, and obtain the first position data of the ball from the team position dataset using a recurrent neural network model that receives the team position dataset as input.

[0069] According to an exemplary embodiment of the present disclosure, a method for obtaining the position of a sports ball may be provided, comprising: obtaining position data of a plurality of players participating in a sports session; obtaining first position data of the ball based on the position data of the plurality of players; setting a region of interest in a video feed of the sports session based on the first position data of the ball; extracting an image segment corresponding to the region of interest from the video feed; detecting the ball from the image segment using a neural network model trained to detect an object included in the image; and obtaining second position data of the ball based on the position of a pixel corresponding to the ball, the position of the camera that obtained the video feed, and the attitude of the camera.

[0070] The step of setting the region of interest may include the step of obtaining a first coordinate within the video feed corresponding to the first position data of the ball, and the step of setting an area extended by a preset range centered on the first coordinate as the region of interest.

[0071] The step of setting the region of interest may include the step of obtaining the distance between the camera and the ball based on the first position data of the ball and the position of the camera, and the step of identifying the preset range based on the distance.

[0073] FIG. 1 is a drawing for explaining a method for obtaining a ball position according to one embodiment of the present disclosure.

[0074] Referring to FIG. 1, there may be multiple players (11, 12, 13, 14, 15) on the field (10) during a sports session. A sports session may include some or all of the events related to sports. Sports events may include sports matches and training. Sports according to the present disclosure mean ball sports. For example, sports according to the present disclosure may include soccer, American football, basketball, hockey, and ice hockey.

[0075] Multiple players (11, 12, 13, 14, 15) may include first-type players (11, 12, 13, 14) and second-type players (15). The first type and the second type may be distinguished by the rules of the sport. The first type and the second type may differ in the scope of actions or activities permitted according to the rules of the sport. For example, a first-type player may be a player who cannot handle the ball or puck with their hands during the game. A second-type player may be a player who can handle the ball or puck with their hands during the game.

[0076] In one embodiment, if the sport is soccer, the first type of players (11, 12, 13, 14) may be field players. The second type of player (15) may be a goalkeeper. In another embodiment, if the sport is ice hockey, the second type of player (15) may be a goaltender.

[0077] During a sports session, the tracking device (T) may be attached to or coupled to the player's body or to clothing worn by the player. The tracking device (T) is configured to acquire the player's location. The tracking device (T) may include a positioning sensor and / or a motion sensor. Here, the positioning sensor may include a GPS sensor and an LPS sensor, and the motion sensor may include an IMU sensor. The tracking device (T) may be detachable or separable from the player's body or clothing worn by the player. For example, players (11, 12, 13, 14) may be active while wearing a vest (200) that accommodates the tracking device (T). However, this is merely one embodiment, and the tracking device (T) may be provided on various items worn by the player. For example, the tracking device (T) may be attached to shoes worn by the player. Meanwhile, the tracking device (T) may be implemented as a wearable device that the player can wear. The wearable device may be implemented in various forms, such as a watch, a bracelet, a hair band, etc.

[0078] The tracking device (T) can obtain location information for the player. For example, if the tracking device (T) is a GPS sensor module, the location information may include latitude, longitude, and altitude corresponding to the player's location. Additionally, the GPS sensor module may detect speed and direction of movement information. For another example, if the tracking device (T) is an IMU sensor module, the location information may include the player's acceleration, angular velocity, and heading direction.

[0079] The tracking device (T) can transmit location information about a player to an external device or electronic device (100). The external device or electronic device (100) can obtain location data (or location values) that reflect the player's location based on the location information. The electronic device (100) can convert the location information about the player into location data. The player's location data may include two-dimensional or three-dimensional coordinates representing the player's location within the field (10). Here, the coordinates may be expressed according to the length direction, width direction, and vertical direction of the stadium.

[0080] The electronic device (100) can obtain a plurality of player location datasets corresponding to a plurality of time points within a sports session from the acquired location data. The player location dataset may include a plurality of location values ​​of a plurality of players. A plurality of location values ​​may each be assigned to a plurality of player identifiers. For example, a first player location dataset corresponding to a first time point may include the location values ​​of each of the players (11, 12, 13, 14) at the first time point. Meanwhile, the electronic device (100) is not required to receive location information about a player from a tracking device (T). For example, the electronic device (100) may receive an image or video captured from one or more cameras installed in the field illustrated in FIG. 1, detect a player from the image using an object detection algorithm, calculate location data about a player by projecting pixel coordinates representing the detected player in the image onto the field, and then obtain a player location dataset from the image.

[0081] The electronic device (100) can generate a team location dataset from a player location dataset. The team location dataset may reflect the placement status of players belonging to the respective team in the field (10). For example, if the first players (13, 14) belong to the first team and the second players (11, 12, 15) belong to the second team, the electronic device (100) can generate a first team location dataset reflecting the placement status of the first players (13, 14) in the field (10) and a second team location dataset reflecting the placement status of the second players (11, 12, 15) in the field (10). The electronic device (100) can generate multiple team location datasets corresponding to multiple time points within a sports session. The multiple team location datasets may include multiple first team location datasets and multiple second team location datasets.

[0082] The electronic device (100) can obtain the position of the ball (1) using a neural network model that receives multiple team position datasets as input. For example, the neural network model may be a series of recurrent neural networks that receive multiple team position datasets with a predetermined sampling rate (e.g., 25 Hz) as input in segment units corresponding to a predetermined time length (e.g., 4 seconds).

[0084] FIG. 2 is a drawing for explaining a method of obtaining the position of a ball according to one embodiment of the present disclosure.

[0085] Referring to FIG. 2, the electronic device (100) can acquire multiple location datasets (21) corresponding to multiple time points within a sports session. For example, the multiple location datasets may be provided in the form of time-series data frames containing location values ​​of multiple players in each frame. The multiple location datasets (21) may be multiple player location datasets or multiple team location datasets.

[0086] The electronic device (100) can input a plurality of location datasets (21) into a first neural network model (M1) to calculate the predicted position of the ball (22). The electronic device (100) can input a plurality of location datasets (21) into the first neural network model (M1) in segment units corresponding to a predetermined time length. The first neural network model (M1) may be a neural network model of the recurrent neural network series trained to output the position of the ball (22) based on the plurality of location datasets (21). The first neural network model (M1) may include a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), and various other Deep Neural Networks (DNN).

[0087] The electronic device (100) can input team location datasets corresponding to each team by separating them into separate input channels. For example, the electronic device (100) can input a first team location dataset, which is the team location dataset of the first team, into the first input channel of the first neural network model (M1). The electronic device (100) can input a second team location dataset, which is the team location dataset of the second team, into the second input channel of the first neural network model (M1). Alternatively, the electronic device (100) can merge the team location datasets of each team to obtain a single team location dataset and input the obtained single team location dataset into the first neural network model (M1).

[0089] FIG. 3 is a diagram showing a player location dataset according to one embodiment of the present disclosure.

[0090] Referring to FIG. 3, the player position dataset (30) may include multiple position values ​​representing the positions of multiple players at multiple time points within a sports session. For example, the player position dataset (30) may include a first player position dataset corresponding to a first time point (t=1). The first player position dataset may include multiple position values ​​((x1_A1, y1_A1), (x1_A2, y1_A2), ..., (x1_B11, y1_B11)) representing the positions of multiple players at the first time point (t=1). For another example, the second player position dataset corresponding to a second time point (t=2) may include multiple position values ​​((x2_A1, y2_A1), (x2_A2, y2_A2), ..., (x2_B11, y2_B11)) representing the positions of multiple players at the second time point (t=2).

[0091] The electronic device (100) can obtain the position of a ball by inputting a player position dataset (30) into a first neural network model (M1). The electronic device (100) can obtain the position of a ball corresponding to a segment by inputting a plurality of player position datasets into the first neural network model (M1) in segment units corresponding to a predetermined time length.

[0092] Meanwhile, the first neural network model (M1) needs to predict the ball's position identically even if the order of input position values ​​changes. For example, two defenders belonging to the same team may change their positions according to tactics. In this case, if the positions of the remaining players other than the two defenders whose positions have changed remain the same, the first neural network model (M1) needs to predict the ball's position identically before and after the position change of the two defenders. That is, the first neural network model (M1) needs to be implemented as a permutation invariant model. To this end, the electronic device (100) can acquire a team position dataset reflecting the placement status of players on the field and input the team position dataset into the first neural network model (M1) to predict the ball's position.

[0094] FIG. 4 is a diagram showing a team location dataset according to one embodiment of the present disclosure.

[0095] Referring to FIG. 4, the team position dataset (40) can reflect the placement status of players in the stadium. Each element of the team position dataset (40) may correspond to a position within the stadium and may indicate whether a player is located at each position. If the value of each element is 0, it may mean that there is no player at the position corresponding to that element. If the value of each element is 1, it may mean that a player belonging to the first team (Team A) is located at the position corresponding to that element. If the value of each element is 2, it may mean that a player belonging to the second team (Team B) is located at the position corresponding to that element. Of course, since the values ​​"1" or "2" are merely trivial indices, it is acceptable to use values ​​of any form that can semantically distinguish teams, such as expressing them as "-1" and "1". Meanwhile, in FIG. 4, the location is represented using a coordinate plane in the form of a grid map with a field of 120x90, but the size of the grid map is not necessarily limited to the example in FIG. 4, and it is also possible to represent the location using vector coordinates instead of grid coordinates.

[0096] For example, the data illustrated in FIG. 4 may reflect that a player belonging to the first team (Team A) is located at the first location corresponding to the first element (41), a player belonging to the second team (Team B) is located at the second location corresponding to the second element (42), and no player exists at the third location corresponding to the third element (43).

[0097] The electronic device (100) can obtain multiple team position datasets from player position datasets corresponding to multiple points in time within a sports session. The electronic device (100) can assign values ​​to elements of the team position datasets based on position values ​​included in the player position datasets.

[0098] Meanwhile, team location datasets may exist for each team. For example, a team location dataset for Team 1 and a team location dataset for Team 2 may exist separately. In particular, when using a grid map-type coordinate system, spatial resolution is often reduced to gain computational speed; in such cases, representing multiple players located on the same element can be counterintuitive or difficult. For instance, if one intends to represent the placement status of two teams simultaneously on a single matrix, issues may arise, such as the need to define a separate representation format when two players belonging to different teams are in close proximity. Therefore, a multi-channel data format that represents the placement status of each team in a separate space may be suitable for compensating for these drawbacks.

[0099] FIG. 5 is a drawing showing team location information according to another embodiment of the present disclosure.

[0100] Referring to FIG. 5, there may be a first team position dataset (51), which is a team position dataset of the first team, and a second team position dataset (52), which is a team position dataset of the second team. The first team position dataset (51) may reflect the placement status of the first team's players in the stadium. The second team position dataset (52) may reflect the placement status of the second team's players in the stadium. If an element of each team's position dataset is "0", there may not be a player at the position corresponding to that element. If an element of each team's position dataset is "1", there may be a player at the position corresponding to that element. It should be noted once again that the index number is arbitrary.

[0101] The electronic device (100) can obtain the position of the ball based on the first team position dataset (51) and the second team position dataset (52). For example, the electronic device (100) can obtain the position of the ball by inputting a plurality of first team position datasets (51) and a plurality of second team position datasets (52) corresponding to a plurality of time points into a first neural network model (M1) at a predetermined time interval.

[0102] Meanwhile, the electronic device (100) can perform several post-processing steps on the team location dataset. For example, the electronic device (100) can apply a smoothing filter to the team location dataset. The smoothing filter may include a Gaussian filter. From another perspective, this post-processing can be understood as converting a binary data form representing the presence or non-presence of a player into a probabilistic model data form. By using a probabilistic model data form, the robustness of the data can be guaranteed even if the accuracy of player positioning is somewhat low, and at the same time, the discreteness caused by the grid map format of the representation can be mitigated. Therefore, by definition, it is not mandatory to use a smoothing filter, and various other probability distribution filters (e.g., z-probability function filters) may also be used. However, for the convenience of explanation below, they will be collectively referred to as smoothing filters.

[0104] FIG. 6 is a diagram illustrating a method for processing a team location dataset according to one embodiment of the present disclosure.

[0105] Referring to FIG. 6, the electronic device (100) can obtain a post-processed team position dataset (63) by applying a smoothing filter (62) to a team position dataset (61). The value of an element included in the team position dataset (63) may represent the probability that a position in the stadium corresponding to that element will be occupied by a player. The larger the value of the element, the higher the probability that a position in the stadium corresponding to that element will be occupied by a player. For example, the value of the first element (64) may be smaller than the value of the second element (65). At this time, the first probability that a first position corresponding to the first element (64) will be occupied by a player may be smaller than the second probability that a second position corresponding to the second element (65) will be occupied by a player.

[0106] Meanwhile, in FIG. 6, the values ​​of some elements of the smoothing filter (62) are given as an example where they exceed 1, but this is merely one embodiment. The smoothing filter (62) may have normalized values ​​such that the sum of the values ​​of all elements that pass through the filter is 1. For example, the elements of the smoothing filter (62) may have values ​​such as [0.0625, 0.125, 0.0625, 0.125, 0.25, 0.125, 0.0625, 0.125, 0.0625].

[0108] FIG. 7 is a diagram illustrating the structure of a first neural network model according to one embodiment of the present disclosure.

[0109] Referring to FIG. 7, the first neural network model (M1) may include a first neural network (NN1), a second neural network (NN2), and a third neural network (NN3). The first neural network (NN1) may output a plurality of first feature vectors (72-1, 72-2, 72-100) based on a plurality of location datasets (71-1, 71-2, 71-100) corresponding to a plurality of frames (or time points), respectively. For example, the first neural network (NN1) may output a first feature vector (72-1) corresponding to a first frame based on a first location dataset (71-1) corresponding to a first frame. In other words, the first neural network (NN1) may function as an embedder or encoder that converts raw input data into feature data. For example, the first neural network (NN1) may be a convolutional autoencoder of the Convolutional Neural Network (CNN) family. However, although FIG. 7 illustrates the first neural network (NN1) being located at the first position of the model, it is not necessary to use a neural network in the process of embedding the location dataset (71-1, 71-2, 71-100) into the first feature vector (72-1, 72-2, 72-100), and it may be possible to apply other embedding algorithms. Also, while the embedding result may typically be in the form of a vector, it may be an embedding in other forms of representation. The multiple location datasets (71-1, 71-2, 71-100) may be player location datasets or team location datasets.

[0110] Meanwhile, the location data input to the first neural network model (M1) does not necessarily have to correspond temporally with the sensed location data. For example, the sampling period during positioning, that is, the time interval between the sensed location data and the sampling period of the location data input to the model, may be different, and there is no need for time synchronization. For example, a location function can be generated using the sensed data acquired at a specific sampling period, and then input location data can be calculated from the location function at a different sampling period. This is also the case with the explanation of Figure 10 below.

[0111] Also, in FIG. 7, the location dataset (71-1, 71-2, 71-100) input to the first neural network model (M1) is shown as being composed of frames, but this is merely one embodiment, and the input data of the first neural network model (M1) may be a plurality of location datasets corresponding to a plurality of frames included in a predetermined time length.

[0112] For example, a predetermined time length may be 100 frames. At this time, the electronic device (100) can input a position dataset into the first neural network model (M1) in units of 100 frames. The electronic device (100) can input the first position dataset (71-1) to the 100th position dataset (71-100) into the first neural network model (M1) all at once. Accordingly, the electronic device (100) can obtain the position of the ball corresponding to 100 frames. In this case, implementation is possible, for example, by using a 3D convolutional neural network, and it is not necessary to use a recurrent neural network series. Of course, an RNN series neural network is still used, but it is also possible to process multiple frames into a single embedding during the embedding process.

[0113] Meanwhile, the position datasets for each time interval input to the first neural network model (M1) may overlap. For example, the electronic device (100) may sequentially input a first position dataset corresponding to a first time interval and a second position dataset corresponding to a second time interval into the first neural network model (M1). At this time, the first time interval may be from the first frame to the 100th frame, and the second time interval may be from the 51st frame to the 150th frame. Accordingly, the first position dataset and the second position dataset may include a common position dataset (i.e., from the 51st frame to the 100th frame).

[0114] The second neural network (NN2) can output a plurality of second feature vectors (73-1, 73-2, 73-100) based on a plurality of first feature vectors (72-1, 72-2, 72-100) corresponding to a plurality of frames, respectively. For example, the second neural network (NN2) can output a second feature vector (73-1) corresponding to a first frame based on a first feature vector (72-1) corresponding to a first frame. The second neural network (NN2) can output a second feature vector (73-2) corresponding to a second frame based on a first feature vector (72-2) corresponding to a second frame. The second neural network (NN2) may be provided as an RNN-type model, for example, a recurrent neural network (RNN) including a forward network and a reverse network, or a unidirectional network including either one of the two, a long short-term memory (LSTM) network, a gated recurrent unit (GRU)-type network, etc.

[0115] The third neural network (NN3) can output the position of the ball (74-1, 74-2, 74-100) based on a plurality of second feature vectors (73-1, 73-2, 73-100) corresponding to a plurality of frames, respectively. For example, the third neural network (NN3) can output the first position (74-1) corresponding to the first frame based on the second feature vector (73-1) corresponding to the first frame. The third neural network (NN3) can output the second position (74-2) corresponding to the second frame based on the second feature vector (73-2) corresponding to the second frame. The third neural network (NN3) may be a Deep Neural Network (DNN).

[0116] Meanwhile, FIG. 7 illustrates an example in which a first feature vector, a second feature vector, and the position of a ball are obtained for each frame, but this is merely one embodiment. The first neural network model (M1) can obtain a first feature vector, a second feature vector, and the position of a ball for time intervals that include multiple frames. For example, the first neural network (NN1) can output a first feature vector corresponding to each time interval based on multiple position datasets corresponding to multiple frames included in each time interval.

[0118] FIG. 8 is a diagram illustrating a learning method for a first neural network model according to one embodiment of the present disclosure.

[0119] Referring to FIG. 8, training data (80) for training a first neural network model (M1) may include position values ​​of players and positions of the ball during multiple time points. For example, the training data (80) may include position values ​​(81) of players belonging to a first team (Team A), position values ​​(82) of players belonging to a second team (Team B), and ground truth (83) indicating the position of the ball.

[0120] The electronic device (100) can train a first neural network model (M1) based on training data (80). For example, the electronic device (100) can obtain a team location dataset (84) from the players' location values ​​(81, 82). The electronic device (100) can obtain the ball's location (85) by inputting the team location dataset (84) into the first neural network model (M1). The electronic device (100) can obtain a loss (86) based on ground truth (83) and the ball's location information (85). The electronic device (100) can update the parameters (or weights) of the first neural network model (M1) so that the loss (86) becomes smaller than a preset value. At this time, the parameters of multiple neural networks (e.g., first neural network (NN1), second neural network (NN2) and third neural network (NN3)) included in the first neural network model (M1) can be updated.

[0121] Meanwhile, some of the players (e.g., goalkeepers) may not wear the vest (200) combined with the tracking device (T). Accordingly, the electronic device (100) may not receive location information of some players from an external device. In this case, the electronic device (100) can obtain the location value of the player not wearing the vest (200) from the location value of the player wearing the vest (200). Accordingly, the electronic device (100) can obtain a more accurate ball location based on the location information of all players.

[0123] FIG. 9 is a drawing for explaining a method for obtaining a ball position according to one embodiment of the present disclosure.

[0124] Referring to FIG. 9, the electronic device (100) can input the location data (91) of the first player into the second neural network model (M2) to obtain the location data (92) of the second player. Here, the first player is a player (e.g., a field player) wearing a vest (200) combined with a tracking device (T), and the second player may be a player (e.g., a goalkeeper) not wearing a vest (200) combined with a tracking device (T). Of course, the first player does not necessarily have to be a field player and the second player must be a goalkeeper, and the first player and the second player may be any combination of all players.

[0125] The electronic device (100) can generate a location dataset representing the placement distribution of multiple first players in the stadium from the location data (91) of multiple first players. Then, the electronic device (100) can input the generated location dataset into a second neural network model (M2) to obtain the location data (92) of the second player.

[0126] The electronic device (100) can obtain a position dataset (93) representing the placement distribution of the first player and the second player in the stadium from the position data (91) of the first player and the position data (92) of the second player. The electronic device (100) can obtain the position (94) of the ball by inputting the position dataset (93) into the first neural network model (M1).

[0127] Meanwhile, in the present disclosure, position information of all players on the field is not necessarily required to obtain the position of the ball, and it is also possible to obtain the position of the ball without considering the position information of some players.

[0129] FIG. 10 is a drawing for explaining the structure of a second neural network model according to one embodiment of the present disclosure.

[0130] Referring to FIG. 10, the second neural network model (M2) may include a fourth neural network (NN4), a fifth neural network (NN5), and a sixth neural network (NN6). The fourth neural network (NN4) may output a plurality of third feature vectors (102-1, 102-2, 102-100) based on a field player location dataset (101-1, 101-2, 101-100) corresponding to a plurality of frames (or time points). For example, the fourth neural network (NN4) may output a third feature vector (102-1) corresponding to a first frame based on a first location dataset (101-1) corresponding to a first frame. In other words, the fourth neural network (NN4) may function as an embedder or encoder that converts raw input data into feature data. For example, the fourth neural network (NN4) may be a convolutional autoencoder of the Convolutional Neural Network (CNN) family. However, although FIG. 10 shows the fourth neural network (NN4) located at the first position of the model, it is not necessary to use a neural network in the process of embedding the position dataset (101-1, 101-2, 101-100) into the first feature vector (72-1, 72-2, 101-100), and it may be possible to apply other embedding algorithms. Also, while the embedding result may typically be in the form of a vector, it may be an embedding in other forms of representation. Multiple location datasets (101-1, 101-2, 101-100) may be the player location dataset described in FIG. 3 or the team location dataset described in FIG. 4.

[0131] Meanwhile, in FIG. 10, for convenience of explanation, the location dataset (101-1, 101-2, , 101-100) is shown as being input to the second neural network model (M2) in frame units, but this is merely one embodiment, and the location dataset (101-1, 101-2, , 101-100) may be input to the second neural network model (M2) in segment units of a predetermined time length. At this time, some of the location datasets for each time interval may overlap.

[0132] The fifth neural network (NN5) can output a plurality of fourth feature vectors (103-1, 103-2, 103-100) based on a plurality of third feature vectors (102-1, 102-2, 102-100) corresponding to a plurality of frames, respectively. For example, the fifth neural network (NN5) can output a fourth feature vector (103-1) corresponding to a first frame based on a third feature vector (102-1) corresponding to a first frame. The fifth neural network (NN5) may be a bidirectional or unidirectional Long Short-Term Memory (LSTM) network or an RNN-type network including a GRU-type network.

[0133] The sixth neural network (NN6) can output the position (104-1, 104-2, , 104-100) of a player (e.g., goalkeepers) without position information based on a plurality of fourth feature vectors (103-1, 103-2, , 103-100) corresponding to a plurality of frames, respectively. For example, the sixth neural network (NN6) can output the first position (104-1) corresponding to the first frame based on the fourth feature vector (103-1) corresponding to the first frame. The sixth neural network (NN6) may be a Deep Neural Network (DNN).

[0134] In the above, regarding the imputation of missing values, a method was applied to infer missing values ​​from values ​​obtained using artificial neural networks; however, other forms of imputation may also be possible. Furthermore, while the above explanation primarily described inferring the goalkeeper's missing values ​​through the positions of field players, it is also possible to infer missing values ​​for at least one arbitrary player from the position values ​​of the remaining players. Additionally, when missing values ​​occur, data augmentation is not strictly necessary; it may also be possible to use only the given values.

[0135] FIG. 11 is a drawing for explaining a learning method of a second neural network model according to one embodiment of the present disclosure.

[0136] Referring to FIG. 11, the electronic device (100) can obtain the location data (112) of the second player by inputting the location data (111) of the first player into the second neural network model (M2). The electronic device (100) can obtain the second loss (124) based on the location data (112) of the second player and the second ground truth (113). The electronic device (100) can repeatedly update the parameters of the second neural network model (M2) so that the second loss (124) becomes smaller than a preset value.

[0137] The electronic device (100) can train a second neural network model (M2) based on the output data of a first neural network model (M1). The electronic device (100) can generate a location dataset (115) based on the location data (111) of a first player and the location data (112) of a second player. The location dataset (115) can represent the placement distribution of the first player and the second player on the field. The electronic device (100) can obtain the ball's location (116) by inputting the location dataset (115) into the first neural network model (M1). The electronic device (100) can obtain a first loss (118) based on the ball's location (116) and the first ground truth (117). The electronic device (100) can iteratively update the parameters of the second neural network model (M2) so that the first loss (118) becomes smaller than a preset value.

[0138] Meanwhile, the training method of the first neural network model (M1) has been described in FIG. 8. The electronic device (100) can retrain the first neural network model (M1) based on the location data (112) of the second player. For example, the electronic device (100) can repeatedly update the parameters of the first neural network model (M1) so that the second loss (114) becomes smaller than a preset value.

[0139] The electronic device (100) can train a first neural network model (M1) and a second neural network model (M2) separately. For example, the electronic device (100) can first train the first neural network model (M1) and then train the second neural network model (M2) using the first neural network model (M1) that has been trained. At this time, the parameters of the first neural network model (M1) can be frozen while the training of the second neural network model (M2) is being performed.

[0140] Alternatively, the electronic device (100) may train the first neural network model (M1) and the second neural network model (M2) together. For example, the electronic device (100) may update the parameters of the first neural network model (M1) and the second neural network model (M2) so that the first loss (118) and the second loss (114) become smaller than preset values.

[0142] Meanwhile, although the above description primarily explained the inference of the ball's position or missing values ​​using only position values, it is also possible to add a channel that reflects the player's velocity to the input data. Of course, while the neural network would potentially infer velocity by inputting position data chronologically, a separate velocity channel can be added to aim for faster learning or improved accuracy. Similar to the position channel, the velocity channel can be represented on a heatmap-type grid, and can be provided in a form where velocity vector values ​​are assigned to elements corresponding to the player's location where the velocity occurred. In addition, a player's heading direction channel may be added. Velocity, speed, and heading direction can be values ​​measured by the GPS module or IMU module of the wearable device.

[0144] FIG. 12 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure.

[0145] Referring to FIG. 12, the electronic device (100) can acquire a plurality of player location datasets corresponding to a plurality of time points within a sports session (S1210). Each of the plurality of player location datasets may include a plurality of location values ​​assigned to a plurality of player identifiers at a corresponding time point.

[0146] The electronic device (100) can generate multiple team location datasets from multiple player location datasets (S1220). The multiple team location datasets may include multiple first team location datasets that are generated from location values ​​assigned to player identifiers corresponding to the first team and reflect the placement status of the first team's players in the stadium. The multiple team location datasets may include multiple second team location datasets that are generated from location values ​​assigned to player identifiers corresponding to the second team and reflect the placement status of the second team's players in the stadium.

[0147] The electronic device (100) can obtain the position of the ball by inputting a plurality of team position datasets into a first neural network model (M1) (S1230). The first neural network model (M1) may be a series of recurrent neural networks that receive a plurality of team position datasets in segment units corresponding to a predetermined time length.

[0149] FIG. 13 is a drawing for explaining a method of obtaining ball ownership information of a team according to one embodiment of the present disclosure.

[0150] Referring to FIG. 13, the electronic device (100) can input a position dataset (131) into a first neural network model (M1) to obtain ball position data (132) representing the position of the ball. The position dataset (131) may be a player position dataset or a team position dataset. The method of obtaining the ball position using the first neural network model (M1) has been described in detail in FIG. 9, so a redundant description will be omitted.

[0151] The electronic device (100) can obtain team ball ownership information (133) by inputting a location dataset (131) and ball location data (132) (or ball location data) into a third neural network model (M3). The third neural network model (M3) may be a series of recurrent neural networks that receive multiple team location datasets and multiple ball location data corresponding to multiple time points within a sports session in segment units corresponding to a predetermined time length.

[0152] Team ball possession information (or information regarding team ball possession) is information indicating which team possesses the ball. In the present disclosure, ball possession information (or information regarding ball possession) may include team ball possession information and player ball possession information. Player ball possession information (or information regarding player ball possession) is information indicating which player possesses the ball. Ball possession information may include the probability that a certain entity (team or player) is determined to possess the ball. For example, team ball possession information (133) may indicate that the probability that the first team possesses the ball is 80% and the probability that the second team possesses the ball is 20%. Meanwhile, in the present disclosure, "possesses the ball" by an entity means that the entity is in a state where it can control the ball.

[0154] FIG. 14 shows input data of a third neural network model according to one embodiment of the present disclosure.

[0155] Referring to FIG. 14, the input data of the third neural network model (M3) may include a first team position dataset (141), a second team position dataset (142), and ball position data (143) corresponding to a first time point (t=1). The input data of the third neural network model (M3) may include a plurality of first team position datasets, a plurality of second team position datasets, and a plurality of ball position data corresponding to a plurality of time points within a sports session.

[0156] The value of an element in the team location dataset may indicate whether the location within the stadium corresponding to that element is occupied by a player. For example, if the value of the element is 1, the location within the stadium corresponding to that element may be occupied by a player. If the value of the element is 0, the location within the stadium corresponding to that element may not be occupied by a player.

[0157] In another embodiment, the input data of the third neural network model (M3) may be data such as the training data (80) of FIG. 8. That is, it may consist of the player position dataset and ball position data of the third neural network model (M3).

[0158] Here, it should be noted that the data described above can be input into the model as a 3-channel data structure and, of course, can be preprocessed according to probability distribution filters, etc.

[0159] FIG. 15 is a diagram illustrating the structure of a third neural network model according to one embodiment of the present disclosure.

[0160] Referring to FIG. 15, the third neural network model (M3) may include a seventh neural network (NN7), an eighth neural network (NN8), and a ninth neural network (NN9). The seventh neural network (NN7) may output a plurality of fifth feature vectors (152-1, 152-2, 152-100) based on input data (151-1, 151-2, 151-100) corresponding to a plurality of frames, respectively. The input data corresponding to each frame may include a position dataset and ball position data. The position dataset may be a player position dataset or a team position dataset. For example, the seventh neural network (NN7) may output a fifth feature vector (152-1) corresponding to the first frame based on the first input data (151-1) corresponding to the first frame. The 7th neural network (NN7) can be a Convolutional Neural Network (CNN).

[0161] Meanwhile, in FIG. 15, for convenience of explanation, input data (151-1, 151-2, , 151-100) is shown as being input to the third neural network model (M3) in frame units, but this is merely one embodiment, and the input data (151-1, 151-2, , 151-100) may be input to the third neural network model (M3) in segment units of a predetermined time length. At this time, some of the input data for each time interval may overlap.

[0162] The eighth neural network (NN8) can output a plurality of sixth feature vectors (153-1, 153-2, 153-100) based on a plurality of fifth feature vectors (152-1, 152-2, 152-100) corresponding to a plurality of frames, respectively. For example, the eighth neural network (NN8) can output a sixth feature vector (153-1) corresponding to a first frame based on a fifth feature vector (152-1) corresponding to a first frame. The eighth neural network (NN5) may utilize various recurrent neural network series networks, including a bidirectional Long Short-Term Memory (LSTM) network comprising a forward network and a backward network.

[0163] The ninth neural network (NN9) can output team ball possession information (154-1, 154-2, , 154-100) in each frame based on a plurality of sixth feature vectors (153-1, 153-2, , 153-100) corresponding to a plurality of frames, respectively. For example, the ninth neural network (NN9) can output team ball possession information (154-1) in the first frame based on the sixth feature vector (153-1) corresponding to the first frame. The ball possession information (154-1) may be a probability value. The ninth neural network (NN6) may be a Deep Neural Network (DNN).

[0165] FIG. 16 is a diagram illustrating a learning method for a third neural network model according to one embodiment of the present disclosure.

[0166] Referring to FIG. 16, training data (160) for training the third neural network model (M3) may include player position values, ball position data, and team ball ownership information during multiple time points. For example, the training data (160) may include position values ​​(161) of players belonging to the first team (Team A), position values ​​(162) of players belonging to the second team (Team B), ball position data (163), and ground truth (164) representing team ball ownership information.

[0167] The electronic device (100) can train a third neural network model (M3) based on training data (160). For example, the electronic device (100) can obtain a team location dataset from the players' location values ​​(161, 162). The electronic device (100) can obtain team ball ownership information (166) by inputting input data (165), including the team location dataset and ball location data, into the third neural network model (M3). The electronic device (100) can obtain a loss (167) based on ground truth (164) and team ball ownership information (166). The electronic device (100) can update the parameters (or weights) of the third neural network model (M3) so that the loss (167) becomes smaller than a preset value. At this time, the parameters of multiple neural networks (e.g., 7th neural network (NN7), 8th neural network (NN8) and 9th neural network (NN9)) included in the 1st neural network model (M3) can be updated.

[0169] FIG. 17 is a flowchart for explaining a method for determining ball ownership information of a team according to one embodiment of the present disclosure.

[0170] Referring to FIG. 17, the electronic device (100) can acquire a position dataset and ball position data (S1710). The position dataset may be a player position dataset or a team position dataset.

[0171] The electronic device (100) can identify the touch state of the ball (S1720). The touch state of the ball may include a first state in which the ball is touched by at least one player and a second state in which it is not touched by anyone. The electronic device (100) can identify the touch state of the ball based on the distance between the player and the ball and the trajectory of the ball. For example, when the distance between the player and the ball is smaller than a preset value, or when the angle along the trajectory of the ball is smaller than a preset value, the electronic device (100) can determine that the touch of the ball is in the first state. When the distance between the player and the ball is greater than or equal to a preset value, or when the angle along the trajectory of the ball is less than or equal to a preset value, the electronic device (100) can determine that the touch of the ball is in the second state.

[0172] When the ball's touch state is in the first state, the electronic device (100) can identify the ball-owning team based on the team ball ownership information obtained by the third neural network model (M3) (S1730). For example, if the team ball ownership information output by the third neural network model (M3) indicates that the first team has the highest probability of owning the ball, the electronic device (100) can identify the first team as the ball-owning team.

[0173] When the ball touch state is in the second state, the electronic device (100) can identify the ball-owning team at the current time based on the ball-owning team at the previous time and the ball-owning team at the subsequent time (S1740). If the ball-owning team at the previous time and the ball-owning team at the subsequent time are the same, the electronic device (100) can determine that the ball-owning team at the current time is also the same as the ball-owning team at the previous time and the ball-owning team at the subsequent time. For example, if the ball-owning team at the first time is the first team and the ball-owning team at the third time after the first time is the first team, the electronic device (100) can determine that the ball-owning team at the second time between the first time and the third time is also the second team.

[0174] If the ball-possessing team at a previous point in time and the ball-possessing team at a subsequent point in time are different, the electronic device (100) can identify the ball-possessing team at the current point in time based on ball-possessing information at the previous point in time and ball-possessing information at the subsequent point in time. For example, the ball-possessing team at the first point in time is Team 1, and the probability that Team 1 possesses the ball at the first point in time may be 90 percent. Also, the ball-possessing team at the third point in time after the first point in time may be Team 2, and the probability that Team 2 possesses the ball at the third point in time may be 80 percent. At this time, the electronic device (100) can determine that the ball-possessing team at the first point in time (i.e., Team 1), which has a higher probability of possessing the ball, is the ball-possessing team at the second point in time between the first point in time and the third point in time.

[0176] FIG. 18 is a drawing for explaining a method for identifying a ball-owning team according to one embodiment of the present disclosure.

[0177] Referring to FIG. 18, the electronic device (100) can acquire the ball touch state (181) and the ball possession team (182). If the value of the cell corresponding to the ball touch state (181) is 1, it means that the ball touch state is in the first state at the time corresponding to that cell. If the value of the cell corresponding to the ball touch state (181) is 0, it means that the ball touch state is in the second state at the time corresponding to that cell. For example, at the first time (t=1), the ball touch state is in the first state, and at the second time (t=2), the ball touch state is in the second state.

[0178] When the ball's touch state is in the first state, the electronic device (100) can identify the team possessing the ball based on the team ball ownership information output by the third neural network model (M3) described above. For example, in the first state (t=1), the team possessing the ball may be Team A.

[0179] When the ball touch is in a second state, the electronic device (100) can identify the ball-owning team based on the ball-owning team at the time points before and after when the ball touch state is in a first state. For example, at the second time point (t=2), the ball touch state may be in a second state. The electronic device (100) can identify the first time point (t=1) and the fourth time point (t=4) at the time points before and after the second time point (t=2) when the ball touch state is in a first state. The electronic device (100) can identify the ball-owning team (i.e., Team A) at the first time point (t=1) and the ball-owning team (i.e., Team A) at the fourth time point (t=4). Since the ball-owning team is the same at the first time point (t=1) and the fourth time point (t=4), the electronic device (100) can identify the ball-owning team at the second time point (t=2) as Team A. Similarly, the electronic device (100) can identify the team possessing the ball as Team A at a third time point (t=3).

[0180] The ball-possessing team at the time points before and after the first state of the ball's touch state may differ. For example, at the fourth time point (t=4), the ball-possessing team may be Team A, and at the eighth time point (t=8), the ball-possessing team may be Team B. When determining the ball-possessing team at a time point existing between the fourth time point (t=4) and the eighth time point (t=8), the electronic device (100) may determine the ball-possessing team at the time point that is closer between the fourth time point (t=4) and the eighth time point (t=8) as the ball-possessing team at each time point. At the fifth time point (t=5), the ball-possessing team may be Team A, which is the ball-possessing team at the fourth time point (t=4). Additionally, at the seventh time point (t=7), the ball-possessing team may be Team B, which is the ball-possessing team at the eighth time point (t=8).

[0181] Meanwhile, when determining the ball-possessing team at the sixth time point (t=6), which is an intermediate time point between the fourth time point (t=4) and the eighth time point (t=8), the electronic device (100) can determine the ball-possessing team based on the team's ball possession information at the fourth time point (t=4) and the team's ball possession information at the eighth time point (t=8). The electronic device (100) can identify the ball-possessing team at the time point where the probability of the team possessing the ball is higher between the fourth time point (t=4) and the eighth time point (t=8) as the ball-possessing team at the sixth time point (t=6). For example, the probability of Team A possessing the ball at the fourth time point (t=4) may be 90 percent, and the probability of Team B possessing the ball at the eighth time point (t=8) may be 80 percent. At this time, the electronic device (100) can identify Team A, which has a higher probability of possessing the ball, as the ball-possessing team at the sixth time point (t=6).

[0182] Meanwhile, FIGS. 17 and 18 describe that when the ball touch state is in the first state, the electronic device (100) identifies the team that owns the ball based on the team ball ownership information output by the second neural network model (M2). However, this is merely one embodiment, and even when the ball touch state is in the second state, the electronic device (100) can identify the team that owns the ball based on the team ball ownership information output by the second neural network model (M2).

[0184] FIG. 19 is a drawing for explaining a method for identifying a ball-owning player according to one embodiment of the present disclosure.

[0185] Referring to FIG. 19, the electronic device (100) can obtain the distance (191) between a plurality of players and the ball. The electronic device (100) can identify the player closest to the ball among the plurality of players as the player who owns the ball.

[0186] The electronic device (100) can identify a ball-possessing player within a team identified as the ball-possessing team. The electronic device (100) can identify the player closest to the ball within the ball-possessing team as the ball-possessing player. For example, if Team A is the ball-possessing team at a first time point (t=1), the electronic device (100) can identify the player closest to the ball among multiple players belonging to Team A as the ball-possessing player.

[0188] FIG. 20 is a drawing for explaining a method for identifying a ball-owning player according to one embodiment of the present disclosure.

[0189] Referring to FIG. 20, the electronic device (100) can obtain a first distance (D) between a plurality of players and a ball. The electronic device (100) can obtain a second distance (L) between a plurality of players and a ball based on the first distance (D) and the team's ball possession information (201). For example, the electronic device (100) can convert the first distance (D) into a second distance (L) based on the mathematical formula (F1). According to the mathematical formula (F1), the second distance (L) may decrease as the team's ball possession probability increases. The second distance (L) may increase as the team's ball possession probability decreases.

[0190] The electronic device (100) can identify the ball-owning player based on a second distance (L). The electronic device (100) can identify the player with the smallest second distance (L) among a plurality of players as the ball-owning player.

[0192] FIG. 21 is a drawing for explaining a method for identifying a ball-owning player according to one embodiment of the present disclosure.

[0193] Referring to FIG. 21, the electronic device (100) can obtain player ball ownership information (213) by inputting a location dataset (211) and ball location data (212) into a fourth neural network model (M4). The location dataset (211) may be a player location dataset or a team location dataset. The player ball ownership information (213) may represent the probability that each of the plurality of players possesses the ball. The electronic device (100) can identify the ball-owning player based on the player ball ownership information (213). The electronic device (100) can identify the player with the highest probability of ball ownership among the plurality of players as the ball-owning player.

[0194] The electronic device (100) can generate a heatmap representing the distribution of players and balls in the stadium based on a location dataset (211) and ball location data (212). The electronic device (100) can input information about the generated heatmap into a fourth neural network model (M4).

[0195] The electronic device (100) can acquire various data based on the ball-possessing team and the ball-possessing player. For example, the electronic device (100) can generate data to evaluate the performance of each team based on the ball-possessing team. Alternatively, the electronic device (100) can generate data to evaluate the performance of a player based on the ball-possessing player. In addition, the electronic device (100) can acquire various data related to sports.

[0197] FIG. 22 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure.

[0198] Referring to FIG. 22, the electronic device (100) can acquire a plurality of player location datasets corresponding to a plurality of points in time within a sports session (S2210). The electronic device (100) can generate a plurality of team location datasets from the plurality of player location datasets (S2220).

[0199] The electronic device (100) can acquire multiple ball position data corresponding to multiple time points (S2230). For example, the electronic device (100) can acquire ball position data by inputting multiple player position datasets into a first neural network model (M1) in segment units.

[0200] The electronic device (100) can obtain information regarding ball ownership of multiple teams from multiple team location datasets and multiple ball location data by using a neural network model of the recurrent neural network series that receives multiple team location datasets and multiple ball location data in segment units corresponding to a predetermined time length (S2240). Here, the neural network model may refer to a third neural network model (M3). The electronic device (100) can identify the ball-owning team based on the information regarding ball ownership of the teams. The electronic device (100) can identify the team with the highest probability of ball ownership among the multiple teams as the ball-owning team.

[0201] Meanwhile, the electronic device (100) can obtain various information related to a sports event based on information regarding the ball's touch status, the team possessing the ball, and the player possessing the ball, corresponding to multiple time points within a sports session. The various information related to the sports event may include indicators related to team performance and indicators related to player performance. Indicators related to team performance may include the team's ball possession rate and pass success rate. Indicators related to player performance may include the player's pass success rate, dribble success rate, and number of effective shots.

[0202] The electronic device (100) can generate a highlight video based on the various information above. For example, the electronic device (100) can receive a video feed of a sports event from an external device. The electronic device (100) can identify the start and end points of a specific player's highlight event based on indicators related to the specific player's performance. Based on the identified start and end points and the video feed, the electronic device (100) can generate a highlight video showing the specific player's performance.

[0204] FIG. 23 is a flowchart illustrating a method for acquiring ball position data according to one embodiment of the present disclosure.

[0205] Referring to FIG. 23, the electronic device (100) can acquire multiple predicted ball position data corresponding to multiple points in time within a sports session (S2310). The electronic device (100) can acquire predicted ball position data using a first neural network model (M1).

[0206] The electronic device (100) can obtain ball velocity information and acceleration information based on a plurality of predicted ball position data (S2320). The electronic device (100) can obtain ball velocity information by differentiating the predicted ball position data. The ball velocity information may include the speed of the ball. The electronic device (100) can obtain ball acceleration information by differentiating the ball velocity information.

[0207] The electronic device (100) can identify the state of the ball related to possession based on the ball's velocity information and acceleration information (S2330). The state of the ball related to possession may include a 'possession state' in which the ball is possessed by at least one of a plurality of players and a 'free state' in which the ball is not possessed by any of the plurality of players. A method for identifying the state of the ball related to possession will be described later with reference to FIG. 25.

[0208] The electronic device (100) can acquire multiple ball position data corresponding to multiple time points based on multiple player position datasets corresponding to multiple time points and the state of the ball related to possession (S2340). The electronic device (100) can correct multiple predicted ball position data based on multiple player position datasets and the state of the ball related to possession. A method for acquiring multiple ball position data will be described later with reference to FIGS. 26 and FIGS. 27.

[0210] FIG. 24 is a graph showing the velocity information and acceleration information of a ball according to one embodiment of the present disclosure.

[0211] Referring to FIG. 24, the electronic device (100) can obtain ball velocity information (241) and ball acceleration information (242). The ball velocity information (241) may represent the speed of the ball over time. The ball acceleration information (242) may represent the acceleration of the ball over time. The electronic device (100) can obtain ball velocity information (241) and ball acceleration information (242) based on predicted ball position data. For example, the ball velocity information (241) may be represented by a first graph (G1) representing the speed of the ball. The ball acceleration information (242) may be represented by a second graph (G2) representing the acceleration of the ball.

[0212] The electronic device (100) can identify the time at which a ball touch occurs based on the ball's acceleration information (242). The electronic device (100) can identify multiple time points at which the rate of change of the ball's acceleration is zero. That is, the electronic device (100) can identify multiple time points at which the slope of the second graph (G2) becomes zero. For example, the electronic device (100) can identify a first time point (t1), a second time point (t2), and a third time point (t3).

[0213] The electronic device (100) can identify the point in time when a ball touch occurs among a plurality of identified points in time based on a preset acceleration range (A). If the acceleration of the ball at the identified point in time (i.e., the point in time when the rate of change of acceleration is 0) is outside the preset acceleration range (A), the electronic device (100) can determine that a ball touch occurred at the identified point in time. If the acceleration of the ball at the identified point in time is within the preset acceleration range (A), the electronic device (100) can determine that a ball touch did not occur at the identified point in time. For example, since the acceleration of the ball at the first point in time (t1) and the second point in time (t2) is outside the preset acceleration range (A), the electronic device (100) can determine that a ball touch occurred at the first point in time (t1) and the second point in time (t2). At the third time point (t3), the acceleration of the ball is within the preset acceleration range (A), so the electronic device (100) can determine that no ball touch occurred at the third time point (t3).

[0214] The preset acceleration range (A) is defined by a lower limit and an upper limit. The magnitude of the lower limit of the preset acceleration range (A) may be greater than the magnitude of the upper limit of the preset acceleration range (A). This is because various resistances (e.g., air resistance or friction with the ground) may act on the moving ball. For example, the lower limit of the preset acceleration range (A) may be -0.4 (m / s²) and the upper limit may be 0.3 (m / s²).

[0215] The above describes how the electronic device (100) detects a ball touch. Below, the method for determining the state of the ball related to ownership in the graph of FIG. 24 will be described.

[0217] FIG. 25 is a drawing for explaining a method for determining the state of a ball related to ownership according to one embodiment of the present disclosure. The graphs (G1, G2) shown in FIG. 25 are identical to the graphs shown in FIG. 24.

[0218] Referring to FIG. 25, the electronic device (100) can identify a plurality of times when the ball is touched. For example, the electronic device (100) can identify a first time point (t1), a second time point (t2), a third time point (t3), a fourth time point (t4), a fifth time point (t5), a sixth time point (t6), and a seventh time point (t7). At the first time point (t1), the fourth time point (t4), the fifth time point (t5), and the seventh time point (t7), the acceleration of the ball may be greater than 0. At the second time point (t2), the third time point (t3), and the sixth time point (t6), the acceleration of the ball may be less than 0.

[0219] The electronic device (100) can identify the state of the ball based on the acceleration of the ball at the time of ball touch. For example, if a ball touch with a magnitude of acceleration greater than 0 is identified, and then a ball touch with a magnitude of acceleration less than 0 is identified, the electronic device (100) can determine that the player's possession of the ball was lost at the time of ball touch with a magnitude of acceleration greater than 0. That is, the electronic device (100) can identify that the state of the ball is free between the time of ball touch with a magnitude of acceleration greater than 0 and the time of ball touch with a magnitude of acceleration less than 0. Referring to FIG. 25, at the first time point (t1), the acceleration of the ball is greater than 0, and at the second time point (t2), the acceleration of the ball is less than 0. The electronic device (100) can determine that the player's possession of the ball was lost at the first time point (t1). The electronic device (100) can identify that the state of the ball is free between the first time point (t1) and the second time point (t2).

[0220] As another example, if a ball touch with a magnitude of acceleration less than 0 is identified, and then a ball touch with a magnitude of acceleration greater than 0 is identified, the electronic device (100) can determine that the player has acquired possession of the ball at the time of the ball touch with a magnitude of acceleration less than 0. That is, the electronic device (100) can identify that the state of the ball is in possession between the time of the ball touch with a magnitude of acceleration less than 0 and the time of the ball touch with a magnitude of acceleration greater than 0. Referring to FIG. 25, at the sixth time point (t6), the acceleration of the ball is less than 0, and at the seventh time point (t7), the acceleration of the ball is greater than 0. The electronic device (100) can determine that the player has acquired possession of the ball at the sixth time point (t6). The electronic device (100) can identify that the state of the ball is in possession between the sixth time point (t6) and the seventh time point (t7).

[0221] As another example, if a plurality of first ball touches with an acceleration magnitude less than 0 are identified consecutively, and then a second ball touch with an acceleration magnitude greater than 0 is identified, the electronic device (100) may determine that the player has acquired possession of the ball at the time when the last ball touch among the plurality of first ball touches occurs. Referring to FIG. 25, at the second time point (t2) and the third time point (t3), the acceleration of the ball is less than 0, and at the fourth time point (t4), the acceleration of the ball is greater than 0. The electronic device (100) may determine that the player has acquired possession of the ball at the third time point (t3). The electronic device (100) may identify that the state of the ball is in possession between the third time point (t3) and the fourth time point (t4). The electronic device (100) may identify that the state of the ball is in a free state between the second time point (t2) and the third time point (t3).

[0222] If, after multiple ball touches with an acceleration magnitude less than 0 are identified, a ball touch with an acceleration magnitude greater than 0 is identified, the electronic device (100) may determine that the player has acquired possession of the ball at the point in time when the ball speed is lowest among the multiple ball touches with an acceleration magnitude less than 0. Referring to FIG. 25, the ball speed (v2) at the second point in time (t2) is greater than the ball speed (v3) at the third point in time (t3). The electronic device (100) may determine that the player has acquired possession of the ball at the third point in time (t3) when the ball speed is lower. The electronic device (100) may identify that the state of the ball is in possession between the third point in time (t3) and the fourth point in time (t4). The electronic device (100) may identify that the state of the ball is in a free state between the second point in time (t2) and the third point in time (t3).

[0223] As another example, if a second ball touch with an acceleration magnitude greater than 0 is identified after a plurality of first ball touches with an acceleration magnitude greater than 0 is identified, the electronic device (100) may determine that ball ownership was lost at the time when the last ball touch among the plurality of first ball touches occurred. Referring to FIG. 25, at the fourth time point (t4) and the fifth time point (t5), the acceleration of the ball is greater than 0, and at the sixth time point (t6), the acceleration of the ball is less than 0. The electronic device (100) may determine that ball ownership was lost at the fifth time point (t5). The electronic device (100) may identify that the state of the ball is in an owned state between the fourth time point (t4) and the fifth time point (t5). The electronic device (100) may identify that the state of the ball is in a free state between the fifth time point (t5) and the sixth time point (t6).

[0224] In the above, a ball touch with a magnitude of acceleration greater than 0 means that at the time the ball touch is identified, the magnitude of the ball's acceleration is greater than 0. For example, the ball touch corresponding to the first time point (t1) is a ball touch with a magnitude of acceleration greater than 0. A ball touch with a magnitude of acceleration less than 0 means that at the time the ball touch is identified, the magnitude of the ball's acceleration is less than 0. For example, the ball touch corresponding to the second time point (t2) is a ball touch with a magnitude of acceleration less than 0. Additionally, a ball touch time with a magnitude of acceleration greater than 0 means that at that time the ball's acceleration is greater than 0. For example, the first time point (t1) is a ball touch time with a magnitude of acceleration greater than 0. A ball touch time with a magnitude of acceleration less than 0 means that at that time the ball's acceleration is less than 0. For example, the second time point (t2) is a ball touch time with a magnitude of acceleration less than 0.

[0226] FIG. 26 is a diagram illustrating a method for acquiring a ball position based on the state of the ball related to possession.

[0227] Referring to FIG. 26, at the first time point (T1), the first player (261) may possess the ball. Therefore, at the first time point (T1), the state of the ball may be in possession. Between the first time point (T1) and the second time point (T2), the first player (261) may pass the ball to the second player (262). Accordingly, the first player (261) loses possession of the ball, and at the second time point (T2), the state of possession of the ball may be free. Subsequently, at the third time point (T3), the second player (262) may possess the ball. Therefore, at the third time point (T3), the state of the ball may be in possession.

[0228] When the ball is in a possession state, the electronic device (100) can acquire the position of the ball-possessing player as the ball position. That is, the electronic device (100) can acquire the position data of the ball-possessing player as ball position data. For example, at a first time point (T1), the electronic device (100) can acquire the position data of the first player (261) as ball position data. Additionally, at a third time point (T3), the electronic device (100) can acquire the position data of the second player (262) as ball position data.

[0229] At a second time point (T2) when the ball is in a free state, the electronic device (100) can obtain ball position data at the second time point (T2) based on ball position data at the first time point (T1) and the third time point (T3), respectively. For example, the electronic device (100) can obtain a position (263) between the ball position data at the first time point (T1) and the ball position data at the third time point (T3) as the ball position data at the second time point (T2). At the second time point (T2), the distance from the position (263) to the first player (261) may be a first distance (d1), and the distance from the position (263) to the second player (262) may be a second distance (d2). The electronic device (100) can determine the first distance (d1) and the second distance (d2) based on ball speed information.

[0231] FIG. 27 is a graph showing ball speed information according to one embodiment of the present disclosure.

[0232] Referring to FIG. 27, the speed of the ball may decrease as it moves from the first time point (T1) to the third time point (T3). The electronic device (100) can obtain the distance traveled by the ball for each time interval based on the speed of the ball. For example, the distance traveled by the ball between the first time point (T1) and the second time point (T2) may be the first distance (d1), and the distance traveled by the ball between the second time point (T2) and the third time point (T3) may be the second distance (d2). The electronic device (100) can obtain the position of the ball at the second time point (T2) based on the distance traveled by the ball.

[0233] Meanwhile, the electronic device (100) can obtain the position of the ball by analyzing an image of the sports session. Below, a method for obtaining the position of the ball through image analysis will be described.

[0235] Meanwhile, to improve the accuracy of the ball position, the following method may be performed. Unless otherwise noted in the following description, it may be understood that the above method is performed by an electronic device (100).

[0236] First, a sports video can be obtained from a camera. The electronic device (100) can detect a ball from an image using an object detection algorithm (e.g., an object detection neural network such as an RCNN-based network or a YOLO network, or an object segmentation-based network such as a CNN-DCNN network) from the sports video, and calculate pixel coordinates as stadium coordinates.

[0237] Next, among the frames of each video feed, anchor frames can be determined as frames where ball detection is believed to have been performed correctly. For example, since only one ball is generally used in sports games, frames with fewer than one ball detection—such as when no ball is detected due to false negatives or when multiple balls are detected due to false positives—can be determined as non-anchor frames. Furthermore, since the movement of the ball in real space must be continuous, even if only one ball is detected, the status of an anchor frame can be determined based on the difference between the position of the ball detected in the previous frame or previous frames and / or subsequent frames and the position of the ball detected in the current frame. Specifically, it can be determined as an anchor frame only if the difference is less than a certain distance. Alternatively, the status of an anchor frame can be determined by considering the trajectory of the ball in previous or subsequent frames.

[0238] Next, the ball's position inferred from the players' positions using an artificial neural network can be corrected by utilizing the ball's positions in the anchor frames. For example, data can be constructed using ball positions calculated from the video for the time period corresponding to the anchor frames, while ball positions for the time period corresponding to non-anchor frames are treated as missing values. These missing values ​​can then be replaced (imputted) with the ball's position inferred from the players' positions for the time period corresponding to the non-anchor frames. During this imputation process, a ball path can be generated from the inferred ball position for the time period corresponding to the non-anchor frames. The starting point of this path is determined as the ball's starting position in the anchor frame immediately preceding that segment, and the ending point as the ball's ending position in the anchor frame immediately following that segment. Finally, the path can be interpolated to connect the starting and ending positions. In other words, the starting and ending points of the path are coordinate-shifted to the respective starting and ending positions. Also, for example, the middle of the path can be interpolated by assigning weights proportional to the distance between the starting point and the starting position and the distance between the ending point and the ending position based on the distance between the starting position and the ending position.

[0239] Alternatively, the positions of the balls in the anchor frame can be used as input data for the artificial neural network illustrated in FIG. 7. Specifically, a channel reflecting the position values ​​of the balls is added to the input data of the artificial neural network in FIG. 7. Here, missing values ​​can be filled in using various data imputation techniques for missing intervals. For example, simple interpolation is also possible.

[0240] Meanwhile, in a method that uses both the ball position calculated based on video and the ball position inferred based on player position, the ball position inferred based on player position may use the result value according to the embodiment described in relation to FIG. 23, which is calculated by additionally considering ownership, etc., instead of the result value according to the embodiment described in relation to FIG. 7.

[0241] FIG. 28 is a flowchart illustrating a method for obtaining a ball position according to one embodiment of the present disclosure.

[0242] Referring to FIG. 28, an electronic device (1000) can acquire position data of multiple players participating in a sports session (S2810). The electronic device (100) can acquire first position data of a ball based on the position data of multiple players (S2820). For example, the electronic device (100) can generate multiple team position datasets from the position data of multiple players. The electronic device (100) can acquire first position data of a ball by inputting the multiple team position datasets into a first neural network model (M1).

[0243] The electronic device (100) can set a region of interest in the video feed of a sports session based on the first position data of the ball (S2830). The electronic device (100) can obtain first coordinates within the video feed corresponding to the first position data of the ball. The electronic device (100) can set an area extended by a preset range centered on the first coordinates as the region of interest. The video feed may be provided from an external device or an external server.

[0244] The electronic device (100) can extract an image segment corresponding to a region of interest from a video feed (S2840). The electronic device (100) can detect a ball from the image segment using a neural network model trained to detect objects included in the image (S2850). The electronic device (100) can obtain second position data of the ball based on the position of the pixel corresponding to the ball, the position of the camera that acquired the video feed, and the pose of the camera (S2860).

[0246] FIGS. 29 and FIGS. 30 are drawings for explaining a method for setting a region of interest according to one embodiment of the present disclosure.

[0247] Referring to FIG. 29, the electronic device (100) can acquire a first video feed (290) and first position data (291) of a ball. The electronic device (100) can set a first region of interest (292) in the first video feed (290) based on the first position data (291) of the ball. The electronic device (100) can acquire the position of a first pixel (293) corresponding to the first position data (291) of the ball based on the position information and pose information of the camera. The electronic device (100) can set an area extended by a first preset range (R1) centered on the first pixel (293) as the first region of interest (292). For example, the first preset range (R1) may correspond to 1.5m.

[0248] Referring to FIG. 30, the electronic device (100) can acquire a second video feed (300) and first position data (301) of a ball. The electronic device (100) can set a second region of interest (302) in the second video feed (300) based on the first position data (301) of the ball. The electronic device (100) can acquire the position of a second pixel (303) corresponding to the first position data (301) of the ball based on the position information and pose information of the camera. The electronic device (100) can set an area extended by a second preset range (R2) centered on the second pixel (303) as the second region of interest (302). For example, the second preset range (R2) may correspond to 1.5m.

[0249] The size of the preset range may vary depending on the position of the pixel within the video feed corresponding to the ball's first position information. For example, as the b-axis value of the pixel corresponding to the ball's first position information increases, the size of the preset range may decrease. The a-axis represents the horizontal axis of the camera's field of view, and the b-axis represents the vertical axis of the camera's field of view.

[0250] When the first video feed (290) and the second video feed (300) are acquired under the same conditions (e.g., captured by a camera located at the same position with the same pose information), the b-axis value of the first pixel (293) may be smaller than the b-axis value of the second pixel (303). Accordingly, the first preset range (R1) may be larger than the second preset range (R2).

[0251] The size of the preset range may vary depending on the distance between the ball and the camera calculated based on the ball's first position data. For example, as the distance between the ball and the camera increases, the size of the preset range may decrease. When the first video feed (290) and the second video feed (300) are acquired under the same conditions (e.g., captured by a camera located at the same position with the same pose information), the distance between the ball and the camera in FIG. 29 may be smaller than the distance between the ball and the camera in FIG. 30. Accordingly, the first preset range (R1) may be larger than the second preset range (R2).

[0252] Meanwhile, although FIGS. 29 and 30 describe the size of the preset range as variable, this is merely one embodiment, and the preset range may have a fixed size.

[0254] FIG. 31 is a drawing for explaining a ball identification method according to one embodiment of the present disclosure.

[0255] Referring to FIG. 31, the electronic device (100) can set a region of interest (312) in a video feed (311). The electronic device (100) can crop an area corresponding to the region of interest (312) in the video feed (311) to extract an image segment (313). The electronic device (100) can input the image segment (313) into a fifth neural network model (M5) to identify a ball (1). The fifth neural network model (M5) may include a Convolutional Neural Network (CNN).

[0256] Meanwhile, there have been conventional attempts to recognize a ball by analyzing a video feed. However, because existing ball recognition methods analyze the entire area of ​​the video feed, they have had problems such as misidentifying objects with characteristics similar to a ball (e.g., a player's head if the player is bald) as a ball.

[0257] In contrast, the electronic device (100) according to the present disclosure recognizes the ball based on a region of interest set based on spatiotemporal information of a plurality of players rather than the entire video feed area, so the ball recognition rate may be higher than that of a conventional ball recognition method.

[0259] FIG. 32 is a drawing for explaining a method of obtaining second position data of a ball according to one embodiment of the present disclosure.

[0260] Referring to FIG. 32, the electronic device (100) can obtain position information (321) and attitude information (322) of the camera (400). Based on the position information (321) and attitude information (322), the electronic device (100) can obtain the optical axis (323) of the camera (400). The optical axis (323) of the camera (400) can pass through the center of the video feed (324).

[0261] The electronic device (100) can project a video feed (324) onto a ground plane based on an optical axis (323). The electronic device (100) can obtain position data (325) of the ball (1) on the ground plane. The z-axis value of the position data (325) of the ball (1) can be obtained based on the z-axis value of the position information (321) of the camera (400). Meanwhile, since the position data (325) of the ball (1) is obtained by a two-dimensional video feed (324), it may be inaccurate when the ball (1) is floating above the ground.

[0262] As described above, the electronic device (100) can obtain first position information (326) of the ball (1) based on the position data of a plurality of players. The electronic device (100) can obtain second position data (327) of the ball (1) based on the first position data (326) of the ball (1) and the optical axis (323) of the camera (400). The electronic device (100) can obtain the z-axis value of the second position data (327) of the ball (1) based on the x-axis value and the y-axis value of the first position data (326) of the ball (1). That is, the electronic device (100) can obtain three-dimensional position data of the ball (1), including the height of the ball (1) from the ground.

[0264] FIG. 33 is a block diagram showing the configuration of an electronic device according to one embodiment of the present disclosure.

[0265] Referring to FIG. 33, the electronic device (100) may include a communication interface (110), a memory (120), and a processor (130). The electronic device (100) may be implemented as a server. However, this is merely one embodiment, and the electronic device (100) may be implemented as a user terminal.

[0266] The communication interface (110) includes at least one communication circuit and can communicate with various types of external devices or external servers. For example, the communication interface (110) can receive location information about a player from an external device. The external device may be a server. The external device may store location information received from the tracking device (T). Alternatively, the external device may acquire location data of the player based on location information about the player and store the acquired location data. The communication interface (110) can receive location information about a player from the tracking device (T).

[0267] The communication interface (110) may include at least one of a Wi-Fi communication module, a cellular communication module, a 3G (3rd generation) mobile communication module, a 4G (4th generation) mobile communication module, a 4th generation LTE (Long Term Evolution) communication module, a 5G (5th generation) mobile communication module, or a wired Ethernet.

[0268] The memory (120) can store an operating system (OS) for controlling the overall operation of the components of the electronic device (100) and commands or data related to the components of the electronic device (100). The memory (120) can store information about a plurality of neural network models. The information about the plurality of neural network models may include information about parameters corresponding to each of the plurality of neural network models and training data for training the plurality of neural network models. The memory (120) may be implemented as non-volatile memory (e.g., hard disk, SSD (Solid State Drive), flash memory) or volatile memory.

[0269] The processor (130) is electrically connected to the memory (120) and can control the overall function and operation of the electronic device (100). The processor (130) can control the electronic device (100) by executing instructions stored in the memory (120).

[0270] For example, the processor (130) may acquire a player location dataset corresponding to multiple time points within a sports session. The player location dataset may include location values ​​representing the locations of multiple players. A sports session may include sessions for various sports-related events, such as match sessions and training sessions. Multiple location values ​​for multiple time points may include coordinates within a field representing the location of each of the multiple players over time.

[0271] The processor (130) can obtain multiple location values ​​in various ways. For example, the processor (130) can receive multiple location values ​​from an external device through a communication interface (110). The external device may be an external server that stores location values ​​obtained based on location information obtained by a tracking device (T). For another example, the processor (130) can receive player location information from the tracking device (T). The processor (130) can obtain location values ​​based on the received location information.

[0272] The processor (130) can generate a team position dataset from a player position dataset. The team position dataset can reflect the placement status of players belonging to a team in the stadium. The team position dataset can be permutation invariant.

[0273] The processor (130) can predict the position of the ball using a neural network model. The processor (130) can obtain the position of the ball by inputting a plurality of team position datasets in segment units into the first neural network model (M1). The processor (130) can input the team position dataset of the first team into the first input channel of the first neural network model (M1) and input the team position dataset of the second team into the second input channel of the first neural network model (M1).

[0274] The processor (130) can obtain a corrected team position dataset by applying a Gaussian filter to the team position dataset. The processor (130) can obtain the position of the ball by inputting the corrected team position information into the first neural network model (M1).

[0275] The processor (130) can obtain information regarding ball ownership of multiple teams based on multiple team location datasets and multiple ball location data corresponding to multiple time points. For example, the processor (130) can obtain information regarding ball ownership of multiple teams from multiple team location datasets and multiple ball location data by using a third neural network model (M3) of a recurrent neural network series that receives multiple team location datasets and multiple ball location data in segment units corresponding to a predetermined time length.

[0276] The processor (130) can identify the team possessing the ball based on information regarding the team's ball possession. The processor (130) can identify the team with the highest probability of possessing the ball among multiple teams as the team possessing the ball.

[0277] The processor (130) can identify the team possessing the ball based on the team's ball possession information and the ball's touch state. The ball's touch state may include a first state in which the ball is touched by any player and a second state in which the ball is not touched by anyone.

[0278] When the ball's touch state is in the first state, the processor (130) can identify the ball-owning team based on the ball ownership probability of each team output by the third neural network model (M3). The processor (130) can identify the team with the highest ball ownership probability as the ball-owning team.

[0279] When the ball's touch state is in the second state, the processor (130) can identify the team possessing the ball based on ball ownership information of each team at a surrounding time. When the ball's touch state is in the second state at the first time, the processor (130) can identify the team possessing the ball at the first time based on ball ownership information of each team corresponding to the second time, which precedes the first time, and ball ownership information of each team corresponding to the third time, which follows the first time. If the team identified as possessing the ball at the second time and the team identified as possessing the ball at the third time are the same, the processor (130) can identify the team identified as possessing the ball at both the second and third times as the team possessing the ball at the first time. For example, if the first team possesses the ball at the second time and the first team also possesses the ball at the third time, the processor (130) can identify the team possessing the ball at the first time as the first team.

[0280] If the team identified as possessing the ball at the second time point and the team identified as possessing the ball at the third time point are different, the processor (130) may identify the team identified as possessing the ball at the time point closer to the first time point among the second and third time points as the team possessing the ball at the first time point. For example, if the first time point is closer to the second time point than the third time point, the processor (130) may identify that the team possessing the ball at the second time point also possesses the ball at the first time point.

[0281] If the difference between the first time point and the second time point is the same as the difference between the first time point and the third time point, the processor (130) can identify the team that possesses the ball at the first time point based on the probability of each team possessing the ball corresponding to the second time point and the probability of each team possessing the ball corresponding to the third time point. For example, the probability of the first team possessing the ball at the second time point may be 95%, and the probability of the second team possessing the ball at the third time point may be 80%. In this case, the processor (130) can identify the first team with the higher probability of possessing the ball as the team that possesses the ball at the first time point.

[0282] The processor (130) can identify the touch state of the ball based on the position information of the ball. For example, if there is a point in time within the first time interval where the angle along the ball's trajectory is smaller than a preset angle, the processor (130) can identify that the touch state of the ball in the first time interval is the first state. If there is no point in time within the first time interval where the angle along the ball's trajectory is smaller than a preset angle, the processor (130) can identify that the touch state of the ball in the first time interval is the second state. The preset angle may be 45 degrees.

[0283] The processor (130) can identify the touch state of the ball based on the ball's movement information. The ball's movement information may include the ball's acceleration information. The processor (130) can obtain the ball's acceleration information based on the ball's position information over time.

[0284] The processor (130) can identify the touch state of the ball based on the acceleration information of the ball. For example, the processor (130) can determine that the ball has been touched by the player at the point where the slope of the acceleration graph representing the acceleration of the ball over time becomes zero. That is, when the slope of the acceleration graph becomes zero, the processor (130) can determine that the touch state of the ball is a first state.

[0285] The processor (130) can identify the touch state of the ball based on the position information of multiple players and the trajectory of the ball. If there exists a point in time within a first time interval where the distance between at least one of the multiple players and the ball is smaller than a preset distance, and if there exists a point in time within the first time interval where the angle according to the trajectory of the ball is smaller than a preset angle, the processor (130) can identify the touch state of the ball in the first time interval as the first state. If there does not exist a point in time within the first time interval where the distance between at least one of the multiple players and the ball is smaller than a preset distance, the processor (130) can identify the touch state of the ball in the first time interval as the second state. The preset distance may be 1m.

[0286] The processor (130) can identify the touch state of the ball based on the acceleration information of the ball. The processor (130) can obtain the acceleration information of the ball based on the position information of the ball. When the acceleration value of the ball is greater than a preset first value (e.g., 0.3 m / s²) and the slope is 0, the processor (130) can determine that the ball has been touched at that point in time. That is, the processor (130) can determine that the touch state of the ball is the first state.

[0287] The processor (130) can identify the player who possesses the ball based on the position data of multiple players and the ball position data.

[0288] The processor (130) can obtain the distance between multiple players and the ball based on the position data of multiple players and the position data of the ball. The processor (130) can identify the player closest to the ball among the multiple players belonging to the team identified as possessing the ball as the ball-possessing player.

[0289] The processor (130) can identify the player possessing the ball among multiple players belonging to the entire team. For example, the processor (130) can obtain a first distance between multiple players and the ball based on the position data of multiple players and the ball position data. The processor (130) can obtain a second distance based on the first distance and information regarding ball possession by multiple teams. The processor (130) can identify the player with the smallest second distance among multiple players as the player possessing the ball. When calculating the second distance between each player and the ball, the processor (130) can apply a weight corresponding to the probability of ball possession by the team to which each player belongs to the first distance corresponding to each player. The larger the weight, the larger the second distance may become. The higher the probability of ball possession by the team, the smaller the weight may become. Therefore, the higher the probability of ball possession by the team to which the player belongs, the smaller the second distance corresponding to that player may become. The higher the probability of ball possession by the team, the higher the probability that a player belonging to that team will be identified as the player possessing the ball.

[0290] Meanwhile, the processor (130) can obtain ball ownership information of a player without the team's ball ownership information. For example, the processor (130) can identify the player closest to the ball among multiple players as the ball-owning player. Alternatively, the processor (130) can use a fourth neural network model (M4) for identifying the ball-owning player. The processor (130) can obtain ball ownership information of multiple players by inputting the position information of multiple players and the position information of the ball into a third neural network model. The ball ownership information of multiple players may represent the probability that each of the multiple players possesses the ball. The processor (130) can identify the player with the highest probability of possessing the ball among the multiple players as the ball-owning player.

[0291] The processor (130) can correct ball position data based on the ball's state related to ownership and a plurality of player position datasets. If the ball's state is in an owned state, the processor (130) can obtain the position of the player who owns the ball among the plurality of players as the ball position. If the ball's state is in a free state, the processor (130) can obtain the corrected ball position based on the ball's position at a preceding and succeeding time point. For example, the processor (130) can obtain a first position of the ball corresponding to a first time point. At this time, the ball's state at the first time point may be in a free state. The processor (130) can identify a second position of the ball at a second time point preceding the first time point and a third position of the ball at a third time point following the first time point. At the second and third time points, the ball's state may be in an owned state.

[0292] The processor (130) can obtain a fourth position of the ball based on the second position and the third position. The processor (130) can obtain a fourth position based on the ball's velocity information between the second position, the third position, the second time point, and the third time point. For example, while the ball moves from the second position corresponding to the second time point to the third position corresponding to the third time point, the ball's velocity may gradually decrease. That is, the distance traveled per unit time of the ball may be greater around the second position than around the third position. Therefore, the processor (130) can determine a fourth position between the second position and the third position, but can determine the fourth position as a position closer to the third position than to the second position. The processor (130) can correct the first position to the fourth position.

[0293] The processor (130) can identify the state of the ball based on the ball's velocity information and acceleration information. For example, the processor (130) can identify the first acceleration of the ball at a first point in time when the rate of change of the ball's acceleration becomes zero, and the second acceleration of the ball at a second point in time when the rate of change of the ball's acceleration is zero and follows the first point in time.

[0294] If the first acceleration is greater than 0, the second acceleration is less than 0, and there is no point in time between the first point in time and the second point in time when the rate of change of the ball's acceleration becomes 0, the processor (130) can identify that the state of the ball between the first point in time and the second point in time is a free state. The processor (130) can determine that at the first point in time, the state of the ball has transitioned from a possessed state to a free state. The processor (130) can determine that the player's possession of the ball has been lost at the first point in time.

[0295] There may exist a third point in time between the first and second points in time when the rate of change of the ball's acceleration becomes zero. If the ball's acceleration is greater than zero at the third point in time, the processor (130) can identify that the state of the ball between the first and third points in time is in a possessed state, and between the third and second points in time is in a free state. That is, the processor (130) can determine that the state of the ball has transitioned from a possessed state to a free state at the third point in time. The processor (130) can determine that the player's possession of the ball has been lost at the third point in time.

[0296] If the first acceleration is less than 0, the second acceleration is greater than 0, and there is no point between the first point and the second point in time where the rate of change of the ball's acceleration becomes 0, the processor (130) can identify that the state of the ball between the first point and the second point in time is in a state of possession. The processor (130) can determine that at the first point in time, the state of the ball has transitioned from a free state to a state of possession. The processor (130) can determine that the player has acquired ownership of the ball at the first point in time.

[0297] There may exist a third point in time between the first and second point in time when the rate of change of the ball's acceleration becomes zero. If the ball's acceleration is less than zero at the third point in time, the processor (130) can identify that the state of the ball between the first and third point in time is free, and the state of the ball between the third and second point in time is possessed. That is, the processor (130) can determine that the state of the ball has transitioned from free to possessed at the third point in time. The processor (130) can determine that the player has acquired possession of the ball at the third point in time. Alternatively, the processor (130) can determine that the state of the ball has transitioned from free to possessed at the point in time when the speed of the ball is lower between the first and third point in time.

[0298] The processor (130) can obtain ball velocity information and acceleration information based on ball position information. The processor (130) can obtain ball velocity information by differentiating ball position information. The ball velocity information may include the speed of the ball. The processor (130) can obtain ball acceleration information by differentiating ball velocity information.

[0299] The processor (130) can set a region of interest in the video feed of a sports session based on the first position data of the ball. For example, the processor (130) can identify a first coordinate corresponding to the first position data of the ball in the video feed. The processor (130) can set a region of interest that is extended by a preset range centered on the first coordinate. The processor (130) can identify a preset range based on the distance between the camera that acquired the video feed of the sports session and the ball. For example, the greater the distance between the camera and the ball, the smaller the preset range identified by the processor (130). The processor (130) can acquire the distance between the camera and the ball based on the first position data of the ball and the position data of the camera. The camera's position information may be stored in advance in the memory (120).

[0300] The processor (130) can identify an object in a region of interest of the video feed. The processor (130) can acquire a region of interest image based on the region of interest. The processor (130) can extract an image segment by cropping the region corresponding to the region of interest in the video feed. The processor (130) can detect a ball from the image segment using a fifth neural network model (M5) trained to detect objects included in the image.

[0301] The processor (130) can obtain second position data of an object based on the position of a pixel corresponding to the detected ball, the position information of the camera that acquired the video feed, and the pose information of the camera. The processor (130) can obtain the relative position of the ball relative to the camera based on the position of the pixel corresponding to the ball. The processor (130) can obtain the absolute position of the ball based on the position information and pose information of the camera and the relative position of the ball. Here, the absolute position of the ball may refer to the second position data of the ball. The position information and pose information of the camera may be stored in the memory (120).

[0302] Meanwhile, the artificial intelligence-related functions according to the present disclosure are operated through a processor (130) and a memory (120). The processor (130) may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as a CPU, AP, DSP (Digital Signal Processor), graphics-dedicated processors such as a GPU, VPU (Vision Processing Unit), or artificial intelligence-dedicated processors such as an NPU. The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in the memory (120). Alternatively, if the one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0303] The predefined rules of operation or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a basic artificial intelligence model is trained using a number of training data by a learning algorithm, thereby creating predefined rules of operation or artificial intelligence models configured to perform desired characteristics (or objectives). Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0304] Artificial intelligence models can be created through learning. An AI model may consist of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through calculations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the AI ​​model. For example, the multiple weights can be updated during the learning process so that the loss or cost values ​​obtained by the AI ​​model are reduced or minimized.

[0305] Artificial neural networks may include deep neural networks (DNNs), such as Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), or Deep Q-Networks, but are not limited to the examples mentioned above.

[0306] The various embodiments described above may be implemented in a recording medium readable by a computer or similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented as the processor itself. Where implemented in software, embodiments such as the procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

[0307] Computer instructions for performing processing operations according to the various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. Computer instructions stored in such a non-transitory computer-readable medium may enable a specific device to perform processing operations according to the various embodiments described above when executed by a processor.

[0308] A non-transient computer-readable medium refers to a medium that stores data semi-permanently and can be read by a device, unlike media that store data for a short period of time such as registers, caches, and memory. Specific examples of non-transient computer-readable media include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0309] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory storage medium' simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily. For example, a 'non-transitory storage medium' may include a buffer in which data is stored temporarily.

[0310] Methods according to the various embodiments disclosed in this document may be provided as part of a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0311] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. Various modifications are possible by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure. Explanation of the symbols

[0312] 100: Electronic device 110: Communication interface 120: Memory 130: Processor

Claims

Claim 1 In an electronic device for predicting ball ownership, a memory for storing at least one instruction; and a processor; wherein the processor, by executing the at least one instruction, acquires a plurality of player position datasets corresponding to a plurality of time points within a sports session, wherein each of the plurality of player position datasets includes a plurality of position values ​​each assigned to a plurality of player identifiers at the corresponding time point, and generates a plurality of team position datasets from the plurality of player position datasets, wherein the plurality of team position datasets include a plurality of first team position datasets generated from position values ​​assigned to a player identifier corresponding to a first team and reflecting the placement status of the players of the first team in the stadium, and a plurality of second team position datasets generated from position values ​​assigned to a player identifier corresponding to a second team and reflecting the placement status of the players of the second team in the stadium, and acquires a plurality of ball position data corresponding to the plurality of time points, wherein each of the plurality of ball position data includes a ball position value at the corresponding time point, and uses a neural network model of the recurrent neural network family that receives the plurality of team position datasets and the plurality of ball position data in segment units corresponding to a predetermined time length to obtain from the plurality of team position datasets and the plurality of ball position data the Information regarding ball possession by multiple teams is obtained, and based on the multiple team location dataset and the multiple ball location data, a touch state of the ball—the touch state being a first state in which the ball is touched by at least one of the multiple players and a second state in which the ball is not touched by the multiple players—is identified, and if the touch state of the ball at a first time is the first state, the team possessing the ball at the first time is identified based on the information regarding ball possession by the multiple teams corresponding to the first time.An electronic device for identifying the team possessing the ball at the first time point based on information regarding ball ownership of the plurality of teams corresponding to the second time point preceding the first time point, and information regarding ball ownership of the plurality of teams corresponding to the third time point following the first time point, if the touch state of the ball at the first time point is the second state. Claim 2 An electronic device according to claim 1, wherein the information regarding ball ownership includes the probability that each of the plurality of teams possesses the ball, and the processor identifies the team with the highest probability among the plurality of teams as the team possessing the ball. Claim 3 An electronic device according to claim 1, wherein the processor obtains a plurality of distances between each of the plurality of players and the ball at a corresponding point in time based on the plurality of team position datasets and the plurality of ball position data, and identifies the player belonging to the team possessing the ball and corresponding to the smallest distance among the plurality of distances as the player possessing the ball. Claim 4 An electronic device according to claim 1, wherein the processor obtains a plurality of distances between each of the plurality of players and the ball at a corresponding point in time based on the plurality of team location datasets and the plurality of ball location data, obtains a plurality of modified distances between each of the plurality of players and the ball based on information regarding the ball ownership of the plurality of teams and the plurality of distances, and identifies a player belonging to the team that owns the ball and corresponding to the smallest modified distance among the plurality of modified distances as the player who owns the ball. Claim 5 delete Claim 6 An electronic device according to claim 1, wherein the processor identifies the touch state of the ball in the first time interval as the first state if there exists a point in time within the first time interval where the angle according to the trajectory of the ball is smaller than the preset angle, and identifies the touch state of the ball in the first time interval as the second state if there does not exist a point in time within the first time interval where the angle according to the trajectory of the ball is smaller than the preset angle. Claim 7 An electronic device according to claim 1, wherein the processor identifies the touch state of the ball in the first time interval as the first state if there exists a point in time within the first time interval where the distance between at least one of the plurality of players and the ball is smaller than the preset distance, and identifies the touch state of the ball in the first time interval as the second state if there does not exist a point in time within the first time interval where the distance between at least one of the plurality of players and the ball is smaller than the preset distance. Claim 8 delete Claim 9 An electronic device according to claim 1, wherein the processor identifies the team identified as possessing the ball at the second time point as the team possessing the ball at the first time point if the team identified as possessing the ball at the second time point and the team identified as possessing the ball at the third time point are the same, and identifies the team identified as possessing the ball at the time point closer to the first time point among the second time point and the third time point as the team possessing the ball at the first time point if the team identified as possessing the ball at the second time point and the team identified as possessing the ball at the third time point are different. Claim 10 An electronic device according to claim 6, characterized in that the above-mentioned preset angle is 45 degrees. Claim 11 In an electronic device for predicting ball ownership, a communication interface; a memory for storing at least one instruction; The apparatus includes a processor; wherein the processor, by executing the at least one instruction, acquires a plurality of player position datasets corresponding to a plurality of time points within a sports session—each of which includes a plurality of position values ​​assigned to a plurality of player identifiers at a corresponding time point—and acquires a plurality of ball position data corresponding to the plurality of time points—each of which includes a ball position value at a corresponding time point—and acquires information regarding ball ownership of the plurality of players from the plurality of player position datasets and the plurality of ball position data using a neural network model of the recurrent neural network family that receives the plurality of player position datasets and the plurality of ball position data in segment units corresponding to a predetermined time length; and identifies a touch state of the ball—the touch state being a first state in which the ball is touched by at least one of the plurality of players and a second state in which the ball is not touched by the plurality of players—based on the plurality of team position datasets and the plurality of ball position data, and if the touch state of the ball at a first time point is the first state, information regarding ball ownership of the plurality of teams corresponding to the first time point An electronic device that identifies the team possessing the ball at the first time point based on information, and if the touch state of the ball at the first time point is the second state, identifies the team possessing the ball at the first time point based on information regarding the ball ownership of the plurality of teams corresponding to the second time point preceding the first time point, and information regarding the ball ownership of the plurality of teams corresponding to the ball ownership of the plurality of teams corresponding to the third time point following the first time point. Claim 12 A method for predicting ball possession comprises: a step of obtaining a plurality of player position datasets corresponding to a plurality of time points within a sports session—each of which includes a plurality of position values ​​assigned to a plurality of player identifiers at the corresponding time point; and a step of generating a plurality of team position datasets from the plurality of player position datasets—the plurality of team position datasets including a plurality of first team position datasets generated from position values ​​assigned to a player identifier corresponding to a first team and reflecting the placement status of the players of the first team in the stadium, and a plurality of second team position datasets generated from position values ​​assigned to a player identifier corresponding to a second team and reflecting the placement status of the players of the second team in the stadium. A step of obtaining a plurality of ball position data corresponding to the plurality of time points—each of the plurality of ball position data includes a ball position value at the corresponding time point—; a step of obtaining information regarding ball ownership of the plurality of teams from the plurality of team position datasets and the plurality of ball position data using a neural network model of the recurrent neural network family that receives the plurality of team position datasets and the plurality of ball position data in segment units corresponding to a predetermined time length; a step of identifying a touch state of the ball—the touch state being a first state in which the ball is touched by at least one of the plurality of players and a second state in which the ball is not touched by the plurality of players—based on the plurality of team position datasets and the plurality of ball position data; and a step of identifying the team that possesses the ball at the first time point based on information regarding ball ownership of the plurality of teams corresponding to the first time point, if the touch state of the ball at the first time point is the first state.A method comprising: a step of identifying the team possessing the ball at the first time point based on information regarding ball ownership of the plurality of teams corresponding to the second time point preceding the first time point, and information regarding ball ownership of the plurality of teams corresponding to the third time point following the first time point, if the touch state of the ball at the first time point is the second state;

Images