Ai-based sports video analysis system with highlight section detection function

Abstract

An artificial intelligence-based sports video analysis system having a highlight section detection function according to the present invention is characterized by comprising: an object detection tracking module that samples a video of a sports game in frame units, detects at least one class among a player, ball, goalpost, goalkeeper, and referee using a YOLO-based vision model in the sampled video, and assigns a unique tracking identifier to each of the detected classes by applying a multi-object tracking algorithm; a feature calculation module that calculates a plurality of frame-unit features in the sampled video based on the tracking identifier; a score calculation module that calculates a frame-specific candidate score by linearly combining a rule-based score and a machine learning model prediction score for the plurality of features; a clip generation module that determines an anchor frame by applying a moving average to the frame-specific candidate score to smooth it and then performing peak detection, and generates a highlight candidate clip centered on the anchor frame; and a contribution calculation module that calculates the contribution of each feature that contributed to the frame-specific candidate score for the machine learning model. According to the system of the present invention, it has the effect of automatically generating highlight candidate clips by detecting objects based on YOLO using only the original game video, calculating the contribution of each feature, and providing the basis for selection in an explainable form.

Description

Technology Field

[0001] The present invention relates to an artificial intelligence-based sports video analysis system having a highlight section detection function, and more specifically, to a sports video analysis system that automatically detects highlight candidate sections from a video of a sports game and provides the basis for calculation in the form of contribution by feature. Background Technology

[0002] Technology for automatically detecting highlight sections in sports game footage has long been researched in the fields of broadcasters, sports media platforms, and game analysis services.

[0003] Initially, methods primarily used to extract highlight sections by utilizing the structural characteristics of broadcast editing, such as replays, zoom-in / zoom-out transitions, and scene transition effects. However, since this method relies on the broadcaster's edited signal, it had limitations in that it was difficult to apply to simply filmed footage, such as amateur matches, youth games, and training videos, which are not broadcast.

[0004] Since then, with the advancement of deep learning technology, a method has been proposed to automatically classify scene types in video and extract highlights by combining them with game log information.

[0005] For example, Korean Registered Patent No. 10-2187208 discloses a technology that classifies each frame of a sports game video into types such as pitch, close-up, and replay using a deep learning-based scene type classification model, and extracts highlight sections by combining them with log information that sequentially records events that occurred in the game, such as text commentary.

[0006] This method is significant in that it improves the accuracy of scene classification by utilizing deep learning, but it has the following limitations.

[0007] First, in game environments where log information such as text commentary is not provided, highlight extraction is impossible, which limits the scope of application.

[0008] Furthermore, since the scene type classification method is limited to classifying match footage into predefined scene types, it cannot quantify the real-time threat intensity of the game—such as the position of the ball, distance from the goal, and density between players—on a frame-by-frame basis.

[0009] Furthermore, since individual objects such as players, the ball, the goalpost, and the goalkeeper are not tracked, it is difficult to detect highlights that precisely reflect the situational significance of the game, such as a specific player's penetration into the goal, pressing defense, or chaotic situations. In addition, there is no configuration that explains the basis for selecting a segment as a highlight in the form of feature contributions, leading to a problem where it is difficult for editors to identify the rationale behind highlight candidates.

[0010] Therefore, there is a need to develop a novel and advanced technology that combines a YOLO-based vision model and a multi-object tracking algorithm using only original game footage without broadcast editing signals or log information to detect classes of players, balls, goalposts, goalkeepers, and referees, assign tracking identifiers, calculate frame-by-frame features, linearly combine rule-based scores and machine learning model prediction scores to automatically generate highlight candidate clips, calculate the contribution of each feature, and provide the basis for selection in an explainable form. Prior art literature

[0011] Korean Registered Patent No. 10-2187208 The problem to be solved

[0012] The present invention was devised to overcome the problems of the above technology, and its main purpose is to provide a system that samples video of a sports game on a frame-by-frame basis, combines a YOLO-based vision model with a multi-object tracking algorithm to detect classes of players, balls, goalposts, goalkeepers, and referees, assigns tracking identifiers, calculates frame-by-frame features, and generates highlight candidate clips by linearly combining rule-based scores and machine learning model prediction scores, and calculates the contribution of each feature.

[0013] Another objective of the present invention is to enable editors to intuitively understand the basis for selecting highlight candidate clips by applying the SHAP method to a machine learning model to calculate feature-specific contributions and classifying them by feature group to generate explanatory information regarding candidate scores of highlight candidate clips.

[0014] Another objective of the present invention is to precisely reflect the actual position of the goalkeeper and whether there is a collision with the goalpost, which is difficult to capture with vision AI alone, into the candidate scores by combining actual physical data measured from a strain gauge sensor attached to the goalpost frame and a UWB sensor worn by the goalkeeper into a non-linear function to first correct the candidate scores per frame. means of solving the problem

[0015] To achieve the above objective, the artificial intelligence-based sports video analysis system having a highlight section detection function according to the present invention comprises: an object detection tracking module that samples a video of a sports game in frame units, detects at least one class among a player, ball, goalpost, goalkeeper, and referee using a YOLO-based vision model in the sampled video, and assigns a unique tracking identifier to each of the detected classes by applying a multi-object tracking algorithm; a feature calculation module that calculates multiple frame-unit features in the sampled video based on the tracking identifier; a score calculation module that calculates frame-specific candidate scores by linearly combining a rule-based score and a machine learning model prediction score for the multiple features; a clip generation module that determines an anchor frame by applying a moving average to the frame-specific candidate scores to smooth them and then performing peak detection, and generates highlight candidate clips centered on the anchor frames; and a contribution calculation module that calculates the contribution of each feature that contributed to the frame-specific candidate scores for the machine learning model.

[0016] In addition, the contribution calculation module is characterized by including a group classification unit that classifies the contribution per feature into at least one feature group among a ball physics group, an attack threat group, a player pressure / density group, a dynamic transition group, and an image quality group, and calculates the contribution per feature group, and an explanation information generation unit that generates explanation information for the candidate score per frame based on the contribution per feature group.

[0017] In addition, the system further includes a strain gauge sensor attached to a goal frame of a sports game to measure the amount of deformation of the goal frame, and a UWB sensor worn on a goalkeeper to measure the real-time position of the goalkeeper, and the score calculation module further includes a first correction unit that first corrects the candidate score per frame based on the amount of deformation of the goal frame and the real-time position of the goalkeeper measured by the UWB sensor. Effects of the invention

[0018] According to the artificial intelligence-based sports video analysis system having a highlight section detection function according to the present invention,

[0019] 1) It has the advantage of automatically generating highlight candidate clips through YOLO-based object detection and tracking, frame-by-frame feature calculation, and a linear combination of rule-based scores and machine learning model prediction scores using only original game footage without broadcast editing signals or log information, and calculating the contribution of each feature to provide the basis for selection in an explainable form,

[0020] 2) By combining the goal collision intensity measured by strain gauge sensors attached to the goal frame and UWB sensors worn by the goalkeeper with the goalkeeper's position and movement speed using a non-linear function to perform primary correction on frame-by-frame candidate scores, it precisely distinguishes between simple goal collision situations that vision AI is prone to misinterpreting and actual threat situations, and

[0021] 3) By calculating a save efficiency index that combines the actual contact pressure, contact time, and contact area measured by the pressure sensor inside the goalkeeper glove with the goalkeeper's movement speed, and by performing a second correction on the first corrected candidate score, the actual save quality of the goalkeeper is reflected in the candidate score, thereby reducing the false positive rate of goalkeeper graze. Brief explanation of the drawing

[0022] FIG. 1 is a block diagram illustrating the configuration of the system of the present invention. FIG. 2 is an example diagram illustrating the state in which object detection and object tracking are performed. FIG. 3 is an example diagram illustrating the state in which frame-unit features are calculated. Figure 4 is an example diagram illustrating the state in which a highlight candidate clip is generated. FIG. 5 is an exemplary diagram illustrating the state in which a sensor for the primary correction of the present invention is placed in a sports stadium. FIG. 6 is an exemplary diagram illustrating the configuration of a goalkeeper glove pressure sensor for secondary correction of the present invention. Specific details for implementing the invention

[0023] Preferred embodiments of the present invention will be described in detail below with reference to the attached drawings. The attached drawings are not drawn to scale, and the same reference numerals in each drawing refer to the same components.

[0024] The artificial intelligence-based sports video analysis system with a highlight section detection function of the present invention (hereinafter referred to as the "system") overcomes the difficulty of accurately identifying situational meanings of sports games, such as shooting, penetration, pressing, and defensive recovery, using simple video analysis methods centered on audio signals, scene transitions, and motion magnitude in the past. By combining a YOLO-based vision model and a multi-object tracking algorithm, the system calculates frame-unit features from video of a game, generates highlight candidate clips by linearly combining rule-based scores and prediction scores from a machine learning model, and provides the basis for calculating candidate scores by calculating the contribution of each feature to the machine learning model. In other words, the main purpose is to automatically detect highlight candidate sections by analyzing video of a game frame by frame and to provide the basis in an explainable form.

[0025] The system of the present invention may be implemented by a central management server (100), and the central management server (100) includes a hardware configuration and a software configuration for receiving video of a sports game and performing an entire analysis pipeline ranging from object detection and tracking, frame-by-frame feature calculation, candidate score calculation, highlight candidate clip generation, and feature contribution calculation.

[0026] Hardware-wise, the central management server (100) includes a central processing unit and memory for receiving and processing video data of a game, and includes at least one of a graphics processing unit and an artificial intelligence dedicated computational processor to perform operations including YOLO-based vision model inference, multi-object tracking, calculation of prediction scores of machine learning models and calculation of contribution per feature, and includes a high-speed storage means to store and manage frame-unit features, candidate score history and highlight candidate clips. The central management server (100) includes at least one of a wired network interface and a wireless network interface for communication with a video input device and a user terminal, and includes an input / output interface for linking with an input device and an output device.

[0027] Software-wise, the central management server (100) is composed of an operating system, a database management system, and an application layer. The operating system manages the central processing unit, memory, and high-speed storage means, and provides an environment in which application programs can be executed. The database management system structures and stores video processing history by game, frame-unit features, candidate score history, highlight candidate clips, and contribution by feature, and, when necessary, queries the corresponding data and provides it to the application layer. The application layer is a layer responsible for processing logic regarding video analysis, feature calculation, score calculation, clip creation, and contribution calculation, and transmits and receives analysis results through API communication with a user terminal.

[0028] Each module and each part is not limited to logical components implemented in software, but may be implemented in a form including a set of instructions executed in a processor and corresponding data structures and storage areas, and some components may be implemented in a hardware form such as programmable logic elements or application-specific integrated circuits. In addition, each module and each part may be implemented as a process, function, procedure, subroutine, program code segment, driver, firmware, circuit, database, data structure, or variable.

[0029] In addition, since the video of the sports match is not limited to a specific sport, the system of the present invention can be applied to any sport that includes players, a ball, a goalpost, a goalkeeper, and a referee.

[0030] Examples of such sports include soccer, field hockey, handball, and water polo.

[0031] However, for the convenience of explanation, the configuration and operation of the system of the present invention will be described below by using a soccer match as an example, in which the composition of players, ball, goalpost, goalkeeper, and referee is most typically observed.

[0032] The specific configuration and operation of the system of the present invention will be described below with reference to the attached drawings.

[0033] FIG. 1 is a block diagram illustrating the configuration of the system of the present invention.

[0034] As can be seen from FIG. 1, the system of the present invention is based on comprising: an object detection tracking module (110) that samples video of a game in frame units, detects objects, and assigns a tracking identifier; a feature calculation module (120) that calculates multiple frame-unit features based on the tracking identifier; a score calculation module (130) that calculates frame-unit candidate scores by linearly combining rule-based scores and machine learning model prediction scores for multiple features; a clip generation module (140) that generates highlight candidate clips by applying a moving average to frame-unit candidate scores to smooth them and then performing peak detection; and a contribution calculation module (150) that calculates the contribution of each feature to the frame-unit candidate scores for the machine learning model.

[0035] Figure 2 is an example diagram illustrating the state in which object detection and object tracking are performed.

[0036] The object detection tracking module (110) samples the video of a sports game frame by frame, detects at least one class among a player, ball, goalpost, goalkeeper, and referee using a YOLO-based vision model in the sampled video, and assigns a unique tracking identifier to each of the detected classes by applying a multi-object tracking algorithm.

[0037] The operating principle of the object detection tracking module (110) is explained sequentially as follows.

[0038] First, the object detection tracking module (110) generates a frame sequence by sampling the video of the input sports game at regular intervals.

[0039] Here, the term "frame sequence" refers to a time-ordered arrangement of still images extracted from the original image at regular intervals.

[0040] For example, if a video captured at 30 frames per second is sampled at intervals of 5 frames, 6 frames per second are extracted and a sequence of frames to be analyzed is generated.

[0041] The sampling interval can be adjusted according to the frame rate of the video and the required precision of analysis. While a narrower sampling interval allows for more detailed tracking of the ball's trajectory and the player's movement, it also increases the amount of computation, so an appropriate interval is selected depending on the analysis environment.

[0042] Next, the object detection tracking module (110) applies a YOLO (You Only Look Once) based vision model to each sampled frame to detect at least one class among a player, ball, goalpost, goalkeeper, and referee.

[0043] YOLO-based vision models stand for You Only Look Once and refer to real-time object detection models that simultaneously predict the location and class of objects in each frame of a video through a single neural network inference process.

[0044] This model operates in a single forward propagation manner that divides a frame into a grid and simultaneously predicts bounding boxes and class probabilities in each grid cell, providing detection speeds suitable for real-time processing compared to the existing two-stage detection method.

[0045] Here, a bounding box refers to a minimum rectangular area surrounding a detected object, which is position information defined by the top-left coordinate, width, and height.

[0046] For example, a YOLO-based vision model detects 13 player classes, 1 ball class, 1 goalpost class, 1 goalkeeper class, and 1 referee class in a specific frame, and for each detected class, calculates bounding box coordinates, center point coordinates, and confidence score.

[0047] The confidence score represents the probability that an object belongs to a specific class as a value between 0 and 1. For example, a player object with a confidence score of 0.95 means that there is a 95% probability that the object within the bounding box is a player.

[0048] Subsequently, the object detection tracking module (110) applies a multi-object tracking algorithm to assign a unique tracking identifier to each detected class.

[0049] A tracking identifier refers to a unique number assigned to identify the same object across consecutive frames.

[0050] As an example of a multi-object tracking algorithm, the ByteTrack algorithm can be used, and other examples include algorithms such as DeepSORT and StrongSORT.

[0051] The ByteTrack algorithm operates by utilizing not only highly reliable detection results but also less reliable detection results for tracking, thereby providing the advantage of maintaining the continuity of the tracking identifier even when a player overlaps with another player or is partially obscured.

[0052] For example, if a tracking identifier 7 is assigned to a specific player, the tracking identifier 7 remains the same even when the player crosses another player or temporarily exits and re-enters the frame boundary, enabling continuous position tracking.

[0053] In this way, the continuity of the tracking identifier must be ensured so that the feature calculation module (120) described later can accurately calculate the ball's velocity, acceleration, and direction change from the center point displacement between frames.

[0054] Meanwhile, if tracking is unstable, the object detection tracking module (110) may switch to detect-only mode.

[0055] Detect-only mode is an operating method that maintains only the bounding box and class information of the corresponding frame without assigning a tracking identifier; it is a safe operating mode designed to prevent errors in the tracking algorithm from propagating and affecting the feature calculation of subsequent frames.

[0056] By switching to detect-only mode in this way, the minimum information necessary for frame-by-frame feature calculation is preserved even in sections where tracking is unstable.

[0057] Figure 3 is an example diagram illustrating the state in which frame-unit features are calculated.

[0058] The feature calculation module (120) provides a function to calculate multiple frame-unit features from a sampled image based on a tracking identifier.

[0059] The frame-unit features produced by the feature production module (120) consist of physical indicators closely related to highlight situations in the flow of the game.

[0060] Specifically, the feature calculation module (120) calculates features regarding the movement of the ball, features regarding the positional relationship between the ball and the goalpost, and features regarding the spatial distribution of the players.

[0061] The feature regarding ball movement quantifies the characteristics of how the ball's movement changes rapidly during decisive moments of the game, such as shooting, strong passes, and dribbling breakthroughs.

[0062] For example, the feature calculation module (120) calculates the ball's speed as 216 pixels / second when the center point of the ball moves 36 pixels compared to the previous frame in a specific frame and the time interval between frames is 1 / 6 second, and calculates the ball's acceleration as 1,116 pixels / second² when the ball's speed changes rapidly from 30 pixels / second to 216 pixels / second in consecutive frames.

[0063] Additionally, the feature output module (120) corresponds a frame in which the speed and acceleration of the ball increase rapidly at the same time to the point in time when a strong shot or long pass occurs, and corresponds a frame in which the speed of the ball converges to 0 to the point in time when the ball is caught by the goalkeeper or the game is temporarily stopped.

[0064] In addition, the feature calculation module (120) calculates the change in direction as 90 degrees when the ball's movement direction changes rapidly between consecutive frames, for example, when the ball's movement vector direction changes from 45 degrees in frame t to 135 degrees in frame t+1, thereby capturing the situation where the ball collides with a player or a goal.

[0065] Regarding the feature concerning the positional relationship between the ball and the goalpost, the feature calculation module (120) calculates the proximity to the goalpost with a structure in which the value increases as the ball approaches the goalpost, and the proximity to the goalpost increases rapidly in the section immediately before the shooting attempt.

[0066] For example, the feature calculation module (120) calculates the proximity of the goal as 1 / 400 = 0.0025 when the distance between the ball and the center of the goal is 400 pixels, and calculates the proximity of the goal as 1 / 50 = 0.020 when the ball is 50 pixels in front of the goal, so that the value increases by 8 times as the ball gets closer to the goal.

[0067] Regarding features concerning the spatial distribution of players, the feature calculation module (120) quantifies situations where multiple players are concentrated at a specific point, such as corner kicks, chaotic situations, and pressing defense, as density.

[0068] For example, the feature calculation module (120) calculates the density as approximately 0.008 by calculating the average distance between players as approximately 120 pixels when 11 players are evenly distributed across the entire field, and calculates the density as approximately 0.067 by calculating the average distance between players as approximately 15 pixels when 8 players are densely packed within a radius of 30 pixels near the goal during a corner kick situation.

[0069] Additionally, the feature calculation module (120) counts the number of players located within a radius of 80 pixels based on the center point of the ball, calculates the number of nearby players feature value as 4 in frames where 4 players are located around the ball, and calculates it as 0 in frames immediately after the ball is caught by the goalkeeper.

[0070] In this way, the feature calculation module (120) calculates multiple frame-unit features simultaneously, thereby quantifying complex highlight situations that are difficult to capture with only one feature in multiple ways.

[0071] For example, the feature calculation module (120) determines that even if the ball speed is calculated as 216 pixels / second, if the proximity to the goal remains at 0.0025, it is a simple midfield pass situation, and conversely, even if the ball speed is only 50 pixels / second, if the proximity to the goal is calculated as 0.020 and the number of nearby players is 5, it is determined that it is a chaotic situation in front of the goal, and thus quantifies the highlight values ​​of the two situations differently.

[0072] Furthermore, to increase the precision of the feature calculation, the feature calculation module (120) can be specialized into a ball physical feature calculation unit (121) and an attack threat feature calculation unit (122).

[0073] The ball physical feature calculation unit (121) calculates the ball's velocity from the center point displacement between consecutive frames of the ball and calculates the ball's acceleration from the change in the ball's velocity to produce a frame-unit feature.

[0074] The ball physical feature calculation unit (121) calculates the center point displacement by comparing the center point coordinates of the ball assigned a tracking identifier between consecutive frames, and calculates the ball's velocity by dividing the calculated center point displacement by the time interval between frames.

[0075] For example, the ball physics feature calculation unit (121) calculates the center point displacement as approximately 43 pixels when the center point coordinates of the ball are (320, 240) in frame t and (356, 264) in frame t+1, and calculates the ball speed as approximately 258 pixels / second by applying a time interval of 1 / 6 second between frames.

[0076] Afterwards, the ball physics feature calculation unit (121) calculates the acceleration of the ball from the amount of change in the ball's velocity between consecutive frames.

[0077] For example, the ball physics feature calculation unit (121) calculates the acceleration of the ball as approximately 1,398 pixels / second² when the ball's speed changes from 25 pixels / second at frame t to 258 pixels / second at frame t+1.

[0078] Referring to FIG. 3, a peak is observed in which the ball speed and ball acceleration calculated by the ball physical feature calculation unit (121) simultaneously rise rapidly to 0.6 or higher during the period of -20 seconds of the representative frame reference time, which indicates that a strong shot or long pass occurred during that period.

[0079] On the other hand, in the section immediately following the 0-second standard, the ball speed and ball acceleration calculated by the ball physical feature calculation unit (121) both converge to 0.05 or less, which corresponds to a situation where the ball is caught by the goalkeeper and the game is temporarily stopped.

[0080] Additionally, the ball physical feature calculation unit (121) calculates the section where the sudden change in direction feature value reaches 1.00 as the point in time when the ball collides with a player or goal and the direction of movement changes abruptly, and in FIG. 3, it can be seen that the sudden change in direction feature frequently forms peaks reaching 1.00 throughout the entire game.

[0081] The attack threat feature calculation unit (122) calculates the proximity of the goal from the inverse of the distance between the center of the ball and the goal, calculates the density based on the distance between players, and calculates the number of players within a certain radius around the ball as a frame-unit feature.

[0082] The attack threat feature calculation unit (122) calculates the Euclidean distance between the center point coordinates of the ball and the center point coordinates of the goalpost, and defines the reciprocal of the calculated distance as the proximity of the goalpost.

[0083] For example, the attack threat feature calculation unit (122) calculates the proximity of the goal as 1 / 500 = 0.002 when the distance between the ball and the center of the goal is 500 pixels, and calculates the proximity of the goal as 1 / 40 = 0.025 when the ball is 40 pixels in front of the goal, showing a numerical difference of about 12.5 times between the two situations.

[0084] Additionally, the attack threat feature calculation unit (122) defines the density as the reciprocal of the average distance between the center point coordinates of the entire detected player class.

[0085] For a specific example, the attack threat feature calculation unit (122) calculates the density as approximately 0.008 by calculating the average distance between players as approximately 120 pixels when 11 players are evenly distributed across the entire field, and calculates the density as approximately 0.067 by calculating the average distance between players as approximately 15 pixels when 8 players are densely packed within a radius of 30 pixels near the goal during a corner kick situation.

[0086] In addition, the attack threat feature calculation unit (122) calculates a proximity player count feature by counting the number of player classes located within a radius of 80 pixels based on the center point of the ball. For example, the attack threat feature calculation unit (122) calculates the proximity player count feature value as 4 in a frame where 4 players are located around the ball, and calculates it as 0 in a frame immediately after the ball is caught by the goalkeeper.

[0087] In FIG. 3, the inverse distance to the goal calculated by the attack threat feature calculation unit (122) is maintained at a level of 0.85 in the -30 second interval and then shows a trend of gradually decreasing to 0.40 in the interval after the 0 second mark, which is calculated as a feature value reflecting the situation where the ball moves away from the vicinity of the goal around the 0 second point.

[0088] In addition, the player density calculated by the attack threat feature calculation unit (122) shows a peak of rapidly rising to 0.30 in the interval immediately after 0 seconds, which is calculated as a feature value reflecting a corner kick or chaotic situation where a large number of players are concentrated near the goal.

[0089] In this way, the feature calculation module (120) is subdivided into a ball physical feature calculation unit (121) and an attack threat feature calculation unit (122), thereby allowing the physical movement characteristics of the ball and the attack threat situation to be independently quantified and provided to the score calculation module (130), and the precision of the candidate score calculation per frame can be increased.

[0090] The score calculation module (130) calculates a candidate score per frame by linearly combining a rule-based score and a machine learning model's prediction score for multiple features.

[0091] First, the score calculation module (130) calculates a rule-based score.

[0092] The term "rule-based score" refers to a score calculated as a weighted sum by applying a preset weight to each of the multiple frame-unit features calculated by the feature calculation module (120).

[0093] The score calculation module (130) defines the rule-based score as a numerical representation of the empirical judgment of a domain expert.

[0094] Additionally, the score calculation module (130) sets weights by reflecting the degree to which each feature contributes to the highlight situation, such as the speed of the ball, proximity to the goal, and density.

[0095] For example, when the score calculation module (130) calculates the ball speed feature value as 0.8, the goal proximity feature value as 0.7, and the density feature value as 0.3 in a specific frame, it applies preset weights of 0.4, 0.4, and 0.2 to each feature to calculate the rule-based score as 0.8×0.4 + 0.7×0.4 + 0.3×0.2 = 0.38 + 0.28 + 0.06 = 0.72.

[0096] Next, the score calculation module (130) calculates the predicted score of the machine learning model.

[0097] Here, the predicted score of a machine learning model refers to a score that predicts the highlight value of a frame from the training data by receiving multiple frame-unit features as input.

[0098] As machine learning models, a tree ensemble model based on XGBoost (eXtreme Gradient Boosting) can be used as an example, and other examples include Random Forest and LightGBM models.

[0099] For example, the score calculation module (130) inputs the feature vector of a specific frame into a machine learning model and calculates the highlight prediction score of the frame as 0.65.

[0100] Afterwards, the score calculation module (130) calculates a candidate score per frame by linearly combining the rule-based score and the machine learning model's prediction score.

[0101] A linear combination refers to a method of summing two scores by multiplying each of them by a combination coefficient.

[0102] For example, the score calculation module (130) applies a combination coefficient of 0.5 to the rule-based score of 0.72 and the machine learning model's prediction score of 0.65, respectively, to calculate the candidate score per frame as 0.72×0.5 + 0.65×0.5 = 0.36 + 0.325 = 0.685.

[0103] In this way, the score calculation module (130) has a structure that linearly combines the rule-based score and the machine learning model's prediction score, thereby allowing for the reflection of explicit game rules that are difficult to capture from training data when only the rule-based score is used, and can compensate for the problem of reduced responsiveness to new game situations not included in training data when only the machine learning model's prediction score is used.

[0104] That is, the score calculation module (130) provides the characteristic of simultaneously securing rule-based explicitness and data-based flexibility by adopting a linear combination of two scores.

[0105] Figure 4 is an example diagram illustrating the state in which a highlight candidate clip is generated.

[0106] The clip generation module (140) applies a moving average to the candidate scores per frame to smooth them, then performs peak detection to determine the anchor frame, and generates a highlight candidate clip centered on the anchor frame.

[0107] First, the clip generation module (140) smooths the frame-by-frame candidate scores calculated by the score calculation module (130) by applying a moving average.

[0108] Moving average refers to a method of mitigating abrupt fluctuations in a score time series by averaging candidate scores of consecutive frames at regular intervals.

[0109] The clip generation module (140) prevents the candidate score in frame units from being mistaken for a peak in a highlight section when the candidate score locally spikes or drops due to noise by applying a moving average.

[0110] For example, when the clip generation module (140) applies a moving average of a window size of 3 seconds, it calculates the average of candidate scores from frame t-45 to frame t+45 as the smoothed score of frame t. For example, if the average of candidate scores in the corresponding interval is calculated as 0.38, the clip generation module (140) determines the smoothed score of frame t as 0.38.

[0111] Next, the clip generation module (140) determines the anchor frame by performing peak detection in the smoothed score time series.

[0112] Peak detection refers to a method of detecting a frame in a smoothed score time series in which the score has a maximum value relative to an adjacent frame.

[0113] The clip generation module (140) determines the frame that has a smoothed score greater than or equal to a preset threshold and represents a maximum value relative to an adjacent frame as the anchor frame.

[0114] For example, when the clip generation module (140) sets the threshold to 0.2, it determines a frame as an anchor frame that has a smoothed score of 0.2 or higher and represents a maximum value relative to the preceding and succeeding frames. When multiple anchor frames are detected adjacently, the clip generation module (140) selects the frame with the highest score among the anchor frames located within a preset minimum interval as the representative anchor frame.

[0115] Afterwards, the clip generation module (140) generates a highlight candidate clip that includes the preceding and succeeding sections centered on the anchor frame.

[0116] For a specific example, the clip generation module (140) generates a highlight candidate clip of a total length of 15 seconds, including a 5-second forward and 10-second backward section based on the anchor frame.

[0117] When the anchor frame is frame number 55020 and the frame rate of the video is 30fps, the clip generation module (140) generates a highlight candidate clip for the section from frame number 54870 to 55320.

[0118] The clip generation module (140) independently generates a highlight candidate clip for each anchor frame when multiple anchor frames are detected, and provides the generated multiple highlight candidate clips sorted in descending order of candidate scores.

[0119] As can be seen from FIG. 4, the clip generation module (140) calculates a smoothed score (blue solid line) with a 3-second moving average applied over the entire duration of the game, and determines the frame that shows a maximum value while exceeding a threshold of 0.2 (yellow dotted line) as the anchor frame (red asterisk).

[0120] The clip generation module (140) determines the peak where the smoothed score rises sharply to 0.80 in the approximately 30-minute interval of the game time as the anchor frame, which indicates that the most intense highlight situation occurred in that interval.

[0121] Additionally, the clip generation module (140) generates a highlight candidate clip section (red shading) centered on each determined anchor frame to produce multiple highlight candidate clips over the entire game.

[0122] In this way, the clip generation module (140) sequentially performs moving average-based smoothing and peak detection, thereby providing the characteristic of accurately generating highlight candidate clips centered on decisive scenes of the game while being robust against frame-unit candidate score noise.

[0123] The contribution calculation module (150) provides a function to calculate the contribution of each feature that contributed to the frame-by-frame candidate score for the machine learning model.

[0124] The contribution calculation module (150) quantifies the weight of which feature determined the candidate score calculated by the machine learning model for a specific frame.

[0125] The contribution calculation module (150) calculates the contribution for each feature, thereby overcoming the limitation that it is difficult for the user to identify the basis for a specific frame being selected as a highlight candidate when the prediction result of a machine learning model is derived in a black-box manner.

[0126] The contribution calculation module (150) can calculate the contribution of each feature in various ways depending on the structure of the machine learning model.

[0127] For example, the contribution calculation module (150) can calculate the contribution per feature by applying the SHAP (SHapley Additive exPlanations) method.

[0128] The SHAP method is a method that applies the concept of the Shapley value from game theory to the prediction explanation of a machine learning model, and refers to quantifying the degree to which each feature contributes to the predicted value as a contribution value.

[0129] For example, the contribution calculation module (150) calculates the contribution of the goal proximity feature as +0.28, the contribution of the ball speed feature as +0.19, the contribution of the density feature as +0.12, the contribution of the ball acceleration feature as +0.08, and the contribution of the direction change feature as -0.03 as a result of applying the SHAP method to a candidate score of 0.685 of a specific frame.

[0130] As another example, the contribution calculation module (150) can calculate the degree to which each feature reduces impurity during the tree branching process in the case of a tree-based machine learning model as feature importance.

[0131] For example, the contribution calculation module (150) calculates that in the tree branching process of the XGBoost model, the proximity feature of the goalpost accounts for 34% of the total impurity reduction, the speed of the ball accounts for 27%, and the density feature accounts for 18%, thereby providing contribution per feature.

[0132] As another example, the contribution calculation module (150) can apply the LIME (Local Interpretable Model-agnostic Explanations) method to construct a linear approximation model in a local area around a specific frame and calculate the coefficients of the linear approximation model as feature-specific contributions.

[0133] For example, the contribution calculation module (150) generates 100 samples around the feature vector of a specific frame and constructs a linear approximation model, and calculates the coefficient of the goal proximity feature as 0.42 and the coefficient of the ball speed feature as 0.31, indicating that the two features had the main influence on the calculation of the candidate score of the frame.

[0134] In addition, the contribution calculation module (150) calculates whether each feature acts in a direction that increases or decreases the candidate score by distinguishing through the sign of the contribution per feature.

[0135] For example, the contribution calculation module (150) indicates that if the contribution of the goalpost proximity feature in a specific frame is calculated as +0.28, the feature has acted to increase the candidate score by 0.28, and if the contribution of the direction change feature is calculated as -0.03, the feature has acted to decrease the candidate score by 0.03.

[0136] The contribution calculation module (150) has the characteristic of clearly providing how a feature has influenced the candidate score in a certain direction, going beyond simply determining which feature is important, by calculating the contribution of each feature that includes directionality.

[0137] Thus, the system of the present invention exhibits the characteristic of not only generating highlight candidate clips but also providing the basis for calculating the candidate score of each clip in the form of contribution by feature.

[0138] In addition, the system of the present invention can utilize the contribution calculated in this way in various ways.

[0139] For example, the system of the present invention visualizes feature-specific contributions in a user interface so that an editor can intuitively grasp the basis for selecting highlight candidate clips and use it to decide whether to adopt them.

[0140] As another example, the system of the present invention can be used to automatically classify the scene type of the corresponding clip into shooting situations, chaotic situations, pressure situations, etc. by extracting the feature that contributed most to the candidate score based on the contribution of each feature.

[0141] As another example, the system of the present invention can progressively improve the precision of candidate score calculation by accumulating the history of an editor adopting or excluding specific highlight candidate clips as training data along with feature-specific contributions, and utilizing this data for the retraining of a machine learning model.

[0142] As an additional example, the system of the present invention can aggregate the calculated feature-specific contributions by player class and utilize them to quantify the extent to which each player contributed to highlight situations in a specific match.

[0143] For example, the system of the present invention can be used to specifically identify the role of each player in a game by classifying a player as an offensive role leading strong shooting or fast passing when the contribution of the ball speed feature in a frame containing tracking identifier player 7 repeatedly shows a high value, and classifying a player as a defensive role leading pressure defense or space blocking when the contribution of the density feature in a frame containing tracking identifier player 3 repeatedly shows a high value.

[0144] Hereinafter, the operation flow of the system of the present invention will be specifically explained under the assumption of a specific situation.

[0145] Assume a scenario in which player number 7 receives the ball and penetrates toward the goal during the 30-minute mark of the second half of a soccer match.

[0146] First, the object detection tracking module (110) samples the video of the game at intervals of 5 frames to extract 6 frames per second, and applies a YOLO-based vision model to each sampled frame to detect 13 players, 1 ball, 1 goalpost, 1 goalkeeper, and 1 referee.

[0147] The object detection tracking module (110) applies the ByteTrack algorithm to assign a unique tracking identifier to each detected class, assigns tracking identifier 7 to player number 7 and tracking identifier 1 to the ball, and maintains the same identifier in subsequent frames.

[0148] Next, the feature calculation module (120) calculates frame-unit features from the sampled image based on the tracking identifier.

[0149] The feature calculation module (120) calculates the ball's speed as approximately 300 pixels / second and the ball's acceleration as approximately 1,500 pixels / second² when the ball's center point coordinates are (320, 340) in frame t, where player 7 receives the ball and starts dribbling toward the goal, and changes to (362, 310) in frame t+1.

[0150] The feature calculation module (120) calculates the proximity of the goal as 1 / 60 = 0.017 when player number 7 penetrates to the front of the goal and the distance between the ball and the center of the goal is narrowed to 60 pixels, and calculates the density as approximately 0.067 when 8 defensive players are densely packed within a radius of 30 pixels near the goal to prevent player number 7 from penetrating.

[0151] Additionally, the feature calculation module (120) calculates the number of nearby players feature value as 5 by counting that 5 players, including player number 7, are located within a radius of 80 pixels around the ball.

[0152] Subsequently, the score calculation module (130) calculates a rule-based score of 0.74 for the multiple features calculated by the feature calculation module (120) and calculates a machine learning model prediction score of 0.68. The score calculation module (130) applies a combination coefficient of 0.5 to each of the two scores to calculate a candidate score per frame as 0.74 × 0.5 + 0.68 × 0.5 = 0.71.

[0153] Next, the clip generation module (140) applies a moving average of a 3-second window to the candidate scores per frame to determine the smoothed score as 0.71, and determines frame number 54900 as the anchor frame, which has a maximum value relative to the preceding and succeeding frames while exceeding a threshold of 0.2.

[0154] The clip generation module (140) generates a highlight candidate clip of a total length of 15 seconds, including a 5-second forward and a 10-second backward section based on the anchor frame. The generated highlight candidate clip includes the entire process of player number 7 receiving the ball, penetrating with a dribble, and attempting a shot.

[0155] Finally, the contribution calculation module (150) calculates the contribution of each feature that contributed to the candidate score of 0.71 of the frame for the machine learning model.

[0156] The contribution calculation module (150) calculates the contribution of the goalpost proximity feature as +0.26, the contribution of the ball speed feature as +0.18, the contribution of the density feature as +0.14, and the contribution of the number of nearby players feature as +0.09.

[0157] The contribution calculation module (150) indicates that the proximity feature of the goalpost due to the penetration of player number 7 into the goalpost contributed most significantly to the calculation of the candidate score of the frame, and connects the calculated contribution to the tracking identifier player number 7 so that it can be specifically identified that the player performed an offensive role leading the penetration into the goalpost and fast shooting during the game.

[0158] In summary, the system of the present invention exhibits the characteristic of performing a series of analysis pipelines collectively within a single system, starting from sampling video of a game frame by frame to detect objects and assign tracking identifiers, sequentially performing frame-by-frame feature calculation, linear combination of rule-based scores and machine learning model prediction scores, moving average-based smoothing and peak detection to generate highlight candidate clips, and calculating contribution per feature to provide the basis for calculating candidate scores in an explainable form.

[0159] Furthermore, in order to increase the classification precision of contribution per feature and systematically generate explanatory information for candidate scores, the contribution calculation module (150) can be specialized into a group classification unit (151) and an explanatory information generation unit (152).

[0160] The group classification unit (151) performs the function of calculating the contribution per feature group by classifying the contribution per feature into at least one feature group among the ball physical group, attack threat group, player pressure / density group, dynamic transition group and image quality group.

[0161] The ball physics group covers features corresponding to the physical movement characteristics of the ball, such as the ball's velocity, acceleration, and change in direction of movement.

[0162] For example, the group classification unit (151) sums the contributions of features belonging to the ball physics group in a specific frame and calculates the contribution of the ball physics group as +0.27. The group classification unit (151) provides this result as a value reflecting that the fast movement and rapid acceleration of the ball in the frame acted as the main factors in the highlight situation.

[0163] The offensive threat group is based on features that reflect the intensity of the threat in an offensive situation, such as proximity to the goal and the number of players near the ball.

[0164] For example, the group classification unit (151) calculates the contribution of the attack threat group as +0.35 in the frame where player number 7 has penetrated to within 60 pixels in front of the goal.

[0165] The group classification unit (151) provides a result in which the attack threat group shows the highest contribution per feature group in the frame as a value reflecting that the goal proximity situation is a key factor in calculating the highlight candidate score.

[0166] The Player Pressure and Congestion group handles features that address spatial concentration situations of players, such as density based on the distance between players.

[0167] For example, the group classification unit (151) calculates the contribution of the player pressure / concentration group as +0.14 in a frame where 8 defensive players are concentrated near the goal.

[0168] The group classification unit (151) provides this result as a value reflecting that high defensive density acted as a factor that further increased the candidate score of the frame.

[0169] The dynamic transition group is defined by features regarding situations where the direction of the ball's movement changes abruptly or the spatial structure between players changes rapidly.

[0170] For a specific example, the group classification unit (151) calculates the contribution of the dynamic change group as +0.08 in a frame where the direction of the ball's movement vector changes abruptly from 45 degrees to 135 degrees. The group classification unit (151) provides this result as a value reflecting that the change in the ball's direction contributed additionally to the candidate score of the frame.

[0171] The image quality group includes features belonging to the reliability of image analysis, such as the confidence score of the detected class and the continuity of the tracking identifier.

[0172] For example, the group classification unit (151) calculates the contribution of the image quality group as -0.05 in a frame where tracking stability is reduced because the reliability score of the detected ball is calculated to be low at 0.45.

[0173] The group classification unit (151) provides this result as a value that reflects that the degradation of image quality acted to slightly lower the candidate score of the corresponding frame.

[0174] In this way, the group classification unit (151) aggregates the contribution of each feature into feature groups of similar nature, thereby enabling the editor to quickly identify the main factors of the highlight situation in a more intuitive form than the contribution at the individual feature level.

[0175] The explanation information generation unit (152) is responsible for generating explanation information about the candidate scores of highlight candidate clips based on the contribution of each feature group.

[0176] The explanation information generation unit (152) receives the contribution of each feature group calculated by the group classification unit (151) and generates explanation information in a form that allows the user to intuitively understand the basis for selecting highlight candidate clips.

[0177] The explanation information generation unit (152) can generate explanation information in various ways depending on the purpose of use and the output form.

[0178] For example, the explanation information generation unit (152) can generate explanation information that visualizes the contribution of each feature group in the form of a bar graph.

[0179] For example, the explanation information generation unit (152) generates a horizontal bar graph sorted in order of contribution size, with the contribution of the attack threat group +0.35, the contribution of the ball physics group +0.27, the contribution of the player pressure / density group +0.14, the contribution of the dynamic transition group +0.08, and the contribution of the video quality group -0.05, so that the editor can see at a glance which feature group contributed the most to the calculation of the candidate score of the clip.

[0180] As another example, the explanation information generation unit (152) can generate explanation information by converting the contribution of each feature group into a natural language explanation sentence.

[0181] For example, when the contribution of the attack threat group is calculated to be highest at +0.35, the explanation information generation unit (152) generates an explanation sentence in the form of “The close-to-goal situation is the main highlight factor. It includes the scene of player number 7 penetrating the goal.” and provides it to the editor.

[0182] As another example, the description information generation unit (152) can generate description information in the form of tags that classify the scene type of a highlight candidate clip based on the contribution of each feature group.

[0183] For example, the explanation information generation unit (152) automatically assigns tags such as 'mess in front of the goal', 'multiple players crowded', and 'shooting attempt' to the clip when the contribution of the attack threat group and the player pressure / congestion group is calculated to be high simultaneously, so that the editor can quickly identify the nature of the clip.

[0184] As an additional example, the explanation information generation unit (152) can generate summary information of a player's in-game contribution by aggregating clips that repeatedly show a high attack threat group contribution of a player with the same tracking identifier.

[0185] For example, the explanation information generation unit (152) generates and provides to the editor player contribution summary information in the form of “Player 7 led 5 goal penetration scenes in this match and the average attack threat contribution is 0.31” when the average contribution of the attack threat group in 5 clips containing tracking identifier player 7 is calculated to be +0.31.

[0186] In this way, the explanation information generation unit (152) converts the contribution of each feature group into various forms of explanation information suitable for the editor's purpose of use, thereby helping the editor make a judgment during the selection process of highlight candidate clips.

[0187] By subdividing the contribution calculation module (150) into a group classification unit (151) and an explanation information generation unit (152) in this way, the contribution per feature is aggregated in group units to generate intuitive explanation information, and the editor has the characteristic of being able to understand the basis for selecting highlight candidate clips from various angles.

[0188] As previously explained, the system of the present invention combines a YOLO-based vision model and a multi-object tracking algorithm to calculate frame-by-frame features from video of a game and calculate candidate scores per frame.

[0189] However, in videos of sports matches such as soccer, false positive situations that are difficult to overcome with vision AI-based video analysis alone can occur frequently.

[0190] For example, in a chaotic situation where multiple players are densely packed near the goal and the ball is partially obscured, the reliability of ball detection decreases, which may lead to inaccurate calculation of the goal proximity feature. Additionally, if the goalkeeper moves rapidly, the vision model may temporarily misidentify the goalkeeper's position, leading to a problem where the positional relationship between the ball and the goalkeeper is incorrectly determined.

[0191] In addition, if the goal's bounding box is temporarily lost or distorted due to camera zoom changes or sudden changes in the shooting angle, a problem may arise where the goal's proximity feature fails to reflect the actual situation.

[0192] Furthermore, it is difficult to distinguish between a ball that has passed near the goal and a ball that has actually collided with the goal using only video, and there are practical limitations to accurately determining whether the goalkeeper made a save during a shooting situation relying solely on vision AI.

[0193] To overcome these limitations, the system of the present invention proposes a novel and advanced technology that directly measures physical events that are difficult for vision AI to capture using actual physical sensors to post-correct image analysis results.

[0194] FIG. 5 is an example diagram illustrating the state in which a sensor for the first correction of the present invention is placed in a sports stadium.

[0195] Specifically, the system of the present invention may additionally include a strain gauge sensor (160) and an Ultra-Wideband sensor (170).

[0196] The strain gauge sensor (160) is attached to the goal frame and has the function of measuring the amount of deformation of the goal frame.

[0197] A strain gauge sensor (160) refers to a sensor attached to the surface of an object that measures the amount of deformation caused by an external force by converting it into a change in electrical resistance.

[0198] The strain gauge sensor (160) is mounted by attaching it to the vertical pillars on both sides of the goal frame and the upper horizontal bar, respectively, and can be implemented in a detachable clamping manner for each stadium, allowing for quick installation and removal in various stadium environments.

[0199] For example, the strain gauge sensor (160) clearly distinguishes the three situations by measuring the deformation of the goal frame as about 850 μm when the ball collides with the upper crossbar of the goal due to a strong punch from the goalkeeper, measuring the deformation as about 120 μm when the ball grazes the vertical post of the goal weakly, and measuring the deformation as about 3 μm or less when the ball only passes near the goal.

[0200] In addition, the strain gauge sensor (160) measures the amount of deformation as approximately 1,200 μm when a strong shot of more than 100 km / h hits the center of the goal, and measures the amount of deformation as approximately 180 μm when a weak pass of about 30 km / h hits the bottom corner of the goal, clearly reflecting the difference in numerical values ​​according to the impact intensity.

[0201] In this way, the strain gauge sensor (160) provides numerical values ​​of whether a collision actually occurred and the collision intensity in situations where it is difficult for the vision AI to distinguish between a ball that passed near the goal and a ball that actually collided with the goal.

[0202] The UWB sensor (170) is worn by the goalkeeper and is responsible for measuring the goalkeeper's real-time position.

[0203] This UWB sensor (170) is defined as a sensor that measures the time of flight between a sensor tag and an anchor node using ultra-wideband wireless communication technology and calculates the position of the sensor tag with a precision of several tens of centimeters based on the measured time of flight.

[0204] The UWB sensor (170) is implemented in the form of a small tag attached to the goalkeeper's vest or uniform and calculates the goalkeeper's real-time position by linking with multiple anchor nodes in the stadium.

[0205] The anchor node is a receiving device installed at a fixed location within the stadium. It can be implemented using a detachable clamp method for each stadium, allowing for rapid installation and removal in various stadium environments.

[0206] For example, the UWB sensor (170) measures the position of the goalkeeper with an update cycle of 100 times per second, and calculates the position with an error range of about ±15cm when the goalkeeper is located 2.5m in front of the goal, and continuously calculates the real-time position with an error range of ±20cm even when the goalkeeper moves to a point 16m outside the penalty box.

[0207] Even if the vision AI misidentifies the goalkeeper's position by 85 pixels relative to the actual position in a player overlap situation, the UWB sensor (170) calculates the goalkeeper's actual position with precision within ±15cm and provides it as basic data for false positive correction.

[0208] In this way, the UWB sensor (170) improves the correction accuracy of candidate scores per frame by providing the actual position of the goalkeeper precisely in situations where the vision AI misidentifies the position of the goalkeeper.

[0209] In response to this, the score calculation module (130) may further include a first correction unit (131).

[0210] The first correction unit (131) performs the function of correcting candidate scores per frame based on the deformation amount of the goalpost frame and the real-time position of the goalkeeper.

[0211] These primary correction units (131) can correct frame-by-frame candidate scores in various ways.

[0212] For example, the first correction unit (131) can determine a correction value based on whether the amount of deformation of the goal frame measured by the strain gauge sensor (160) exceeds a preset threshold value and the distance between the real-time position of the goalkeeper and the position of the ball measured by the UWB sensor (170), and perform the correction by adding it to the frame-by-frame candidate score calculated by the score calculation module (130).

[0213] For example, the first correction unit (131) determines a correction value of 0.25 when the deformation amount of the goalpost frame exceeds a threshold of 50 μm and the distance between the goalkeeper's real-time position and the ball's position is less than 1.0 m, and adds this to the frame-specific candidate score of 0.45 calculated by the score calculation module (130) to calculate the corrected candidate score as 0.45 + 0.25 = 0.70, and when the deformation amount exceeds a threshold of 50 μm but the distance is 5.0 m or more, it determines a correction value of 0.05 to calculate the corrected candidate score as 0.45 + 0.05 = 0.50, and when the deformation amount is 3 μm or less and the distance is 5.0 m or more, it determines a correction value of 0 to maintain the candidate score of 0.45 as is.

[0214] As another example, the first correction unit (131) can perform correction by calculating a correction value by dividing the amount of deformation of the goal frame measured by the strain gauge sensor (160) by a preset reference deformation amount and the distance between the goalkeeper's real-time position and the ball's position measured by the UWB sensor (170) by a preset reference distance, and adding this value to the candidate score per frame calculated by the score calculation module (130).

[0215] For example, the first correction unit (131) calculates the deformation ratio as 850 / 1,000 = 0.85 when the deformation amount of the goalpost frame is 850 μm and the reference deformation amount is 1,000 μm, and calculates the distance ratio as 0.8 / 5.0 = 0.16 when the distance between the goalkeeper's real-time position and the ball's position is 0.8 m and the reference distance is 5.0 m, and then calculates the correction value as 0.85 × 0.16 = 0.136 and adds it to the frame-specific candidate score of 0.52 calculated by the score calculation module (130) to calculate the corrected candidate score as 0.52 + 0.136 = 0.656.

[0216] As another example, the primary correction unit (131) can correct the candidate score per frame based on a correction table that combines the deformation amount of the goal frame measured by the strain gauge sensor (160) and the real-time position of the goalkeeper measured by the UWB sensor (170) by classifying them into sections.

[0217] For example, the first correction unit (131) determines that a case where the deformation amount of the goalpost frame is 120 μm or more and the goalkeeper's real-time position is within 1.5 m of the front of the goalpost is an actual save situation, adds a correction value of 0.25, and calculates the candidate score per frame of 0.58 calculated by the score calculation module (130) as 0.58 + 0.25 = 0.83, determines that a case where the deformation amount is 120 μm or more and the goalkeeper's real-time position is 8.0 m or more away from the goalpost is a simple goalpost collision situation, adds a correction value of 0.10, and calculates the corrected candidate score as 0.58 + 0.10 = 0.68, determines that a case where the deformation amount is 3 μm or less and the goalkeeper's real-time position is 8.0 m or more away from the goalpost is a situation unrelated to highlights, determines the correction value as 0, and maintains the candidate score of 0.58 as is.

[0218] In this way, the first correction unit (131) can accurately reflect the actual physical data measured from the strain gauge sensor (160) and the UWB sensor (170) into the frame-by-frame candidate score calculated by the score calculation module (130), thereby accurately reflecting the goalpost collision and the actual position of the goalkeeper in the candidate score, which are difficult to capture with Vision AI alone.

[0219] As previously explained, the first correction unit (131) first corrects the candidate scores per frame based on the amount of deformation of the goal frame measured by the strain gauge sensor (160) and the real-time position of the goalkeeper measured by the UWB sensor (170).

[0220] However, threshold-based addition methods or segment-based correction table methods simplify the relationship between sensor measurements and candidate scores, and thus the precision of correction may decrease in situations where the deformation of the goal frame and the goalkeeper's movement speed interact in combination.

[0221] To overcome these limitations, the system of the present invention proposes a first-order correction algorithm based on a special mathematical formula that combines the deformation amount of the goalpost frame and the position and movement speed of the goalkeeper into a non-linear function.

[0222] Specifically, the first correction unit (131) first corrects the candidate scores per frame according to the following mathematical formula 1.

[0223] Mathematical formula 1.

[0224]

[0225] Here, S_1(t) is the first corrected candidate score in frame t, S_0(t) is the frame-specific candidate score in frame t, F_strain(t) is the deformation amount of the goal frame in frame t (μm), F_th is the threshold value of the deformation amount of the goal frame (μm), d_gk(t) is the distance between the goalkeeper position and the ball position in frame t (m), d_ref is the distance reference value (m), v_gk(t) is the goalkeeper movement speed in frame t (m / s), v_ref is the goalkeeper movement speed reference value (m / s), α is the correction sensitivity coefficient, and if F_strain(t) is less than F_th, the tanh term is set to 0 to maintain S_1(t) as S_0(t).

[0226] Mathematical formula 1 has a structure that calculates a first-corrected candidate score S_1(t) by multiplying the frame-specific candidate score S_0(t) calculated by the score calculation module (130) by a tanh function-based correction term. The correction term of mathematical formula 1 consists of the product of three physical indicators, and the physical meaning of each term is explained as follows.

[0227] The (F_strain(t) - F_th) / F_th term is a value that normalizes the degree to which the deformation of the goal frame exceeds a threshold value, and has a structure that converges to 0 as the deformation approaches the threshold value and increases in the positive direction as the deformation exceeds the threshold value significantly.

[0228] For example, if F_th is set to 50μm, when F_strain(t) is measured as 850μm, it is calculated as (850 - 50) / 50 = 16.0, and when F_strain(t) is measured as 55μm, it is calculated as (55 - 50) / 50 = 0.10, so the magnitude of the correction term changes sensitively depending on the goal collision intensity.

[0229] The term e^(-d_gk(t) / d_ref) is an exponential damping function that dampens the correction strength according to the distance between the goalkeeper and the ball, and has a structure in which the correction term converges to 1 as the goalkeeper gets closer to the ball and the correction term converges to 0 as the goalkeeper gets further away from the ball.

[0230] For example, if d_ref is set to 5.0m, if d_gk(t) is 0.8m, e^(-0.8 / 5.0) = e^(-0.16) = approximately 0.852 is calculated, and if d_gk(t) is 8.0m, e^(-8.0 / 5.0) = e^(-1.6) = approximately 0.202 is calculated, so the correction strength is automatically suppressed when the goalkeeper is far from the ball.

[0231] The v_gk(t) / v_ref term is a value normalized by the goalkeeper's movement speed to a reference speed, and has a structure in which the correction term increases as the goalkeeper reacts and moves faster. For example, if v_ref is set to 3.0 m / s, if v_gk(t) is 4.5 m / s, it is calculated as 4.5 / 3.0 = 1.5, and if v_gk(t) is 0.5 m / s, it is calculated as 0.5 / 3.0 = approximately 0.167, so the correction strength is amplified in situations where the goalkeeper reacts rapidly toward the ball.

[0232] The first correction unit (131) inputs the value obtained by applying a correction sensitivity coefficient α to the product of these three terms into the tanh function, thereby limiting the maximum value of the correction term to 1 and securing a convergence structure in which S_1(t) does not exceed a maximum of 2 times S_0(t).

[0233] The first correction unit (131) prevents the candidate score from rising excessively even when the sensor measurement value shows an outlier.

[0234] Conventional sensor data-based correction technologies were often limited to methods that compared a single sensor measurement of either deformation or distance with a threshold value to add a correction value or multiply by a correction coefficient, and this presented a problem in that it could not distinguish between a situation where the ball collides with the goalpost when the goalkeeper has already moved far away and an actual save situation where the goalkeeper blocks the ball directly in front of the goal.

[0235] Mathematical formula 1 non-linearly combines the goalpost collision intensity, the distance between the goalkeeper and the ball, and the goalkeeper's movement speed within a single formula, so that the correction intensity is maximized only when all three conditions are simultaneously satisfied, and the correction intensity is automatically suppressed when any one of the conditions is not satisfied, thereby increasing the precision of the correction compared to conventional technology.

[0236] To give a specific numerical example, the first correction unit (131) is set to α = 1.0, F_th = 50μm, d_ref = 5.0m, v_ref = 3.0m / s, and when S_0(t) = 0.52, F_strain(t) = 850μm, d_gk(t) = 0.8m, and v_gk(t) = 4.5m / s, the internal value of the correction term is calculated as 1.0 × 16.0 × 0.852 × 1.5 = 20.45, and tanh(20.45) = approximately 1.0, so S_1(t) = 0.52 × (1 + 1.0) = 1.04.

[0237] On the other hand, when S_0(t) = 0.52, F_strain(t) = 55μm, d_gk(t) = 3.0m, and v_gk(t) = 1.0m / s, the first correction unit (131) calculates the internal value of the correction term as 1.0 × 0.10 × e^(-0.6) × 0.333 = 1.0 × 0.10 × 0.549 × 0.333 = approximately 0.018, and calculates tanh(0.018) = approximately 0.018, thereby calculating S_1(t) = 0.52 × (1 + 0.018) = approximately 0.529.

[0238] According to the first correction unit (131) according to mathematical formula 1, when the goalpost collision intensity, the distance between the goalkeeper and the ball, and the goalkeeper's movement speed simultaneously indicate a highlight situation, S_1(t) increases to 1.04, whereas when only some of the three conditions are satisfied, S_1(t) is 0.529, which is almost the same as S_0(t), thus providing a structure in which correction is suppressed.

[0239] As previously explained, the first correction unit (131) first corrects the candidate score per frame by non-linearly combining the deformation amount of the goal frame measured by the strain gauge sensor (160) and the position and movement speed of the goalkeeper measured by the UWB sensor (170) according to Equation 1.

[0240] However, mathematical formula 1 is a structure that performs correction based on the collision intensity of the goal frame and the position and movement speed of the goalkeeper, so there is a possibility that it may not directly reflect whether the goalkeeper actually came into contact with the ball and the intensity of the contact.

[0241] For example, even if the goalkeeper moves quickly in front of the goal, it is difficult to distinguish with only mathematical formula 1 whether the ball was actually blocked with the hand or if the ball grazed the goalkeeper's body.

[0242] To improve this aspect, the system of the present invention proposes a novel and advanced technique that further reflects actual contact data measured from a pressure sensor (180) attached inside the goalkeeper's glove to perform a second correction on a first-corrected candidate score.

[0243] FIG. 6 is an exemplary diagram illustrating the configuration of a goalkeeper glove pressure sensor for secondary correction of the present invention.

[0244] The pressure sensor (180) is responsible for measuring the contact area calculated from the contact pressure of the ball, contact time, and the number of pressure cells detected, with multiple pressure cells arranged inside the goalkeeper's glove.

[0245] The pressure sensor (180) is implemented as a plurality of pressure cells arranged in a grid shape on the front inner surface of the goalkeeper's glove, and each pressure cell independently measures the pressure acting on the cell through a change in electrical resistance when an external force is applied.

[0246] The pressure sensor (180) can be easily attached and detached during the game by the way a goalkeeper wears gloves, allowing for quick application in various game environments.

[0247] The pressure sensor (180) measures the contact pressure, contact time, and contact area of ​​the ball as follows. The contact pressure of the ball is the value measured as pressure per unit area (kPa) of the force applied to the pressure cell at the moment of collision between the ball and the glove.

[0248] The ball's contact time is a value measured as the time interval (ms) from the point when the pressure cell's measurement exceeds a preset detection threshold to the point when it returns to below the detection threshold.

[0249] The contact area of ​​the ball is a value (cm²) calculated by multiplying the number of pressure cells exceeding the detection threshold by the unit cell area.

[0250] For example, when a goalkeeper blocks a strong shot head-on, the pressure sensor (180) measures the contact pressure as approximately 85 kPa, the contact time as approximately 120 ms, and the number of detected pressure cells as 24, and calculates the contact area as approximately 48 cm².

[0251] On the other hand, when the ball brushes past the end of the goalkeeper's glove, the pressure sensor (180) measures the contact pressure as approximately 12 kPa, the contact time as approximately 18 ms, and the number of detected pressure cells as 3, and calculates the contact area as approximately 6 cm².

[0252] In this way, the pressure sensor (180) clearly distinguishes between actual line contact and simple contact situations, which are difficult for vision AI to distinguish, by numerical values ​​of contact pressure, contact time, and contact area.

[0253] In response to this, the score calculation module (130) may further include a secondary correction part including an efficiency index calculation part (132) and a secondary correction part (133).

[0254] That is, the second correction unit is intended to quantify the goalkeeper's save efficiency based on actual contact data measured from the pressure sensor (180), and to calculate the second corrected candidate score by reflecting the quantified save efficiency index in the first corrected candidate score.

[0255] Specifically, the efficiency index calculation part (132) performs the function of calculating the save efficiency index according to the following mathematical formula 2 based on the ball's contact pressure, contact time, and contact area measured by the pressure sensor (180) and the goalkeeper's movement speed measured by the UWB sensor (170).

[0256] Mathematical formula 2.

[0257]

[0258] Here, E_save(t) is the save efficiency index (dimensionless) in frame t, p(t) is the contact pressure of the ball in frame t (kPa), p_ref is the reference value of the contact pressure (kPa), A_contact(t) is the contact area of ​​the ball (cm²), A_ref is the reference value of the contact area (cm²), T_contact(t) is the contact time of the ball (ms), T_ref is the reference value of the contact time (ms), v_gk(t) is the movement speed of the goalkeeper measured by the UWB sensor (170) in frame t (m / s), and v_ref is the reference value of the goalkeeper's movement speed (m / s).

[0259] The save efficiency index E_save(t) is a dimensionless index that quantifies the physical characteristics of the moment the goalkeeper actually contacts the ball, and has a fundamentally different significance from the method of considering a situation where the ball is located near the goal or the goalkeeper is moving quickly as a save.

[0260] Conventional save determination methods were often limited to vision AI determining a save when the goalkeeper's bounding box and the ball's bounding box overlapped, or considering a save when the distance between the goalkeeper and the ball was below a threshold.

[0261] In other words, the conventional method had the problem of failing to distinguish between a situation where the ball grazed the goalkeeper's body and an actual save situation where the goalkeeper strongly blocked the ball.

[0262] In contrast, E_save(t) in Equation 2 combines contact pressure, contact area, contact time, and goalkeeper's movement speed into a single exponent, so that E_save(t) has a value greater than 1 only when all four conditions are simultaneously satisfied, and E_save(t) is automatically suppressed when any one of the conditions is not satisfied.

[0263] The physical meaning of each term constituting mathematical formula 2 is as follows.

[0264] The p(t) / p_ref term is a value obtained by normalizing the ball's contact pressure to the reference pressure, and it has a structure in which the contact pressure increases E_save(t) in proportion to the reference value.

[0265] For example, when p_ref is set to 50kPa, if p(t) is measured at 85kPa, it is calculated as 85 / 50 = 1.70, and if p(t) is measured at 12kPa, it is calculated as 12 / 50 = 0.24, showing a numerical difference of about 7.1 times between a strong line and a simple graze situation.

[0266] The A_contact(t) / A_ref term is a value normalized by the ball's contact area to the reference area, and it has a structure where E_save(t) increases as the contact area increases.

[0267] For example, if A_ref is set to 20cm², A_contact(t) is calculated as 48 / 20 = 2.40 if A_contact(t) is 48cm², and 6 / 20 = 0.30 if A_contact(t) is 6cm², clearly distinguishing between a situation where the front of the armor blocks and a situation where the armor grazes.

[0268] The term (1 - e^(-T_contact(t) / T_ref)) is a saturation function that converges between 0 and 1 depending on the contact time, and has a structure that shows a value close to 0 when the contact time is very short and converges to 1 as the contact time increases.

[0269] For example, if T_ref is set to 50ms, if T_contact(t) is 120ms, it is calculated as 1 - e^(-120 / 50) = 1 - e^(-2.4) = 1 - 0.091 = 0.909, and if T_contact(t) is 18ms, it is calculated as 1 - e^(-18 / 50) = 1 - e^(-0.36) = 1 - 0.698 = 0.302, which non-linearly reflects the difference in contact time between the instantaneous brush and the actual line.

[0270] The term (1 - e^(-v_gk(t) / v_ref)) is a saturation function that converges between 0 and 1 depending on the goalkeeper's movement speed, and has a structure in which E_save(t) increases as the goalkeeper reacts rapidly toward the ball.

[0271] For example, if v_ref is set to 3.0 m / s, if v_gk(t) is 4.5 m / s, it is calculated as 1 - e^(-4.5 / 3.0) = 1 - e^(-1.5) = 1 - 0.223 = 0.777, and if v_gk(t) is 0.5 m / s, it is calculated as 1 - e^(-0.5 / 3.0) = 1 - e^(-0.167) = 1 - 0.846 = 0.154, distinguishing between a situation where the goalkeeper makes contact with the ball while near a standstill and a situation where he reacts rapidly to make a save.

[0272] Mathematical Equation 2 has a structure that combines a linear term normalized by the contact pressure and contact area to reference values ​​with a non-linear term converted by the contact time and the goalkeeper's movement speed into a saturation function. By processing four measurements with different physical characteristics into a function form suited to each characteristic, it accurately reflects the physical meaning of each variable compared to a simple weighted sum method.

[0273] In addition, since mathematical formula 2 consists of the product of four terms, if any one term shows a value close to zero, the entire E_save(t) has a structure that converges to zero. For example, if the contact time is extremely short and T_contact(t) = 1ms, it is calculated as 1 - e^(-1 / 50) = approximately 0.020. Even if the contact pressure and contact area are temporarily measured to be high, the efficiency index calculation part (132) calculates E_save(t) as a value close to zero, thereby preventing E_save(t) from rising excessively in situations where the ball momentarily brushes against the glove.

[0274] Furthermore, since Equation 2 has a structure that calculates E_save(t) as a dimensionless exponent by normalizing all variables to a reference value, the calculation structure of E_save(t) can be maintained identically by adjusting only the reference value even in environments where the sensitivity or measurement unit of the sensor differs from stadium to stadium, so it can be universally applied in various game environments.

[0275] As a numerical example for mathematical formula 2, the efficiency index calculation part (132) is set to p_ref = 50kPa, A_ref = 20cm², T_ref = 50ms, v_ref = 3.0m / s, and in the case of a strong frontal situation where p(t) = 85kPa, A_contact(t) = 48cm², T_contact(t) = 120ms, v_gk(t) = 4.5m / s, E_save(t) = 1.70 × 2.40 × 0.909 × 0.777 = approximately 2.89. On the other hand, in the case of a simple contact situation where p(t) = 12kPa, A_contact(t) = 6cm², T_contact(t) = 18ms, and v_gk(t) = 0.5m / s, the efficiency index calculation part (132) calculates E_save(t) = 0.24 × 0.30 × 0.302 × 0.154 = approximately 0.003, thereby calculating E_save(t) which shows a numerical difference of approximately 963 times between the two situations.

[0276] The second correction part (133) performs the function of secondly correcting the candidate score that was first corrected according to the following mathematical formula 3, based on the line efficiency index calculated by the efficiency index calculation part (132).

[0277] Mathematical formula 3.

[0278]

[0279] Here, S_2(t) is the second-order corrected candidate score in frame t, S_1(t) is the first-order corrected candidate score in frame t, E_save(t) is the line efficiency index calculated by Equation 2, and λ is the second-order correction sensitivity coefficient.

[0280] Mathematical formula 3 has a structure for calculating a second corrected candidate score S_2(t) by multiplying a first corrected candidate score S_1(t) by a correction term that reflects the first correction efficiency index E_save(t). The second correction part (133) has a structure in which, when E_save(t) is close to 0, the correction term converges to 1 so that S_2(t) has a value almost identical to S_1(t), and as E_save(t) increases, the correction term increases so that S_2(t) rises relative to S_1(t).

[0281] The second correction part (133) controls the intensity of the influence of the line efficiency index on the second-corrected candidate score according to the value of λ.

[0282] The second correction part (133) provides a structure in which each correction step has a different physical basis by sequentially applying mathematical formula 1 and mathematical formula 3 to independently perform a first correction based on whether there is a collision with the goalpost and the position of the goalkeeper and a second correction based on the actual contact quality.

[0283] To explain the advantages of Equation 3, since Equation 3 has a simple linear combination structure that takes E_save(t) calculated in Equation 2 as input and multiplies it by S_1(t), it provides the characteristic of clearly separating the roles of the two equations without transferring the non-linear complexity of Equation 2 to Equation 3.

[0284] In addition, since Equation 3 has a structure that converges to S_2(t) = S_1(t) when E_save(t) is 0, the secondary correction does not affect the result of the primary correction in frames where actual contact by the goalkeeper does not occur, thereby ensuring the independence of the correction.

[0285] Furthermore, by adjusting the λ value, the reflection intensity of the actual contact data provided by the pressure sensor (180) can be flexibly adjusted according to the type of game or sensor sensitivity, so it can be universally applied in various game environments.

[0286] Conventional line detection methods have limitations in that they rely on the boundary box overlap or distance threshold comparison of Vision AI, failing to reflect actual contact strength and contact quality in candidate scores. Equation 3 provides a structure that precisely reflects actual line quality in candidate scores compared to conventional methods by directly reflecting the line efficiency index based on actual contact data calculated in Equation 2 into the first-corrected candidate scores.

[0287] To give a specific numerical example for mathematical formula 3, the second correction part (133) is set to λ = 0.15, and in the case of a strong preemptive situation where S_1(t) = 1.04 and E_save(t) = 2.89, S_2(t) = 1.04 × (1 + 0.15 × 2.89) = 1.04 × 1.434 = approximately 1.491. On the other hand, in the case of a simple graze situation where S_1(t) = 0.529 and E_save(t) = 0.003, the second correction part (133) calculates S_2(t) = 0.529 × (1 + 0.15 × 0.003) = 0.529 × 1.0005 = approximately 0.530, so that in a strong frontal situation, S_2(t) rises to 1.491, whereas in a simple graze situation, S_2(t) is 0.530, which is almost the same as S_1(t), thus providing a structure in which correction is suppressed.

[0288] Thus, the system of the present invention has the characteristic of increasing the detection accuracy of highlight candidate clips and reducing the false positive rate by sequentially performing first and second corrections that reflect actual physical data measured from strain gauge sensors (160), UWB sensors (170), and pressure sensors (180) in stages, in addition to an analysis pipeline that includes vision AI-based object detection and tracking, frame-unit feature calculation, linear combination of rule-based scores and machine learning model prediction scores, generation of highlight candidate clips through moving average-based smoothing and peak detection, and calculation of contribution per feature, thereby precisely reflecting the goal collision, the goalkeeper's actual position, and the actual save quality, which are difficult to capture with vision AI alone, into the candidate scores.

[0289] Below, experimental results for verifying the performance of the correction algorithm according to Equations 1, 2, and 3 are described.

[0290] <Example 1>

[0291] Example 1 is a configuration that first corrects the candidate score per frame by non-linearly combining the deformation amount of the goalpost frame measured by the strain gauge sensor (160) and the position and movement speed of the goalkeeper measured by the UWB sensor (170) according to Equation 1, and is set to α = 1.0, F_th = 50μm, d_ref = 5.0m, and v_ref = 3.0m / s.

[0292] <Example 2>

[0293] Example 2 is configured to calculate a save efficiency index by combining the ball's contact pressure, contact time, and contact area measured by the pressure sensor (180) according to Equations 2 and 3 and the goalkeeper's movement speed measured by the UWB sensor (170) in addition to the first correction of Example 1, and to second correct the first corrected candidate score, with p_ref = 50kPa, A_ref = 20cm², T_ref = 50ms, and λ = 0.15.

[0294] <Comparative Example 1>

[0295] Comparative Example 1 uses a strain gauge sensor (160), a UWB sensor (170), and a pressure sensor (180) in the same way as Example 2, but the values ​​measured from the three sensors are normalized to a reference value and then combined by a weighted sum to correct the candidate score per frame, and the correction formula is set as S_correction(t) = S_0(t) + w_1 × (F_strain(t) / F_th) + w_2 × (1 - d_gk(t) / d_max) + w_3 × (p(t) / p_ref). Here, w_1 = 0.10, w_2 = 0.08, w_3 = 0.07, F_th = 50μm, d_max = 10.0m, and p_ref = 50kPa.

[0296] <Comparative Example 2>

[0297] Comparative Example 2 uses a strain gauge sensor (160) and a UWB sensor (170) in the same manner as Example 1, but is configured to correct candidate scores per frame by normalizing the values ​​measured from the two sensors to a reference value and combining them by a simple weighted sum, and the correction formula is set as S_correction(t) = S_0(t) + w_1 × (F_strain(t) / F_th) + w_2 × (1 - d_gk(t) / d_max). Here, w_1 = 0.12, w_2 = 0.10, F_th = 50μm, and d_max = 10.0m.

[0298] The evaluation data consisted of 1,240 highlight candidate clips extracted from 28 soccer match videos.

[0299] Three evaluators (football analysts with an average of 7.3 years of experience) distinguished between goal scenes, shots on target, goalkeeper saves, collisions with the goalpost, and general attack scenes and assigned correct labels, and the inter-rater agreement rate was calculated as 0.84 based on Cohen's Kappa, ensuring inter-rater reliability.

[0300] [Experiment 1: Comparison of Highlight Candidate Clip Detection Accuracy]

[0301] Experiment 1 was conducted to compare and verify the accuracy of four highlight candidate clip detection methods based on precision, recall, and F1-score.

[0302] The measurement items were set to precision (the proportion of calculated highlight candidate clips corresponding to actual highlight scenes), recall (the proportion of actual highlight scenes detected as highlight candidate clips), and F1-score (the harmonic mean of precision and recall).

[0303] Precision was calculated by dividing the number of clips matching the correct label assigned by the evaluator among the 1,240 highlight candidate clips generated by each method by the total number of generated clips, and recall was calculated by dividing the number of clips detected as highlight candidate clips by each method among the actual highlight scenes assigned the correct label by the evaluator by the total number of correct labels, and the F1-score was calculated as the harmonic mean of precision and recall.

[0304] Table 1 below shows the results for precision, recall, and F1-score.

[0305] Precision Recall rate F1-score Example 1 0.827 0.786 0.806 Example 2 0.884 0.841 0.862 Comparative Example 1 0.762 0.724 0.742 Comparative Example 2 0.713 0.681 0.697

[0306] Example 1 showed a result in which the precision increased from 0.713 to 0.827 and the F1-score increased from 0.697 to 0.806 compared to Comparative Example 2, which combined the same sensors as a simple weighted sum, by simultaneously reflecting the scene where the deformation amount of the goal frame exceeds a threshold and the scene where the goalkeeper actually reacts.

[0307] Example 2 showed that the precision increased from 0.762 to 0.884 and the F1-score increased from 0.742 to 0.862 compared to Comparative Example 1, which combined the same sensors by a simple weighted sum, by quantifying the actual contact data measured from the pressure sensor (180) into a linear efficiency index and reflecting it in the secondary correction.

[0308] As a result of Experiment 1, Example 1 and Example 2 showed a statistically significant difference compared to their respective control groups at the p < 0.01 level.

[0309] [Experiment 2: Comparison of Reduction Rates by False Positive Type]

[0310] Experiment 2 was conducted to classify false positives occurring during the highlight candidate clip detection process by type and to compare and verify the false positive reduction effects of four methods.

[0311] As measurement items, the following were set: false positive rate for passing near the goal (the rate at which the ball passed near the goal but was incorrectly detected as a highlight candidate clip), false positive rate for simple goalpost collision (the rate at which a scene where the ball collided with the goal but no goalkeeper save was incorrectly detected as a highlight candidate clip), and false positive rate for grazing the goalkeeper (the rate at which a scene where the ball grazed the goalkeeper's body was incorrectly detected as an actual save).

[0312] The false positive rate for each false positive type was calculated as the ratio of the number of events incorrectly detected as highlight candidate clips by each method to the total number of events of that type.

[0313] A ball passing near the goal event is defined as a case where the amount of deformation measured by the strain gauge sensor (160) is 3 μm or less, a simple goalpost collision event is defined as a case where the amount of deformation is 50 μm or more and the position of the goalkeeper measured by the UWB sensor (170) is 5.0 m or more away from the goalpost, and a goalkeeper graze event is defined as a case where the contact pressure measured by the pressure sensor (180) is 15 kPa or less and the contact time is 30 ms or less.

[0314] Table 2 below shows the results for false positive rates by detailed false positive type.

[0315] Passing False Positive Rate Simple goalpost collision false positive rate Goalkeeper False Detect Rate Example 1 8.4% 11.2% 12.8% Example 2 7.8% 10.1% 6.8% Comparative Example 1 9.2% 15.8% 10.2% Comparative Example 2 10.8% 19.4% 14.3%

[0316] Example 1 showed a result in which the simple goal collision false detection rate decreased to 11.2% compared to 19.4% in Comparative Example 2 due to a structure that suppresses correction by setting the tanh term to 0 when the amount of deformation of the goal frame measured by the strain gauge sensor (160) is less than the threshold value. The false detection rate of ball passing near the goal decreased to 8.4% compared to 10.8% in Comparative Example 2. The false detection rate of goalkeeper graze was calculated to be 12.8% in Example 1, showing a result of decrease compared to 14.3% in Comparative Example 2; however, due to the limitation that Example 1 does not reflect actual contact data, the reduction effect compared to 10.2% in Comparative Example 1 was limited.

[0317] Comparative Example 1 showed a result in which the goalkeeper graze false detection rate decreased to 10.2% compared to 14.3% in Comparative Example 2 by reflecting the contact pressure measured from the pressure sensor (180) in a weighted sum manner. However, Comparative Example 1 showed a result in which the false detection rate for simple goal collision was calculated to be 15.8%, which is a result of limited false detection suppression effect compared to 11.2% in Example 1. This is due to the limitation that the simple weighted sum structure of Comparative Example 1 cannot suppress correction even when the amount of deformation of the goal frame exceeds a threshold value.

[0318] Example 2 showed that the goalkeeper graze false positive rate decreased from 10.2% in Comparative Example 1 to 6.8% in Example 1 and from 12.8% in Example 1 by quantifying the actual contact data measured from the pressure sensor (180) into a save efficiency index and reflecting it in the secondary correction. The false positive rate of passing near the goal and the false positive rate of simple goalpost collision decreased from 8.4% to 7.8% and from 11.2% to 10.1%, respectively, compared to Example 1, showing that the key improvement effect of Example 2 was concentrated on reducing the goalkeeper graze false positive rate.

[0319] As a result of Experiment 2, Example 1 and Example 2 showed a statistically significant difference compared to their respective control groups at the p < 0.01 level.

[0320] Thus, according to Example 1, compared to Comparative Example 2 which combines the same sensors as a simple weighted sum, the false positive rate of a simple goalpost collision can be intensively reduced by a first correction that non-linearly combines whether there is a collision with the goalpost and the goalkeeper's actual reaction, and according to Example 2, the effect of intensively reducing the false positive rate of a goalkeeper graze is provided by a second correction that additionally reflects actual save contact data.

[0321] As explained above, the configuration and operation of the AI-based sports video analysis system with a highlight section detection function according to the present invention have been described and illustrated in the above description and drawings; however, this is merely an example, and the concept of the present invention is not limited to the above description and drawings. It is understood that various changes and modifications are possible within the scope of the technical concept of the present invention. Explanation of the symbols

[0322] 100: Sports Video Analysis System 110: Object Detection and Tracking Module 120: Feature Calculation Module 121: Ball Physics Feature Calculation Unit 122: Attack Threat Feature Calculation Unit 130: Score Calculation Module 131: Primary Correction Unit 132: Efficiency Index Calculation Part 133: Secondary Correction Part 140: Clip Creation Module 150: Contribution Calculation Module 151: Group Classification Unit 152: Description information generation unit 160: Strain gauge sensor 170: UWB sensor 180: Pressure sensor

Claims

Claim 1 An artificial intelligence-based sports video analysis system having a highlight section detection function, comprising: an object detection tracking module that samples a sports game video in frame units, detects at least one class among a player, ball, goalpost, goalkeeper, and referee using a YOLO-based vision model in the sampled video, and assigns a unique tracking identifier to each of the detected classes by applying a multi-object tracking algorithm; a feature calculation module that calculates multiple frame-unit features in the sampled video based on the tracking identifiers; a score calculation module that calculates frame-specific candidate scores by linearly combining rule-based scores and machine learning model prediction scores for the multiple features; a clip generation module that determines an anchor frame by applying a moving average to the frame-specific candidate scores to smooth them and then performing peak detection, and generates highlight candidate clips centered on the anchor frames; and a contribution calculation module that calculates the contribution of each feature to the frame-specific candidate scores for the machine learning model. Claim 2 A sports video analysis system according to claim 1, wherein the feature calculation module comprises: a ball physical feature calculation unit that calculates the velocity of the ball from the displacement of the center point between consecutive frames of the ball and calculates the acceleration of the ball from the amount of change in the velocity of the ball to calculate the frame-unit feature; and an attack threat feature calculation unit that calculates the proximity of the goal from the reciprocal of the distance between the center of the ball and the goal, calculates the density based on the distance between players, and counts the number of players within a certain radius around the ball to calculate the frame-unit feature. Claim 3 A sports video analysis system according to claim 1, wherein the contribution calculation module comprises a group classification unit that classifies the contribution per feature into at least one feature group among a ball physics group, an attack threat group, a player pressure / density group, a dynamic transition group, and an image quality group, and calculates the contribution per feature group, and an explanation information generation unit that generates explanation information for the candidate score per frame based on the contribution per feature group. Claim 4 A sports video analysis system according to claim 1, wherein the system further comprises a strain gauge sensor attached to a goal frame of a sports game to measure the amount of deformation of the goal frame, and a UWB sensor worn on a goalkeeper to measure the real-time position of the goalkeeper, and wherein the score calculation module further comprises a first correction unit that first corrects the candidate scores per frame based on the amount of deformation of the goal frame and the real-time position of the goalkeeper measured by the UWB sensor. Claim 5 In Clause 4, the first correction unit first corrects the candidate scores per frame according to the following mathematical formula 1, and mathematical formula 1. (Here, S_1(t) is the first corrected candidate score in frame t, S_0(t) is the frame-specific candidate score in frame t, F_strain(t) is the deformation amount of the goalpost frame in frame t (μm), F_th is the threshold value of the deformation amount of the goalpost frame (μm), d_gk(t) is the distance between the goalkeeper position and the ball position in frame t (m), d_ref is the distance reference value (m), v_gk(t) is the goalkeeper movement speed in frame t (m / s), v_ref is the goalkeeper movement speed reference value (m / s), and α is the correction sensitivity coefficient) A sports video analysis system characterized by setting the tanh term to 0 when F_strain(t) is less than F_th to maintain S₁(t) as S_0(t). Claim 6 In claim 5, the system further comprises a pressure sensor in which a plurality of pressure cells are arranged inside the glove of the goalkeeper to measure the contact pressure of the ball, the contact time, and the contact area calculated from the number of detections of the pressure cells, and the score calculation module comprises an efficiency index calculation part that calculates a save efficiency index according to the following mathematical formula 2 based on the contact pressure, contact time, and contact area of ​​the ball measured by the pressure sensor and the movement speed of the goalkeeper measured by the UWB sensor, and mathematical formula 2. (wherein E_save(t) is a save efficiency index in frame t (dimensionless), p(t) is the contact pressure of the ball in frame t (kPa), p_ref is a reference value of the contact pressure (kPa), A_contact(t) is the contact area of ​​the ball (cm²), A_ref is a reference value of the contact area (cm²), T_contact(t) is the contact time of the ball (ms), T_ref is a reference value of the contact time (ms), v_gk(t) is the movement speed of the goalkeeper measured by the UWB sensor in frame t (m / s), v_ref is a reference value of the goalkeeper movement speed (m / s)) A sports video analysis system further comprising a secondary correction part that performs a secondary correction of the primary corrected candidate score according to the following mathematical formula 3 based on the save efficiency index. Mathematical Formula 3. (Here, S₂(t) is the second-order corrected candidate score at frame t, S₁(t) is the first-order corrected candidate score at frame t, E_save(t) is the preemptive efficiency index calculated by Equation 2, and λ is the second-order correction sensitivity coefficient)

Images