Distribution device, distribution method, and program

Abstract

To provide a distribution device, distribution method, and program capable of distributing a video obtained by imaging a competition in a manner that makes it easy for beginners to understand the competition.SOLUTION: A distribution device comprises: a tracking unit that tracks, for each player, the position of the player on the basis of coordinates of the player in time-series frames of a video obtained by imaging players competing in an athletic field; a measurement unit that measures, for each player, the movement speed of the player on the basis of differences in the tracked positions of the players; an energy estimation unit that estimates, for each player, the energy exerted by the player on the basis of the movement speed of the player; a physical strength estimation unit that estimates, for each player, the remaining physical strength of the player on the basis of a cumulative value of the energy exerted by the player; an image processing unit that superimposes, on the time-series frames, a physical strength gauge image representing the remaining physical strength of each player in association with the player; and a communication unit that distributes the time-series frames with the physical strength gauge image superimposed thereon to a user terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a distribution device, a distribution method, and a program. [Background technology]

[0002] Patent Document 1 discloses a television station server and a judgment device that distributes video footage of sporting matches and provides users with the results of judging the merits and demerits of teams in the matches. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-168258 Summary of the Invention [Problem to be solved by the invention]

[0004] Some sports and publicly managed games are difficult for beginners to understand due to complex rules, etc. However, there is a problem in that it is not possible to distribute video footage of the games so that beginners can easily understand the games.

[0005] In view of the above circumstances, the present invention aims to provide a distribution device, distribution method, and program that are capable of distributing video footage of a sport so that even beginners can easily understand the sport. [Means for solving the problem]

[0006] One aspect of the present invention is a distribution device comprising: a tracking unit that tracks the position of one or more players based on the coordinates of the players in time-series frames of a video captured of the players competing in a stadium; a measurement unit that measures the movement speed of the players for each player based on the difference in the tracked positions of the players; an energy estimation unit that estimates the energy exerted by the players for each player based on the movement speed of the players; a physical strength estimation unit that estimates the remaining physical strength of the players for each player based on the cumulative value of the energy exerted by the players; an image processing unit that associates a physical strength gauge image representing the remaining physical strength of the players with each player and superimposes it on the time-series frames; and a communication unit that distributes the time-series frames on which the physical strength gauge image is superimposed to one or more user terminals.

[0007] One aspect of the present invention is a distribution method executed by a distribution device, the distribution method including: a tracking step of tracking the position of one or more players for each player based on the coordinates of the player in a time series of frames of a video captured of the players competing in a stadium; a measurement step of measuring the movement speed of the player for each player based on a difference in the tracked positions of the players; an energy estimation step of estimating the energy exerted by the player for each player based on the movement speed of the player; a stamina estimation step of estimating the remaining stamina of the player for each player based on a cumulative value of the energy exerted by the player; an image processing step of associating a stamina gauge image representing the remaining stamina of the player with the player and superimposing the stamina gauge image on the time series of frames; and a communication step of distributing the time series of frames with the stamina gauge image superimposed to one or more user terminals.

[0008] One aspect of the present invention is a program for causing a computer to execute the following steps: a tracking procedure for tracking the position of one or more players based on the coordinates of the players in time-series frames of a video captured of the players competing in a stadium; a measurement procedure for measuring the movement speed of the players for each player based on the difference in the tracked positions of the players; an energy estimation procedure for estimating the energy exerted by the players for each player based on the movement speed of the players; a stamina estimation procedure for estimating the remaining stamina of the players for each player based on the cumulative value of the energy exerted by the players; an image processing procedure for associating a stamina gauge image representing the remaining stamina of the players with each player and superimposing it on the time-series frames; and a communication procedure for distributing the time-series frames with the stamina gauge image superimposed to one or more user terminals. [Effects of the Invention]

[0009] According to the present invention, it is possible to distribute video footage of a sport so that even beginners can easily understand the sport. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a distribution system in a first embodiment. [Figure 2] FIG. 10 is a diagram showing an example of definition of the relative position of a player in the direction of travel in the first embodiment. [Figure 3] FIG. 10 is a diagram showing an example of definition of the relative positions of players in a direction perpendicular to the direction of travel in the first embodiment. [Figure 4] FIG. 2 is a diagram showing an example of the definition of coordinates of a stadium in the first embodiment. [Figure 5] FIG. 2 is a diagram showing an example of definition of coordinates of a player running along a lane in a rectangular area of ​​a stadium in the first embodiment. [Figure 6] FIG. 2 is a diagram showing an example of definition of coordinates of a player running along a lane in a semicircular area of ​​a stadium in the first embodiment. [Figure 7]FIG. 10 is a diagram illustrating an example of the formation effect in the traveling direction in the first embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of the formation effect in a direction perpendicular to the traveling direction in the first embodiment. [Figure 9] FIG. 10 is a diagram showing an example of past positions of other players in the first embodiment. [Figure 10] FIG. 10 is a diagram showing an example of the formation effect of a plurality of players in the first embodiment. [Figure 11] FIG. 10 is a diagram showing a first example of the formation effect between players in the first embodiment. [Figure 12] FIG. 10 is a diagram showing a second example of the formation effect between players in the first embodiment. [Figure 13] FIG. 3 is a diagram showing an example of a stamina gauge image in the first embodiment. [Figure 14] 5 is a flowchart illustrating an example of the operation of the distribution device in the first embodiment. [Figure 15] FIG. 2 is a diagram showing a first example of a display of a stamina gauge image in the first embodiment. [Figure 16] FIG. 10 is a diagram showing a second example of a display of a stamina gauge image in the first embodiment. [Figure 17] FIG. 10 is a diagram showing a third example of a display of a stamina gauge image in the first embodiment. [Figure 18] FIG. 10 is a diagram showing a fourth example of a display of a stamina gauge image in the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to the drawings. (First embodiment) FIG. 1 is a diagram showing an example of the configuration of a distribution system 1 in the first embodiment. The distribution system 1 is a system that distributes images (video) captured of athletes competing in a competition to a user terminal. The images may be moving images or one or more still images. The still images to be distributed may be one or more frames from a moving image (frames in time series) to be distributed. In the following, the competition is not limited to a specific sport (e.g., soccer) or a publicly managed race (e.g., bicycle racing). In the first embodiment, the competition is bicycle racing as an example.

[0012] The distribution system 1 includes an imaging device 2, a relay device 3, a communication line 4, and a distribution device 5. One or more user terminals 6 are connected to the communication line 4 so as to be able to communicate with the distribution device 5.

[0013] The distribution device 5 includes a communication unit 51, a storage device 52, a decoding unit 53, a tracking unit 54, a measurement unit 55, an energy estimation unit 56, a physical strength estimation unit 57, an image processing unit 58, and an encoding unit 59. The user terminal 6 includes a terminal communication unit 61, a terminal decoding unit 62, and a terminal display unit 63.

[0014] Some or all of the functional units of the distribution device 5 and the user terminal 6 are realized as software by a processor such as a CPU (Central Processing Unit) executing a program stored in a storage device having a non-volatile storage medium (non-transitory storage medium). The program may be recorded on a computer-readable storage medium. Examples of computer-readable storage media include portable media such as flexible disks, magneto-optical disks, ROMs (Read Only Memory), and CD-ROMs (Compact Disc Read Only Memory), and non-transitory storage media such as hard disks built into computer systems.

[0015] Some or all of the functional units of the distribution device 5 and the user terminal 6 may be realized using hardware including electronic circuits (electronic circuits or circuitry) using, for example, an LSI (Large Scale Integrated circuit), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array).

[0016] The imaging device 2 generates a time series of frames of a video in which one or more athletes (participating athletes) competing in a race at the racetrack 100 (velodrome). The frame rate of the video is, for example, 30 frames per second. The imaging device 2 captures images of each athlete, for example, from before the start to after the finish. The imaging device 2 may capture images of the race guides as well as each keirin athlete. Multiple imaging devices 2 may be installed at the racetrack 100. The angle of view of the frame (the captured range) can be estimated, for example, based on parameters of the imaging device 2 (for example, the orientation and imaging magnification of the imaging device 2).

[0017] The relay device 3 transmits time-series frames of the captured video to the communication unit 51 using a communication line 4. The communication line 4 is, for example, the Internet. The distribution device 5 is a device that distributes images (for example, video (time-series frames)) captured of athletes competing in a competition to a user terminal 6.

[0018] The communication unit 51 acquires time-series frames of the captured moving image from the relay device 3. The time-series frames of the moving image acquired from the relay device 3 have been subjected to compression encoding processing in advance by the relay device 3. The communication unit 51 records the acquired time-series frames of the moving image in the storage device 52.

[0019] The communication unit 51 acquires time-series frames on which the stamina gauge image of each player is superimposed from the image processing unit 58 or the encoding unit 59. The time-series frames on which the stamina gauge image is superimposed have been subjected to compression encoding processing in advance by the encoding unit 59. The communication unit 51 distributes the time-series frames on which the stamina gauge image is superimposed to one or more user terminals 6.

[0020] The decoding unit 53 performs a decoding process on the time-series frames of the video that have been subjected to the compression encoding process. The decoding unit 53 outputs the time-series frames of the video to the tracking unit 54. The tracking unit 54 tracks the position of each player based on the coordinates of the player in the time-series frames of the video in which one or more players competing in the stadium 100 are captured.

[0021] The measurement unit 55 measures the movement speed of each player based on the tracked position of the player. For example, the measurement unit 55 measures the movement speed of the player based on the frame rate and the difference between the position in the stadium 100 corresponding to the coordinates of the player captured in the nth frame and the position in the stadium 100 corresponding to the coordinates of the player captured in the (n-1)th frame.

[0022] The energy estimation unit 56 estimates the energy exerted by each athlete based on the athlete's moving speed. The energy estimation unit 56 estimates the energy of each resistance force element acting on the moving object (= athlete + bicycle) when the moving object moves. The energy (driving energy) exerted by the athlete is used to move the moving object (= athlete + bicycle). Four types of resistance forces act on a moving moving object: acceleration resistance force, climbing resistance force, air resistance force, and rolling resistance force. Therefore, if the energy of each of these four types of resistance forces can be estimated, the energy exerted by the athlete can be estimated.

[0023] At the speed of the moving object (= cyclist + bicycle) in a Keirin race, the Reynolds number of the air around the moving object is relatively large, so it is thought that the inertial resistance force outweighs the viscous resistance force among the air resistance forces. Therefore, in the following, as an example, we will assume that the inertial resistance force, which is proportional to the square of the moving object's speed, is the main air resistance force acting on the moving object.

[0024] If the speed of a moving object is constant, the energy of the acceleration resistance force is zero. The energy of the acceleration resistance force can be estimated using an equation that represents the physical law of kinetic energy. For example, the energy of the acceleration resistance force can be estimated based on the difference between the speed of the athlete captured in the nth frame and the speed of the athlete captured in the (n-1)th frame. The mass of the moving object is determined in advance based on, for example, the weight listed in the athlete's profile and the weight listed in the bicycle specifications.

[0025] When a moving object is moving on a flat surface, the energy of the climbing resistance force (the force resisting gravity) is 0. The energy of the climbing resistance force can be estimated using an equation that represents the physical law of potential energy. For example, it is possible to estimate the energy of the climbing resistance force based on the difference between the potential energy of a player captured in the nth frame and the potential energy of that player captured in the (n-1)th frame.

[0026] The energy of air resistance can be estimated using an equation that expresses the physical laws of air resistance. The air density and the frontal projection area (running posture) of the moving object may each be a constant value that is predetermined as a parameter used in estimating the energy of air resistance. The energy estimation unit 56 may estimate the energy of air resistance taking into account the formation effect (wind-shielding effect). For example, the energy estimation unit 56 determines whether other athletes are running in a low-pressure space that has occurred directly behind a running athlete, based on the relative positions of the athletes. The energy estimation unit 56 reflects this determination result in the process of estimating the energy of air resistance.

[0027] A low-pressure space that occurs directly behind a moving object creates a suction force, and other moving objects chasing that object can be pulled into that low-pressure space. This effect is called the formation effect (slipstream). In Keirin, the method in which riders ride in formation is called "drafting." By riding in formation, the following riders experience less air resistance than normal. When a rider rides directly behind another rider, the effect of reducing the energy of air resistance is greatest. The further a rider is from directly behind another rider, the smaller the effect of reducing the energy of air resistance that the rider experiences.

[0028] The energy estimation unit 56 determines whether or not there is another player ahead of the player based on the coordinates of each player. If there is another player ahead of the player, the energy estimation unit 56 estimates the direction and distance of the other player relative to the player. Here, to determine whether or not the direction in which the other player is located is ahead of the player, the direction of travel of each player in the stadium 100 needs to be determined in advance. Therefore, by defining the general shape (approximate shape) of the stadium 100, the direction of travel of each player in the time-series frames is defined. Furthermore, the energy estimation unit 56 estimates the formation effect of the players based on the direction and distance of the other players.

[0029] The energy of the rolling resistance force can be estimated using an equation that represents the physical law of friction resistance force between the lane (road surface) of the stadium 100 and the tires. The friction coefficient due to the materials of the road surface and the tires may be a predetermined constant value. The energy estimation unit 56 estimates the energy of the rolling resistance force for each athlete.

[0030] The physical strength estimation unit 57 estimates the amount of physical strength consumed per predetermined time based on the energy exerted by the player. For example, the physical strength estimation unit 57 estimates the amount of physical strength consumed per 1 / 30th of a second (the display time of one frame) based on the frame rate of the video. The physical strength estimation unit 57 estimates the remaining physical strength (amount of remaining energy) of each player based on the cumulative value of the energy exerted by the player (amount of physical strength consumed per hour). Here, the physique and basic physical strength of each player are determined in advance based on the profile and past performance of each player.

[0031] A rider's remaining stamina (remaining energy) is expressed as the result of subtracting the cumulative energy exerted by the rider from his / her maximum stamina (MAX HP). Maximum stamina is determined for each rider based on various factors, such as past performance and tactics. Generally, the better a rider's performance, the stronger his / her maximum stamina. Furthermore, because keirin is based on the concept of teams, a rider's maximum stamina (maximum energy) varies depending on his / her role on the team. Generally, younger riders have relatively strong maximum stamina, so they ride at the front of the pack, cutting through the wind. Veteran riders have relatively weaker maximum stamina, so they ride at the back of the pack rather than at the front, in order to conserve their remaining stamina. Veteran riders also make full use of tactics (techniques), such as blocking other teams.

[0032] The image processing unit 58 associates a stamina gauge image, which indicates the remaining stamina of the player, with the player and superimposes it on the time-series frames. For example, the image processing unit 58 superimposes the stamina gauge image of the player near the image of the player in the time-series frames. The encoding unit 59 performs a compression encoding process on the time-series frames of the video on which the stamina gauge image is superimposed. The encoding unit 59 outputs the time-series frames of the video on which the compression encoding process has been performed to the communication unit 51.

[0033] Next, the distribution device 5 will be described in detail. The energy estimation unit 56 estimates the energy of each resistance force, including acceleration resistance force, climbing resistance force, air resistance force, and rolling resistance force, at a predetermined period (for example, frame period) based on an equation that represents the physical law of each resistance force.

[0034] When the moving object is accelerating, the acceleration resistance force is a positive value, when the moving object is moving at a constant velocity the acceleration resistance force is 0, and when the moving object is decelerating the acceleration resistance force is a negative value. The energy estimation unit 56 estimates the energy of the acceleration resistance force at the time of the nth frame based on the difference between the kinetic energy at the time of the nth frame and the kinetic energy at the time of the (n-1)th frame. Here, the energy estimation unit 56 estimates the energy of the acceleration resistance force at the time of the nth frame based on the mass "m" of the moving object (= athlete + bicycle) and the velocity "V" at the time of the nth frame. sn ” and the velocity at the time of the (n-1)th frame “V sn-1 The energy of the acceleration resistance force at the time of the nth frame is estimated based on the above. The mass of the player in the estimation process is determined in advance based on the weight recorded in the player's profile.

[0035] The energy estimation unit 56 estimates the energy of the climbing resistance force at the time of the nth frame based on the difference between the potential energy at the time of the nth frame and the potential energy at the time of the (n-1)th frame. The energy estimation unit 56 estimates the energy of the rolling resistance force at the time of the nth frame and the energy of the air resistance force at the time of the nth frame in a frame cycle. Here, the energy estimation unit 56 first estimates the energy of the air resistance force without taking the formation effect into consideration, and then estimates the final energy of the air resistance force based on the energy of the air resistance force thus estimated, taking the formation effect into consideration.

[0036] Regarding kinetic energy, the mass "m" (unit: kg) corresponding to the athlete's body weight and bicycle weight is set to a constant value as a numerical value associated with the athlete. s (unit: m / s) is set to a constant value based on the environment. s" (unit: m / s) is determined for each player based on the measurement results of the speed of the moving object. The symbol "n" represents the frame number in the time series. The kinetic energy is estimated, for example, to one decimal place. The energy estimation unit 56 estimates the kinetic energy based on equation (1). The energy estimation unit 56 may use the moving average of the kinetic energy in the most recent 30 frames (1 second) as the kinetic energy in the current frame.

[0037]

number

[0038] Regarding potential energy, the mass "m" (unit: kg) corresponding to the athlete's body weight and bicycle weight is set to a constant value as a numerical value associated with the athlete. The coefficient of gravitational acceleration "g" (unit: m / s 2 ) is set to a constant value of "9.8". The coordinates (x, y, z) are the measurement results of the coordinates of the player in the stadium 100. Here, "z" represents the coordinate in the height direction (unit: m). The velocity "V s " (unit: m / s) is the measurement result of the speed of the moving object (= athlete + bicycle). The symbol "n" represents the frame number in the time series. The potential energy is estimated, for example, to one decimal place. The energy estimation unit 56 estimates the potential energy based on equation (2). The energy estimation unit 56 may use the moving average of the potential energy in the most recent 30 frames (1 second) as the potential energy in the current frame.

[0039]

number

[0040] Regarding rolling resistance, the mass "m" (unit: kg) corresponding to the athlete's body weight and bicycle weight is set to a constant value as a numerical value associated with the athlete. The coefficient of gravitational acceleration "g" (unit: m / s 2 ) is set to a constant value of "9.8". The rolling resistance coefficient "U r" is set to a constant value of "0.004". The coordinates (x, y, z) are the measurement results of the coordinates of the player in the stadium 100. The symbol "n" represents the frame number in the time series. The energy of the rolling resistance force (the energy of the driving force of the moving body) is the velocity "V" of the moving body per display time of one frame. s " is the rolling resistance "U r mg". The energy of the rolling resistance force is estimated, for example, to one decimal place. The energy estimation unit 56 estimates the energy of the rolling resistance force based on equation (3). Here, the energy estimation unit 56 may take the moving average of the energy of the rolling resistance force in the most recent 30 frames (1 second) as the energy of the rolling resistance force in the current frame.

[0041]

number

[0042] Regarding air resistance (excluding the formation effect), the mass "m" (unit: kg) corresponding to the athlete's body weight and bicycle weight is set to a constant value as a numerical value associated with the athlete. The coefficient of air resistance "C d ", the frontal projection area of ​​the moving object "A", the air density "R", and the wind speed "v s (unit: m / s) is set to a constant value. s " (unit: m / s) is the measurement result of the speed of the moving object (= rider + bicycle). The symbol "n" represents the frame number in the time series. In keirin, the inertial resistance force, which is proportional to the square of the speed, acts on the moving object as the main air resistance force, so the air resistance force is expressed as the speed "V s " and wind speed "v s " Addition result "V s +v s " to the square of the air resistance coefficient "C d " is proportional to the product of the air resistance force "R" and the air density "R" multiplied by the frontal projection area "A" of the moving object. The energy of the air resistance force is estimated, for example, to one decimal place. The energy estimation unit 56 estimates the air resistance force (excluding the formation effect) based on equation (4).

[0043]

number

[0044] The energy of air resistance (excluding the formation effect) is calculated by multiplying the energy of air resistance (excluding the formation effect) by the velocity of the moving object (= athlete + bicycle) per frame of display time, V s The energy estimation unit 56 estimates the energy of the air resistance force (excluding the formation effect) based on the formula (5).

[0045]

number

[0046] FIG. 2 is a diagram (bird's-eye view) showing an example of defining the relative positions of players in the direction of travel in the first embodiment. FIG. 3 is a diagram (bird's-eye view) showing an example of defining the relative positions of players in the direction perpendicular to the direction of travel in the first embodiment. The formation effect is the effect of a low-pressure space (air current) occurring directly behind a moving object traveling at the front of the formation reducing the air resistance force on a moving object at the rear of the formation. When a low-pressure space occurs directly behind a moving object, surrounding moving objects are drawn into that low-pressure space.

[0047] The measurement unit 55 measures the distance (relative distance) from the player to other players in the player's direction of movement "X." Furthermore, the measurement unit 55 measures the distance (relative distance) from the player to other players in the direction "Y" (horizontal direction) perpendicular to the player's direction of movement. The energy estimation unit 56 estimates the formation effect that the player receives from other players based on the distance from the player to the other players. Furthermore, the energy estimation unit 56 estimates the formation effect that the player receives from all other players based on the estimated results of the formation effect that each other player has on the player.

[0048] Next, the method for estimating the platoon effect (slipstream) will be described in detail. (A) Definition of the general shape (approximate shape) of the stadium (definition of the direction of movement of athletes) 4 is a diagram (bird's-eye view) showing an example of how coordinates of a stadium are defined in the first embodiment. In order to measure the distance in the player's forward direction and the distance to the player's side, it is necessary to measure the direction in which the player is moving in the stadium 100 based only on the coordinates in the captured frame. Therefore, the shape of the lanes of the stadium 100 is defined as being divided into straight lines and semicircles (corners).

[0049] In the linear region, the moving objects (= athletes and bicycles) move in a linear direction. In the semicircular region, the moving objects (= athletes and bicycles) move in a tangent direction to the semicircle. The shape of the stadium 100 may be determined based on a transition curve.

[0050] Regarding the coordinates (x_a, y_a) of player 101 and the coordinates (x_b, y_b) of player 102, the y coordinate is in the negative direction on the home stretch side of the stadium 100. The y coordinate is in the positive direction on the back stretch side of the stadium 100. The shape of the stadium 100 is divided into two semicircular areas and a rectangular area (-40≦x≦40).

[0051] (B) Measurement of the distance from one player to another (distance in the direction of travel "v", distance in the direction perpendicular to the direction of travel "h" (horizontal)) 5 is a diagram (bird's-eye view) showing an example of the definition of coordinates of players running along a lane in a rectangular area of ​​a stadium in the first embodiment. The distance from player 101 to player 102 (another player) in the lane in the rectangular area is measured based on the difference in coordinates. One of player 101 or player 102 may run in the lane in the rectangular area, and the other may run in the lane in the semicircular area.

[0052]

number

[0053] 6 is a diagram (bird's-eye view) showing an example of the definition of the coordinates of a player running along a semicircular lane in a stadium in the first embodiment. When either player 101 or player 102 is running in a semicircular lane, the tangent direction of the semicircular lane is defined as the direction of travel. In this case, distance is measured by arithmetic operations using the dot product and cross product.

[0054]

number

[0055] (C) Estimate the energy of the formation effect based on each distance. The energy estimation unit 56 estimates the energy of the formation effect based only on the distance "v" in the direction of travel, and corrects the result based on the distance in the lateral direction. Basically, the longer the lateral distance "h" between one player and another player, the lower the formation effect.

[0056] 7 is a diagram (side view) showing an example of the formation effect in the direction of travel in the first embodiment. Even if player 102 is positioned ahead of player 101, if the distance in the direction of travel is less than 0.5 m, it is unlikely that player 101 will be in a position where the formation effect can be obtained, even if the length of the bicycle (bicycle body length) is taken into consideration, and there will be no formation effect in the direction of travel (slipstream(v) = 0, (v < 0.5)).

[0057] When the distance in the direction of travel is greater than 0.5 cm and less than 3 m, air resistance decreases to 50%, so the formation effectiveness in the direction of travel is 50% (= 0.5) (slipstream(v) = 0.5, (0.5 <= v < 3.0)). When the distance in the direction of travel is 3 m or more, the formation effectiveness in the direction of travel decreases gradually (slipstream(v) = 1 / 2 / exp(v-3)^(1 / 4) (v >= 3.0)).

[0058] The formation effect based only on the direction of travel is expressed as, for example, equation (8).

[0059]

number

[0060] For example, the platooning effectiveness in the direction of travel is 50% (maximum effect) at a distance of 0.5 vehicle lengths, 43% at a distance of 1 vehicle length in the direction of travel, 27% at a distance of 2 vehicle lengths in the direction of travel, and 10% at a distance of 4 vehicle lengths in the direction of travel.

[0061] FIG. 8 is a diagram (bird's-eye view) showing an example of the formation effect in the direction perpendicular to the direction of travel (lateral direction) in the first embodiment. When the lateral distance "h" is between 0 and 0.15 m, the formation effect in the direction of travel is 100%. Therefore, up to a distance of 0.15 m behind another player, it is considered to be the same as being directly behind that player (0 m from behind the other player).

[0062] When the lateral distance is 1m or more (h = 1.15m or more), there is no formation effect "slipstream(v)" in the direction of travel (slipstream(v, h) = 0, (h >= 1.0)). When the lateral distance is less than 1m (less than h = 1.15m), the longer the lateral distance, the less effective the formation in the direction of travel becomes (slipstream(v, h) = slipstream(v)*(1-h)^3, (0.0 <= h < 1.0)).

[0063] The formation effect in which the lateral direction is taken into account in addition to the formation effect in the direction of travel, that is, the formation effect in the direction perpendicular to the direction of travel (lateral direction), is expressed as, for example, equation (9).

[0064]

number

[0065] For example, when player 101 is positioned directly behind player 102, the horizontal formation effect is 50% (=0.5) of the formation effect in the forward direction (maximum formation effect). Even when player 101 is positioned just 0.1 m to the side of player 102, the horizontal formation effect is 50% (maximum effect). When player 101 is positioned just 0.25 m to the side of player 102, the horizontal formation effect is 45%. When player 101 is positioned 1.15 m or more to the side of player 102, the horizontal formation effect is 0% (no formation effect).

[0066] (D) Improvements to ensure accurate estimation of semicircular lane areas and their slopes. Even in cases where the players are far apart on a semicircular lane (corner), by taking into consideration not only the coordinates of player 102 at the first time relative to the coordinates of player 101 at the first time, but also the coordinates of player 102 (another player) at a second time that is earlier than the first time, it is possible to compare the trajectory of the position of player 102 with the trajectory of the position of player 101. In other words, it is possible to compare the relative positions of the players in the past.

[0067] FIG. 9 is a diagram (bird's-eye view) showing an example of the past positions of other players in the first embodiment. The tracking unit 54 tracks the position of each player for multiple frames including past frames in a time series. In FIG. 9, the player 102 designated by the reference symbol "102-(n-4)" represents the player 102 four frames in the past relative to the player 102 designated by the reference symbol "102-n."

[0068] The measurement unit 55 measures, for example, the distance (v0, v1, v2, v3, v4) in the direction of travel from the player 101 to the player 102 for a total of five frames from the present (nth frame) to the past ((n-4)th frame). The measurement unit 55 measures the distance (h0, h1, h2, h3, h4) in the lateral direction from the player 101 to the player 102 for these five frames.

[0069] (E) Estimate the formation effect of all other players on a player based on the formation effect of each other player (integrated processing). FIG. 10 is a diagram (bird's-eye view) showing an example of the formation effect of multiple players in the first embodiment. Even when the players in the formation are lined up in a single file, the formation effect decreases the further back in the formation. The formation effect of players other than the lead player is uniformly reduced by about 50%, with no significant difference between players. The energy estimation unit 56 estimates the energy of the air resistance force, including the formation effect, that a player receives from all other players for each frame (time point), based on the energy of the air resistance force, including the formation effect, that a player receives from each of the other players.

[0070] 11 is a diagram (bird's-eye view) showing a first example of the formation effect between players in the first embodiment. In this example, the formation effect decreases by 50% or more when a player 101 is surrounded by multiple players 102.

[0071] FIG. 12 is a diagram (bird's-eye view) showing a second example of the formation effect between riders in the first embodiment. In Keirin, it is unlikely that a rider 101 will be surrounded by multiple riders 102. Therefore, when multiple riders 102 are causing a formation effect on a rider 101, the maximum formation effect among the formation effects of the multiple riders 102 is adopted as the integrated value of the formation effect received by the rider 101 (slipstream_total = max(slipstream(v, h))). The energy estimation unit 56 multiplies the energy of the air resistance force (excluding the formation effect) by the formation effect coefficient "1 - slipstream_total" (unit: dimensionless).

[0072] The energy estimation unit 56 estimates the energy of the formation effect that the player 101 receives from the current and past positions of the player 102. For example, the energy estimation unit 56 estimates the energy of the formation effect based on a total of five frames, the current (0th frame) and past (1st to 4th frames), based on the distances in the forward direction (v0, v1, v2, v3, v4) based on a total of five frames and the distances in the lateral direction (h0, h1, h2, h3, h4) based on a total of five frames.

[0073] Here, when the energy estimation unit 56 estimates the energy of the formation effect based on the distance "v" in the direction of travel based on past frames of the player 102, if any of the distances from "v1" to "v4" (for example, "v3") is a negative value, the energy estimation unit 56 estimates that there is no formation effect (0%), as shown in equation (10), even if the distance "v0" is a positive value.

[0074]

number

[0075] The energy of the platoon effect (slipstream(v3)) estimated based on, for example, distance "v3" in the direction of travel only may be corrected using the platoon effect based on the lateral distance "h3" as exemplified in equation (11).

[0076]

number

[0077] When multiple players 102 are causing a formation effect on player 101, the maximum formation effect among the formation effects of those multiple players 102 is adopted as the integrated value of the formation effect received by player 101, as shown in equation (12).

[0078]

number

[0079] In this way, the energy estimation unit 56 estimates the energy (amount of physical energy consumed) exerted by the athlete for each frame (time). The physical strength estimation unit 57 subtracts the energy estimated for each frame from the athlete's maximum physical strength (maximum energy) as time passes. The result of this subtraction is displayed on the terminal display unit 63 as a physical strength gauge image. Here, the maximum physical strength is determined in advance based on past performance such as race scores (for example, results over the past four months).

[0080] FIG. 13 is a diagram showing an example of a stamina gauge image (life gauge image) in the first embodiment. The stamina gauge image is an image that shows what percentage of the remaining stamina (energy) corresponds to, with maximum stamina (maximum energy) being 100%. The length of the stamina gauge image may be normalized so that it is the same length for all riders. A standard maximum stamina may be determined for each rider with reference to measurements from past Keirin races. The difference in maximum stamina between riders may be determined with reference to race results.

[0081] Even after a Keirin race, riders have not yet fully utilized their maximum physical strength. Therefore, the maximum physical strength of each rider may be estimated based on their performance in past Functional Threshold Power (FTP) tests and past races.

[0082] The maximum physical strength (MAX HP) of a player is expressed using the deviation value of the player's maximum physical strength, for example, as shown in equation (13). Here, the deviation value of the maximum physical strength is expressed as shown in equation (14).

[0083]

number

[0084]

number

[0085] Next, an example of the operation of the distribution device 5 will be described. FIG. 14 is a flowchart showing an example of the operation of the distribution device 5 in the first embodiment. The tracking unit 54 tracks the position of each player based on the coordinates of the player in time-series frames of the video in which the player is captured (step S101). The measurement unit 55 measures the movement speed of each player based on the difference in the player's position (step S102). The energy estimation unit 56 estimates the energy exerted by each player based on the player's movement speed (step S103). The physical strength estimation unit 57 estimates the remaining physical strength of each player based on the cumulative value of the energy exerted by the player (step S104). The image processing unit 58 superimposes a physical strength gauge image on the time-series frames in association with the player (step S105). The communication unit 51 distributes the time-series frames on which the physical strength gauge image is superimposed to the user terminal 6 (step S106).

[0086] As described above, the tracking unit 54 tracks the position of each player based on the coordinates of the player in time-series frames of a video in which one or more players are competing in a competition (a race, a soccer match, etc.) in the stadium 100. The measurement unit 55 measures the movement speed of each player based on the difference in the tracked positions of the player. The energy estimation unit 56 estimates the energy exerted by each player based on the movement speed of the player. The physical strength estimation unit 57 estimates the remaining physical strength of each player based on the cumulative value of the energy exerted by the player. The image processing unit 58 associates a physical strength gauge image representing the remaining physical strength of the player with each player and superimposes it on the time-series frames. The communication unit 51 distributes the time-series frames on which the physical strength gauge image is superimposed to one or more user terminals 6.

[0087] In this way, the stamina gauge image is associated with each player and superimposed on a time-series frame of the video. This makes it possible to distribute video footage of the sport so that even beginners can easily understand the sport. Because the player's stamina is estimated based on the video footage, there is no need for the player to wear a wearable device to measure their stamina, and this does not affect the player's performance.

[0088] The image processing unit 58 may superimpose a stamina gauge image near the image of the player in the time-series frames. The image processing unit 58 may superimpose a stamina gauge image in a color associated with the player on the time-series frames.

[0089] The physical strength estimation unit 57 may estimate the maximum physical strength of the player based on at least one of the player's physique, weight, and past performance. The physical strength estimation unit 57 may estimate the remaining physical strength of the player by subtracting the cumulative value of the energy exerted by the player from the player's maximum physical strength.

[0090] The tracking unit 54 may track the position of each player based on the coordinates of the player in time-series frames of a video in which multiple players are captured. The energy estimation unit 56 may estimate the air resistance force for each player based on the positions of multiple players in the formation and the movement speed of the player. The energy estimation unit 56 may estimate the energy exerted by the player based at least on the air resistance force.

[0091] The tracking unit 54 may divide the stadium 100 into a plurality of areas according to the shape of the stadium 100, and track the positions of the players for each divided area. The pattern in which the tracking unit 54 divides the stadium 100 is not limited to a specific pattern. The tracking unit 54 may divide the stadium 100 into, for example, semicircular areas and rectangular areas.

[0092] Next, a display example of a stamina gauge image (HP (Hit Point) gauge image) will be described. FIG. 15 is a diagram showing a first example of the display of a stamina gauge image in the first embodiment. The terminal communication unit 61 acquires the distributed video (video of a bicycle race) from the communication unit 51. The terminal decoding unit 62 decodes the video distributed from the communication unit 51. The terminal display unit 63 displays the video distributed from the communication unit 51. The image processing unit 58 superimposes (adds) a stamina gauge image near an image of a player in a time-series frame. In FIG. 15, the image processing unit 58 superimposes a stamina gauge image 201 near (within a predetermined distance from) a player who is previously associated with the stamina gauge image 201. If the player is at the front of the line, the image processing unit 58 may include the character string "It's windy" in the stamina gauge image 201.

[0093] Fig. 16 is a diagram showing a second example of display of a stamina gauge image in the first embodiment. In Fig. 16, the image processing unit 58 superimposes a stamina gauge image 201 near the player who has been previously associated with the stamina gauge image 201. The image processing unit 58 superimposes a stamina gauge image 202 near the player who has been previously associated with the stamina gauge image 202. The image processing unit 58 superimposes a stamina gauge image 203 near the player who has been previously associated with the stamina gauge image 203. The image processing unit 58 may include the character string "60%" in the stamina gauge image 203 of a player whose stamina gauge is 60% or less.

[0094] Fig. 17 is a diagram showing a third example of display of a stamina gauge image in an embodiment. In Fig. 17, the image processing unit 58 superimposes (adds) a stamina gauge image 204 near the player who has been previously associated with the stamina gauge image 204. The image processing unit 58 superimposes a stamina gauge image 205 near the player who has been previously associated with the stamina gauge image 205. The image processing unit 58 superimposes a stamina gauge image 206 near the player who has been previously associated with the stamina gauge image 206. The image processing unit 58 may include the character string "Conserving OK" in the stamina gauge image 204 of a player whose stamina gauge exceeds 60%. The same applies to the stamina gauge image 205 and the stamina gauge image 206.

[0095] Fig. 18 is a diagram showing a fourth example of display of a stamina gauge image in the first embodiment. In Fig. 18, the image processing unit 58 superimposes a stamina gauge image 207 near the player previously associated with the stamina gauge image 207. The image processing unit 58 superimposes a stamina gauge image 208 near the player previously associated with the stamina gauge image 208. If the distance between the player previously associated with the stamina gauge image 207 and the player previously associated with the stamina gauge image 208 is less than a threshold and the difference between the stamina gauges (remaining stamina) of these players is equal to or greater than a predetermined difference, the image processing unit 58 may include the character string "CHANCE!!" in the stamina gauge image 208.

[0096] (Second embodiment) In track events, athletes move along tracks (lanes) that are defined along the outer edge of a stadium (field). The second embodiment differs from the first embodiment mainly in that the event is not a track event. The second embodiment will be described focusing on the differences from the first embodiment.

[0097] As mentioned above, the competition is not limited to a specific sport or public competition. The competition may be other than track events, such as soccer, as mentioned above. The tracking unit 54 may divide a rectangular stadium into, for example, two rectangular areas, or into, for example, four triangular areas.

[0098] In soccer, fouls are more likely to occur when multiple players are crowded together in a small area around the edge of the field or near the goalposts, competing for the ball. Therefore, the more densely packed the players are in such a small area of ​​the field, the more likely they are to tire. Furthermore, the longer the players are packed together, the more likely they are to tire.

[0099] In this way, not only the elements of a player's movement distance and movement speed (air resistance), but also other elements have a significant impact on a player's remaining physical strength (fatigue level). Therefore, the tracking unit 54 may track the positions of players moving within each divided area of ​​a soccer or other sports field. The divided areas may be weighted. Small areas near the outer edge of the sports field or the goal posts may be weighted relatively more heavily than other areas.

[0100] The physical strength estimation unit 57 may estimate the remaining physical strength of the player for each divided area based on at least one of the player's moving distance, the player's moving distance, the density of multiple players, and a weighting (coefficient) set for the divided area.

[0101] The physical strength estimating unit 57 may make the cumulative value of energy (amount of physical strength consumed per hour) exerted by players in areas with high weighting (for example, small areas close to the goal posts) greater than the cumulative value of energy exerted by players in areas with low weighting. The physical strength estimating unit 57 may make the cumulative value of energy (amount of physical strength consumed per hour) exerted by players in areas with high player density greater than the cumulative value of energy exerted by players in areas with low player density. Furthermore, the physical strength estimating unit 57 may make the cumulative value of energy (amount of physical strength consumed per hour) exerted by players who spend a long time crowded together greater than the cumulative value of energy exerted by players who spend a short time crowded together.

[0102] As described above, the physical strength estimation unit 57 estimates the remaining physical strength of the player for each divided area based on at least one of the player's moving distance, the player's moving distance, the density of multiple players, and the weighting (coefficient) set for the divided area.

[0103] This makes it possible to distribute video footage of sports other than track events so that beginners can easily understand them. Based on the video footage, the physical strength of the athletes is accurately estimated by, for example, weighting each divided area, eliminating the need for athletes to wear wearable devices to measure their physical strength, and preventing any impact on the athletes' performance.

[0104] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0105] 1...Distribution system, 2...Imaging device, 3...Relay device, 4...Communication line, 5...Distribution device, 6...User terminal, 51...Communication unit, 52...Storage device, 53...Decoding unit, 54...Tracking unit, 55...Measurement unit, 56...Energy estimation unit, 57...Stamina estimation unit, 58...Image processing unit, 59...Encoding unit, 61...Terminal communication unit, 62...Terminal decoding unit, 63...Terminal display unit, 100...Stadium, 101...Athlete, 102...Athlete, 201...Stamina gauge image, 202...Stamina gauge image, 203...Stamina gauge image, 204...Stamina gauge image, 205...Stamina gauge image, 206...Stamina gauge image, 207...Stamina gauge image, 208...Stamina gauge image

Claims

1. a tracking unit that tracks the position of each of one or more athletes competing in a stadium based on coordinates of the athletes in time-series frames of a video in which the athletes are captured; a measuring unit that measures the movement speed of each player based on the difference in the tracked positions of the players; an energy estimation unit that estimates the energy exerted by each player based on the movement speed of the player; a physical strength estimation unit that estimates the remaining physical strength of each player based on the cumulative value of energy exerted by the player; an image processing unit that associates a stamina gauge image representing the remaining stamina of the player with the player and superimposes the image on the time series of frames; a communication unit that distributes the time-series frames on which the stamina gauge image is superimposed to one or more user terminals; A distribution device comprising:

2. The distribution device according to claim 1 , wherein the image processing unit superimposes the stamina gauge image near an image of the player in the time-series frames.

3. The distribution device according to claim 2 , wherein the image processing unit superimposes the stamina gauge image in a color associated with the player on the time-series frames.

4. 2. The distribution device according to claim 1, wherein the physical strength estimation unit estimates the maximum physical strength of the player based on at least one of the player's physique, weight, and past performance, and estimates the remaining physical strength of the player by subtracting a cumulative value of energy exerted by the player from the player's maximum physical strength.

5. the tracking unit tracks the position of each player based on coordinates of the player in the time-series frames of a video in which a plurality of players are captured; 2. The distribution device according to claim 1, wherein the energy estimation unit estimates air resistance energy for each of the players based on the positions of the players constituting the formation and the movement speeds of the players, and estimates energy exerted by the players based on at least the air resistance energy.

6. The distribution device according to claim 1 , wherein the tracking unit divides the stadium into a plurality of areas according to a shape of the stadium, and tracks the position of the player for each divided area.

7. The distribution device according to claim 1 , wherein the tracking unit divides the stadium into semicircular and rectangular areas.

8. A distribution method executed by a distribution device, a tracking step of tracking the positions of one or more athletes playing in a stadium based on coordinates of the athletes in time-series frames of a video captured of the athletes; a measuring step of measuring a moving speed of each player based on a difference in the tracked positions of the players; an energy estimation step of estimating the energy exerted by each player based on the movement speed of the player; a physical strength estimation step of estimating the remaining physical strength of each of the players based on the cumulative value of the energy exerted by the players; an image processing step of superimposing a stamina gauge image representing the remaining stamina of the player on the time series frames in association with the player; a communication step of distributing the time-series frames on which the stamina gauge image is superimposed to one or more user terminals; Delivery methods including.

9. On the computer, a tracking step of tracking the positions of one or more athletes playing in a stadium based on coordinates of the athletes in time-series frames of a video captured of the athletes; a measuring step of measuring a moving speed of each player based on a difference in the tracked positions of the players; an energy estimation step of estimating, for each player, the energy exerted by the player based on the player's movement speed; a physical strength estimation step of estimating the remaining physical strength of each player based on the cumulative energy exerted by the player; an image processing step of superimposing a stamina gauge image representing the remaining stamina of the player on the time series frames in association with the player; a communication procedure for distributing the time-series frames on which the stamina gauge image is superimposed to one or more user terminals; A program to execute.

Images