Animation Creation Device

Abstract

To provide an animation creating device for creating animation that can be used for coaching a player. [Solution] An animation creation device acquires video data generated by filming a game. The animation creation device includes a teacher data storage unit that stores teacher data. The animation creation device detects the positions of multiple players and the ball in the video data. The animation creation device assigns identification information to each of the multiple players. The animation creation device creates images representing the following (a) to (c), and creates an animation using these images: (a) the positions of the multiple players and the ball; (b) the identification information assigned to the multiple players; and (c) the positions and identification information of the multiple players included in the teacher data.

Description

[Technical Field]

[0001] The present disclosure relates to an animation creation device. [Background technology]

[0002] Patent Document 1 describes an event log generator. The event log generator detects events by analyzing video images generated by shooting a soccer match. Events include kicking the ball, tackling another player, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-41725 Summary of the Invention [Problem to be solved by the invention]

[0004] It is conceivable to use video data generated by filming a sport such as soccer to instruct players in a sport. In particular, it is desirable to instruct players so that they can move to the correct position depending on the situation of the sport. However, such instruction has been difficult because the video data does not show the correct position of the player.

[0005] In one aspect of the present disclosure, it is preferable to provide an animation creation device that creates animations that can be used to teach players. [Means for solving the problem]

[0006] One aspect of the present disclosure is an animation creation device, comprising: a video data acquisition unit configured to acquire video data generated by filming a game played on a field by a plurality of players using a ball;

[0007] a teacher data storage unit that stores, for each position of the ball on the field, multiple pieces of teacher data that associate the position of the ball on the field, the positions of the multiple players on the field, and identification information assigned to each of the multiple players.

[0008] a position detection unit configured to detect positions of the players and the ball in each of a plurality of frames included in the video data, and a position conversion unit configured to convert, in each of the plurality of frames, the positions of the players and the ball in the frame into positions of the players and the ball on the field.

[0009] a first identification information assignment unit configured to assign the identification information to each of the plurality of players in a specific frame that is any one of frames included in the video data; and a tracking unit configured to associate the same player detected in each of the plurality of frames by tracking.

[0010] A second identification information assignment unit configured to assign the identification information assigned to the player of the specific frame by the first identification information assignment unit to players in frames other than the specific frame who are associated with the player of the specific frame through the tracking.

[0011] an image creation unit configured to create an image representing the following (a) to (c) for each of the plurality of frames; and an animation creation unit configured to create an animation using the plurality of images created by the image creation unit.

[0012] (a) the positions of the players and the ball on the field; (b) the identification information assigned to the plurality of players; (c) In the training data, the positions of the multiple players associated with the position of the ball in (a), and the identification information assigned to each of the multiple players in the training data.

[0013] An animation creation device according to one aspect of the present disclosure can create animations that can be used to teach players. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a plan view showing a half court. [Figure 2] FIG. 1 is a block diagram showing a configuration of an animation creation device. [Figure 3] FIG. 2 is a block diagram showing the functional configuration of a calculation unit. [Figure 4] FIG. 2 is an explanatory diagram showing the contents of teacher data. [Figure 5] 10 is a flowchart showing a process executed by a calculation unit. [Figure 6] Fig. 6A is an explanatory diagram showing a process for detecting multiple players in a frame, and Fig. 6B is an explanatory diagram showing a process for detecting a ball in a frame. [Figure 7] Fig. 7A is an explanatory diagram showing a process for identifying the positions of the four corners of a half court in a frame, and Fig. 7B is an explanatory diagram showing a process for mapping the positions of the four corners in a drawing of a half court seen from directly above. [Figure 8] FIG. 2 is an explanatory diagram illustrating an image created by an image creating unit. DETAILED DESCRIPTION OF THE INVENTION

[0015] Exemplary embodiments of the present disclosure will be described with reference to the drawings. First Embodiment 1. Configuration of animation creation device 1 The configuration of the animation creation device 1 will be described with reference to Figs. 1 to 4. The animation creation device 1 is a device that creates animation from video data generated by filming a soccer match or practice. Soccer is a sport played on a field by multiple players using a ball.

[0016] The video data was generated by capturing images of a half court 101 shown in Fig. 1, which is a field where soccer is played. The camera that generates the video data is located outside the half court 101, at a certain height.

[0017] 2, the animation creation device 1 includes a calculation unit 3 and a teacher data storage unit 5. The calculation unit 3 is connected to a cloud 7 and the teacher data storage unit 5. The calculation unit 3 includes a microcomputer having a CPU 11 and a semiconductor memory such as RAM or ROM (hereinafter referred to as memory 13).

[0018] Each function of the calculation unit 3 is realized by the CPU 11 executing a program stored in a non-transitory physical recording medium. In this example, the memory 13 corresponds to the non-transitory physical recording medium storing the program. Furthermore, the execution of this program results in the execution of a method corresponding to the program. The calculation unit 3 may include one microcomputer or multiple microcomputers.

[0019] As shown in FIG. 3, the calculation unit 3 functionally comprises a video data acquisition unit 21, a frame extraction unit 23, a position detection unit 25, a position conversion unit 27, a first identification information assignment unit 29, a tracking unit 31, a second identification information assignment unit 33, an image creation unit 35, and an animation creation unit 37.

[0020] As shown in FIG. 2, the teacher data storage unit 5 stores a plurality of pieces of teacher data 41. As shown in FIG. 4, the teacher data 41 is data that associates the position of the ball 109 on the half court 101, the positions of multiple players 107 on the half court 101, and identification information assigned to each of the multiple players 107. In this embodiment, there are eight players 107. The teacher data storage unit 5 stores a plurality of pieces of teacher data 41 that differ for each position of the ball 109.

[0021] For example, the teacher data 41 at the top in Fig. 4 is the teacher data 41 when the position of the ball 109 is "6-I". Also, the teacher data 41 second from the top in Fig. 4 is the teacher data 41 when the position of the ball 109 is "6-H".

[0022] In the teacher data 41 at the top of Figure 4, the player 107 at position "3-C" is associated with the identification information "1". In addition, in the teacher data 41, the player 107 at position "7-D" is associated with the identification information "2". For example, the teacher data 41 can be created by someone with extensive knowledge of soccer.

[0023] The position of the ball 109 and the position of the player 107 on the half court 101 are represented by positions in the X and Y directions shown in FIG. 1. In the X direction, the half court 101 is divided into rows numbered 1 to 15. In the Y direction, the half court 101 is divided into columns numbered A to I. The half court 101 is divided into 135 grids 102 by the rows in the X direction and the columns in the Y direction. A position on the half court 101 is represented by which grid 102 the player is in. For example, a position represented by "3-C" is a position in the grid 102 whose row in the X direction is "3" and whose column in the Y direction is "C" as shown in FIG. 1.

[0024] One piece of identification information is assigned to each of the multiple players 107. The identification information is different for each player 107. The identification information assigned to any one player 107 does not change over time. In this embodiment, the identification information corresponds to the position of the player 107 in the game (for example, the far right forward, the central midfielder, the far left defender, etc.). The multiple players 107 assigned identification numbers 1 to 8 are players on the defensive team who are guarding the goal 103 in the half court 101.

[0025] 2. Processing performed by the animation creation device 1 The processing executed by the animation creating device 1 will be described with reference to FIGS. In step 1, the video data acquisition unit 21 acquires video data that has been uploaded to the cloud 7. The video data has 24 frames 105 per second. Note that the user can save the video data in the cloud 7 in advance.

[0026] In step 2, the frame extraction unit 23 extracts a portion of the frames 105 from all of the frames 105 of the video data. Specifically, all of the frames 105 of the video data are divided into groups of three consecutive frames 105. One frame 105 is then extracted from each group, and the other two frames 105 are deleted. As a result, the extracted video data has eight frames 105 per second.

[0027] In step 3, the position detection unit 25 detects the positions of the players 107 and the ball 109 in each of the frames 105 included in the video data extracted in step 2.

[0028] For example, as shown in Fig. 6A, the position detection unit 25 recognizes multiple players 107 in each frame 105 and detects the positions of the multiple players 107 in the frame 105. The position in the frame 105 is a position represented by coordinates in the vertical direction x and horizontal direction y of the frame 105, as shown in Fig. 6A.

[0029] The position detection unit 25 can distinguish between the players 107 and the players 108 of the offensive team attacking toward the goal 103 in the following manner. The position detection unit 25 recognizes the colors of all detected players and divides the players into two groups according to their colors. Of the two groups, the player in the group with the largest number of players in the half court 101 is designated as the player 107.

[0030] Since the uniforms, bibs, etc. of the two teams are usually different in color, the players can be divided into two groups based on color. One of the two groups is the offensive team attacking toward goal 103, and the other is the defensive team protecting goal 103. Furthermore, since the number of players on the defensive team is usually greater than the number of players on the offensive team in half court 101, it is highly likely that the players in the larger group are players on the defensive team (i.e., players 107).

[0031] Furthermore, the position detection unit 25 recognizes the ball 109 in each frame 105 and detects the position of the ball 109 in the frame 105, as shown in FIG. 6B.

[0032] In step 4, the positions of the multiple players 107 and the ball 109 are converted in each of the multiple frames 105. The conversion of the positions means converting the positions in the frames 105 into positions in the half court 101.

[0033] This transformation can be performed, for example, as follows: As shown in Fig. 7A, the positions of the four corners 111a to 111d of the half court 101 are identified in the frame 105. Next, as shown in Fig. 7B, the positions of the corners 111a to 111d are mapped in a drawing of the half court 101 as seen from directly above. Next, affine transformation and perspective transformation matrices are calculated. Using these matrices, the positions in the frame 105 can be transformed into positions on the half court 101.

[0034] In step 5, the first identification information assignment unit 29 acquires the first frame 105 (hereinafter referred to as a specific frame 105A) from the video data. The specific frame 105A is a frame 105 at the start of the competition or at the start of the second half of the competition. The first identification information assignment unit 29 assigns identification information to each of the multiple players 107 detected in the specific frame 105A.

[0035] At the start of the game or the start of the second half of the game, there is a high probability that each of the multiple players 107 will be in a position corresponding to the identification information. For example, at the start of the game or the start of the second half of the game, there is a high probability that a forward player 107 will be in a position further forward (i.e., a position corresponding to the forward's identification information) than other players 107. Therefore, there is a high probability that the correct identification information can be assigned to each of the multiple players 107 detected in the specific frame 105A, depending on the position of the player 107 in the specific frame 105A.

[0036] In step 6, the tracking unit 31 tracks the player 107 in the multiple frames 105, thereby associating the same player 107 detected in each of the multiple frames 105.

[0037] Tracking can be performed by a known method. For example, if a player 107 detected in a certain frame 105 and a player 107 detected in the next frame 105 are close in position, movement direction, etc., the tracking unit 31 determines that there is a high probability that those players 107 are the same, and associates them.

[0038] In step 7, the second identification information assignment unit 33 assigns identification information to the players 107 detected in frames 105 other than the specific frame 105A. Specifically, the following process is performed.

[0039] For example, suppose that identification information a is assigned to one player 107a detected in a specific frame 105A in step 5. The second identification information assignment unit 33 assigns the same identification information a to the players 107 detected in each frame 105 other than the specific frame 105A and associated with the player 107a in step 6. This process is performed for all players 107 and for all frames 105. By performing step 7, one identification information is assigned to the same player 107 throughout all of the multiple frames 105.

[0040] In step 8, the image creation unit 35 creates an image 113 shown in FIG. 8 for each of the multiple frames 105. The number of images 113 created is equal to the number of frames 105 extracted in step 2. One image 113 is created from any one frame 105. Any one image 113 is created using information obtained for the corresponding one frame 105 in the processes of steps 3 to 7. The image 113 is a drawing that represents the half court 101 as seen from directly above.

[0041] The image 113 displays as many real marks 115 as there are multiple players 107. Each real mark 115 corresponds to one player 107. The real mark 115 represents the real position of the corresponding player 107 and the identification information of that player 107. The real mark 115 is displayed at the position of the corresponding player 107 in the half court 101. The position of the real mark 115 is the position of the player 107 calculated in the processing of step 4. The numerical value included in the real mark 115 is the identification information assigned to the corresponding player 107 in step 5 or 7.

[0042] The real marks 115 represent the positions of the multiple players 107 on the half court 101 and identification information assigned to the multiple players 107. The image 113 displays teacher marks 117 in the same number as the multiple players 107. Each teacher mark 117 corresponds to one player 107. The numerical value included in the teacher mark 117 is identification information assigned to the corresponding player 107.

[0043] The image creation unit 35 displays the teacher mark 117 on the image 113 in the following manner. First, the image creation unit 35 reads out the teacher data 41 corresponding to the position of the ball 109 obtained by conversion in step 4 from the teacher data storage unit 5. For example, if the position of the ball 109 is "6-I," the image creation unit 35 reads out the teacher data 41 located at the top in FIG. 4.

[0044] Next, the image creation unit 35 displays teacher marks 117 in the image 113 at the positions of the multiple players 107 included in the read teacher data 41. For example, when the teacher data 41 at the top in FIG. 4 is read, teacher marks 117 are displayed at the positions "3-C," "7-D," "12-C," "4-E," "8-E," "13-E," "6-H," and "12-C."

[0045] Furthermore, each teacher mark 117 displays identification information associated with the position of that teacher mark 117 (i.e., the position of the corresponding player 107) in the read teacher data 41. For example, when the teacher data 41 at the top in FIG. 4 is read, the teacher mark 117 at the position "3-C" displays "1," which is the identification information associated with the position of that teacher mark 117 (i.e., the position of the corresponding player 107) in the read teacher data 41.

[0046] The teacher marks 117 represent the positions of the multiple players 107 associated with the position of the ball 109 obtained by conversion in step 4 in the teacher data 41, and the identification information assigned to each of the multiple players 107 in the teacher data 41. One ball mark 119 is displayed in the image 113. The ball mark 119 is displayed at the position of the ball 109 obtained by conversion in step 4.

[0047] In step 9, the animation creation unit 37 creates an animation using the plurality of images 113 created in step 8. When the animation is displayed, the plurality of images 113 are displayed one by one in order. When the animation is displayed, the real mark 115, the teacher mark 117, and the ball mark 119 appear to move continuously on the half court 101 over time.

[0048] 3. Effects of Animation Creation Device 1 (1A) The animation created by the animation creating device 1 is composed of images 113 representing the following (a) to (c):

[0049] (a) The positions of multiple players 107 and the ball 109 in the half court 101 (i.e., the position of the real mark 115 and the position of the ball mark 119). (b) Identification information assigned to multiple players 107 (i.e., identification information attached to reality marks 115).

[0050] (c) In the teacher data 41, the positions of multiple players 107 associated with the position of the ball 109 in (a) above, and the identification information assigned to each of the multiple players 107 in the teacher data 41 (i.e., the position of the teacher mark 117 and the identification information assigned thereto).

[0051] Therefore, the animation can simultaneously display the real position of each player 107 (i.e., the position of the real mark 115) and the correct position of each player 107 (i.e., the position of the teacher mark 117). For example, a user of the animation creation device 1 can use this animation to instruct each player 107 so that each player 107 moves to the correct position.

[0052] Since the real marks 115 and the teacher marks 117 are each provided with identification information, viewers of the animation can easily use the identification information as a clue to recognize which real marks 115 correspond to which teacher marks 117 (i.e., whether they are the same player 107).

[0053] (1B) The sport is, for example, soccer. In this case, animation can be used to provide instruction to soccer players 107. (1C) Image 113 is an image representing a half court 101 on a soccer field. Furthermore, multiple players 107 are players on the defensive team who are guarding a goal 103 on the half court 101. In this case, animation can be used to instruct the players 107 on the defensive team.

[0054] (1D) In ​​the processing from step 3 onwards, a portion of frames 105 extracted from all of the frames 105 included in the video data is used. This reduces the processing load on the calculation unit 3 when performing the processing from step 3 onwards.

[0055] <Other embodiments> Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments and can be implemented in various modified forms.

[0056] (1) In the above embodiment, the sport is soccer, but it is not limited to this. For example, the sport may be futsal, rugby, American football, etc. (2) The image 113 may represent the entire field of play. The players 107 may be players on an attacking team attacking the goal 103. The players 107 may also represent all players on a team.

[0057] (3) The appearance of the reality mark 115 and the teacher mark 117 is not particularly limited, and their shape, size, color, pattern, etc. can be set appropriately. For example, the reality mark 115 or the teacher mark 117 may be a grid 102 with a color or pattern, etc., that is different from that of the other grids 102.

[0058] (4) The image 113 may further include other elements. For example, the image 113 may further include an indication that associates the reality mark 115 and the teacher mark 117, which have the same identification information. For example, an example of the association indication is a line connecting the reality mark 115 and the teacher mark 117.

[0059] The display may further include a display showing the distance between the real mark 115 and the teacher mark 117, which have the same identification information. Examples of the display showing the distance include a display of a numerical value of the distance. (5) The animation creation device 1 may further include other functions. For example, a moving image may be created by combining a plurality of frames 105 in which markers are attached to a plurality of players 107, as shown in FIG. 3A. In the moving image, a marker may also be displayed on the ball 109, as shown in FIG. 6B. The moving image may be in MP4 format, for example.

[0060] (6) The specific frame 105A may be any frame 105 that shows the relationship between the position of the player 107 and the identification information, and does not have to be the frame 105 at the start of the game or the start of the second half of the game.

[0061] (7) The calculation unit 3 and the method described herein may be implemented by a special-purpose computer configured by configuring a processor and memory programmed to execute one or more functions embodied in a computer program. Alternatively, the calculation unit 3 and the method described herein may be implemented by a special-purpose computer configured by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the calculation unit 3 and the method described herein may be implemented by one or more special-purpose computers configured by combining a processor and memory programmed to execute one or more functions with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory tangible recording medium. The method for implementing the functions of each unit included in the calculation unit 3 does not necessarily need to include software; all of the functions may be implemented using one or more hardware.

[0062] (8) The function of one component in each of the above embodiments may be shared among multiple components, or the functions of multiple components may be performed by one component. Also, part of the configuration of each of the above embodiments may be omitted. Furthermore, at least part of the configuration of each of the above embodiments may be added to or substituted for the configuration of another of the above embodiments.

[0063] (9) In addition to the animation creation device 1 described above, the present disclosure can also be realized in various forms, such as a system including the animation creation device 1 as a component, a program for causing a computer to function as the calculation unit 3, a non-transient physical recording medium such as a semiconductor memory on which this program is recorded, a method for creating animation, a method for processing frames 105, etc.

[0064] [Technical idea disclosed in this specification] [Item 1] a video data acquisition unit configured to acquire video data generated by filming a game played on a field by a plurality of players using a ball; a teacher data storage unit that stores, for each position of the ball on the field, a plurality of pieces of teacher data that associate the position of the ball on the field, the positions of the plurality of players on the field, and identification information assigned to each of the plurality of players; a position detection unit configured to detect positions of the players and the ball in each of a plurality of frames included in the video data; a position conversion unit configured to convert, in each of the plurality of frames, positions of the plurality of players and the ball in the frame into positions of the plurality of players and the ball on the field; a first identification information assignment unit configured to assign the identification information to each of the plurality of players in a specific frame that is any one of frames included in the video data; a tracking unit configured to perform tracking to associate the same player detected in each of the plurality of frames; a second identification information assignment unit configured to assign the identification information assigned to the player of the specific frame by the first identification information assignment unit to players in frames other than the specific frame who are associated with the player of the specific frame through the tracking; an image creation unit configured to create an image representing the following (a) to (c) for each of the plurality of frames; an animation creation unit configured to create an animation using the plurality of images created by the image creation unit; An animation creation device comprising: (a) the positions of the players and the ball on the field; (b) the identification information assigned to the plurality of players; (c) In the training data, the positions of the multiple players associated with the position of the ball in (a), and the identification information assigned to each of the multiple players in the training data. [Item 2] The animation creation device according to item 1, The sport is soccer, Animation creation device. [Item 3] The animation creation device according to item 2, the image is an image representing a half court, the plurality of players are players of a team defending a goal in the half court; Animation creation device. [Item 4] The animation creation device according to any one of items 1 to 3, The specific frame is a frame at the start of the competition or a frame at the start of the second half of the competition. Animation creation device. [Item 5] The animation creation device according to any one of items 1 to 4, the plurality of frames are a portion of frames extracted from all frames included in the video data; Animation creation device. [Explanation of symbols]

[0065] 1...animation creation device, 3...computation unit, 5...teaching data storage unit, 7...cloud, 11...CPU, 13...memory, 21...video data acquisition unit, 23...frame extraction unit, 25...position detection unit, 27...position conversion unit, 29...first identification information assignment unit, 31...tracking unit, 33...second identification information assignment unit, 35...image creation unit, 37...animation creation unit, 41...teaching data, 101...half court, 102...grid, 103...goal, 105...frame, 107, 108...player, 109...ball, 111a to 111d...corners, 113...image, 115...real mark, 117...teaching mark, 119...ball mark

Claims

1. a video data acquisition unit configured to acquire video data generated by filming a game played on a field by a plurality of players using a ball; a teacher data storage unit that stores, for each position of the ball on the field, a plurality of pieces of teacher data that associate the position of the ball on the field, the positions of the plurality of players on the field, and identification information assigned to each of the plurality of players; a position detection unit configured to detect positions of the players and the ball in each of a plurality of frames included in the video data; a position conversion unit configured to convert, in each of the plurality of frames, positions of the plurality of players and the ball in the frame into positions of the plurality of players and the ball on the field; a first identification information assignment unit configured to assign the identification information to each of the plurality of players in a specific frame that is any one of frames included in the video data; a tracking unit configured to perform tracking to associate the same player detected in each of the plurality of frames; a second identification information assignment unit configured to assign the identification information assigned to the player of the specific frame by the first identification information assignment unit to players in frames other than the specific frame who are associated with the player of the specific frame through the tracking; an image creation unit configured to create, for each of the plurality of frames, an image representing: an animation creation unit configured to create an animation using the plurality of images created by the image creation unit; An animation creation device comprising: (a) the positions of the players and the ball on the field; (b) the identification information assigned to the plurality of players; (c) In the training data, the positions of the multiple players associated with the position of the ball in (a), and the identification information assigned to each of the multiple players in the training data.

2. 2. The animation creation device according to claim 1, The sport is soccer, Animation creation device.

3. 3. The animation creation device according to claim 2, the image is an image representing a half court, the plurality of players are players of a team defending a goal in the half court; Animation creation device.

4. 3. The animation creation device according to claim 1, The specific frame is a frame at the start of the competition or a frame at the start of the second half of the competition. Animation creation device.

5. 3. The animation creation device according to claim 1, the plurality of frames are a portion of frames extracted from all frames included in the video data; Animation creation device.

Images