Information processing system, information processing device, and information processing method

Abstract

The information processing system includes one or more cameras installed looking diagonally down on a soccer field and an information processing device, and uses the position of the soccer ball on its two-dimensional coordinate system to identify, on the two-dimensional coordinate system, sections in which the soccer ball rolled on the ground, sections in which it was dribbled, and sections in which it flew through the air, respectively, and stores a predetermined reference height in association with the two-dimensional coordinate positions of the soccer ball in the sections in which it rolled on the ground and the sections in which it was dribbled, recalculates the position of the soccer ball on its two-dimensional coordinate system in the sections in which it flew through the air and calculates its height position, updates the position of the soccer ball on its two-dimensional coordinate system in the sections in which it flew through the air, and stores the calculated height position in association with the position of the soccer ball in its two-dimensional coordinate system in the updated sections.

Description

[Technical field]

[0001] The present invention relates to measuring the position of a soccer ball. [Background technology]

[0002] An invention has been proposed that uses multiple imaging devices to identify the position of an object such as a soccer ball (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-160233 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional methods for measuring the position of a soccer ball require multiple cameras installed at considerable distances from each other to increase the parallax between the cameras, which poses the problem of high costs for purchasing and installing the cameras.

[0005] An object of one embodiment of the present invention is to provide an information processing system, an information processing device, and an information processing method that can reduce the number of installed cameras. Also, an object of one embodiment of the present invention is to provide an information processing system, an information processing device, and an information processing method that can arrange multiple cameras closer together than before with smaller parallax than before when multiple cameras are used. [Means for solving the problem]

[0006] The present invention includes the following embodiment.

[0007] An information processing device, Using a position of the soccer ball on a two-dimensional coordinate system, a section in which the soccer ball moves as if rolling on the ground, a section in which the soccer ball is dribbled, and a section in which the soccer ball moves as if flying through the air are identified on the two-dimensional coordinate system, storing a predetermined reference height in association with a position on a two-dimensional coordinate system of the soccer ball in the section in which the soccer ball rolls on the ground and in the section in which the soccer ball is dribbled; an information processing device that recalculates the position of the soccer ball in two-dimensional coordinates during the section where it moves as if flying through the air, calculates its position in a height direction, updates the position of the soccer ball in two-dimensional coordinates during the section where it moves as if flying through the air, and stores the calculated position in the height direction in correspondence with the position of the soccer ball in the two-dimensional coordinates during the updated section.

[0008] One or more cameras are installed at an angle overlooking the soccer field, An information processing system including an information processing device having a calculation device and a storage device, The computing device includes: calculating a two-dimensional coordinate position of the soccer ball at each predetermined time interval by performing image analysis on the images captured by the one or more cameras; storing the calculated position of the soccer ball on the two-dimensional coordinate system in the storage device; using the position of the soccer ball on a two-dimensional coordinate system stored in the storage device, identifying, on the two-dimensional coordinate system, a section in which the soccer ball moved as if rolling on the ground, a section in which the soccer ball was dribbled, and a section in which the soccer ball moved as if flying in the air; storing a predetermined reference height in the storage device in association with positions on a two-dimensional coordinate system of the soccer ball in a section in which the soccer ball moves so as to roll on the ground and in a section in which the soccer ball is dribbled; read from the storage device a position of the soccer ball in two-dimensional coordinates during a section in which the soccer ball moves as if flying through the air, a position of the soccer ball in two-dimensional coordinates at a start time of the section in which the soccer ball moves as if flying through the air, and a position of the soccer ball in two-dimensional coordinates at an end time of the section in which the soccer ball moves as if flying through the air, and using the read positions, recalculate the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moves as if flying through the air and calculate a position in a height direction of the soccer ball during the section in which the soccer ball moves as if flying through the air; updating the position of the soccer ball in the two-dimensional coordinate system during the section in which the soccer ball moves as if flying through the air, which is stored in the storage device, using the recalculated position in the two-dimensional coordinate system during the section in which the soccer ball moves as if flying through the air; the calculated position in a height direction in a section in which the soccer ball has traveled flying in the air is associated with the updated position in two-dimensional coordinates of the soccer ball in the section in which the soccer ball has traveled flying in the air, and stored in the storage device; an information processing system that outputs, in response to an output request, the position in the two-dimensional coordinate system and the position in the height direction of the soccer ball stored in the storage device within a range corresponding to the output request.

[0009] An information processing device having a calculation device and a storage device, The computing device includes: calculating a position of the soccer ball in two-dimensional coordinates at each predetermined time interval by performing image analysis on images captured by one or more cameras installed diagonally overlooking the soccer field; storing the calculated position of the soccer ball on the two-dimensional coordinate system in the storage device; using the position of the soccer ball on a two-dimensional coordinate system stored in the storage device, identifying, on the two-dimensional coordinate system, a section in which the soccer ball moved as if rolling on the ground, a section in which the soccer ball was dribbled, and a section in which the soccer ball moved as if flying in the air; storing a predetermined reference height in the storage device in association with positions on a two-dimensional coordinate system of the soccer ball in a section in which the soccer ball moves so as to roll on the ground and in a section in which the soccer ball is dribbled; read from the storage device a position of the soccer ball in two-dimensional coordinates during a section in which the soccer ball moves as if flying through the air, a position of the soccer ball in two-dimensional coordinates at a start time of the section in which the soccer ball moves as if flying through the air, and a position of the soccer ball in two-dimensional coordinates at an end time of the section in which the soccer ball moves as if flying through the air, and using the read positions, recalculate the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moves as if flying through the air and calculate a position in a height direction of the soccer ball during the section in which the soccer ball moves as if flying through the air; updating the position of the soccer ball in the two-dimensional coordinate system during the section in which the soccer ball moves as if flying through the air, which is stored in the storage device, using the recalculated position in the two-dimensional coordinate system during the section in which the soccer ball moves as if flying through the air; the calculated position in a height direction in a section in which the soccer ball has traveled flying in the air is associated with the updated position in two-dimensional coordinates of the soccer ball in the section in which the soccer ball has traveled flying in the air, and stored in the storage device; an information processing device that outputs, in response to an output request, the position in the two-dimensional coordinate system and the position in the height direction of the soccer ball stored in the storage device within a range corresponding to the output request.

[0010] An information processing method executed in an information processing device having a calculation device and a storage device, The computing device calculates the position of the soccer ball in two-dimensional coordinates at each predetermined time interval by performing image analysis on images captured by one or more cameras installed diagonally overlooking the soccer field; The calculation device stores the calculated position of the soccer ball on the two-dimensional coordinate system in the storage device, the arithmetic device uses the position of the soccer ball on the two-dimensional coordinate system stored in the storage device to identify, on the two-dimensional coordinate system, a section in which the soccer ball moves as if rolling on the ground, a section in which the soccer ball is dribbled, and a section in which the soccer ball moves as if flying in the air; the calculation device stores in the storage device a predetermined reference height in association with positions on a two-dimensional coordinate system of the soccer ball in a section in which the soccer ball moves so as to roll on the ground and in a section in which the soccer ball is dribbled; the arithmetic device reads from the storage device a position of the soccer ball in two-dimensional coordinates during a section in which the soccer ball moves as if flying through the air, a position of the soccer ball in two-dimensional coordinates at a start time of the section in which the soccer ball moves as if flying through the air, and a position of the soccer ball in two-dimensional coordinates at an end time of the section in which the soccer ball moves as if flying through the air, and uses the read positions to recalculate the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moves as if flying through the air and calculate a position in a height direction of the soccer ball during the section in which the soccer ball moves as if flying through the air; the arithmetic device updates the position of the soccer ball in the two-dimensional coordinate system during the section in which the soccer ball moves as if flying through the air, the position being stored in the storage device, using the recalculated position in the two-dimensional coordinate system during the section in which the soccer ball moves as if flying through the air; the calculation device associates the calculated position in a height direction in a section in which the soccer ball moves so as to fly through the air with the updated position in two-dimensional coordinates of the soccer ball in the section in which the soccer ball moves so as to fly through the air, and stores the position in the storage device; An information processing method in which, in response to an output request, the calculation device outputs the position in two-dimensional coordinates and the position in the height direction of the soccer ball stored in the storage device within a range corresponding to the output request. Effect of the Invention

[0011] According to one embodiment of the present invention, the costs required for purchasing and installing cameras can be significantly reduced. [Brief description of the drawings]

[0012] [Figure 1] 1 is a schematic diagram showing a configuration of an information processing system 1 according to a first embodiment. [Diagram 2] FIG. 2 is a schematic diagram illustrating a method for installing the camera 10. [Diagram 3] 3 is a cross-sectional view taken along line AA in FIG. 2. [Figure 4] FIG. 2 is a schematic diagram showing a configuration of an information processing device 20. [Diagram 5] 2 is a schematic diagram showing a first example of data storage in a storage device 24. FIG. [Figure 6] 13 is a schematic diagram showing a second example of data storage in the storage device 24. FIG. [Figure 7] 10 is a flowchart illustrating an example of the operation of the information processing device 20. [Figure 8] 10 is a flowchart illustrating an example of the operation of the information processing device 20. [Figure 9] FIG. 1 is a schematic diagram illustrating an example of a trajectory of a soccer ball. [Figure 10] 1 is a schematic diagram illustrating an example of a position on a two-dimensional coordinate system stored in a storage device 24. FIG. [Figure 11] 1 is a schematic diagram illustrating a state in which positions on two-dimensional coordinates stored in storage device 24 are plotted on an xy plane. [Figure 12] 13 is a flowchart illustrating a specific example of step S3. [Figure 13] 13 is a diagram for explaining how the slope is associated with a section and stored in the storage device 24. FIG. [Figure 14] 13 is a graph illustrating the change in the slope stored in the storage device 24. FIG. [Figure 15] FIG. 13 is a schematic diagram illustrating a switching point. [Figure 16] 1 is a diagram illustrating switching points (black circles) and each section identified in step S3 on an xy plane. [Figure 17] FIG. 11 is a diagram showing an example of storage in the storage device 24 after execution of step S4. [Figure 18] 13 is a schematic diagram for explaining recalculation of a position on a two-dimensional coordinate system and calculation of a position in a height direction in step S5. FIG. [Figure 19] FIG. 13 is a diagram showing an example of storage in the storage device 24 after steps S6 and S7 are executed. [Figure 20] FIG. 13 is a diagram showing a state in which the positions in the height direction stored in step S7 are plotted in the range from time t1 to time t26. [Figure 21] FIG. 11 is a schematic diagram illustrating an information processing system 2 according to a second embodiment. [Figure 22] 2 is a schematic diagram illustrating the parallax between the cameras 12 and 14. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] [Information Processing System According to the First Embodiment] Fig. 1 is a schematic diagram showing the configuration of an information processing system 1 according to embodiment 1. As shown in Fig. 1, the information processing system 1 according to this embodiment is an information processing system including one or more cameras 10 and an information processing device 20. The following will explain in detail.

[0014] (Camera 10) Fig. 2 is a schematic diagram for explaining a method for installing the camera 10 according to the embodiment 1. Fig. 2 is a diagram of a soccer field as viewed from directly above, and Fig. 3 is a cross-sectional view taken along the line AA in Fig. 2.

[0015] The camera 10 is installed, for example, outside the touchlines and / or outside the goal lines. Fig. 2 shows an example of installing the camera 10 outside the touchlines. In soccer, the direction parallel to the touchlines of the field is sometimes called the vertical direction, and the direction parallel to the goal lines is sometimes called the horizontal direction, but in Fig. 2, the touchlines are drawn parallel to the horizontal direction of the drawing, and the goal lines are drawn parallel to the vertical direction of the drawing.

[0016] 3, the camera 10 is installed so as to look down diagonally from above onto the soccer field. The field refers to the ground on which a soccer match is played, and is sometimes called the pitch.

[0017] The number of cameras 10 is one or more. In this embodiment, one camera 10 is used. As described later, it is also possible to use a plurality of cameras.

[0018] The camera 10 may be a monocular camera or a compound eye camera. When a monocular camera is used, it is possible to reduce the cost required for measuring the position of a soccer ball compared to when a compound eye camera is used. In this embodiment, a monocular camera is used.

[0019] (Information processing device 20) Fig. 4 is a schematic diagram showing the configuration of the information processing device 20. As shown in Fig. 4, the information processing device 20 includes a calculation device 22 and a storage device 24. The information processing device 20 operates when the calculation device 22 executes a program.

[0020] Examples of the arithmetic device 22 include a CPU and a GPU. The arithmetic device included in the information processing device 20 includes the arithmetic device 22 (see FIG. 4) built into the information processing device 20, as well as a arithmetic device on a cloud, that is, a computer that is connected to the information processing device 20 via a network and executes calculations in response to requests from the information processing device 20. The case where the process according to this embodiment is executed by such a arithmetic device on a cloud is also included in the case where the information processing device 20 includes a arithmetic device.

[0021] Examples of the storage device 24 include a RAM and a hard disk. The storage device provided in the information processing device 20 includes the storage device 24 (see FIG. 4) built into the information processing device 20, as well as a storage device on a cloud, that is, a storage device connected to the information processing device 20 via a network and storing data in response to a request from the information processing device 20. Such a case where data is stored in a storage device on a cloud is also included in the case where the information processing device 20 is provided with a storage device.

[0022] Fig. 5 is a schematic diagram showing a first example of data storage in storage device 24. As shown in Fig. 5, storage device 24 stores a time t, positions x and y of the soccer ball on the two-dimensional coordinate system, and a position z of the soccer ball in the height direction in association with each other.

[0023] (time t) The time t column contains the times t1, t2, t3, etc. n is stored. n is an integer equal to or greater than 1. At time t n For example, the time t is 14:30:20 on April 1, 2024. n+1 -time t n = Δt n There is a relationship between Δt nindicates a "predetermined time interval." For example, if the "predetermined time interval" is 10 seconds, and if t1 is 15:00:00 on April 3, 2024, then t2 will be 15:00:10 on April 3, 2024. The "predetermined time interval" is equal to, for example, a sampling period. The sampling period refers to the predetermined time interval when the position of a soccer ball is detected from an image captured by the camera 10 at a predetermined time interval.

[0024] (position x, position y, position z) The position x and the position y of the soccer ball are an example of the position of the soccer ball on the two-dimensional coordinate system, and can also be expressed as the two-dimensional coordinates (x, y) of the soccer ball. The position x, the position y, and the height z are an example of the position of the soccer ball on the three-dimensional coordinate system, and can also be expressed as the three-dimensional coordinates (x, y, z) of the soccer ball. The position x, the position y, and the position z are numerical values ​​such as 10, 5, and 20, and examples of units of these numerical values ​​include meters (m), centimeters (cm), and pixels. The position x is a position in the x-axis direction, the position y is a position in the y-axis direction, and the position z is a position in the z-axis direction, which is a direction perpendicular to the xy plane defined by the x-axis and the y-axis. The direction of the x-axis may or may not be parallel to the touchline or the goal line. The direction of the y-axis may or may not be parallel to the touchline or the goal line.

[0025] Fig. 6 is a schematic diagram showing a second example of data storage in the storage device 24. As shown in Fig. 6, the storage device 24 stores the intervals and the slope a in association with each other.

[0026] (section) In the interval column, "n → n+1" such as "1 → 2", "2 → 3", etc. are stored. n is an integer equal to or greater than 1. For example, "n → n+1" is n From time t n+1For example, if n=1, time t1=2024-04-03 15:00:00, and t2=2024-04-03 15:00:10, then "time t1 -> time t2" means the time interval from 2024-04-03 15:00:00 to 2024-04-03 15:00:10.

[0027] (Tilt a) In the slope a column, 1→2 , a 2→3 , a 3→4 Such as, a n→n+1 is stored. n is an integer equal to or greater than 1. The slope a n→n+1 For example, at time t n+1 The position of the soccer ball in two-dimensional coordinates (x n+1 , y n+1 ) and time t n The position of the soccer ball in two-dimensional coordinates (x n , y n For example, if time t1 = 2024-04-03 15:00:00 and t2 = 2024-04-03 15:00:10, then "a 1→2 " is the slope of the line connecting the position (x2, y2) of the soccer ball on the two-dimensional coordinate system at 15:00:10 on April 3, 2024 and the position (x1, y1) of the soccer ball on the two-dimensional coordinate system at 15:00:00 on April 3, 2024. n→n+1 The calculation method of is not limited, and the calculation device 22 may calculate a by executing a known method, for example. n→n+1 can be calculated.

[0028] (Example of operation of information processing device 20) Fig. 7 and Fig. 8 are both flowcharts illustrating an example of the operation of the information processing device 20. Fig. 9 is a schematic diagram illustrating an example of the trajectory of a soccer ball. The white circle in Fig. 9 indicates the position (x, y) of the soccer ball on the two-dimensional coordinate system.

[0029] (Soccer ball trajectory) FIG. 9 shows the movement of a soccer ball as follows: (1) From time t1 to time t5, a soccer ball moves as if it is rolling on the ground. For example, a soccer ball is passed by a ground pass. (2) From time t5 to time t9, the soccer ball moves as if it is rolling on the ground. For example, the soccer ball is passed by a ground pass. (3) From time t9 to time t 16 For example, a soccer ball is passed so that it flies through the air. (4) Time 16 From time 22 A soccer ball is dribbled towards the goal. (5) Time 22 From time 26 For example, a soccer ball is shot toward a goal post as it rolls along the ground. An example of a rolling movement on the ground includes a ground pass as well as a rolling shot, and an example of a flying movement includes an airborne pass as well as an airborne shot.

[0030] In the following, an information processing method executed by the information processing device 20 will be described with reference to the flowcharts shown in Fig. 7 and Fig. 8. In other words, when a soccer ball moves along the trajectory shown in Fig. 9, how the information processing device 20 operates to calculate the position of the soccer ball in three-dimensional coordinates (in other words, the position of the soccer ball in two-dimensional coordinates and the height direction position) will be described. In this embodiment, for ease of understanding, the position x n and position x n+1 The absolute value of the difference between the positions x and x is assumed to be "1". However, the magnitude of the absolute value is not limited to "1" and may be a value other than 1, including 0, and such cases are also included in this embodiment. For example, n and position x n+1 The absolute value of the difference may be 0, 1.3, 2, or 10.

[0031] (Step S1) As shown in FIG. 7, first, the calculation device 22 calculates the position (x, y) of the soccer ball on the two-dimensional coordinate system at each "predetermined time interval" by performing image analysis on the video captured by the camera 10. A publicly known method can be used as the image analysis method. The "predetermined time interval" is, for example, a sampling period as described above. In other words, the calculation device 22 determines the position (x, y) of the soccer ball on the two-dimensional coordinate system at, for example, a predetermined sampling period. With this step, the calculation device 22 calculates the position (x, y) of the soccer ball on the two-dimensional coordinate system at, for example, a time t n The position on the two-dimensional coordinate system (x n , y n ) is calculated, where n is an integer equal to or greater than 1. In this embodiment, 26 26 2D coordinate positions (x n , y n ) is calculated.

[0032] Calculation of the soccer ball's two-dimensional coordinate position (x, y) in this step includes cases where the calculation device itself performs image analysis to determine the soccer ball's two-dimensional coordinate position (x, y), as well as cases where the calculation device acquires or receives input of the two-dimensional coordinate position (x, y) determined by another computer through image analysis from another computer or storage medium. The latter case also includes cases where the calculation device calculates the soccer ball's two-dimensional coordinate position (x, y) at predetermined time intervals by performing image analysis on video captured by a camera. Note that in this embodiment, the calculation device 22 itself performs image analysis to determine the soccer ball's two-dimensional coordinate position (x, y).

[0033] (Step S2) FIG. 10 is a schematic diagram illustrating an example of a position on a two-dimensional coordinate system stored in the storage device 24. FIG. 11 is a schematic diagram illustrating a state in which the positions on the two-dimensional coordinate system stored in the storage device 24 are plotted on an xy plane. The calculation device 22 stores the position (x, y) on the two-dimensional coordinate system of the soccer ball calculated in step S1 in the storage device 24. In this embodiment, as shown in FIG. 10, the calculation device 22 calculates the position (x, y) of the soccer ball on the two-dimensional coordinate system at time t nThe position (x n , y n ) is stored in the storage device 24, where n is an integer of 1 or more.

[0034] Immediately after this step is executed, the soccer ball is still at its height position z n has not been calculated. For this reason, nothing has been entered in the z column in FIG. 10 yet (i.e., it is blank). Note that the z column may not be blank, but may store a predetermined initial value such as a reference height. The reference height is the height at which the soccer ball is considered to be in contact with the ground. The reference height may be 0 or a value other than 0.

[0035] (Step S3) The calculation device 22 calculates the position (x n , y n ) is used to identify, on two-dimensional coordinates, a section in which the soccer ball moves as if rolling on the ground, a section in which the soccer ball is dribbled, and a section in which the soccer ball moves as if flying through the air. Examples of a section in which the soccer ball moves as if rolling on the ground include a section in which the soccer ball is rolling on the ground due to a ground pass, and a section in which the soccer ball is rolling on the ground because it is shot as if rolling on the ground. Examples of a section in which the soccer ball moves as if flying through the air include a section in which the soccer ball moves as if flying through the air because it is passed as if flying through the air, and a section in which the soccer ball moves as if flying through the air because it is shot as if flying through the air.

[0036] 12 is a flow chart for explaining a specific example of step S3. Hereinafter, a specific example of step S3 will be explained with reference to FIG.

[0037] (Step S31) As shown in FIG. 12, first, the calculation device 22 calculates the position (x n , y n) at time t n+1 The two-dimensional coordinate position of the soccer ball in (x n+1 , y n+1 ) and time t n The two-dimensional coordinate position of the soccer ball in (x n , y n ) and the slope of the line connecting n→n+1 Calculate.

[0038] (Step S32) 13 is a diagram for explaining how the slope is associated with the interval and stored in the storage device 24. As shown in FIG. 13, the calculation device 22 stores the slope a calculated in step S31. n→n+1 is stored in the storage device 24 in association with the interval (n→n+1). n is an integer equal to or greater than 1. As an example, in the case of n=3, the calculation device 22 calculates the slope a 3→4 The number "5" is calculated as the interval "3→4" and stored.

[0039] FIG. 14 is a diagram for explaining, in a graph, the change in the slope stored in the storage device 24. In FIG. 14, the horizontal axis indicates the section, and the vertical axis indicates the slope. As shown in FIG. 14, in the example shown in FIG. 13, the section from section 1→2 to section 4→5 (i.e., between x1 and x5), the section from section 5→6 ​​to section 8→9 (i.e., between x5 and x9), and the section from section 22→23 to section 25→26 (i.e., between x 22 x 26 The slope of each n→n+1 In addition, the section from section 16→17 to section 21→22 (i.e., x 16 x 22 The slope of the n→n+1 In addition, the section from section 9 to 10 to section 15 to 16 (i.e., x9 to x 16 The position of the soccer ball in the two-dimensional coordinate system (x n , y n ) draws a parabola with a slope an→n+1 This is the section where changes.

[0040] The term "constant slope" includes not only the case where the slope is strictly constant mathematically, but also the case where the slope is substantially constant.

[0041] The position of the soccer ball in two-dimensional coordinates (x n , y n If the slope of the line changes like a parabola, the position of the soccer ball on the two-dimensional coordinate system (x n , y n ) changes in a mathematically exact parabola on a two-dimensional coordinate system (i.e., on the xy plane). In addition, the position of the soccer ball on the two-dimensional coordinate system (x n , y n For example, when the trajectory of the soccer ball describes a parabola on a two-dimensional coordinate system (i.e., on the xy plane), the calculation device 22 calculates the position (x n , y n ) can be determined to be a section where the slope changes like a parabola.

[0042] There is no particular limitation on the method of determining whether the inclination is constant, whether the inclination changes discontinuously, or whether the position of the soccer ball on the two-dimensional coordinate system changes so as to draw a parabola. For example, the arithmetic device 22 can determine that the inclination changes discontinuously if the inclination is not constant and the position of the soccer ball on the two-dimensional coordinate system does not change so as to draw a parabola. That is, the arithmetic device 22 can determine, for example, whether the inclination is constant and whether the position of the soccer ball on the two-dimensional coordinate system changes so as to draw a parabola, and can determine that the inclination changes discontinuously if the inclination does not fit either of the cases. In other words, the arithmetic device 22 can determine whether the inclination changes discontinuously by a process of elimination.

[0043] (Step S33) Returning to FIG. 12, the calculation device 22 calculates the slope a stored in the storage device 24 in step 32. n→n+1 In this embodiment, the slope a n→n+1 The section where the slope a is constant is defined as the section where the soccer ball moves as if rolling on the ground. The calculation device 22 according to the present embodiment defines the slope a as the slope a in the section from section 1→2 to section 4→5, the section from section 5→6 ​​to section 8→9, and the section from section 22→23 to section 25→26. n→n+1 is determined to be constant, and these sections are identified as sections in which a soccer ball moves as if rolling on the ground.

[0044] (Step S34) Next, the calculation unit 22 calculates the slope a stored in the storage unit 24 in step S32. n→n+1 In this embodiment, the section in which the soccer ball is dribbled is identified using the slope a n→n+1 The section where the slope a changes discontinuously is defined as the section where the soccer ball is dribbled. n→n+1 is discontinuously changing, and this section is identified as a section in which the soccer ball is dribbled.

[0045] (Step S35) Next, the calculation unit 22 calculates the slope a stored in the storage unit 24 in step S32. n→n+1 In this embodiment, the section in which the soccer ball flew through the air is identified by using the position (x n , y n ) draws a parabola with a slope a n→n+1 The section in which the position of the soccer ball on the two-dimensional coordinate system (x n , y n ) draws a parabola with a slope a n→n+1is changed, and this section is identified as a section in which a soccer ball moves as if flying through the air.

[0046] 15 is a schematic diagram illustrating the switching points. In this embodiment, the slope, which had been constant until time t5, changes, and the slope, which had been constant until time t9, changes again at time t 16 The position of the soccer ball in the two-dimensional coordinate system (x n , y n ) changes like a parabola, and the slope changes at time t 22 At times t5, t9, and t 16 , t 22 is identified as a switching point. A switching point means a point where the type of movement of the soccer ball changes, and such a switching occurs, for example, when a player touches the soccer ball by kicking or heading it. For example, the time and position at which the soccer ball switches from rolling on the ground to flying in the air is an example of a point where the type of movement of the soccer ball changes.

[0047] FIG. 16 is a diagram illustrating the switching points (black circles) and each section identified in step S3 on an xy plane. In FIG. 16, the switching points described above are indicated by black circles. Also, in FIG. 16, the section identified in step S3 where the ball moved as if rolling on the ground is indicated by "G", the section where the ball was dribbled is indicated by "D", and the section where the ball moved as if flying through the air is indicated by "F". In section F in FIG. 16, the position of the soccer ball on the two-dimensional coordinate system describes a parabola.

[0048] (Step S4) Returning to FIG. 7, the calculation device 22 calculates the predetermined reference height by calculating the position (x n , y n) and stored in the storage device 24. In this embodiment, the calculation device 22 sets the predetermined reference height=0, and stores the time intervals from time t1 to time t5, from time t5 to time t9, and the time intervals from time t 16 From time t 22 Each section up to time t 22 From time t 26 An example of the data stored in the storage device 24 after step S4 is executed is shown in FIG.

[0049] (Step S5) Next, the calculation device 22 calculates the position (x, y) of the soccer ball in the two-dimensional coordinate system during the section in which the soccer ball moves as if flying through the air, the position (x9, y9) of the soccer ball in the two-dimensional coordinate system at the start time t9 of the section in which the soccer ball moves as if flying through the air, and the end time t 16 The position of the soccer ball in two-dimensional coordinates (x 16 , y 16 ) are read from the storage device 24, and using these read positions, the position (x, y) in the two-dimensional coordinate system of the soccer ball during the section in which the soccer ball moved as if flying through the air is recalculated, and the position z in the height direction of the soccer ball during the section in which the soccer ball moved as if flying through the air is calculated.

[0050] 18 is a schematic diagram for explaining the recalculation of the position on the two-dimensional coordinate system and the calculation of the position in the height direction in step S5. The method of recalculating the position on the two-dimensional coordinate system and the method of calculating the position in the height direction are not limited. For example, as shown in FIG. 18, in step S5, the arithmetic unit 22 calculates the position on the two-dimensional coordinate system at time t9 and time t 16 Find the line that connects time t9 to time t 16 The intersection point (for example, time t 12 For example, the position on the two-dimensional coordinate system in step S5 (time t 12 About x 12 , y 12) and recalculate it as follows. 16 The calculation device 22 calculates the position z in the height direction by using the difference (for example, the length of the perpendicular line) between the trajectory of the soccer ball (i.e., the parabola) and the position z in the height direction. The calculation device 22 calculates the position z in the height direction of the soccer ball by using the difference itself or a predetermined calculation performed on the difference.

[0051] (Step S6) Returning to FIG. 8, the calculation device 22 updates the two-dimensional coordinate position of the soccer ball during the section where the soccer ball moves as if flying through the air, which is stored in the storage device 24, using the two-dimensional coordinate position of the soccer ball during the section where the soccer ball moves as if flying through the air, which is recalculated in step S5. For example, at time t 12 The position on the two-dimensional coordinate system (x 12 , y 12 ), then (xo, yo) will be changed to (xp, yp).

[0052] (Step S7) Next, the calculation device 22 associates the height position of the soccer ball in the section where it moved as if flying through the air, calculated in step S5, with the updated position of the soccer ball in two-dimensional coordinates in the section where it moved as if flying through the air, and stores this in the storage device 24.

[0053] 19 is a diagram showing an example of the storage of the storage device 24 after steps S6 and S7 are executed. As shown in FIG. 19, in step S6, the calculation device 22 converts the position on the two-dimensional coordinate system recalculated in step S5 from (x9, y9), for example, to (x 16 , y 16 19, the calculation device 22 stores in the storage device 24 the position on the two-dimensional coordinate system from time t9 to time t 16 The value of z in (i.e. z9 to z 16 ) is stored in the storage device 24.

[0054] (Step S8) 8, in response to the output request, the calculation device 22 outputs the position in the two-dimensional coordinate system and the position in the height direction of the soccer ball stored in the storage device 24, that is, the position in the three-dimensional coordinate system, within a range corresponding to the output request. For example, the calculation device 22 determines whether the range corresponding to the output request is from time t3 to time t 18 If the range is 3D coordinates, the position in this range is from (x3, y3, z3) to (x 18 , y 18 , z 18 ) is output.

[0055] The target to which the calculation device 22 outputs (hereinafter referred to as the "output-receiving device") is not particularly limited. For example, the calculation device 22 can output a position on a three-dimensional coordinate system to a monitor, an analysis device, or the like. In the output-receiving device, the output position on the three-dimensional coordinate system can be used to display the trajectory of the soccer ball on a screen or to analyze it. In addition, by using the output position on the three-dimensional coordinate system as learning data for an AI such as a generation AI, various soccer ball trajectories can be generated.

[0056] FIG. 20 shows the height direction position stored in step S7 from time t1 to time t 26 As shown in FIG. 20, according to the present embodiment, the soccer ball is plotted in the range from time t1 to t9 and the range from time t 16 From time t 26 In the section, the height z is 0, and from time t9 to time 16 In other words, the height z is greater than 0 in the section from time t9 to time 16 It can be seen that the soccer ball is floating in the air in the section up to .

[0057] [Information Processing System According to the Second Embodiment] Fig. 21 is a schematic diagram illustrating an information processing system 2 according to embodiment 2. Fig. 22 is a schematic diagram illustrating the parallax between cameras 12, 14. As shown in Figs. 21 and 22, the information processing system 2 according to embodiment 2 is a form that uses a plurality of cameras 12, 14, and in this respect differs from the information processing system 1 according to embodiment 1 that uses one camera 10, but has the same configuration in other respects.

[0058] The parallax θ between the cameras 12 and 14 is at least 0 degrees or more and 19 degrees or less, preferably 0 degrees or more and 5 degrees or less, more preferably 0 degrees or more and 1 degree or less, and further preferably 0 degrees or more and 0.7 degrees or less. According to this embodiment, even though the multiple cameras are arranged with such a small parallax, the position of the soccer ball in the height direction can be calculated with relatively high accuracy. In the conventional stereo method, the multiple cameras are arranged at intervals of 100 m so that the parallax becomes large, but according to this embodiment, it is not necessary to make the parallax between the cameras large in order to increase the accuracy of the position measurement in the height direction. In other words, even if the parallax between the cameras is small, the position of the soccer ball in the height direction can be measured with high accuracy. Therefore, the distance between the cameras 12 and 14 (for example, the shortest distance) is, for example, 1 m or more and 3 m or less. In other words, according to this embodiment, even though the cameras 12 and 14 are only, for example, 1 m or more and 3 m or less apart, the position of the soccer ball in the three-dimensional coordinate system, including the position of the soccer ball in the height direction, can be measured with high accuracy. According to this embodiment, the cameras 12 and 14 can be placed close together so that one person can adequately manage them. This significantly reduces the burden of operating and managing the cameras 12 and 14, and can greatly reduce the cost of measuring the position of a soccer ball.

[0059] Although two cameras 12 and 14 are shown in FIG. 22, the number of cameras may be three or more as long as the parallax between the cameras is within the above-mentioned range. For example, when three cameras are used, the parallax between the first camera and the second camera is 0 degrees or more and 19 degrees or less, the parallax between the second camera and the third camera is 0 degrees or more and 19 degrees or less, and the parallax between the first camera and the third camera is 0 degrees or more and 19 degrees or less. Note that, for example, if a fourth camera is installed, but the parallax between this fourth camera and at least one of the three cameras is not within the above-mentioned range, this fourth camera does not correspond to the camera according to this embodiment. However, even in this case, the remaining three cameras correspond to the cameras according to this embodiment.

[0060] As described above, in the information processing system of embodiments 1 and 2, the information processing device uses the position of the soccer ball on the two-dimensional coordinates to identify, on the two-dimensional coordinates, the sections where the soccer ball moved as if rolling on the ground, the sections where it was dribbled, and the sections where it moved as if flying through the air, respectively, and stores a predetermined reference height in association with the two-dimensional coordinate positions of the soccer ball in the sections where it moved as if rolling on the ground and the sections where it was dribbled, recalculates the two-dimensional coordinate position of the soccer ball in the sections where it moved as if flying through the air and calculates the height position, updates the two-dimensional coordinate position of the soccer ball in the sections where it moved as if flying through the air, and stores the calculated height position in association with the two-dimensional coordinate position of the soccer ball in this updated section.

[0061] According to the above-described first and second embodiments, the trajectory of the soccer ball on the two-dimensional coordinate system is divided into a section where the soccer ball moves as if rolling on the ground, a section where the soccer ball is dribbled, and a section where the soccer ball moves as if flying in the air, and the position of the soccer ball on the three-dimensional coordinate system is calculated based on the characteristics of each section thus divided. Such calculation can be performed by one camera or by multiple cameras with a parallax of 0 degrees or more and 19 degrees or less. Therefore, according to the first and second embodiments, it is possible to reduce the number of cameras. Even when multiple cameras are used, the multiple cameras can be arranged close to each other so that the parallax between the cameras is smaller than that of the conventional method. For example, the distance between the multiple cameras can be set to 1 m or more and 3 m or less. Therefore, according to the first and second embodiments, it is possible to significantly reduce the cost required for purchasing and installing an imaging device.

[0062] According to the first and second embodiments, the movement of a ball can be tracked and digitized from a video of a soccer game. In the first embodiment, the digitization is performed using one camera 10, and in the second embodiment, the digitization is performed using multiple cameras 12 and 14. Even if multiple cameras are used, the parallax between the cameras does not need to be large, and the parallax may be in the range of 0 degrees or more and 19 degrees or less. For this reason, the multiple cameras only need to be installed at a distance of, for example, 1 meter or more and 3 meters or less, and do not need to be installed at 100 meter intervals as in the past, and can be installed in one location (or a location close enough to be considered as one location). Therefore, no large-scale preparation or equipment is required to install the cameras. While a camera used in the stereo system costs, for example, several million yen, the cameras 10, 12, and 14 used in the first and second embodiments can be cameras that are sold for, for example, several tens of thousands of yen and are commonly used in daily life. Therefore, according to the first and second embodiments, it is also possible to reduce the cost required for measuring the position of a soccer ball to, for example, about one hundredth of the conventional cost.

[0063] People basically move on a flat surface (the pitch) and do not fly in the air. For this reason, tracking of people is possible with existing technology. In contrast, a ball can either roll on the pitch or fly in the air. In particular, since a photo taken with a camera is two-dimensional, it is difficult to grasp the "height." Therefore, in the first and second embodiments, attention is paid to the fact that the movement of a soccer ball can be roughly classified into three types. The first type is a rolling movement (i.e., movement along a flat surface), the second type is a flying movement in the air (i.e., movement that draws a parabola), and the third type is a dribble (movement with fine zigzag or other movements). In the first and second embodiments, the position of the soccer ball is calculated based on these classifications, so that the height of the soccer ball can be grasped even with one camera or multiple cameras with a parallax of 0 degrees or more and 19 degrees or less.

[0064] According to the first and second embodiments, for example, it is possible to perform analysis on the position measurement of a soccer ball in real time (or almost real time). Therefore, using the position information of the soccer ball measured by applying the first and second embodiments, it is possible to plan a strategy using the analysis data of the first half, for example, at halftime.

[0065] Soccer is a sport in which players mainly use their feet to carry the ball to a desired location. The rules prohibit players other than the goalkeeper from using their hands to move the ball during the game. There are three ways to carry the ball to a desired location: shooting, passing, and dribbling. Dribbling is a method of moving the ball by controlling it mainly using the player's feet. Passing is a method of moving the ball by kicking it to a specific destination. Passes are divided into "ground passes" where the soccer ball rolls on the ground and "air passes" where the ball moves in a parabolic arc through the air. Shooting is what creates goals in soccer and is a way to move the ball into the goal area. Like passing, shooting can be divided into two methods: one where the soccer ball rolls on the ground and one where the ball moves in a parabolic arc through the air.

[0066] Although the embodiment has been described above, these descriptions are merely examples, and the configurations described in the claims are not limited in any way by these descriptions. Even if the embodiment differs from the above description, as long as it has the configurations described in the claims, it will have the effect of the present invention and be included in the present invention. [Explanation of symbols]

[0067] 1, 2 Information Processing System 10, 12, 14 Camera 20 Information processing device 22 Arithmetic unit 24 Storage device

Claims

1. An information processing device, Using a position of the soccer ball on a two-dimensional coordinate system, a section in which the soccer ball moves as if rolling on the ground, a section in which the soccer ball is dribbled, and a section in which the soccer ball moves as if flying through the air are identified on the two-dimensional coordinate system, storing a predetermined reference height in association with a position on a two-dimensional coordinate system of the soccer ball in the section in which the soccer ball rolls on the ground and in the section in which the soccer ball is dribbled; an information processing device that recalculates the position of the soccer ball in two-dimensional coordinates during the section where it moves as if flying through the air, calculates its position in a height direction, updates the position of the soccer ball in two-dimensional coordinates during the section where it moves as if flying through the air, and stores the calculated position in the height direction in correspondence with the position of the soccer ball in the two-dimensional coordinates during the updated section.

2. One or more cameras are installed to look down diagonally on the soccer field, An information processing system including an information processing device having a calculation device and a storage device, The computing device includes: calculating a position of the soccer ball in two-dimensional coordinates at each predetermined time interval by performing image analysis on the images captured by the one or more cameras; storing the calculated position of the soccer ball on the two-dimensional coordinate system in the storage device; using the position of the soccer ball on a two-dimensional coordinate system stored in the storage device, identifying, on the two-dimensional coordinate system, a section in which the soccer ball moved as if rolling on the ground, a section in which the soccer ball was dribbled, and a section in which the soccer ball moved as if flying in the air; storing a predetermined reference height in the storage device in association with positions on a two-dimensional coordinate system of the soccer ball in a section in which the soccer ball moves so as to roll on the ground and in a section in which the soccer ball is dribbled; read from the storage device a position of the soccer ball in two-dimensional coordinates during a section in which the soccer ball moves as if flying through the air, a position of the soccer ball in two-dimensional coordinates at a start time of the section in which the soccer ball moves as if flying through the air, and a position of the soccer ball in two-dimensional coordinates at an end time of the section in which the soccer ball moves as if flying through the air, and using the read positions, recalculate the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moves as if flying through the air and calculate a position in a height direction of the soccer ball during the section in which the soccer ball moves as if flying through the air; updating the position of the soccer ball in the two-dimensional coordinate system during the section in which the soccer ball moved as if flying through the air, which is stored in the storage device, using the recalculated position in the two-dimensional coordinate system during the section in which the soccer ball moved as if flying through the air; the calculated position in a height direction in a section in which the soccer ball has traveled flying through the air is associated with the updated position in two-dimensional coordinates of the soccer ball in the section in which the soccer ball has traveled flying through the air, and stored in the storage device; an information processing system that outputs, in response to an output request, the position in the two-dimensional coordinate system and the position in the height direction of the soccer ball stored in the storage device within a range corresponding to the output request.

3. 3. The information processing system according to claim 2, The computing device includes: The position (x ｎ , y ｎ ) at time t ｎ＋１ The two-dimensional coordinates of the soccer ball in (x ｎ＋１ , y ｎ＋１ ) and time t ｎ The two-dimensional coordinates of the soccer ball in (x ｎ , y ｎ ) the slope of the line connecting ｎ→ｎ＋１ Calculate The calculated slope a ｎ→ｎ＋１ is stored in a storage device in association with the interval (n→n+1); The slope a stored in the storage device ｎ→ｎ＋１ The section where is constant is identified as the section where the soccer ball moves as if rolling on the ground, The slope a stored in the storage device ｎ→ｎ＋１ A section where the distance changes discontinuously is identified as a section where the soccer ball is dribbled, The position of the soccer ball on the two-dimensional coordinate system (x ｎ , y ｎ ) is stored in the storage device so that it draws a parabola ｎ→ｎ＋１ The information processing system identifies the section in which the value changes as a section in which a soccer ball moves as if flying through the air.

4. 4. The information processing system according to claim 2, The number of the cameras is two or more, An information processing system, wherein the parallax between two or more of the cameras is greater than or equal to 0 degrees and less than or equal to 19 degrees.

5. An information processing device having a calculation device and a storage device, The computing device includes: The position of the soccer ball in two-dimensional coordinates is calculated at each predetermined time interval by performing image analysis on images taken by one or more cameras installed diagonally overlooking the soccer field; storing the calculated position of the soccer ball on the two-dimensional coordinate system in the storage device; using the position of the soccer ball on a two-dimensional coordinate system stored in the storage device, identifying, on the two-dimensional coordinate system, a section in which the soccer ball moved as if rolling on the ground, a section in which the soccer ball was dribbled, and a section in which the soccer ball moved as if flying in the air; storing a predetermined reference height in the storage device in association with positions on a two-dimensional coordinate system of the soccer ball in a section in which the soccer ball moves so as to roll on the ground and in a section in which the soccer ball is dribbled; read from the storage device a position of the soccer ball in two-dimensional coordinates during a section in which the soccer ball moves as if flying through the air, a position of the soccer ball in two-dimensional coordinates at a start time of the section in which the soccer ball moves as if flying through the air, and a position of the soccer ball in two-dimensional coordinates at an end time of the section in which the soccer ball moves as if flying through the air, and using the read positions, recalculate the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moves as if flying through the air and calculate a position in a height direction of the soccer ball during the section in which the soccer ball moves as if flying through the air; updating the position of the soccer ball in the two-dimensional coordinate system during the section in which the soccer ball moved as if flying through the air, which is stored in the storage device, using the recalculated position in the two-dimensional coordinate system during the section in which the soccer ball moved as if flying through the air; the calculated position in a height direction in a section in which the soccer ball has traveled flying through the air is associated with the updated position in two-dimensional coordinates of the soccer ball in the section in which the soccer ball has traveled flying through the air, and stored in the storage device; an information processing device that outputs, in response to an output request, the position in the two-dimensional coordinate system and the position in the height direction of the soccer ball stored in the storage device within a range corresponding to the output request.

6. 6. The information processing device according to claim 5, The computing device includes: The position (x ｎ , y ｎ ) at time t ｎ＋１ The two-dimensional coordinates of the soccer ball in (x ｎ＋１ , y ｎ＋１ ) and time t ｎ The two-dimensional coordinates of the soccer ball in (x ｎ , y ｎ ) the slope of the line connecting ｎ→ｎ＋１ Calculate The calculated slope a ｎ→ｎ＋１ is stored in a storage device in association with the interval (n→n+1); The slope a stored in the storage device ｎ→ｎ＋１ The section where is constant is identified as the section where the soccer ball moves as if rolling on the ground, The slope a stored in the storage device ｎ→ｎ＋１ A section where the distance changes discontinuously is identified as a section where the soccer ball is dribbled, The position of the soccer ball on the two-dimensional coordinate system (x ｎ , y ｎ ) is stored in the storage device so that it draws a parabola ｎ→ｎ＋１ The information processing device identifies a section in which the time period changes as a section in which a soccer ball moves as if flying through the air.

7. 7. The information processing device according to claim 5, The number of the cameras is two or more, An information processing device, wherein the parallax of the two or more cameras is greater than or equal to 0 degrees and less than or equal to 19 degrees.

8. An information processing method executed in an information processing device having a calculation device and a storage device, the computing device calculates the position of the soccer ball in two-dimensional coordinates at each predetermined time interval by performing image analysis on images captured by one or more cameras installed diagonally overlooking the soccer field; the calculation device stores the calculated position of the soccer ball on the two-dimensional coordinate system in the storage device; the arithmetic device uses the position of the soccer ball on the two-dimensional coordinate system stored in the storage device to identify, on the two-dimensional coordinate system, a section in which the soccer ball moves as if rolling on the ground, a section in which the soccer ball is dribbled, and a section in which the soccer ball moves as if flying in the air; the computing device stores in the storage device a predetermined reference height in association with positions on a two-dimensional coordinate system of the soccer ball in a section in which the soccer ball moves so as to roll on the ground and in a section in which the soccer ball is dribbled; the arithmetic device reads from the storage device a position of the soccer ball in two-dimensional coordinates during a section in which the soccer ball moves as if flying through the air, a position of the soccer ball in two-dimensional coordinates at a start time of the section in which the soccer ball moves as if flying through the air, and a position of the soccer ball in two-dimensional coordinates at an end time of the section in which the soccer ball moves as if flying through the air, and uses the read positions to recalculate the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moves as if flying through the air and calculates a position in a height direction of the soccer ball during the section in which the soccer ball moves as if flying through the air; the calculation device updates the position of the soccer ball in the two-dimensional coordinate system during the section in which the soccer ball moves as if flying through the air, ... the calculation device associates the calculated position in a height direction in a section in which the soccer ball moves so as to fly through the air with the updated position in two-dimensional coordinates of the soccer ball in the section in which the soccer ball moves so as to fly through the air, and stores the position in the storage device; The information processing method, in response to an output request, wherein the arithmetic device outputs the position in two-dimensional coordinates and the position in the height direction of the soccer ball stored in the storage device within a range corresponding to the output request.

9. 9. The information processing method according to claim 8, The calculation device calculates the position (x ｎ , y ｎ ) at time t ｎ＋１ The two-dimensional coordinates of the soccer ball in (x ｎ＋１ , y ｎ＋１ ) and time t ｎ The two-dimensional coordinates of the soccer ball in (x ｎ , y ｎ ) the slope of the line connecting ｎ→ｎ＋１ Calculate The calculation device calculates the slope a ｎ→ｎ＋１ is stored in a storage device in association with the interval (n→n+1); The arithmetic unit calculates the slope a stored in the storage device. ｎ→ｎ＋１ The section where is constant is identified as the section where the soccer ball moves as if rolling on the ground, The arithmetic unit calculates the slope a stored in the storage device. ｎ→ｎ＋１ A section where the distance changes discontinuously is identified as a section where the soccer ball is dribbled, The calculation device calculates the position (x ｎ , y ｎ ) is stored in the storage device so that it draws a parabola ｎ→ｎ＋１ The information processing method identifies a section in which the value of the time series changes as a section in which a soccer ball moves as if flying through the air.

10. 10. The information processing method according to claim 8, further comprising: The number of the cameras is two or more, An information processing method, wherein the parallax of two or more of the cameras is greater than or equal to 0 degrees and less than or equal to 19 degrees.