Information processing system, information processing device, and information processing method

Abstract

The present invention provides an information processing system comprising one or more cameras installed so as to look down on a soccer field obliquely from above, and an information processing device. A section through which the soccer ball moves so as to roll on the ground, a section through which the ball is dribbled, and a section in which the soccer ball moves so as to fly in the air are respectively specified in two-dimensional coordinates using the two-dimensional coordinate position of a soccer ball; a prescribed reference height is stored in association with the two-dimensional coordinate position of the soccer ball in the section through which the soccer ball moves so as to roll on the ground and the section through which the ball is dribbled; the two-dimensional coordinate position of the soccer ball in the section through which the soccer ball moves so as to fly in the air is recalculated and the position of the soccer ball in the height direction is calculated; the two-dimensional coordinate position of the soccer ball in the section through which the soccer ball moves so as to fly in the air is updated; and the calculated position in the height direction is stored in association with the two-dimensional coordinate position of the soccer ball in the updated section.

Description

Information processing system, information processing device, and information processing method

[0001] The present invention relates to measuring the position of a soccer ball.

[0002] An invention has been proposed that uses multiple imaging devices to identify the position of an object such as a soccer ball (Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2022-160233

[0004] Conventional methods for measuring the position of a soccer ball often require multiple cameras installed at considerable distances from each other to increase the parallax between the cameras, which poses a problem of high costs for purchasing and installing the cameras.

[0005] An object of one embodiment of the present invention is to provide an information processing system, an information processing device, and an information processing method that can reduce the number of installed cameras. Also, an object of one embodiment of the present invention is to provide an information processing system, an information processing device, and an information processing method that, when using multiple cameras, can arrange the multiple cameras closer together than before with smaller parallax than before.

[0006] The present invention includes the following embodiment.

[0007] an information processing device that uses the position of a soccer ball on two-dimensional coordinates to identify, on the two-dimensional coordinates, sections in which the soccer ball rolls on the ground, sections in which the soccer ball is dribbled, and sections in which the soccer ball flies through the air; stores a predetermined reference height in association with the two-dimensional coordinate positions of the soccer ball in the sections in which the soccer ball rolls on the ground and the sections in which the soccer ball is dribbled; recalculates the two-dimensional coordinate position of the soccer ball in the sections in which the soccer ball flies through the air and calculates a height position, updates the two-dimensional coordinate position of the soccer ball in the sections in which the soccer ball flies through the air, and stores the calculated height position in association with the two-dimensional coordinate position of the soccer ball in the updated sections.

[0008] An information processing system comprising: one or more cameras installed diagonally overlooking a soccer field; and an information processing device having an arithmetic unit and a storage device, wherein the arithmetic unit calculates the position of the soccer ball in two-dimensional coordinates at predetermined time intervals by performing image analysis on images captured by the one or more cameras; stores the calculated position of the soccer ball in two-dimensional coordinates in the storage device; identifies, using the position of the soccer ball in two-dimensional coordinates stored in the storage device, a section in which the soccer ball rolls on the ground, a section in which the soccer ball is dribbled, and a section in which the soccer ball flies through the air, respectively; and stores a predetermined reference height in the storage device in association with the positions of the soccer ball in two-dimensional coordinates in the section in which the soccer ball rolls on the ground and the section in which the soccer ball is dribbled. read from the storage device the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, the position of the soccer ball in two-dimensional coordinates at the start time of the section in which the soccer ball moved as if flying through the air, and the position of the soccer ball in two-dimensional coordinates at the end time of the section in which the soccer ball moved as if flying through the air, and use the read positions to recalculate the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air and calculate the position of the soccer ball in the height direction during the section in which the soccer ball moved as if flying through the air; update the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, which is stored in the storage device, using the recalculated position in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air; associate the calculated position in the height direction during the section in which the soccer ball moved as if flying through the air with the updated position in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, and store it in the storage device;an information processing system that, in response to an output request, outputs the position of the soccer ball in two-dimensional coordinates and the position in the height direction stored in the storage device within a range corresponding to the output request.

[0009] An information processing device having a calculation device and a storage device, wherein the calculation device calculates the position of the soccer ball on two-dimensional coordinates at predetermined time intervals by performing image analysis on images taken by one or more cameras installed diagonally above a soccer field, stores the calculated position of the soccer ball on two-dimensional coordinates in the storage device, identifies, using the position of the soccer ball on two-dimensional coordinates stored in the storage device, a section in which the soccer ball moved as if rolling on the ground, a section in which the soccer ball was dribbled, and a section in which the soccer ball moved as if flying through the air, and stores a predetermined reference height in the storage device in association with the positions of the soccer ball on two-dimensional coordinates in the section in which the soccer ball moved as if rolling on the ground and the section in which the soccer ball was dribbled, read from the storage device the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, the position of the soccer ball in two-dimensional coordinates at the start time of the section in which the soccer ball moved as if flying through the air, and the position of the soccer ball in two-dimensional coordinates at the end time of the section in which the soccer ball moved as if flying through the air, and use the read positions to recalculate the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air and calculate the position of the soccer ball in the height direction during the section in which the soccer ball moved as if flying through the air; update the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, which is stored in the storage device, using the recalculated position in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air; associate the calculated position in the height direction during the section in which the soccer ball moved as if flying through the air with the updated position in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, and store it in the storage device; an information processing device that, in response to an output request, outputs the position of the soccer ball in the two-dimensional coordinate system and the position in the height direction stored in the storage device within a range corresponding to the output request.

[0010] An information processing method executed in an information processing device having a calculation device and a storage device, wherein the calculation device calculates the position of the soccer ball in two-dimensional coordinates at predetermined time intervals by image analysis of video captured by one or more cameras installed diagonally above a soccer field, the calculation device stores the calculated position of the soccer ball in two-dimensional coordinates in the storage device, the calculation device uses the position of the soccer ball in two-dimensional coordinates stored in the storage device to identify, on the two-dimensional coordinates, a section in which the soccer ball moved as if rolling on the ground, a section in which the soccer ball was dribbled, and a section in which the soccer ball moved as if flying through the air, and the calculation device stores, in the storage device, a predetermined reference height associated with the positions of the soccer ball in two-dimensional coordinates in the section in which the soccer ball moved as if rolling on the ground and the section in which the soccer ball was dribbled, the arithmetic device reads from the storage device the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, the position of the soccer ball in two-dimensional coordinates at the start time of the section in which the soccer ball moved as if flying through the air, and the position of the soccer ball in two-dimensional coordinates at the end time of the section in which the soccer ball moved as if flying through the air, and uses the read positions to recalculate the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air and calculate the position of the soccer ball in the height direction during the section in which the soccer ball moved as if flying through the air; the arithmetic device updates the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, which is stored in the storage device, using the recalculated position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air; the calculation device associates the calculated position in the height direction in the section in which the soccer ball travels as if flying through the air with the updated position on the two-dimensional coordinates of the soccer ball in the section in which the soccer ball travels as if flying through the air, and stores the result in the storage device;An information processing method in which, in response to an output request, the arithmetic device outputs the position of the soccer ball in two-dimensional coordinates and the position in the height direction stored in the storage device within a range corresponding to the output request.

[0011] According to one embodiment of the present invention, the costs required for purchasing and installing cameras can be significantly reduced.

[0012] 1 is a schematic diagram showing the configuration of an information processing system 1 according to a first embodiment. It is a schematic diagram illustrating a method for installing a camera 10. It is a cross-sectional view taken along the line A-A in FIG. 2. It is a schematic diagram showing the configuration of an information processing device 20. It is a schematic diagram showing an example (part 1) of data storage in a storage device 24. It is a schematic diagram showing an example (part 2) of data storage in a storage device 24. It is a flowchart illustrating an example of the operation of the information processing device 20. It is a flowchart illustrating an example of the operation of the information processing device 20. It is a schematic diagram illustrating an example of the trajectory of a soccer ball. It is a schematic diagram illustrating an example of positions on two-dimensional coordinates stored in the storage device 24. It is a schematic diagram illustrating positions on two-dimensional coordinates stored in the storage device 24 plotted on an xy plane. It is a flowchart illustrating a specific example of step S3. It is a diagram illustrating how slopes are stored in the storage device 24 in association with sections. It is a diagram illustrating, in a graph, changes in the slope stored in the storage device 24. It is a schematic diagram illustrating switching points. It is a diagram illustrating switching points (black circles) and each section identified in step S3, shown on an xy plane. FIG. 10 is a diagram showing an example of data stored in the storage device 24 after execution of step S4. FIG. 11 is a schematic diagram for explaining recalculation of the position on the two-dimensional coordinate system and calculation of the position in the height direction in step S5. FIG. 11 is a diagram showing an example of data stored in the storage device 24 after execution of steps S6 and S7. １ From time t ２６ Fig. 10 is a diagram illustrating plots in a range of Fig. 11. Fig. 11 is a schematic diagram illustrating an information processing system 2 according to a second embodiment. Fig. 12 is a schematic diagram illustrating the parallax between cameras 12 and 14.

[0013] [Information Processing System According to Embodiment 1] Fig. 1 is a schematic diagram showing the configuration of an information processing system 1 according to Embodiment 1. As shown in Fig. 1, the information processing system 1 according to this embodiment is an information processing system including one or more cameras 10 and an information processing device 20. This will be described in detail below.

[0014] (Camera 10) Fig. 2 is a schematic diagram for explaining a method for installing the camera 10 according to embodiment 1. Fig. 2 is a diagram of a soccer field as seen from directly above, and Fig. 3 is a cross-sectional view taken along line AA in Fig. 2.

[0015] The camera 10 is installed, for example, outside the touchlines and / or outside the goal lines. Figure 2 shows an example of installing the camera 10 outside the touchlines. In soccer, the direction parallel to the touchlines of the field is sometimes called the vertical direction, and the direction parallel to the goal lines is sometimes called the horizontal direction. In Figure 2, the touchlines are drawn parallel to the horizontal direction of the drawing, and the goal lines are drawn parallel to the vertical direction of the drawing.

[0016] 3, the camera 10 is installed so as to look down diagonally from above onto the soccer field. The field refers to the ground on which a soccer match is played, and is also called the pitch.

[0017] The number of cameras 10 is one or more. In this embodiment, one camera 10 is used. As will be described later, it is also possible to use a plurality of cameras.

[0018] The camera 10 may be a monocular camera or a compound camera. When a monocular camera is used, it is possible to reduce the cost required for measuring the position of a soccer ball compared to when a compound camera is used. In this embodiment, a monocular camera is used.

[0019] (Information processing device 20) Fig. 4 is a schematic diagram showing the configuration of the information processing device 20. As shown in Fig. 4, the information processing device 20 includes a calculation device 22 and a storage device 24. The information processing device 20 operates when the calculation device 22 executes a program.

[0020] Examples of the arithmetic device 22 include a CPU and a GPU. The arithmetic device included in the information processing device 20 includes the arithmetic device 22 (see FIG. 4 ) built into the information processing device 20, as well as a arithmetic device on a cloud, that is, a computer that is connected to the information processing device 20 via a network and performs calculations in response to requests from the information processing device 20. The case where the processing according to this embodiment is executed by such a arithmetic device on a cloud is also considered to be included in the case where the information processing device 20 includes a arithmetic device.

[0021] Examples of the storage device 24 include RAM, a hard disk, etc. The storage device provided in the information processing device 20 includes the storage device 24 (see FIG. 4) built into the information processing device 20, as well as a storage device on a cloud, that is, a storage device connected to the information processing device 20 via a network and storing data in response to a request from the information processing device 20. The case where data is stored in such a storage device on a cloud is also considered to be included in the case where the information processing device 20 is provided with a storage device.

[0022] Fig. 5 is a schematic diagram showing an example (part 1) of data storage in the storage device 24. As shown in Fig. 5, the storage device 24 stores a time t, positions x and y of the soccer ball on the two-dimensional coordinate system, and a position z of the soccer ball in the height direction, in association with each other.

[0023] (Time t) In the time t column, time t １ , t ２ , t ３ and other t ｎ is stored, where n is an integer equal to or greater than 1. ｎ is a numerical value such as 14:30:20 on April 1, 2024. ｎ＋１ - time t ｎ =Δt ｎ There is a relationship between Δt ｎ indicates a "predetermined time interval." For example, the "predetermined time interval" is 10 seconds, and t １ = 15:00:00 on April 3, 2024, t ２= 15:00:10 on Apr. 3, 2024. The "predetermined time interval" is equal to, for example, the sampling period. The sampling period refers to the predetermined time interval when the position of the soccer ball is detected from the image captured by the camera 10 at the predetermined time interval.

[0024] (Position x, Position y, Position z) The position x and position y of the soccer ball are an example of the position of the soccer ball on a two-dimensional coordinate system, and can also be expressed as the two-dimensional coordinates (x, y) of the soccer ball. Furthermore, the position x, position y, and height z are an example of the position of the soccer ball on a three-dimensional coordinate system, and can also be expressed as the three-dimensional coordinates (x, y, z) of the soccer ball. The position x, position y, and position z are numerical values ​​such as 10, 5, and 20, and examples of units of these numerical values ​​include meters (m), centimeters (cm), and pixels. Note that the position x is the position in the x-axis direction, the position y is the position in the y-axis direction, and the position z is the position in the z-axis direction, which is perpendicular to the x-y plane defined by the x-axis and y-axis. The direction of the x-axis may or may not be parallel to the touchline or goal line. Furthermore, the direction of the y-axis may or may not be parallel to the touchline or goal line.

[0025] 6 is a schematic diagram showing a second example of data storage in the storage device 24. As shown in FIG. 6, the storage device 24 stores the intervals and the slopes a in association with each other.

[0026] (Interval) In the interval column, "n → n+1" such as "1 → 2" and "2 → 3" are stored. n is an integer equal to or greater than 1. For example, "n → n+1" is a time t ｎ From time t ｎ＋１ For example, n=1 and time t １ = 15:00:00 on April 3, 2024, and t ２ = 15:00:10 on April 3, 2024, "time t １ →Time t ２ " means the time period from 15:00:00 on April 3, 2024 to 15:00:10 on April 3, 2024.

[0027] (Slope a) In the slope a column, a １→２ , a ２→３ , a ３→４ Such as, a ｎ→ｎ＋１ is stored. n is an integer equal to or greater than 1. ｎ→ｎ＋１ For example, at time t ｎ＋１ The position of the soccer ball on the two-dimensional coordinate system (x ｎ＋１ , y ｎ＋１ ) and time t ｎ The position of the soccer ball on the two-dimensional coordinate system (x ｎ , y ｎ ) means the slope of the line connecting the time t １ = 15:00:00 on April 3, 2024, and t ２ = 2024 / 4 / 3 15:00:10, １→２ " is the position of the soccer ball on the two-dimensional coordinate system (x ２ , y ２ ) and the position of the soccer ball on the two-dimensional coordinate system at 15:00:00 on April 3, 2024 (x １ , y １ ) is the slope of the line connecting these two points. ｎ→ｎ＋１ The calculation method of is not limited, and the calculation device 22 may calculate a by executing a known method, for example. ｎ→ｎ＋１ can be calculated.

[0028] (Example of operation of information processing device 20) Figures 7 and 8 are both flowcharts explaining an example of operation of information processing device 20. Figure 9 is a schematic diagram explaining an example of the trajectory of a soccer ball. The white circle in Figure 9 indicates the position (x, y) of the soccer ball on the two-dimensional coordinate system.

[0029] (Trajectory of a soccer ball) FIG. 9 shows the movement of a soccer ball as follows: (1) At time t １ From time t ５ For example, a soccer ball is passed by a ground pass. (2) At time t ５ From time t ９ The soccer ball moves as if it is rolling on the ground. For example, the soccer ball is passed by a ground pass. (3) At time t９ From time t １６ A soccer ball moves as if flying through the air. For example, a soccer ball is passed as if flying through the air. (4) Time １６ From time ２２ A soccer ball is dribbled around the time. (5) Time ２２ From time ２６ A soccer ball moves as if it is rolling on the ground. For example, a soccer ball is shot towards a goal post as if it is rolling on the ground. Examples of movements that roll on the ground include a ground pass and a shot that rolls on the ground. Examples of movements that fly through the air include a pass that flies through the air and a shot that flies through the air.

[0030] The following describes an information processing method executed by the information processing device 20, with reference to the flowcharts shown in Figures 7 and 8. In other words, when a soccer ball moves along the trajectory shown in Figure 9, how the information processing device 20 operates to calculate the position of the soccer ball in three-dimensional coordinates (in other words, the position of the soccer ball in two-dimensional coordinates and the position in the height direction) will be described. In this embodiment, for ease of understanding, the position x ｎ and position x ｎ＋１ The absolute value of the difference between the positions x and x is "1". However, the magnitude of the absolute value is not limited to "1" and may be a value other than 1, including 0, and such cases are also included in this embodiment. For example, ｎ and position x ｎ＋１ The absolute value of the difference may be 0, 1.3, 2, or 10.

[0031] (Step S1) As shown in FIG. 7, first, the calculation device 22 calculates the position (x, y) of the soccer ball on the two-dimensional coordinate system at each "predetermined time interval" by performing image analysis on the video captured by the camera 10. A known method can be used for the image analysis method. As described above, the "predetermined time interval" is, for example, a sampling period. In other words, the calculation device 22 obtains the position (x, y) of the soccer ball on the two-dimensional coordinate system at, for example, a predetermined sampling period. In this step, the calculation device 22 calculates the position (x, y) of the soccer ball on the two-dimensional coordinate system at, for example, a time t ｎ The position on the two-dimensional coordinate system (x ｎ , y ｎ ) is calculated, where n is an integer equal to or greater than 1. In this embodiment, 26 26 positions on the two-dimensional coordinate system (x ｎ , y ｎ ) is calculated.

[0032] Calculation of the soccer ball's two-dimensional coordinate position (x, y) in this step includes cases where the calculation device itself performs image analysis to determine the soccer ball's two-dimensional coordinate position (x, y), as well as cases where the calculation device acquires or receives input from another computer or storage medium the two-dimensional coordinate position (x, y) determined by another computer's image analysis. The latter case also includes cases where the calculation device calculates the soccer ball's two-dimensional coordinate position (x, y) at predetermined time intervals by analyzing images captured by a camera. In this embodiment, it is assumed that the calculation device 22 itself performs image analysis to determine the soccer ball's two-dimensional coordinate position (x, y).

[0033] (Step S2) Fig. 10 is a schematic diagram illustrating an example of a position on a two-dimensional coordinate system stored in the storage device 24. Fig. 11 is a schematic diagram illustrating a state in which the positions on the two-dimensional coordinate system stored in the storage device 24 are plotted on an xy plane. The calculation device 22 stores the position (x, y) of the soccer ball on the two-dimensional coordinate system calculated in step S1 in the storage device 24. In this embodiment, as shown in Fig. 10, the calculation device 22 calculates the position (x, y) of the soccer ball on the two-dimensional coordinate system at time t ｎ , and the position (xｎ , y ｎ ) is stored in the storage device 24, where n is an integer of 1 or greater.

[0034] Immediately after this step is executed, the soccer ball's height position z ｎ has not been calculated. Therefore, nothing has been entered in the z column in FIG. 10 yet (i.e., it is blank). Note that the z column may not be blank, but may store a predetermined initial value such as a reference height. The reference height is the height at which the soccer ball is considered to be in contact with the ground. The reference height may be 0 or a value other than 0.

[0035] (Step S3) The calculation device 22 calculates the position (x ｎ , y ｎ ) is used to identify, on two-dimensional coordinates, sections in which the soccer ball moves as if rolling on the ground, sections in which the soccer ball is dribbled, and sections in which the soccer ball moves as if flying through the air. Examples of sections in which the soccer ball moves as if rolling on the ground include sections in which the soccer ball is rolling on the ground due to a ground pass, and sections in which the soccer ball is rolling on the ground because it is shot as if it is rolling on the ground. Examples of sections in which the soccer ball moves as if it is flying through the air include sections in which the soccer ball moves as if it is passed through the air, and sections in which the soccer ball moves as if it is shot as if it is flying through the air.

[0036] 12 is a flowchart illustrating a specific example of step S3. A specific example of step S3 will be described below with reference to FIG.

[0037] (Step S31) As shown in FIG. 12, first, the calculation device 22 calculates the position (x) of the soccer ball on the two-dimensional coordinate system stored in the storage device 24. ｎ , y ｎ ) at time t ｎ＋１ The two-dimensional coordinate position of the soccer ball (x ｎ＋１ , y ｎ＋１ ) and time t ｎThe two-dimensional coordinate position of the soccer ball (x ｎ , y ｎ ) and the slope of the line connecting ｎ→ｎ＋１ Calculate.

[0038] (Step S32) Fig. 13 is a diagram illustrating how the slope is associated with the section and stored in the storage device 24. As shown in Fig. 13, the calculation device 22 stores the slope a calculated in step S31. ｎ→ｎ＋１ is stored in the storage device 24 in association with the interval (n → n+1). n is an integer equal to or greater than 1. For example, in the case of n=3, the calculation device 22 stores ４ The two-dimensional coordinates of the soccer ball (x ４ , y ４ ) and time t ３ The two-dimensional coordinates of the soccer ball (x ３ , y ３ ) the slope of the line connecting ３→４ The value "5" is calculated as the interval "3→4" and stored.

[0039] FIG. 14 is a graph illustrating the change in the slope stored in the storage device 24. In FIG. 14, the horizontal axis indicates the interval and the vertical axis indicates the slope. As shown in FIG. 14, in the example shown in FIG. 13, the interval from interval 1 → 2 to interval 4 → 5 (i.e., x １ From x ５ Between section 5 → 6 and section 8 → 9 (i.e., x ５ From x ９ ), and the section from section 22 → 23 to section 25 → 26 (i.e., x ２２ From x ２６ The slope of each ｎ→ｎ＋１ In addition, the section from section 16 to 17 to section 21 to 22 (i.e., x １６ From x ２２ The slope a ｎ→ｎ＋１ In addition, the section from section 9 to 10 to section 15 to 16 (i.e., x ９ From x １６ The position of the soccer ball on the two-dimensional coordinate system (x ｎ , y ｎ) draws a parabola with a slope a ｎ→ｎ＋１ This is the section where changes.

[0040] The term "constant slope" includes not only cases where the slope is mathematically strictly constant, but also cases where the slope is substantially constant.

[0041] The position of the soccer ball on the two-dimensional coordinate system (x ｎ , y ｎ If the slope of the ball changes like a parabola, the position of the soccer ball on the two-dimensional coordinate system (x ｎ , y ｎ ) on the two-dimensional coordinate system (i.e., on the x-y plane) changes its slope so that it draws a mathematically exact parabola. In addition, the position of the soccer ball on the two-dimensional coordinate system (x ｎ , y ｎ For example, when the trajectory of the soccer ball is a parabola on the two-dimensional coordinate system (i.e., on the xy plane), the calculation device 22 calculates the position (x ｎ , y ｎ ) can be judged to be a section where the slope changes like a parabola.

[0042] The method for determining whether the slope is constant, whether the slope changes discontinuously, or whether the position of the soccer ball on the two-dimensional coordinate system is changing in a parabolic manner is not particularly limited. For example, the calculation device 22 can determine that the slope is changing discontinuously if the slope is not constant and the position of the soccer ball on the two-dimensional coordinate system is not changing in a parabolic manner. That is, the calculation device 22 can determine, for example, whether the slope is constant and whether the position of the soccer ball on the two-dimensional coordinate system is changing in a parabolic manner, and can determine that the slope is changing discontinuously if the slope does not fit either of these cases. In other words, the calculation device 22 can determine whether the slope is changing discontinuously by a process of elimination.

[0043] (Step S33) Returning to FIG. 12, the calculation device 22 calculates the slope a stored in the storage device 24 in step S32. ｎ→ｎ＋１ In this embodiment, the slope a ｎ→ｎ＋１ The section where the slope a is constant is defined as the section where the soccer ball moves as if rolling on the ground. The calculation device 22 according to this embodiment calculates the slope a in the section from section 1 → 2 to section 4 → 5, the section from section 5 → 6 to section 8 → 9, and the section from section 22 → 23 to section 25 → 26. ｎ→ｎ＋１ is determined to be constant, and these sections are identified as sections in which a soccer ball moves as if rolling on the ground.

[0044] (Step S34) Next, the calculation unit 22 calculates the slope a stored in the storage unit 24 in step S32. ｎ→ｎ＋１ In this embodiment, the slope a ｎ→ｎ＋１ The calculation device 22 according to the present embodiment defines the section in which the slope a ｎ→ｎ＋１ is determined to be changing discontinuously, and this section is identified as a section in which the soccer ball is dribbled.

[0045] (Step S35) Next, the calculation device 22 calculates the slope a stored in the storage device 24 in step S32. ｎ→ｎ＋１ In this embodiment, the section in which the soccer ball moves as if flying through the air is identified by using the position (x ｎ , y ｎ ) draws a parabola with a slope a ｎ→ｎ＋１ The section in which the position of the soccer ball on the two-dimensional coordinate system (x ｎ , y ｎ ) draws a parabola with a slope a ｎ→ｎ＋１ is determined to be changing, and this section is identified as a section in which a soccer ball is flying through the air.

[0046] 15 is a schematic diagram illustrating the switching point. ５ At time t ９ At time t １６ The soccer ball's position on the two-dimensional coordinate system (x ｎ , y ｎ ) changes in a parabolic curve, and at time t ２２ The gradient, which had been discontinuously changing until then, changes at time t ５ , t ９ , t １６ , t ２２ are identified as switching points. A switching point refers to a point at which the type of movement of a soccer ball changes, and such a switch occurs, for example, when a player touches the soccer ball by kicking or heading it. For example, the time and position at which a soccer ball switches from rolling on the ground to flying in the air is an example of a point at which the type of movement of a soccer ball changes.

[0047] FIG. 16 is a diagram illustrating the switching points (black circles) and each section identified in step S3 on the xy plane. In FIG. 16, the above-mentioned switching points are indicated by black circles. Also, in FIG. 16, the section identified in step S3 where the ball moved as if rolling on the ground is indicated by "G," the section where the ball was dribbled is indicated by "D," and the section where the ball moved as if flying through the air is indicated by "F." In section F in FIG. 16, the position of the soccer ball on the two-dimensional coordinate system describes a parabola.

[0048] (Step S4) Returning to FIG. 7, the calculation device 22 calculates the predetermined reference height by calculating the position (x) of the soccer ball on the two-dimensional coordinate system in the section where the soccer ball moves as if rolling on the ground and the section where the soccer ball is dribbled. ｎ , y ｎ ) and stored in the storage device 24. In this embodiment, the calculation device 22 sets the predetermined reference height to 0 and stores the １ From time t ５ Each section up to time t５ From time t ９ Each section up to time t １６ From time t ２２ Each section up to time t ２２ From time t ２６ An example of the data stored in the storage device 24 after step S4 is executed is shown in FIG.

[0049] (Step S5) Next, the calculation device 22 calculates the position (x, y) of the soccer ball on the two-dimensional coordinate system during the section in which the soccer ball moves as if flying through the air, and the start time t ９ The position of the soccer ball on the two-dimensional coordinate system (x ９ , y ９ ) and the end time t １６ The position of the soccer ball on the two-dimensional coordinate system (x １６ , y １６ ) are read from the storage device 24, and the positions thus read are used to recalculate the position (x, y) of the soccer ball in two-dimensional coordinates during the section in which the soccer ball travels as if flying through the air, and also to calculate the position z of the soccer ball in the height direction during the section in which the soccer ball travels as if flying through the air.

[0050] 18 is a schematic diagram illustrating the recalculation of the position on the two-dimensional coordinate system and the calculation of the position in the height direction in step S5. The method of recalculating the position on the two-dimensional coordinate system and the method of calculating the position in the height direction are not limited. For example, as shown in FIG. 18, in step S5, the calculation device 22 calculates the position on the two-dimensional coordinate system at time t ９ and time t １６ Find the line connecting the time t ９ From time t １６ The intersection point (for example, time t １２ xp, yp) is calculated as the position on the two-dimensional coordinate system in step S5 (for example, time t １２ About x １２ , y １２ ) and recalculate it as follows.９ From time １６ The calculation device 22 calculates the position z of the soccer ball in the height direction by calculating the difference (for example, the length of the perpendicular line) between the trajectory of the soccer ball (i.e., the parabola) and the position z of the soccer ball in the height direction. The calculation device 22 calculates the position z of the soccer ball in the height direction by using the calculated difference itself or by performing a predetermined calculation on the calculated difference.

[0051] (Step S6) Returning to FIG. 8 , the calculation device 22 updates the position of the soccer ball on the two-dimensional coordinate system during the section where the soccer ball travels as if flying through the air, which is stored in the storage device 24, using the position on the two-dimensional coordinate system during the section where the soccer ball travels as if flying through the air, which was recalculated in step S5. For example, １２ The position on the two-dimensional coordinate system (x １２ , y １２ ), then (xo, yo) is changed to (xp, yp).

[0052] (Step S7) Next, the calculation device 22 stores in the storage device 24 the height position of the soccer ball in the section where it moved as if flying through the air, calculated in step S5, in association with the updated position of the soccer ball in the two-dimensional coordinate system in the section where it moved as if flying through the air.

[0053] 19 is a diagram showing an example of data stored in the storage device 24 after steps S6 and S7 are executed. As shown in FIG. 19, in step S6, the calculation device 22 calculates the position on the two-dimensional coordinate system recalculated in step S5 as, for example, (x ９ , y ９ ) to (x １6 , y １６ 19, in step S7, the calculation unit 22 calculates the position z in the height direction calculated in step S5, and stores the position z in the height direction in the storage device 24 as a position on the two-dimensional coordinate system up to time t ９ From time t １６ The value of z at (i.e. z ９ From Z １６ ) is stored in the storage device 24.

[0054] (Step S8) Returning to Fig. 8, in response to the output request, the calculation device 22 outputs the position in the two-dimensional coordinate system and the position in the height direction of the soccer ball stored in the storage device 24, i.e., the position in the three-dimensional coordinate system, within a range according to the output request. For example, the calculation device 22 outputs the range according to the output request at time t ３ From time t １８ If the range is , the position on the three-dimensional coordinate system in this range is (x ３ , y ３ , z ３ ) to (x １８ , y １８ , z １８ ) is output.

[0055] The target to which the calculation device 22 outputs (hereinafter referred to as the "output-receiving device") is not particularly limited. For example, the calculation device 22 can output a position in three-dimensional coordinates to a monitor, an analysis device, or the like. The output-receiving device can use the outputted position in three-dimensional coordinates to display or analyze the trajectory of the soccer ball on a screen. Furthermore, by using the outputted position in three-dimensional coordinates as learning data for an AI such as a generation AI, various soccer ball trajectories can be generated.

[0056] FIG. 20 shows the height direction position stored in step S7 at time t １ From time t ２６ As shown in FIG. 20, according to this embodiment, the soccer ball is １ From t ９ and the interval up to time t １６ From time t ２6 In the intervals, the height z is 0 and the time t ９ From time １６ It can be seen that the height z is greater than 0 in the section up to time t ９ From time １６ It can be seen that the soccer ball is floating in the air in the section up to .

[0057] [Information Processing System According to Embodiment 2] Fig. 21 is a schematic diagram illustrating an information processing system 2 according to embodiment 2. Fig. 22 is a schematic diagram illustrating the parallax between cameras 12 and 14. As shown in Figs. 21 and 22, the information processing system 2 according to embodiment 2 uses a plurality of cameras 12 and 14, and in this respect differs from the information processing system 1 according to embodiment 1 that uses a single camera 10, but has the same configuration in other respects.

[0058] The parallax θ between cameras 12 and 14 is at least 0 degrees or more and 19 degrees or less, preferably 0 degrees or more and 5 degrees or less, more preferably 0 degrees or more and 1 degree or less, and even more preferably 0 degrees or more and 0.7 degrees or less. According to this embodiment, even when multiple cameras are arranged with such small parallax, the height position of the soccer ball can be calculated relatively accurately. In conventional stereo methods, multiple cameras are arranged at intervals of 100 meters to increase the parallax, but according to this embodiment, there is no need to increase the parallax between cameras to improve the accuracy of height position measurement. In other words, even when the parallax between cameras is small, the height position of the soccer ball can be measured accurately. Therefore, the distance (e.g., shortest distance) between cameras 12 and 14 is, for example, 1 meter or more and 3 meters or less. In other words, according to this embodiment, even when cameras 12 and 14 are separated by only, for example, 1 meter or more and 3 meters or less, the three-dimensional position of the soccer ball, including the height position of the soccer ball, can be measured accurately. According to this embodiment, the cameras 12 and 14 can be placed close together so that they can be managed by one person. This significantly reduces the burden of operating and managing the cameras 12 and 14, and significantly reduces the cost of measuring the position of a soccer ball.

[0059] Although two cameras 12 and 14 are shown in FIG. 22 , the number of cameras may be three or more as long as the parallax between the cameras is within the above-mentioned range. For example, when three cameras are used, the parallax between the first camera and the second camera is 0 degrees or more and 19 degrees or less, the parallax between the second camera and the third camera is 0 degrees or more and 19 degrees or less, and the parallax between the first camera and the third camera is 0 degrees or more and 19 degrees or less. Note that, for example, if a fourth camera is installed but the parallax between this fourth camera and at least one of the three cameras is not within the above-mentioned range, this fourth camera does not qualify as a camera according to this embodiment. However, even in this case, the remaining three cameras qualify as cameras according to this embodiment.

[0060] As described above, in the information processing systems according to the first and second embodiments, the information processing device uses the position of the soccer ball on two-dimensional coordinates to identify, on the two-dimensional coordinates, the sections in which the soccer ball rolled on the ground, the sections in which it was dribbled, and the sections in which it flew through the air, respectively, and stores a predetermined reference height in association with the positions of the soccer ball on the two-dimensional coordinates in the sections in which it rolled on the ground and the sections in which it was dribbled, recalculates the position of the soccer ball on two-dimensional coordinates in the sections in which it flew through the air and calculates its height position, updates the position of the soccer ball on two-dimensional coordinates in the sections in which it flew through the air, and stores the calculated height position in association with the position of the soccer ball on two-dimensional coordinates in this updated section.

[0061] According to the first and second embodiments described above, the trajectory of a soccer ball on a two-dimensional coordinate system is divided into a section in which the soccer ball rolls on the ground, a section in which the soccer ball is dribbled, and a section in which the soccer ball flies through the air. The position of the soccer ball on a three-dimensional coordinate system is calculated based on the characteristics of each divided section. This calculation can be performed using a single camera or multiple cameras with a parallax of 0 degrees or more and 19 degrees or less. Therefore, according to the first and second embodiments, the number of cameras can be reduced. Furthermore, even when multiple cameras are used, the multiple cameras can be positioned closely together so that the parallax between the cameras is smaller than in the past. For example, the distance between the multiple cameras can be set to 1 meter or more and 3 meters or less. Therefore, according to the first and second embodiments, the costs required for purchasing and installing imaging devices can be significantly reduced.

[0062] According to the first and second embodiments, the movement of a ball can be tracked and digitized from a video of a soccer game. In the first embodiment, this digitization is performed using a single camera 10, whereas in the second embodiment, it is performed using multiple cameras 12 and 14. Even when multiple cameras are used, the parallax between the cameras does not need to be large; it is sufficient that the parallax be in the range of 0 degrees to 19 degrees. Therefore, the multiple cameras only need to be installed, for example, 1 meter to 3 meters apart. They do not need to be installed 100 meters apart as in the past, and can be installed in a single location (or in locations close enough to be considered essentially a single location). Therefore, extensive preparation and equipment are not required for camera installation. While cameras used in stereo systems cost, for example, several million yen, the cameras 10, 12, and 14 used in the first and second embodiments can be cameras commonly used in everyday life, sold for, for example, several tens of thousands of yen. Therefore, according to the first and second embodiments, it is possible to reduce the cost required for measuring the position of a soccer ball to, for example, about one-hundredth of the conventional cost.

[0063] People generally move on a flat surface (the pitch) and do not fly in the air. For this reason, tracking people is possible with existing technology. In contrast, a ball can either roll on the pitch or fly in the air. Since photos taken with a camera are two-dimensional, it is particularly difficult to grasp its "height." Therefore, in the first and second embodiments, attention is focused on the fact that the movement of a soccer ball can be broadly classified into three types. The first type is rolling movement (i.e., movement along a flat surface), the second type is flying movement through the air (i.e., movement that describes a parabola), and the third type is dribbling (movement with small zigzag or other movements). In the first and second embodiments, the position of the soccer ball is calculated based on these classifications, making it possible to grasp the height of the soccer ball using a single camera or multiple cameras with a parallax of 0 degrees or more and 19 degrees or less.

[0064] According to the first and second embodiments, for example, it is possible to perform analysis relating to the measurement of the position of a soccer ball in real time (or almost real time). Therefore, using the position information of the soccer ball measured by applying the first and second embodiments, it is possible to plan a strategy using the analysis data from the first half, for example, at halftime.

[0065] Soccer is a sport in which players primarily use their feet to move the ball to a specific location. The rules prohibit players other than the goalkeeper from using their hands to move the ball during a game. Methods for moving the ball to a specific location include shooting, passing, and dribbling. Dribbling is a method of moving the ball, primarily using the player's feet to control the ball. Passing is a method of movement in which a player kicks the ball to move it to a specific destination. Passes can be divided into "ground passes," in which the soccer ball rolls on the ground, and "air passes," in which the ball moves in a parabolic arc through the air. Shots are used to score goals in soccer and are a way to move the ball into the goal area. Like passes, shots can be divided into two types: one in which the soccer ball rolls on the ground, and one in which the ball moves in a parabolic arc through the air.

[0066] Although the embodiments have been described above, these descriptions are merely examples and do not limit the configurations described in the claims. Even if the configuration differs from the above description, as long as it has the configurations described in the claims, it will still achieve the effects of the present invention and be included in the present invention.

[0067] 1, 2 Information processing system 10, 12, 14 Camera 20 Information processing device 22 Arithmetic device 24 Storage device

Claims

1. An information processing device that uses the position of a soccer ball on two-dimensional coordinates to identify, on the two-dimensional coordinates, sections in which the soccer ball rolls on the ground, sections in which it is dribbled, and sections in which it flies through the air; stores a predetermined reference height in association with the two-dimensional coordinate positions of the soccer ball in the sections in which it rolls on the ground and the sections in which it is dribbled; recalculates the two-dimensional coordinate position of the soccer ball in the sections in which it flies through the air and calculates its height position, updates the two-dimensional coordinate position of the soccer ball in the sections in which it flies through the air, and stores the calculated height position in association with the two-dimensional coordinate position of the soccer ball in the updated sections.

2. An information processing system comprising one or more cameras installed diagonally above a soccer field, and an information processing device having an arithmetic unit and a storage device, wherein the arithmetic unit calculates the position of the soccer ball on two-dimensional coordinates at predetermined time intervals by performing image analysis on the images captured by the one or more cameras, stores the calculated position of the soccer ball on two-dimensional coordinates in the storage device, identifies, using the position of the soccer ball on two-dimensional coordinates stored in the storage device, sections in which the soccer ball rolls on the ground, sections in which the soccer ball is dribbled, and sections in which the soccer ball flies in the air, respectively, and stores a predetermined reference height in the storage device in association with the positions of the soccer ball on two-dimensional coordinates in the sections in which the soccer ball rolls on the ground and the sections in which the soccer ball is dribbled, read from the storage device the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, the position of the soccer ball in two-dimensional coordinates at the start time of the section in which the soccer ball moved as if flying through the air, and the position of the soccer ball in two-dimensional coordinates at the end time of the section in which the soccer ball moved as if flying through the air, and use the read positions to recalculate the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air and calculate the position of the soccer ball in the height direction during the section in which the soccer ball moved as if flying through the air; update the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, which is stored in the storage device, using the recalculated position in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air; associate the calculated position in the height direction during the section in which the soccer ball moved as if flying through the air with the updated position in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, and store it in the storage device;an information processing system that, in response to an output request, outputs the position of the soccer ball in two-dimensional coordinates and the position in the height direction stored in the storage device within a range corresponding to the output request.

3. An information processing system according to claim 2, wherein the calculation device calculates the position (x, y, z) of the soccer ball on the two-dimensional coordinate system stored in the storage device. ｎ , y ｎ ) at time t ｎ＋１ The two-dimensional coordinates of the soccer ball (x ｎ＋１ , y ｎ＋１ ) and time t ｎ The two-dimensional coordinates of the soccer ball (x ｎ , y ｎ ) the slope of the line connecting ｎ→ｎ＋１ The calculated slope a ｎ→ｎ＋１ is stored in a storage device in association with the interval (n → n+1), and the slope a ｎ→ｎ＋１ is determined as a section in which a soccer ball moves as if rolling on the ground, and the slope a ｎ→ｎ＋１ The section where the position of the soccer ball on the two-dimensional coordinate system (x ｎ , y ｎ ) is stored in the storage device so that it draws a parabola ｎ→ｎ＋１ The information processing system identifies a section where the value changes as a section where a soccer ball moves like flying through the air.

4. An information processing system according to claim 2 or 3, wherein the number of cameras is two or more, and the parallax of the two or more cameras is between 0 degrees and 19 degrees. 5.An information processing device having a calculation device and a storage device, wherein the calculation device calculates the position of the soccer ball on two-dimensional coordinates at predetermined time intervals by performing image analysis on images taken by one or more cameras installed diagonally above a soccer field, stores the calculated position of the soccer ball on two-dimensional coordinates in the storage device, identifies, using the position of the soccer ball on two-dimensional coordinates stored in the storage device, a section in which the soccer ball moved as if rolling on the ground, a section in which the soccer ball was dribbled, and a section in which the soccer ball moved as if flying through the air, and stores a predetermined reference height in the storage device in association with the positions of the soccer ball on two-dimensional coordinates in the section in which the soccer ball moved as if rolling on the ground and the section in which the soccer ball was dribbled, read from the storage device the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, the position of the soccer ball in two-dimensional coordinates at the start time of the section in which the soccer ball moved as if flying through the air, and the position of the soccer ball in two-dimensional coordinates at the end time of the section in which the soccer ball moved as if flying through the air, and use the read positions to recalculate the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air and calculate the position of the soccer ball in the height direction during the section in which the soccer ball moved as if flying through the air; update the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, which is stored in the storage device, using the recalculated position in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air; associate the calculated position in the height direction during the section in which the soccer ball moved as if flying through the air with the updated position in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, and store it in the storage device; In response to an output request, the information processing device outputs the position of the soccer ball in two-dimensional coordinates and the position in the height direction stored in the storage device within a range corresponding to the output request.

6. An information processing device according to claim 5, wherein the calculation device calculates the position (x, y, z) of the soccer ball on the two-dimensional coordinate system stored in the storage device. ｎ , y ｎ ) at time t ｎ＋１ The two-dimensional coordinates of the soccer ball (x ｎ＋１ , y ｎ＋１ ) and time t ｎ The two-dimensional coordinates of the soccer ball (x ｎ , y ｎ ) the slope of the line connecting ｎ→ｎ＋１ The calculated slope a ｎ→ｎ＋１ is stored in a storage device in association with the interval (n → n+1), and the slope a ｎ→ｎ＋１ is determined as a section in which a soccer ball moves as if rolling on the ground, and the slope a ｎ→ｎ＋１ The section where the position of the soccer ball on the two-dimensional coordinate system (x ｎ , y ｎ ) is stored in the storage device so that it draws a parabola ｎ→ｎ＋１ The information processing device identifies a section where the value changes as a section where a soccer ball moves as if flying through the air.

7. An information processing device according to claim 5 or 6, wherein the number of the cameras is two or more, and the parallax of the two or more cameras is between 0 degrees and 19 degrees.

8. An information processing method executed in an information processing device having a calculation device and a storage device, wherein the calculation device calculates the position of the soccer ball on two-dimensional coordinates at predetermined time intervals by image analysis of images taken by one or more cameras installed diagonally above a soccer field, the calculation device stores the calculated position of the soccer ball on two-dimensional coordinates in the storage device, the calculation device uses the position of the soccer ball on two-dimensional coordinates stored in the storage device to identify, on the two-dimensional coordinates, a section where the soccer ball moved as if rolling on the ground, a section where the soccer ball was dribbled, and a section where the soccer ball moved as if flying through the air, and the calculation device stores, in the storage device, a predetermined reference height associated with the positions of the soccer ball on two-dimensional coordinates in the section where the soccer ball moved as if rolling on the ground and the section where the soccer ball was dribbled, the arithmetic device reads from the storage device the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, the position of the soccer ball in two-dimensional coordinates at the start time of the section in which the soccer ball moved as if flying through the air, and the position of the soccer ball in two-dimensional coordinates at the end time of the section in which the soccer ball moved as if flying through the air, and uses the read positions to recalculate the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air and calculate the position of the soccer ball in the height direction during the section in which the soccer ball moved as if flying through the air; the arithmetic device updates the position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air, which is stored in the storage device, using the recalculated position of the soccer ball in two-dimensional coordinates during the section in which the soccer ball moved as if flying through the air; the calculation device associates the calculated position in the height direction in the section in which the soccer ball travels as if flying through the air with the updated position on the two-dimensional coordinates of the soccer ball in the section in which the soccer ball travels as if flying through the air, and stores the result in the storage device;An information processing method in which, in response to an output request, the arithmetic device outputs the position of the soccer ball in two-dimensional coordinates and the position in the height direction stored in the storage device within a range corresponding to the output request.

9. An information processing method according to claim 8, wherein the calculation device calculates the position (x, ｎ , y ｎ ) at time t ｎ＋１ The two-dimensional coordinates of the soccer ball (x ｎ＋１ , y ｎ＋１ ) and time t ｎ The two-dimensional coordinates of the soccer ball (x ｎ , y ｎ ) the slope of the line connecting ｎ→ｎ＋１ The calculation device calculates the calculated slope a ｎ→ｎ＋１ is stored in a storage device in association with the interval (n → n+1), and the arithmetic unit calculates the slope a stored in the storage device. ｎ→ｎ＋１ The calculation device determines a section in which the slope a is constant as a section in which the soccer ball moves as if rolling on the ground, and ｎ→ｎ＋１ The calculation device determines a section in which the position (x ｎ , y ｎ ) is stored in the storage device so that it draws a parabola ｎ→ｎ＋１ The information processing method identifies a section where the value changes as a section where a soccer ball moves like flying through the air.

10. An information processing method according to claim 8 or 9, wherein the number of cameras is two or more, and the parallax of the two or more cameras is between 0 degrees and 19 degrees.

Images