Orbit calculation device, orbit calculation method, and computer readable recording medium

Abstract

The present invention provides a trajectory calculation device, a trajectory calculation method, and a computer-readable recording medium. In sports competitions such as figure skating that involve jumping, there is a need to calculate the trajectory of a moving object with high precision based on dynamic images captured by a simple single-lens reflex camera, thereby measuring the height and flight distance of the jump. The device, method, and program provided by the present invention calculate the three-dimensional trajectory of a moving object that is a target of a jump based on information from multiple image frames captured by a single-lens reflex camera, and are characterized in that a specific point of the moving object that is a target is detected based on information from the image frame, a change amount related to the three-dimensional position of the specific point is calculated based on information from multiple consecutive image frames, and the three-dimensional trajectory of the specific point is calculated based on the positional relationship between a curve that can describe parabolic motion through a take-off point and a landing point and a straight line where the specific point exists.

Description

Technical Field

[0001] The invention relates to an orbit calculation device, an orbit calculation method and a computer-readable recording medium. Background Art

[0002] In sports competitions involving jumping, such as figure skating, there is a need to calculate the trajectory of the moving object based on the captured dynamic images to measure the jumping height and flying distance.

[0003] In sports competitions conducted on the field, there is known a technique for calculating the position of a moving object in three-dimensional space based on image data captured by cameras installed at positions within the field suitable for measuring the height of a jump or the distance of a flight. For example, there is a technique described in Patent Document 1.

[0004] In general, the trajectory of a moving object is calculated by obtaining continuous three-dimensional positions of the moving object from stereoscopic image data captured by a plurality of cameras in synchronization. For example, there is a technique described in Patent Document 2.

[0005] In addition, a technique for estimating a two-dimensional human posture from image data is disclosed in Non-Patent Document 1. Non-Patent Document 1 is mentioned in the section on modes for carrying out the invention.

[0006] However, in sports competitions such as figure skating, where cameras cannot be installed in the ice rink, the technology described in Patent Document 1 cannot be used. In addition, in the stereo matching described in Patent Document 2, multiple cameras need to be synchronized to shoot, but if the trajectory of the moving object can be calculated with high accuracy based on the dynamic image captured by the single-lens camera, it is also possible to simplify the shooting equipment used to measure the height of the jump and the flying distance.

[0007] Patent Document 1: Japanese Patent Application Publication No. 2001-336917

[0008] Patent Document 2: Japanese Patent Application Publication No. 2005-235104

[0009] Non-patent literature 1: Zhe Cao Je, et al., 'Realtime Multi-Person 2D PoseEstimation using Part Affinity Fields' arXiv:1611.08050v2 [cs.CV] 14Apr 2017 Summary of the invention

[0010] Therefore, an object of the present invention is to provide an apparatus, method, and program for calculating the trajectory of a moving object with high accuracy based on a dynamic image captured by a single-lens reflex camera in a sports competition involving jumping, such as figure skating.

[0011] One embodiment of the present invention is a trajectory calculation device that calculates a three-dimensional trajectory of a moving object that is a jumping target based on information of a plurality of image frames captured by a single-lens reflex camera, wherein the trajectory calculation device comprises: a detection unit that detects a specific point of the moving object that is a jumping target based on information of the image frame; an existence line calculation unit that calculates an existence line that connects the three-dimensional position of an image of the specific point captured in the image frame with the three-dimensional position of an optical center of a lens of the single-lens reflex camera; a starting point calculation unit that calculates a change amount related to the three-dimensional position of the specific point based on information of a plurality of consecutive image frames, determines a starting point from which the moving object that is a jumping target jumps based on the calculated change amount, and calculates the location of the specific point at the starting point. a three-dimensional position; a landing point calculation unit, which calculates a change amount related to the three-dimensional position of a specific point based on information of a plurality of continuous image frames, determines the landing point where the target moving object lands based on the calculated change amount, and calculates the three-dimensional position of the specific point at the landing point; and a trajectory calculation unit, which calculates the three-dimensional trajectory of the specific point based on the positional relationship between the curve and the existing straight line, wherein the curve is a curve capable of describing the parabolic motion of the three-dimensional position of the specific point at the take-off point calculated by the take-off point calculation unit and the three-dimensional position of the specific point at the landing point calculated by the landing point calculation unit, and the existing straight line is related to the image of the specific point captured in one or more image frames after the target moving object takes off and before landing.

[0012] Here, the specific point of the moving object as the target refers to a point determined in order to calculate the trajectory of the moving object as the target based on the information of the image frame. For example, in a figure skating competition described as an embodiment of the present invention described later, the front end of the edge of the skates worn by the skater is set as the specific point. The specific point is the same in the specification of this application.

[0013] In this specification, as described above, a straight line connecting the three-dimensional position of the image of a specific point captured in an image frame and the three-dimensional position of the optical center of the lens of the single-lens camera is referred to as an existence straight line.

[0014] Here, continuous image frames refer to a state where there are no other image frames between two image frames, and the image frame numbers do not necessarily need to be continuous. For example, even if two image frames are continuous, there may be a situation where the image frame numbers are discontinuous due to the image frame that originally existed between them being dropped. In addition, multiple image frames that are set to be continuous have multiple image frames that are continuous in the order of moments along the same time stream. Therefore, in the present invention, in addition to the captured image information, the moment when the image frame was captured is also included as information in the image frame information. This is the same in the specification of this application.

[0015] In a trajectory calculation device as one embodiment of the present invention, a take-off point calculation unit calculates a change amount related to a three-dimensional position of a specific point based on information of a plurality of image frames continuous in positive order of shooting time. When the calculated change amount is greater than a threshold value, it is considered that the moving object as the target has taken off and the take-off point is determined, and the three-dimensional position of the specific point at the take-off point is calculated.

[0016] As a trajectory calculation device of one embodiment of the present invention, the landing point calculation unit calculates the change amount related to the three-dimensional position of a specific point based on the information of a plurality of image frames continuous in reverse order of the shooting time. When the calculated change amount is greater than a threshold value, it is considered that the moving object as the target has landed and the landing point is determined, and the three-dimensional position of the specific point at the landing point is calculated.

[0017] As a trajectory calculation device of one embodiment of the present invention, a landing point calculation unit calculates a change amount related to the three-dimensional position of a specific point based on information of a plurality of image frames that are continuous in positive order after the moment when the take-off point is photographed, and when the calculated change amount is reversed in sign, it is determined that the moving object as the target has reached the apex of the jump and the moment when the apex is photographed is obtained, and the change amount related to the three-dimensional position of the specific point is calculated based on information of a plurality of image frames that are continuous in reverse order from the moment when the take-off point is photographed after a period of time, and when the calculated change amount is greater than a threshold value, it is determined that the moving object as the target has landed and the landing point is determined, and the three-dimensional position of the specific point at the landing point is calculated, and the above-mentioned time is a time obtained by subtracting twice the time obtained by taking the take-off point from the moment when the apex is photographed plus a certain time.

[0018] In a trajectory calculation device as one embodiment of the present invention, a trajectory calculation unit calculates a curve as a three-dimensional trajectory of a specific point, wherein the curve is a curve capable of describing the parabolic motion of the three-dimensional position of the specific point at the take-off point calculated by the take-off point calculation unit and the three-dimensional position of the specific point at the landing point calculated by the landing point calculation unit, and the total distance of the curve to the image of one or more specific points captured in one or more image frames after the moving object as the target takes off and before landing is minimized.

[0019] In a trajectory calculation device as one embodiment of the present invention, a trajectory calculation unit calculates a curve as a three-dimensional trajectory of a specific point, wherein the curve is a curve capable of describing the parabolic motion of the three-dimensional position of the specific point at the take-off point calculated by the take-off point calculation unit and the three-dimensional position of the specific point at the landing point calculated by the landing point calculation unit, and the curve is a curve that calculates the distance between an existing straight line related to an image of the specific point captured in an image frame at the time of shooting and a predicted position on the curve at the time when the specific point is captured for one or more specific points of the moving object as a target after take-off and before landing, and minimizes the sum of the calculated one or more distances to the predicted position.

[0020] The present invention is a trajectory calculation method, which calculates the three-dimensional trajectory of a moving object as a jumping target based on information of multiple image frames captured by a single-lens reflex camera, wherein the trajectory calculation method comprises the following steps: a detection step, which detects a specific point of the moving object as a target based on information of the image frame; an existence straight line calculation step, which calculates an existence straight line formed by connecting the three-dimensional position of the image of the specific point captured in the image frame and the three-dimensional position of the optical center of the lens of the single-lens reflex camera; a starting point calculation step, which calculates a change amount related to the three-dimensional position of the specific point based on information of multiple consecutive image frames, and determines the starting point after the moving object as the target jumps based on the calculated change amount, and calculates the three-dimensional position of the specific point at the starting point. a landing point calculation step, calculating a change amount related to the three-dimensional position of a specific point based on information of a plurality of continuous image frames, determining the landing point where the moving object as a target lands based on the calculated change amount, and calculating the three-dimensional position of the specific point at the landing point; and a trajectory calculation step, calculating the three-dimensional trajectory of the specific point based on the positional relationship between the curve and the existing straight line, wherein the curve is a curve capable of describing the parabolic motion of the three-dimensional position of the specific point at the take-off point calculated in the take-off point calculation step and the three-dimensional position of the specific point at the landing point calculated in the landing point calculation step, and the existing straight line is related to the image of the specific point captured in one or more image frames after the moving object as the target takes off and before landing.

[0021] The present invention is a computer-readable recording medium, which records a trajectory calculation program, and the trajectory calculation program enables a computer to calculate a three-dimensional trajectory of a moving object as a jumping target based on information of a plurality of image frames captured by a single-lens reflex camera, wherein the trajectory calculation program enables the computer to execute the following steps: a detection step, in which a specific point of the moving object as a target is detected based on information of the image frame; an existence straight line calculation step, in which an existence straight line is calculated that connects the three-dimensional position of the image of the specific point captured in the image frame with the three-dimensional position of the optical center of the lens of the single-lens reflex camera; a starting point calculation step, in which a change amount related to the three-dimensional position of the specific point is calculated based on information of a plurality of continuous image frames, and a starting point is determined after the moving object as a target jumps based on the calculated change amount, The three-dimensional position of a specific point at the take-off point is calculated; a landing point calculation step is performed to calculate a change amount related to the three-dimensional position of the specific point based on information from a plurality of consecutive image frames, and the landing point where the target moving object lands is determined based on the calculated change amount, and the three-dimensional position of the specific point at the landing point is calculated; and a trajectory calculation step is performed to calculate the three-dimensional trajectory of the specific point based on the positional relationship between the curve and the existing straight line, wherein the curve is a curve that can describe the parabolic motion through the three-dimensional position of the specific point at the take-off point calculated in the take-off point calculation step and the three-dimensional position of the specific point at the landing point calculated in the landing point calculation step, and the existing straight line is related to the image of the specific point captured in one or more image frames after the target moving object takes off and before landing.

[0022] According to the present invention, in sports competitions such as figure skating involving jumping, the trajectory of a moving object can be calculated with high accuracy based on a moving image captured by a single-lens reflex camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a block diagram of the orbit calculation device.

[0024] Figure 2 It is the hardware structure diagram of the orbit calculation device.

[0025] Figure 3 The figure schematically depicts a skater jumping in a figure skating competition.

[0026] Figure 4 This is a flowchart showing the operation of the trajectory calculation device according to the embodiment.

[0027] Figure 5 This is a diagram showing a coordinate system used in the description of the embodiment.

[0028] Figure 6 It means in Figure 5 A diagram showing a situation in which a specific point serving as a target in a three-dimensional space is photographed using a camera in a coordinate system.

[0029] Figure 7 It is a diagram for explaining the take-off point calculation process and the landing point calculation process.

[0030] Figure 8 It is a diagram showing an example of orbit calculation processing.

[0031] Fig. 9 It is a diagram showing another example of orbit calculation processing.

[0032] Explanation of symbols

[0033] 1 orbit calculation device,

[0034] 101 Inspection Department,

[0035] 102 There is a straight line calculation unit,

[0036] 103 Jumping point calculation department,

[0037] 104 Landing point calculation department,

[0038] 105 Orbital Calculation Department,

[0039] 201 Central Processing Unit (CPU),

[0040] 202 input device,

[0041] 203 output device,

[0042] 204 main storage device (RAM / ROM),

[0043] 205 auxiliary storage device,

[0044] 301 The front edge of the skate worn by the jumping skater (a specific point at the take-off point),

[0045] 302 The front end of the edge of the skate worn by the jumping skater (a specific point at the landing point),

[0046] 303 The front edge of the skate worn by the jumping skater (a specific point in the air),

[0047] 304 The front edge of the skate worn by the jumping skater (a specific point in the air),

[0048] 305 The front edge of the skate worn by the jumping skater (a specific point in the air),

[0049] 501The origin of the three-dimensional orthogonal coordinate system ∑XYZ,

[0050] 502The optical center of the camera lens,

[0051] 503The origin of the two-dimensional orthogonal coordinate system ∑uv,

[0052] 504 image frames (photographing element surface),

[0053] 505 reference points,

[0054] 506 reference points,

[0055] 507 benchmark points,

[0056] 508 benchmark points,

[0057] The ice surface of the 509 ice rink,

[0058] 701 The point at the front end of the edge of the skate worn by the skater,

[0059] 702 The image of the point at the front end of the edge of the skate worn by the skater,

[0060] 801 about the existence of a line at a specific point 303,

[0061] 802 about the existence of a straight line of a specific point 304,

[0062] 803 The distance between the curve 805 and the existing straight line 801,

[0063] 804 The distance between the curve 805 and the existence straight line 802,

[0064] 805 can describe the curve of parabolic motion through the take-off point and the landing point,

[0065] 901 is the predicted position of the specific point on the curve 905 at the time when the specific point 303 is photographed,

[0066] 902 is the predicted position of the specific point on the curve 905 at the time when the specific point 304 is photographed,

[0067] 903 The distance between the predicted position 901 of a specific point on the curve 905 and the existence straight line 801,

[0068] 904 The distance between the predicted position 902 of a specific point on the curve 905 and the existence straight line 802,

[0069] 905 can describe a curve of parabolic motion passing through the take-off point and the landing point. DETAILED DESCRIPTION

[0070] Embodiments of the present invention will be described with reference to the accompanying drawings. In addition, overlapping descriptions are omitted, and the same or corresponding parts in each drawing are denoted by the same reference numerals.

[0071] The trajectory calculation device, method, and program according to the present embodiment can calculate the trajectory of a moving object with high accuracy from a moving image captured by a single-lens camera in sports competitions involving jumping, such as figure skating.

[0072] Figure 1 1 is a block diagram of a trajectory calculation device. The trajectory calculation device 1 may be configured as a single device or may be used in combination with other devices. Other devices in which the trajectory calculation device 1 is assembled may be electronic products such as smart phones, portable information terminals, digital cameras, game terminals, and televisions. Figure 2 As shown, the trajectory calculation device 1 is physically configured as a computer including a central processing unit (CPU) 201 , an input device 202 , an output device 203 , a main storage device (RAM / ROM) 204 , and an auxiliary storage device 205 .

[0073] The functions of the orbit calculation device 1 are realized by: Figure 2 The central processing unit (CPU) 201, main storage device (RAM / ROM) 204, etc. shown read the program that enables the computer to function, so as to track the position changes of a specific point of a moving object as a target based on the dynamic image captured by the SLR camera, and calculate the trajectory of the three-dimensional position without losing it even if the specific point of the moving object as a target is blocked. Therefore, the input device 202 and the output device 203 are activated through the control of the central processing unit (CPU) 201, and data is read and written with the main storage device (RAM / ROM) 204 and the auxiliary storage device 205.

[0074] like Figure 1 As shown in FIG. 1 , the trajectory calculation device 1 includes a detection unit 101, an existence line calculation unit 102, a take-off point calculation unit 103, a landing point calculation unit 104, and a trajectory calculation unit 105. In the trajectory calculation device 1, information of an image frame related to a dynamic image of a target moving object captured by a single-lens reflex camera is provided as an input from the outside. The information of the image frame includes information of the time when the image frame was captured in addition to the captured image information. In addition, the trajectory calculation device 1 outputs information related to the trajectory of a specific point of the target moving object to the outside.

[0075] according to Figure 1 The function of each module of the trajectory calculation device 1 is explained with reference to the block diagram of FIG. In addition, the detailed operation of each module will be described later in the embodiments.

[0076] The detection unit 101 detects a specific point of a target moving object based on information of an image frame.

[0077] The existence line calculation unit 102 calculates an existence line connecting the three-dimensional position of the image of the specific point captured in the image frame and the three-dimensional position of the optical center of the lens of the single-lens camera. The specific calculation process will be described later in the embodiment.

[0078] The jump point calculation unit 103 calculates the variation related to the three-dimensional position of the specific point based on the information of the continuous plurality of image frames, determines the jump point of the moving object as the target based on the calculated variation, and calculates the three-dimensional position of the specific point at the jump point. The specific calculation process will be described later in the embodiment.

[0079] The landing point calculation unit 104 calculates the change amount related to the three-dimensional position of the specific point based on the information of the continuous plurality of image frames, determines the landing point where the moving object as the target lands based on the calculated change amount, and calculates the three-dimensional position of the specific point at the landing point. The specific calculation process will be described later in the embodiment.

[0080] The trajectory calculation unit 105 calculates the three-dimensional trajectory of the specific point based on the positional relationship between the curve and the existing straight line, wherein the curve is a curve that can record the parabolic motion of the three-dimensional position of the specific point at the take-off point calculated by the take-off point calculation unit and the three-dimensional position of the specific point at the landing point calculated by the landing point calculation unit, and the existing straight line is a straight line related to the image of the specific point captured in one or more image frames after the moving object as the target takes off and before landing. The specific calculation process will be described later in the embodiment.

[0081] Next, the operation of the trajectory calculation device 1 of the present embodiment will be described. Considering the ease of understanding of the description, the operation of the trajectory calculation device 1 when calculating the trajectory of the tip of the edge of the skate worn by a jumping skater based on an image of a figure skating competition captured by a single-lens camera will be described as an example. In addition, this is only an example, and the present invention is not limited to being applicable only to figure skating competitions.

[0082] [Example]

[0083] Figure 3 FIG. 2 is a diagram schematically illustrating a situation in which a skater jumps in a figure skating competition described as a preferred example in this embodiment. Figure 3 301 to 305 respectively represent the front edge of the skates worn by the jumping skater. In this embodiment, the front edge of the skates is used as a specific point. 301 represents a specific point at the take-off point, 302 represents a specific point at the landing point, and 303 to 305 represent specific points in the air during the jump. In particular, 304 represents a specific point at the vertex in the jumping track.

[0084] In figure skating competitions, the height of the jump and the distance of flight are important factors that determine the quality of the performance, and there is a great demand for obtaining the skater's jump trajectory from the images taken of the figure skating competition. On the other hand, in recent years, the cameras built into devices such as so-called smart phones have also become more high-performance, and it has become common to simply shoot dynamic images with a single-lens reflex camera. If the trajectory of a moving object can be calculated with high precision based on the dynamic images captured by a single-lens reflex camera in sports competitions accompanied by jumps such as figure skating, it is possible to meet the demand for measuring the height of the jump and the distance of flight using simple equipment such as smart phones. Therefore, the present invention is suitable for use in figure skating competitions. This is only an example and does not limit the present invention to being applicable only to figure skating competitions.

[0085] Figure 4 is a flowchart showing the operation of the trajectory calculation device 1 of the embodiment. Figure 4 The operation of the trajectory calculation device 1 in the embodiment is explained with reference to the flowchart.

[0086] The information of a plurality of image frames captured by the SLR camera is input to the trajectory calculation device 1. In this embodiment, the information of the image frames includes the time when the image frames were captured in addition to the captured image information. The trajectory calculation device 1 starts the operation after all the information of the image frames to be processed is input. The operation can be started automatically after the image frame information is input, or it can be started based on an explicit command. The trajectory calculation device 1 processes the input image frame information. Figure 4 Flowchart of the process.

[0087] When the trajectory calculation device 1 starts to operate, the detection unit 101 performs a detection process (S401). In the detection process (S401), a specific point of the moving object captured as a target is detected from the input image data. In this embodiment, as described above, the point at the front end of the edge of the skates worn by the skater is taken as the specific point.

[0088] Regarding detecting a specific point from the image data, any method is not limited as long as the coordinates of the specific point on the two-dimensional plane in the image frame can be detected from the image data. For example, the technique of estimating the posture of a two-dimensional person from image data disclosed in non-patent document 1, etc., can be used. In addition, a method of detecting a mark pre-attached to a specific point of the skater as the photographed object from the image data can also be used. When the detection processing (S401) of the detection unit 101 is completed, the existence line calculation unit 102 starts the existence line calculation processing (S402).

[0089] Here, before explaining the existence line calculation process (S402) performed by the existence line calculation unit 102, Figure 4The coordinate system used in the description of this embodiment is described with reference to the flowchart of FIG. Figure 5 This is a diagram showing a coordinate system used in the description of this embodiment.

[0090] The coordinate system ΣXYZ is a three-dimensional orthogonal coordinate system in an actual three-dimensional space. In the present embodiment, the coordinate system ΣXYZ is a three-dimensional orthogonal coordinate system in which the shape of the ice surface 509 of the ice rink for figure skating competitions is set to be an oblong symmetrical in the major axis direction and the minor axis direction, the intersection of the major axis and the minor axis on the ice surface 509 is set to the origin O(0, 0, 0) 501, and the X axis is the major axis direction and the Y axis is the minor axis direction.

[0091] In addition, in this embodiment, as a plurality of points (hereinafter referred to as reference points) whose three-dimensional coordinates are known in the coordinate system ΣXYZ, three-dimensional coordinates are set as follows for two intersection points (505 and 506) between the outer edge of the ice surface 509 and the X axis, and two intersection points (507 and 508) between the outer edge of the ice surface 509 and the Y axis. Here, the length of the long axis direction of the ice rink is 2A, and the length of the short axis direction is 2B. A and B are constants.

[0092] Reference point 505: (X, Y, Z) = (A, 0, 0)

[0093] Reference point 506: (X, Y, Z) = (-A, 0, 0)

[0094] Reference point 507: (X, Y, Z) = (0, B, 0)

[0095] Reference point 508: (X, Y, Z) = (0, -B, 0)

[0096] In addition, this is only an example, and the setting of the three-dimensional coordinate system and the reference point is not limited to this. The setting of the three-dimensional coordinate system and the reference point is preferably appropriately set to match the motion characteristics of the target moving object, environmental conditions, etc. In this embodiment, as described later, in order to obtain the three-dimensional position coordinates of the optical center of the camera lens and the external parameters of the lens, the reference point with a known three-dimensional position coordinate is used.

[0097] The coordinate system Σxyz is a three-dimensional orthogonal coordinate system having an origin 502 at the optical center of the camera lens. The coordinate system Σuv is a two-dimensional orthogonal coordinate system having an origin 503 fixed to the image frame (camera element surface) 504. The coordinate system Σuv is a coordinate system obtained by parallel translation of the coordinate system Σxyz, and the xy coordinate axis is parallel to the uv coordinate axis. In addition, the intersection of the z axis of the coordinate system Σxyz and the image frame 504 is the origin 503 of the coordinate system Σuv. The z axis of the coordinate system Σxyz is orthogonal to the image frame 504.

[0098] The above is the description of the coordinate system used in the description of this embodiment.

[0099] Return to Figure 4 Now, we return to the description of the existence line calculation process performed by the existence line calculation unit 102. Figure 6 Calculation of a three-dimensional straight line formed by connecting the three-dimensional position of the image of a specific point as a target in the image frame and the optical center of the camera lens in the existence straight line calculation processing (S402) will be described.

[0100] Figure 6 It means in Figure 5 602. The figure shows a situation where a specific point P as a target in a three-dimensional space is photographed by a camera in a coordinate system of Σxyz. The three-dimensional position of the specific point P as a target in the three-dimensional orthogonal coordinate system Σxyz is set to P(X, Y, Z). The specific point P as a target is photographed as an image P' in the image frame 504 by a camera having an optical center of a lens at the origin 602 of the three-dimensional orthogonal coordinate system Σxyz. The two-dimensional position of the image P' in the two-dimensional orthogonal coordinate system Σuv in the image frame 504 is set to P'(u, v).

[0101] exist Figure 6 , the position of the optical center of the lens (the origin 502 of the three-dimensional orthogonal coordinate system Σxyz) in the three-dimensional orthogonal coordinate system ΣXYZ is set to P0 (X0, Y0, Z0). The z-axis of the three-dimensional orthogonal coordinate system Σxyz with the optical center of the lens as the origin 502 is equivalent to the optical axis. The optical axis is orthogonal to the image frame 504, and the intersection thereof becomes the origin 503 of the two-dimensional orthogonal coordinate system Σuv in the image frame. The two-dimensional position of the origin 503 is set to Q (u0, v0). The distance between the origin 502 of the three-dimensional orthogonal coordinate system Σxyz and the origin 503 of the two-dimensional orthogonal coordinate system Σuv is the focal length, and the length is set to d.

[0102] exist Figure 6 , the position of the image P' observed from the three-dimensional orthogonal coordinate system Σxyz with the optical center of the lens as the origin 502 is P'(u-u0, v-v0, d). Here, the proportionality constant is set to k, and the coordinate transformation matrix from the three-dimensional orthogonal coordinate system Σxyz with the optical center of the lens as the origin 502 to the three-dimensional orthogonal coordinate system ΣXYZ in the three-dimensional space is set to R. In this embodiment, if the position coordinates and rotation of the camera in the three-dimensional orthogonal coordinate system ΣXYZ in the three-dimensional space are provided, the coordinate transformation matrix R can be determined.

[0103] Rotation is the so-called extrinsic parameter of the lens and is Figure 6 The rotation angle parameters of the three axes when the coordinate system Σxyz is transformed into the coordinate system ΣXYZ. By determining the rotation value, the optical axis of the lens is determined. Figure 6 In the coordinate system Σxyz, the z-axis is the optical axis.

[0104] In addition, in the embodiment, the so-called internal parameters of the lens (focal length, lens distortion) are known. Figure 6 In the example, the distance between the optical center of the camera lens (the origin 502 of the coordinate system Σxyz) and the origin 503 of the coordinate system Σuv is the focal length. In addition, when the distortion of the lens affects the result, an image with the distortion removed is generated, and the subsequent processing is performed on the image after the distortion is removed. For example, the distortion correction function disclosed in OpenCV can be used for distortion removal.

[0105] In the three-dimensional orthogonal coordinate system Σxyz with the optical center of the lens as the origin 502, the optical center of the lens (origin 502), the image P' and the target specific point P are located on the same straight line, so the following equation (Equation 1) holds.

[0106] [Mathematical formula 1]

[0107] P(X,Y,Z)=P0(X0,Y0,Z0)+k RP′(u-u0,v-v0,d)

[0108] In this embodiment, u-u0 and v-v0 as the X and Y elements of P' can be obtained from the coordinates of the two-dimensional position given by the detection unit 101 to the existence line calculation unit 102, and d is the focal length and is known.

[0109] The above formula (Formula 1) is a formula for a straight line in the three-dimensional orthogonal coordinate system ΣXYZ connecting the optical center of the lens, the image P', and the target specific point P. When the specific point P is photographed using a camera having the optical center of the lens at the origin 502 of the three-dimensional orthogonal coordinate system Σxyz, if the two-dimensional coordinates of the image P' of the image frame 504 in the two-dimensional orthogonal coordinate system Σuv can be determined, the above formula (Formula 1) can be calculated, and in the three-dimensional orthogonal coordinate system ΣXYZ, the target specific point P exists on the straight line shown in the above formula (Formula 1).

[0110] When the three-dimensional position coordinates and rotation of the optical center of the camera lens are not provided in advance, the three-dimensional position coordinates of the optical center of the camera lens and the external parameters of the lens are obtained by using the reference points whose three-dimensional position coordinates are known, and the coordinate transformation matrix R is determined. As described above, in this embodiment, for the four points of reference points 505, 506, 507 and 508, the three-dimensional coordinates in the three-dimensional orthogonal coordinate system ΣXYZ are known as follows, and the two-dimensional coordinates of the image of each reference point in the two-dimensional orthogonal coordinate system Σuv can also be measured.

[0111] Reference point 505: (X, Y, Z) = (A, 0, 0)

[0112] Reference point 506: (X, Y, Z) = (-A, 0, 0)

[0113] Reference point 507: (X, Y, Z) = (0, B, 0)

[0114] Reference point 508: (X, Y, Z) = (0, -B, 0)

[0115] Use the above Figure 6 In the three-dimensional orthogonal coordinate system Σxyz with the optical center of the lens as the origin 502, the optical center of the lens (origin 502), the image P' and the target P are located on the same straight line, so the formula (Mathematical Formula 1) of the straight line in the three-dimensional orthogonal coordinate system ΣXYZ connecting the optical center of the lens in the three-dimensional orthogonal coordinate system Σxyz, the image P' and the specific point P as the target is calculated. In the formula (Mathematical Formula 1) of the straight line, P (X, Y, Z) is used as each reference point, P' (u, v) is used as the image of the image frame corresponding to each reference point, and d is known, so the optical center P0 (X0, Y0, Z0) of the lens of the camera in the three-dimensional orthogonal coordinate system ΣXYZ and the rotation constituting the coordinate transformation matrix R are calculated.

[0116] When calculating the optical center P0 (X0, Y0, Z0) of the lens of the camera in the three-dimensional orthogonal coordinate system ΣXYZ and the rotation constituting the coordinate transformation matrix R (rotation angle parameters of the three axes when the coordinate transformation is performed from the coordinate system Σxyz to the coordinate system ΣXYZ), the number of unknown elements is 6, and the image of the moving object as the target of the image frame is provided by the elements of the two-dimensional orthogonal coordinate system Σuv, so the number of reference points can be 3. However, considering that the calculation accuracy can be improved by increasing the number of reference points, in this embodiment, the number of reference points is set to 4. The number of reference points is preferably an appropriate number that matches the space to be measured and the configuration of the available reference points, and the number of reference points is not limited to 4.

[0117] In this embodiment, the position coordinates and rotation of the camera in the three-dimensional coordinate system are calculated by the nonlinear least square method. In addition, this is an example, and the processing method is not limited to this. As long as the position coordinates and rotation of the camera in the three-dimensional coordinate system can be calculated with high accuracy, it can also be performed by other processing methods. For example, it is also possible to consider calculations based on the LM method (Levenberg-Marquardt Method) that can expect high accuracy, and calculations based on the DLT method (Direct Linear Transformation method) after increasing the number of reference points.

[0118] The above is the description of the existence line calculation process (S402) performed by the existence line calculation unit 102. When the existence line calculation process (S402) by the existence line calculation unit 102 is completed, the starting point calculation unit 103 starts the starting point calculation process (S403).

[0119] In the take-off point calculation process (S403), the change amount related to the three-dimensional position of the specific point is calculated based on the information of multiple consecutive image frames, the take-off point of the moving object as the target is determined based on the calculated change amount, and the three-dimensional position of the specific point at the take-off point is calculated.

[0120] In this embodiment, in the take-off point calculation process (S403), the change amount related to the three-dimensional position of the specific point is calculated based on the information of multiple image frames continuous in positive order at the time of shooting. When the calculated change amount is greater than a threshold, the take-off point is determined as the take-off of the target moving object, and the three-dimensional position of the specific point at the take-off point is calculated.

[0121] use Figure 7 Next, the take-off point calculation process (S403) performed by the take-off point calculation unit 103 is described. In the take-off point calculation process (S403), it is assumed that the skater has not jumped, and the position of the specific point is calculated.

[0122] exist Figure 7 In the example, the coordinates of the two-dimensional position of the image 702 of the tip of the edge of the skate worn by the skater in the image frame 504 in the two-dimensional orthogonal coordinate system Σuv are delivered from the detection unit 101 to the existence line calculation unit 102. Figure 6 As described above, in the straight line calculation process (S402), the straight line equation (Formula 1) is used to find Figure 7 There is a straight line connecting the optical center (502) of the lens in and the image 702 of the specific point 701.

[0123] Next, in the take-off point calculation process (S403), it is assumed that the skater does not jump, and the position of the specific point is calculated. In the case where the skater does not jump, the specific point is the contact point 701 between the front end of the edge of the skates worn by the skater and the ice rink. There is a contact point 701 between the front end of the edge of the skates worn by the skater and the ice rink on the ice surface 509 of the ice rink. At this time, the specific point (the contact point 701 between the edge of the skates worn by the skater and the ice rink) exists on the existence line calculated by the existence line calculation process (S402) and exists on the ice surface 509 of the ice rink, so the intersection of the two is set as the specific point to calculate the coordinates of the three-dimensional position. In addition, the value of the Z coordinate of the calculated specific point in the three-dimensional orthogonal coordinate system ΣXYZ is 0.

[0124] As described above, in the jump point calculation process (S403), the position of the specific point is calculated on the assumption that the skater does not jump, but the skater actually jumps. When jumping, the tip of the edge of the skates worn by the skater does not actually contact the ice surface 509 of the ice rink. Therefore, if the intersection of the ice surface 509 of the ice rink and the straight line calculated in the existence straight line calculation process (S402) is set as the specific point, at the moment when the skater takes off, the values ​​of the X element and the Y element of the three-dimensional coordinates of the specific point change sharply in the direction away from the optical center of the camera lens.

[0125] Thus, as an example, the jump point calculation process (S403) adopts the following process. The predicted values ​​of the X and Y elements of the three-dimensional coordinates of the specific point based on the polynomial approximation are calculated, and the values ​​of the X and Y elements of the three-dimensional coordinates of the specific point are obtained based on the values ​​of the X and Y elements of the three-dimensional coordinates of the specific point, i.e., the contact point 509 of the edge tip of the skate worn by the skater and the ice rink, in the past five consecutive frames at the time of photographing. For two specific points adjacent in the positive order of the photographing time, the difference between the above-mentioned predicted values ​​of the X and Y elements of the three-dimensional coordinates is obtained. Based on the difference between the predicted values ​​of the X and Y elements of the three-dimensional coordinates, the movement amount of the specific point in the direction away from the optical center of the camera lens is calculated and used as the variation. For example, the case where the variation amount is 50 or more in the direction away from the optical center of the camera lens is determined as a variation exceeding the threshold value. In the jump point calculation process (S403), the change amount is calculated for the specific point in the positive order of the shooting time. When the calculated change amount is greater than a predetermined threshold, the target moving object jumps and the jump point is determined. The three-dimensional position of the specific point at the jump point is calculated and handed over to the next process. In addition, this is an example, and the jump point calculation process (S403) and the method of obtaining the threshold are not limited to this.

[0126] After the take-off point calculation process (S403) is completed, the landing point calculation unit 104 starts the landing point calculation process (S404). In this embodiment, in the landing point calculation process (S404), the change amount related to the three-dimensional position of the specific point is calculated based on the information of the continuous plurality of image frames, and the take-off point of the moving object as the target is determined based on the calculated change amount, and the three-dimensional position of the specific point at the take-off point is calculated.

[0127] The landing point calculation process (S404) calculates the three-dimensional position of the specific point at the landing point by performing the same process as the above-mentioned starting point calculation process (S403) on the information of the plurality of image frames that are continuous in reverse order of the shooting time. The starting point calculation process (S403) and the landing point calculation process (S404) differ in that the time when the plurality of continuous image frames are shot is in positive order and in reverse order, but the other processes are the same, so the description is omitted.

[0128] In addition, the landing point calculation process (S404) can also be performed by the process described below. The characteristics of the skater's jump being a parabolic motion are utilized to reduce the processing of the landing point calculation process (S404). Based on the information of a plurality of image frames that are continuous in the positive order of the shooting time after the time of shooting the take-off point, the change amount related to the three-dimensional position of the specific point is calculated. Here, as described above, the change amount refers to the movement amount of the specific point in the direction away from the optical center of the camera lens calculated based on the difference in the predicted values ​​of the values ​​of the X element and the Y element of the three-dimensional coordinates, and the change amount is used as the change amount. Next, the time of the image frame when the calculated change amount is shot is obtained when the positive and negative inversions are reversed. This time is determined as the time when the vertex of the jump of the target moving object is shot. The time obtained by adding a certain time to the time when the take-off point is shot and the time obtained by subtracting the time obtained by shooting the vertex from the time of shooting the take-off point twice is calculated. The change amount related to the three-dimensional coordinates of the position of the specific point is calculated based on the information of a plurality of image frames that are continuous in the reverse order of the shooting time from the calculated time. When the calculated amount of change is larger than a predetermined threshold value, the moving object as the target has landed, the landing point is determined, and the three-dimensional position of the specific point at the landing point is calculated.

[0129] Reference Figure 3 As mentioned above, Figure 3 Schematically depicts a situation in which a skater jumps in a figure skating competition described as a preferred example in this embodiment. Figure 3 As shown schematically, the skater's jump has the characteristics of a parabolic motion. Figure 3301 to 305 respectively represent the edge front end of the skates worn by the jumping skater. In this embodiment, the edge front end of the skates is a specific point. 301 represents a specific point at the take-off point, 302 represents a specific point at the landing point, and 303 to 305 represent specific points in the air during the jump. In particular, 304 represents a specific point at the vertex in the jumping track. In the landing point calculation process (S404), the characteristics of parabolic motion are used, and the specific point 301 at the take-off point and the specific point 302 at the landing point are roughly located at symmetrical positions with the specific point 304 at the vertex as the center. Since it is estimated that the time from the take-off point 301 to the specific point 304 at the vertex and the time from the vertex 304 to the specific point 302 at the landing point are roughly the same, if the time when the specific point 301 at the take-off point is photographed is added to twice the time when the specific point 301 at the take-off point is photographed to the specific point 304 at the vertex, the time when the specific point 302 at the landing point is photographed can be roughly calculated. Starting from the time when a certain buffer time (e.g., 1 second) is added to the time, the landing point is calculated based on the information of a plurality of image frames that are continuous in reverse order of the time. As a result, the specific point 302 at the landing point can be efficiently searched from the vicinity of the specific point 302 at the landing point, thereby reducing the processing. In addition, the specific point 304 at the vertex is determined by the change in the value of the X element and the Y element of the three-dimensional coordinates of the specific point from the direction away from the optical center of the camera lens to the direction close to the change. In addition, this is an example, and the landing point calculation process (S404), the method of obtaining the threshold value, etc. are not limited to this.

[0130] When the landing point calculation process (S404) is completed, the trajectory calculation unit 105 starts the trajectory calculation process (S405). In the trajectory calculation process (S405), the three-dimensional trajectory of the specific point is calculated based on the positional relationship between the curve that can describe the parabolic motion of the three-dimensional position of the specific point at the take-off point calculated by the take-off point calculation process (S403) and the three-dimensional position of the specific point at the landing point calculated by the landing point calculation process (S404) and the existing straight line related to the image of the specific point captured in one or more image frames after the take-off and before the landing of the target moving object.

[0131] Figure 8 4 is a diagram showing an example of orbit calculation processing (S405). Figure 8An example of trajectory calculation processing (S405) is described. In this example, the trajectory calculation processing (S405) calculates a curve as a three-dimensional trajectory of a specific point, wherein the curve is a curve that can describe the parabolic motion of the three-dimensional position of the specific point at the take-off point calculated by the take-off point calculation processing (S403) and the three-dimensional position of the specific point at the landing point calculated by the landing point calculation processing (S404), and the curve minimizes the total distance of the existing straight lines related to the image of one or more specific points captured in one or more image frames after the take-off and before the landing of the moving object as the target.

[0132] exist Figure 8 In the figure, considering the ease of explanation and understanding, the specific points are set to four points, namely, the take-off point 301, the landing point 302, and the specific points 303 and 304 in the air. This is also true even if the number of specific points in the air increases. 801 represents an existing straight line related to the specific point 303, and 802 represents an existing straight line related to the specific point 304. In addition, the curve 805 shown by the single-point chain line is a curve that can describe the parabolic motion passing through the specific point 301 at the take-off point and the specific point 302 at the landing point.

[0133] For example, the curve 805 can be represented by the three equations shown in the following equation (Equation 2). Here, the equation (Equation 2) is represented by a three-dimensional orthogonal coordinate system ΣXYZ. The following notation regulations apply only to the description of the equation (Equation 2). t represents time, m represents mass, g represents gravitational acceleration, v represents velocity, and k represents viscosity coefficient. In addition, the subscripts x, y, and z represent the X element, Y element, and Z element in the three-dimensional orthogonal coordinate system ΣXYZ.

[0134] The formulas representing the X element, Y element, and Z element in the three-dimensional orthogonal coordinate system ΣXYZ shown in the formula (Math. 2) each have three parameters. In the first formula representing the X element, x 0 、v x0 , k x / m, these three are y in the second formula representing the Y element. 0 、v y0 , k y / m, in the second formula representing the Z element, z 0 、v z0 , k z By substituting the coordinate value of the take-off point 301 and the time of take-off, and the coordinate value of the landing point 302 and the time of landing into the equation (Math. 2), two of the three parameters of each equation can be determined.

[0135] [Mathematical formula 2]

[0136]

[0137]

[0138]

[0139] In the orbit calculation process (S405), the value of the remaining parameter is calculated for each equation using the above equation (Equation 2) so that Figure 8 The sum of the distance 803 between the curve 805 and the existing straight line 801 and the distance 804 between the curve 805 and the existing straight line 802 shown is minimized, and the calculated curve is determined as the three-dimensional trajectory of the specific point.

[0140] in addition, Fig. 9 is a diagram showing another example of the orbit calculation process (S405). Fig. 9 Another example of trajectory calculation processing (S405) is described. In this example, in the trajectory calculation processing (S405), a curve is calculated as a three-dimensional trajectory of a specific point, wherein the curve is a curve that can describe the parabolic motion of the three-dimensional position of the specific point at the take-off point calculated by the take-off point calculation processing (S403) and the three-dimensional position of the specific point at the landing point calculated by the landing point calculation processing (S404), and the curve calculates the distance between the existence straight line of the image of the specific point captured in the image frame at the shooting time and the predicted position on the curve at the shooting time of the specific point for one or more specific points after the take-off and before the landing of the moving object as the target, so that the total of the calculated one or more distances to the predicted position is minimized.

[0141] exist Fig. 9 In the figure, considering the ease of explanation, the specific points are set to four points: the take-off point 301, the landing point 302, and the specific points 303 and 304 in the air. This is also true even if the number of specific points in the air increases. 801 represents an existence straight line about the specific point 303, and 802 represents an existence straight line about the specific point 304. In addition, the curve 905 shown by the single-point dashed line is a curve that can describe the parabolic motion passing through the specific point 301 at the take-off point and the specific point 302 at the landing point. Figure 8 Similarly to the curve 805 in , the curve 905 can be expressed by the above-mentioned formula (Formula 2), for example.

[0142] In the orbit calculation process (S405), the predicted position 901 of the specific point on the curve 905 at the time when the specific point 303 in the sky is photographed is calculated by the above-mentioned formula (Formula 2). Similarly, the predicted position 902 of the specific point on the curve 905 at the time when the specific point 304 in the sky is photographed is calculated by the above-mentioned formula (Formula 2).

[0143] In the orbit calculation process (S405), the above-mentioned formula (Formula 2) is used to calculate the value of the remaining parameter of each formula so that the sum of the distance 903 between the predicted position 901 of the specific point on the curve 905 and the existing straight line 801 and the distance 904 between the predicted position 902 of the specific point on the curve 905 and the existing straight line 802 is minimized, and the calculated curve is determined as the three-dimensional orbit of the specific point.

[0144] The above is the description of the embodiment.

[0145] Next, the orbit calculation program for making the computer function as an orbit calculation device is described. Figure 2 shown.

[0146] The orbit calculation program includes a main module, an input / output module, and an operation processing module. The main module is a part that performs unified control processing. The input / output module enables the computer to obtain input information such as image data, and outputs the information of the calculated orbit to the computer in the form of numerical values ​​and images. The operation processing module includes a detection module, an existence straight line calculation module, a take-off point calculation module, a landing point calculation module, and an orbit calculation module. The functions implemented by executing the main module, the input module, and the operation processing module are respectively the same as the functions of the detection unit 101, the existence straight line calculation unit 102, the take-off point calculation unit 103, the landing point calculation unit 104, and the orbit calculation unit 105 of the orbit calculation device 1.

[0147] The orbit calculation program is provided by, for example, a storage medium such as a ROM or a semiconductor memory. Alternatively, the orbit calculation program may be provided via a network.

[0148] As described above, according to the trajectory calculation device, trajectory calculation method, and trajectory calculation program involved in this embodiment, in sports competitions involving jumping such as figure skating, the trajectory of a moving object can be calculated with high accuracy based on a dynamic image captured by a single-lens reflex camera.

Claims

1. A trajectory calculation device, which calculates a three-dimensional trajectory of a moving object as a jumping target based on information of a plurality of image frames captured by a single-lens camera, characterized in that: The orbit calculation device comprises: a detection unit, which detects a specific point of the moving object as the target based on information of the image frame; an existence line calculation unit that calculates an existence line connecting the three-dimensional position of the image of the specific point captured in the image frame and the three-dimensional position of the optical center of the lens of the single-lens camera; a take-off point calculation unit, which calculates a change amount related to the three-dimensional position of the specific point based on information of a plurality of continuous image frames, determines a take-off point from which the moving object as the target takes off based on the calculated change amount, and calculates the three-dimensional position of the specific point at the take-off point; a landing point calculation unit, which calculates a change amount related to the three-dimensional position of the specific point based on information of a plurality of continuous image frames, determines a landing point where the moving object as the target lands based on the calculated change amount, and calculates the three-dimensional position of the specific point at the landing point; as well as a trajectory calculation unit that calculates a three-dimensional trajectory of the specific point based on a positional relationship between a curve and the existing straight line, wherein the curve is a curve that can describe a parabolic motion of the three-dimensional position of the specific point at the take-off point calculated by the take-off point calculation unit and the three-dimensional position of the specific point at the landing point calculated by the landing point calculation unit, and the existing straight line is related to an image of the specific point captured in one or more of the image frames after the moving object as the target takes off and before landing, The take-off point calculation unit includes the following units: based on the information of the plurality of image frames that are continuous in the positive order of the shooting time, the change amount related to the three-dimensional position of the specific point is calculated; when the calculated change amount is greater than a threshold value, it is considered that the moving object as the target has taken off and the take-off point is determined; the three-dimensional position of the specific point at the take-off point is calculated; The landing point calculation unit includes the following units: based on the information of the multiple image frames that are continuous in reverse order of the shooting time, the change amount related to the three-dimensional position of the specific point is calculated. When the calculated change amount is greater than a threshold, it is considered that the moving object as the target has landed and the landing point is determined, and the three-dimensional position of the specific point at the landing point is calculated.

2. A trajectory calculation device, which calculates a three-dimensional trajectory of a moving object as a jumping target based on information of a plurality of image frames captured by a single-lens camera, characterized in that: The orbit calculation device comprises: a detection unit, which detects a specific point of the moving object as the target based on information of the image frame; an existence line calculation unit that calculates an existence line connecting the three-dimensional position of the image of the specific point captured in the image frame and the three-dimensional position of the optical center of the lens of the single-lens camera; a take-off point calculation unit, which calculates a change amount related to the three-dimensional position of the specific point based on information of a plurality of continuous image frames, determines a take-off point from which the moving object as the target takes off based on the calculated change amount, and calculates the three-dimensional position of the specific point at the take-off point; a landing point calculation unit, which calculates a change amount related to the three-dimensional position of the specific point based on information of a plurality of continuous image frames, determines a landing point where the moving object as the target lands based on the calculated change amount, and calculates the three-dimensional position of the specific point at the landing point; as well as a trajectory calculation unit that calculates a three-dimensional trajectory of the specific point based on a positional relationship between a curve and the existing straight line, wherein the curve is a curve that can describe a parabolic motion of the three-dimensional position of the specific point at the take-off point calculated by the take-off point calculation unit and the three-dimensional position of the specific point at the landing point calculated by the landing point calculation unit, and the existing straight line is related to an image of the specific point captured in one or more of the image frames after the moving object as the target takes off and before landing, The take-off point calculation unit includes the following units: based on the information of the plurality of image frames that are continuous in the positive order of the shooting time, the change amount related to the three-dimensional position of the specific point is calculated; when the calculated change amount is greater than a threshold value, it is considered that the moving object as the target has taken off and the take-off point is determined; the three-dimensional position of the specific point at the take-off point is calculated; The landing point calculation unit includes the following units: calculating a change amount related to the three-dimensional position of the specific point based on information of a plurality of image frames that are continuous in positive order after the moment when the take-off point is photographed, and in a case where the calculated change amount is reversed in sign, determining that the moving object as the target has reached the apex of the jump and obtaining the moment when the apex is photographed, calculating a change amount related to the three-dimensional position of the specific point based on information of a plurality of image frames that are continuous in reverse order from the moment when the take-off point is photographed after a period of time has passed, and in a case where the calculated change amount is greater than a threshold value, determining the landing point by considering that the moving object as the target has landed, and calculating the three-dimensional position of the specific point at the landing point, the above-mentioned time being a time obtained by subtracting a time obtained by photographing the take-off point from a time obtained by photographing the apex plus a certain time.

3. The orbit calculation device according to claim 1 or 2, characterized in that: The orbit calculation unit includes the following units: A curve is calculated as the three-dimensional trajectory of the specific point, wherein the curve is a curve capable of describing the parabolic motion of the three-dimensional position of the specific point at the take-off point calculated by the take-off point calculation unit and the three-dimensional position of the specific point at the landing point calculated by the landing point calculation unit, and the total distance of the curve from the images of one or more specific points captured in one or more image frames after the moving object as the target takes off and before landing is minimized.

4. The orbit calculation device according to claim 1 or 2, characterized in that: The orbit calculation unit includes the following units: A curve is calculated as a three-dimensional trajectory of the specific point, wherein the curve is a curve capable of describing the parabolic motion of the three-dimensional position of the specific point at the take-off point calculated by the take-off point calculation unit and the three-dimensional position of the specific point at the landing point calculated by the landing point calculation unit, and the curve is a curve that calculates the distance between an existing straight line related to an image of the specific point captured in the image frame at the shooting moment and a predicted position on the curve at the moment when the specific point is captured for one or more specific points of the moving object as the target after take-off and before landing, and minimizes the sum of the calculated one or more distances to the predicted position.

5. A trajectory calculation method, which calculates a three-dimensional trajectory of a moving object as a jumping target based on information of a plurality of image frames captured by a single-lens camera, characterized in that: The orbit calculation method comprises the following steps: A detection step, detecting a specific point of the moving object as the target according to information of the image frame; an existence line calculation step of calculating an existence line connecting the three-dimensional position of the image of the specific point captured in the image frame and the three-dimensional position of the optical center of the lens of the single-lens camera; a take-off point calculation step, calculating a change amount related to the three-dimensional position of the specific point based on information of a plurality of continuous image frames, determining a take-off point after the moving object as the target takes off based on the calculated change amount, and calculating the three-dimensional position of the specific point at the take-off point; a landing point calculation step, calculating a change amount related to the three-dimensional position of the specific point based on information of a plurality of consecutive image frames, determining a landing point where the moving object as the target lands based on the calculated change amount, and calculating the three-dimensional position of the specific point at the landing point; as well as a trajectory calculation step of calculating a three-dimensional trajectory of the specific point based on a positional relationship between a curve and the existing straight line, wherein the curve is a curve capable of describing a parabolic motion passing through the three-dimensional position of the specific point at the take-off point calculated in the take-off point calculation step and the three-dimensional position of the specific point at the landing point calculated in the landing point calculation step, and the existing straight line is related to an image of the specific point captured in one or more of the image frames after the moving object as the target takes off and before landing, In the take-off point calculation step, a change amount related to the three-dimensional position of the specific point is calculated based on information of a plurality of image frames that are continuous in positive order of shooting time. When the calculated change amount is greater than a threshold, it is considered that the moving object as the target has taken off and the take-off point is determined. The three-dimensional position of the specific point at the take-off point is calculated. In the landing point calculation step, based on the information of the multiple image frames that are continuous in reverse order of the shooting time, the change amount related to the three-dimensional position of the specific point is calculated. When the calculated change amount is greater than a threshold, it is considered that the moving object as the target has landed and the landing point is determined, and the three-dimensional position of the specific point at the landing point is calculated.

6. A computer-readable recording medium having a trajectory calculation program recorded thereon, the trajectory calculation program causing a computer to calculate a three-dimensional trajectory of a moving object as a jumping target based on information of a plurality of image frames captured by a single-lens camera, characterized in that: The orbit calculation program causes the computer to execute the following steps: A detection step, detecting a specific point of the moving object as the target according to information of the image frame; an existence line calculation step of calculating an existence line connecting the three-dimensional position of the image of the specific point captured in the image frame and the three-dimensional position of the optical center of the lens of the single-lens camera; a take-off point calculation step, calculating a change amount related to the three-dimensional position of the specific point based on information of a plurality of continuous image frames, determining a take-off point after the moving object as the target takes off based on the calculated change amount, and calculating the three-dimensional position of the specific point at the take-off point; a landing point calculation step, calculating a change amount related to the three-dimensional position of the specific point based on information of a plurality of consecutive image frames, determining a landing point where the moving object as the target lands based on the calculated change amount, and calculating the three-dimensional position of the specific point at the landing point; as well as a trajectory calculation step of calculating a three-dimensional trajectory of the specific point based on a positional relationship between a curve and the existing straight line, wherein the curve is a curve capable of describing a parabolic motion passing through the three-dimensional position of the specific point at the take-off point calculated in the take-off point calculation step and the three-dimensional position of the specific point at the landing point calculated in the landing point calculation step, and the existing straight line is related to an image of the specific point captured in one or more of the image frames after the moving object as the target takes off and before landing, In the take-off point calculation step, a change amount related to the three-dimensional position of the specific point is calculated based on information of a plurality of image frames that are continuous in positive order of shooting time. When the calculated change amount is greater than a threshold, it is considered that the moving object as the target has taken off and the take-off point is determined. The three-dimensional position of the specific point at the take-off point is calculated. In the landing point calculation step, based on the information of the multiple image frames that are continuous in reverse order of the shooting time, the change amount related to the three-dimensional position of the specific point is calculated. When the calculated change amount is greater than a threshold, it is considered that the moving object as the target has landed and the landing point is determined, and the three-dimensional position of the specific point at the landing point is calculated.

Images