Image processing device, image processing method, image processing system, and program

Abstract

To more easily estimate a rotation angle of the body of a subject in a swing motion of the subject.SOLUTION: An image processing device 10 comprises: a position specification unit 131 which specifies positions of a plurality of representative points of the body of a subject 1 in a moving image obtained by imaging the subject 1 who performs a swing motion of a club 2; and an angle estimation unit 132 which estimates a rotation angle around a vertical axis of the body of the subject 1 on the basis of a vector defined from the representative points whose positions are specified by the position specification unit 131 or the length of a line segment that connects the representative points.SELECTED DRAWING: Figure 1

Description

[[Technical Field]]

[0001] The present disclosure relates to an image processing apparatus, an image processing method, an image processing system, and a program. [[Background Art]]

[0002] A golf swing is an exercise performed by rotating a golfer's body and a golf club to hit a ball. Various methods for evaluating a swing motion performed by a subject (golfer) have been studied.

[0003] As one of the methods for evaluating a swing motion performed by a subject, there is a method using optical motion capture. In this method, the swing motion of the subject is imaged with markers attached to a plurality of representative points on the subject's body, the position of the marker, that is, the position of the representative point on the subject's body is specified from the captured image, and the swing motion is evaluated.

[0004] Further, Patent Document 1 describes a method in which a subject performing a swing motion is imaged from different angles by a plurality of imaging devices, and the swing motion of the subject is evaluated using images captured by each of the plurality of imaging devices.

[0005] Further, Patent Document 2 describes, as one of methods for evaluating hip movement during a swing motion, a method for evaluating the swing motion of a subject by causing the subject to perform a swing motion with an inertial sensor attached to the subject's body. [[Prior Art Documents]] [[Patent Documents]]

[0006] [[Patent Document 1]] Japanese Patent No. 4290462 [[Patent Document 2]] Japanese Patent No. 7069662 [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0007] Optical motion capture methods require time-consuming steps such as pre-measurement calibration and the placement of markers on representative points on the subject's body. Furthermore, this method is limited to locations where a motion capture system can be used.

[0008] Furthermore, the method described in Patent Document 1 requires pre-measurement calibration and the effort of simultaneously capturing control points with the subject. Additionally, this method is limited to locations where imaging can be performed using multiple imaging devices.

[0009] Furthermore, the method described in Patent Document 2 requires attaching sensors to different parts of the subject's body depending on the location to be measured.

[0010] Thus, conventional methods have various limitations, and there was a need for a method that could more easily estimate the rotation angle of the subject's body during their swing motion.

[0011] In view of the above-mentioned problems, the purpose of this disclosure is to provide an image processing device, an image processing method, an image processing system, and a program that can more easily estimate the rotation angle of a subject's body during a subject's swing motion. [Means for solving the problem]

[0012] (1) An image processing device in one aspect of the present disclosure is an image processing device that estimates the rotation angle of a subject's body around the vertical axis from a video image of a subject performing a club swing motion, comprising: a position identification unit that identifies the positions of a plurality of representative points of the subject's body in the video image; and an angle estimation unit that estimates the rotation angle of the subject's body around the vertical axis based on a vector defined from the representative points whose positions are identified by the position identification unit or the length of a line segment connecting the representative points. With this configuration, the rotation angle of the subject's body during a swing motion can be more easily estimated from moving images captured by a single imaging device, without the need to attach markers or sensors to the subject's body or perform prior calibration.

[0013] (2) In the image processing apparatus of (1), the moving image is a moving image of the subject taken from the front, or a moving image of the subject taken from the direction of the ball flight. With this configuration, it is possible to estimate the rotation angle of the subject's body when viewed from the front or in the direction of the ball's flight path.

[0014] (3) In the image processing apparatus of (1) or (2), the position identification unit identifies the positions of the pair of representative points on the left and right sides of the subject's body as the positions of the plurality of representative points. With this configuration, it is possible to estimate the rotation angle of the body parts of subject 1 that correspond to paired representative points on the left and right sides.

[0015] (4) In any of the image processing devices described in (1) to (3), the position identification unit identifies the positions of the plurality of representative points as a pair of representative points on the left and right sides of the subject's upper body, or the positions of a pair of representative points on the left and right sides of the subject's lower body. With this configuration, it is possible to estimate the rotation angles of parts corresponding to paired representative points on the left and right sides of the subject's upper or lower body.

[0016] (5) In any of the image processing devices described in (1) to (3), the position identification unit identifies the positions of a pair of representative points on the left and right sides of the subject's upper body and the positions of a pair of representative points on the left and right sides of the subject's lower body as the positions of the plurality of representative points; the angle estimation unit estimates the rotation angle of the subject's upper body based on a vector defined from the pair of representative points on the left and right sides of the subject's upper body or the length of a line segment connecting the representative points; estimates the rotation angle of the subject's lower body based on a vector defined from the pair of representative points on the left and right sides of the subject's lower body or the length of a line segment connecting the representative points; and calculates the torsion difference, which is the difference between the estimated rotation angle of the subject's upper body and the rotation angle of the subject's lower body. With this configuration, it is possible to calculate the torsional difference, which is the difference between the rotation angle of the subject's upper body and the rotation angle of the subject's lower body.

[0017] (6) In any of the image processing devices described in (1) to (5), the angle estimation unit estimates the rotation angle at each time point based on the ratio of the reference length of the line segment connecting the representative points during the swing motion period to the length of the vector or line segment defined from the representative points at each time point of the swing motion. With this configuration, it is possible to estimate the rotation angle of a subject's body from a two-dimensional image.

[0018] (7) In any of the image processing devices described in (1) to (5), the angle estimation unit estimates the rotation angle at each time point based on the ratio of the reference length of the line segment in the first period of the swing motion to the length of the vector or line segment defined from the representative point at each time point within the first period, and estimates the rotation angle at each time point based on the ratio of the reference length of the line segment in the second period following the first period of the swing motion to the length of the vector or line segment defined from the representative point at each time point within the second period. This configuration allows for improved accuracy in estimating the rotation angle.

[0019] (8) An image processing method according to one aspect of the present disclosure is an image processing method performed by an image processing apparatus that estimates a rotation angle of a subject's body about a vertical axis from a moving image obtained by capturing the subject performing a club swing motion, the method comprising: specifying positions of a plurality of representative points on the subject's body in the moving image; and estimating the rotation angle of the subject's body about the vertical axis based on a vector defined from the representative points whose positions are specified or a length of a line segment connecting the representative points. According to this configuration, the rotation angle of the subject's body during the swing motion can be estimated more easily from a moving image captured by a single imaging device without requiring attachment of markers or sensors to the subject's body or prior calibration.

[0020] (9) An image processing system according to one aspect of the present disclosure is an image processing system comprising: a terminal that captures an image of a subject performing a club swing motion; and an image processing apparatus that estimates a rotation angle of the subject's body about a vertical axis from a moving image captured by the terminal, wherein the image processing apparatus comprises: a position specifying unit that specifies positions of a plurality of representative points on the subject's body in the moving image; and an angle estimation unit that estimates the rotation angle of the subject's body about the vertical axis based on a vector defined from the representative points whose positions are specified by the position specifying unit or a length of a line segment connecting the representative points. According to this configuration, the rotation angle of the subject's body during the swing motion can be estimated more easily from a moving image captured by a single terminal without requiring attachment of markers or sensors to the subject's body or prior calibration.

[0021] (10) A program according to one aspect of the present disclosure causes a computer to operate as the image processing apparatus according to any one of (1) to (7). Effects of the Invention

[0022] According to the image processing apparatus, image processing method, image processing system, and program according to the present disclosure, the rotation of the subject's body during the subject's swing motion can be estimated more easily. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] [Figure 1] FIG. 1 is a diagram showing a configuration example of an image processing apparatus according to an embodiment of the present disclosure. [Figure 2A] FIG. 2 is a diagram schematically showing a state where a subject performing a swing motion is imaged from the front. [Figure 2B] FIG. 3 is a diagram schematically showing a state where a subject performing a swing motion is imaged from the ball flight line direction. [Figure 3A] FIG. 4 is a diagram for explaining representative points of the subject's body specified by the position specifying unit shown in FIG. 1. [Figure 3B] FIG. 5 is a diagram for explaining representative points of a club specified by the position specifying unit shown in FIG. 1. [Figure 4] FIG. 6 is a diagram showing an example of representative points of the subject's body specified by the position specifying unit shown in FIG. 1. [Figure 5] FIG. 7 is a diagram showing rotation of the subject's body caused by a swing motion. [Figure 6] FIG. 8 is a diagram for explaining the rotation angle θ estimated by the angle estimating unit shown in FIG. 1. [Figure 7] FIG. 9 is a diagram for explaining estimation of the rotation angle θ using arccosine. [Figure 8] FIG. 10 is a diagram showing an example of rotation of the subject's body in a series of swing motions. [Figure 9] FIG. 11 is a flowchart showing an example of the operation of the image processing apparatus shown in FIG. 1. [Figure 10] FIG. 12 is a diagram showing a configuration example of an image processing system according to an embodiment of the present disclosure. MODE FOR CARRYING OUT THE INVENTION

[0024] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0025] Figure 1 is a diagram showing an example of the configuration of an image processing device 10 according to one embodiment of the present disclosure. The image processing device 10 according to this embodiment estimates the rotation angle θ of the body of subject 1 around the vertical axis from a moving image taken by an imaging device from the front (a direction perpendicular to the direction of the ball flight) of subject 1 performing a swing motion of a club 2, as shown in Figure 2A, or from a moving image taken by an imaging device from behind in the direction of the ball flight, as shown in Figure 2B.

[0026] As shown in Figure 1, the image processing apparatus 10 according to this embodiment includes a communication unit 11, a storage unit 12, and a control unit 13.

[0027] The communication unit 11 is comprised of one or more communication modules. The communication unit 11 may include a communication module that supports mobile communication standards such as 4G (4th Generation) and 5G (5th Generation). The communication unit 11 may also include a communication module that supports wireless LAN standards (for example, IEEE 802.11). Furthermore, the communication unit 11 may also include a communication module that supports wired LAN standards.

[0028] The storage unit 12 is one or more memories. The memories are, for example, semiconductor memories, magnetic memories, or optical memories, but are not limited to these and can be any type of memory. The storage unit 12 is, for example, built into the image processing device 10, but it is also possible to configure it so that it can be accessed from the outside of the image processing device 10 via any interface.

[0029] The storage unit 12 stores various data used in various processes performed by the control unit 13. The storage unit 12 may also store the results and intermediate data of the various processes performed by the control unit 13.

[0030] The control unit 13 is one or more processors. The processors are, for example, general-purpose processors or dedicated processors specialized for specific processing, but are not limited to these and can be any processor. The control unit 13 controls the overall operation of the image processing device 10.

[0031] The image processing device 10 may have the following software configuration. One or more programs according to this disclosure, used to control the operation of the image processing device 10, are stored in the storage unit 12. When the programs stored in the storage unit 12 are read by the processor of the control unit 13, the control unit 13 is made to function as a position identification unit 131 and an angle estimation unit 132.

[0032] The position identification unit 131 receives a video image of subject 1 performing a swing motion with club 2. As described above, the video image is either a video image of subject 1 performing a swing motion taken from the front, as shown in Figure 2A, or a video image of subject 1 performing a swing motion taken from behind in the direction of the ball flight, as shown in Figure 2B. By using such a video image, it is possible to estimate the rotation angle θ of subject 1's body around the vertical axis, as viewed from the front or side (direction of the ball flight). It is preferable that the video image is taken with the imaging device parallel to the ground. It is also preferable that the video image is taken with the entire body of subject 1 within the field of view and subject 1 located near the center of the field of view. By doing so, it is possible to obtain an image of subject 1 with less distortion, and the estimation accuracy can be improved.

[0033] The position identification unit 131 identifies the positions of multiple representative points on the subject 1's body in the input video image. Here, the positions of the identified representative points are two-dimensional positions on the image.

[0034] Figure 3A shows an example of a representative point on the body of subject 1 identified by the position identification unit 131.

[0035] As shown in Figure 3A, the position identification unit 131 identifies the positions of representative points, for example, the top of the head, the left and right tragus points, the left and right shoulders, the upper edge of the sternum (or the midpoint of both shoulders), the left and right elbows, the left and right wrists, the left and right hands, the left and right lower ends of the ribs, the midpoint of the left and right lower ends of the ribs, the left and right hip joints (greater trochanter), the left and right knee joints, the left and right ankle joints, and the left and right toes or left and right heels of the subject 1. In particular, the position identification unit 131 identifies the positions of paired representative points on the left and right sides of the subject 1's body (for example, the left and right shoulders, left and right hip joints, etc.). By identifying the positions of paired representative points on the left and right sides, the rotation angle θ of the parts corresponding to the paired representative points can be estimated.

[0036] The position identification unit 131 may identify the positions of representative points of the subject 1, specifically the positions of a pair of representative points on the left and right sides of the subject 1's upper body, or the positions of a pair of representative points on the left and right sides of the subject 1's lower body. By doing so, the rotation angle θ of the parts corresponding to the pair of representative points on the left and right sides of the subject 1's upper or lower body can be estimated.

[0037] Furthermore, the position identification unit 131 may identify the positions of representative points of the subject 1, specifically the positions of paired representative points on the left and right sides of the subject 1's upper body, and the positions of paired representative points on the left and right sides of the subject 1's lower body. By doing so, the difference (torsion difference) between the rotation angle θ of the parts corresponding to the paired representative points on the left and right sides of the subject 1's upper body (e.g., the left and right shoulders) and the rotation angle θ of the parts corresponding to the paired representative points on the left and right sides of the subject 1's lower body (e.g., the left and right hip joints) can be determined.

[0038] Furthermore, the position identification unit 131 may identify the position of a representative point of the club 2. For example, as shown in Figure 3B, the position identification unit 131 may identify the grip end, the connection point between the grip and the shaft, the connection point between the shaft and the head, and the center of the head as representative points of the club 2.

[0039] The position identification unit 131 identifies the positions of the representative points mentioned above, for example, by image recognition of the images constituting the input video. Alternatively, the positions of the representative points may be input to the images constituting the video of subject 1 using a digitizer or the like, and the position identification unit 131 may acquire the input result. In the following, as shown in Figure 4, the position identification unit 131 identifies the positions of subject 1's left and right shoulders and left and right hip joints as representative points of subject 1's body.

[0040] Referring again to Figure 1, the angle estimation unit 132 estimates the rotation angle θ of the subject 1's body around the vertical axis based on a vector defined from representative points identified by the position identification unit 131 or the length of a line segment connecting the representative points. In this embodiment, as shown in Figure 4, the position identification unit 131 identifies the positions of paired representative points (left and right shoulders) on the upper body of the subject 1, and the positions of paired representative points (left and right hip joints) on the lower body of the subject 1. In this case, the angle estimation unit 132 may estimate the rotation angle θ of the subject 1's upper body (left and right shoulders) based on a vector defined from paired representative points on the upper body of the subject 1, or the length of a line segment connecting the representative points. Alternatively, the angle estimation unit 132 may estimate the rotation angle θ of the subject 1's lower body (left and right hip joints) based on a vector defined from paired representative points on the lower body of the subject 1, or the length of a line segment connecting the representative points. Furthermore, the angle estimation unit 132 may calculate the torsional difference, which is the difference between the estimated rotation angle θ of the upper body and the rotation angle θ of the lower body of the subject 1.

[0041] Furthermore, after the position identification unit 131 identifies the position of the representative point, and before the angle estimation unit 132 estimates the rotation angle θ, a real-space length conversion may be performed to determine the distance between the representative points in real space. Alternatively, the real-space length conversion may be performed after the angle estimation unit 132 estimates the rotation angle θ. The real-space length conversion can be performed based on the real-space length of one pixel in the image of the subject 1 performing the swing motion.

[0042] Next, we will explain the estimation of the rotation angle θ by the angle estimation unit 132. In the following, as shown in Figure 5, subject 1 as viewed from directly above (Z-axis direction) is shown by a dashed circle, the dotted line within the dashed circle represents the line segment connecting the left and right representative points on the left and right sides of subject 1's body (in the example in Figure 5, the left and right hip joints (hipR, hipL)), and the solid line within the dashed circle represents the vector defined from the representative points or the line segment connecting the representative points as viewed from the front (X-axis direction) of the analysis direction of subject 1's swing motion.

[0043] As shown in Figure 6, the angle estimation unit 132 defines the rotation angle θ=0 when the Y-axis, which is orthogonal to the X-axis and Z-axis, is parallel to the line segment connecting the pair of representative points on the left and right sides of the subject's body (for example, the left and right hip joints (hipR, hipL)), and estimates the rotation angle θ(t) at each time point t during the swing motion. If the direction of a right-hand screw is considered positive around the vertically upward axis (Z-axis direction), then as shown in Figure 6, -180°≦θ(t)≦180°.

[0044] The angle estimation unit 132 estimates the rotation angle θ(t) at time t using the following equation (1). P(t)=arccos(A)×180 / π[deg](0≦|A|≦1) Equation (1)

[0045] In equation (1), arccos (arccosine) is the inverse function of cosine. However, generally, considering y = arccos(x), when -1 ≤ x ≤ 1, 0 ≤ y ≤ π [rad], and estimating angles less than 0° in the domain of θ(t) is difficult by simply applying equation (1). Therefore, we consider extending the scope of application of the concept so that estimation can also be performed when θ(t) < 0° by making case distinctions according to each time point t during the swing motion. Details of the case distinctions in the calculation formula for estimating θ(t) will be described later. Also, as shown in Figure 2A, when subject 1 is viewed from the front, A is the maximum value of the norm of the left-right component (Y-axis direction) of the vector from the right representative point to the left representative point of the pair at time t / the left-right component (Y-axis direction) of the vector from the right representative point to the left representative point of the pair. To give a concrete example, when θ=0° as shown in Figure 5, we can conclude that A is 1 based on the magnitude and direction of the left-right (Y-axis direction) component of the vector pointing from the right representative point to the left representative point, which are a pair of left and right vectors.

[0046] Furthermore, as shown in Figure 2B, when subject 1 is viewed from the direction of the ball flight, A is the maximum value of the norm of the front-to-back (X-axis direction) component of the vector from the right representative point to the left representative point in a pair of left-to-right representative points at time t. In other words, the angle estimation unit 132 estimates the rotation angle θ(t) at each time t based on the ratio of the reference length (maximum value) of the line segment connecting the left and right representative points in a pair during the swing motion and the length of the vector or line segment defined from the representative point at each time t of the swing motion. In this way, the rotation angle θ of subject 1's body can be estimated from a two-dimensional image. Note that the average value of the line segment connecting the left and right representative points in a pair during a predetermined period of the swing motion may be used as the reference length.

[0047] Here, we will explain with reference to Figure 7 why the arccosine can be applied to estimate the body posture (rotation angle θ) of subject 1, as shown in equation (1).

[0048] Figure 7 shows the swing motion as viewed from the front and rear. As shown in the leftmost image of Figure 7, when viewing Subject 1 performing the swing motion from the front, Subject 1 often takes a stance parallel to the direction of the ball flight at address. In this case, the line segment connecting paired representative points on the left and right sides of Subject 1 (for example, the left and right shoulders and the left and right hip joints) is parallel to the ground and also parallel to the direction of the ball flight. Furthermore, the line segment connecting paired representative points on the left and right sides of Subject 1 is longest when the optical axis of the imaging device is directly facing Subject 1's body. Then, as Subject 1's body rotates, the length of the line segment visible from the front shortens, and the lateral positional relationship between the paired representative points on the left and right also changes. In addition, a golf swing is a whole-body rotational movement in place, with the standing position remaining almost unchanged during the swing.

[0049] Therefore, the rotation angle θ(t) at each time point t can be estimated from the reference length of the line segment connecting the paired representative points on the left and right during the swing motion, and the length of the vector or line segment defined from the representative point at each time point t of the swing motion.

[0050] Figure 8 shows an example of the change in the rotation angle θ of the left and right hip joints (hipR, hipL) of Subject 1 during a swing motion. In Figure 8, an example of the change in the rotation angle θ is shown when Subject 1 performing the swing motion is viewed from the front.

[0051] At address, Subject 1's body is directly facing the imaging direction of the imaging device (i.e., the X-axis direction), so the rotation angle θ = 0°. As Subject 1 begins the backswing, the rotation angle θ decreases (θ < 0°). As Subject 1 continues the backswing, the left and right hip joints overlap when viewed from the front of Subject 1, and θ = -90°. At the top of the swing, the rotation angle θ becomes even smaller (θ < -90°). Note that in subjects with little torso rotation, θ may not be ≤ -90°.

[0052] When Subject 1 begins the downswing from the top position, the rotation angle θ increases, and near impact, the rotation angle θ = 0°. After impact, when Subject 1 follows through, the rotation angle θ increases further (θ > 0). At a certain point in the follow-through, when viewed from the front of Subject 1, the left and right hip joints coincide, resulting in a rotation angle θ = 90°, and at the end of the follow-through, the rotation angle θ increases further (θ > 90°). Note that in subjects with less torso rotation, θ may not be ≥ 90°.

[0053] The following describes in detail the estimation (calculation) of the rotation angle θ by the angle estimation unit 132 in a series of swing motions as explained with reference to Figure 8. First, we will explain the calculation of the rotation angle θ when subject 1 is imaged from the front. In the following, t1 is defined as the time when the length of the line segment connecting the paired representative points on the left and right sides of the swing motion is maximum, and t2 is defined as the time when P(t) = arccos(A) × 180 / π is minimum near impact after the take-back. A is the maximum value of the norm of the left-right (Y-axis direction) component of the vector from the right representative point to the left representative point of the paired representative points on the left and right sides / the left-right (Y-axis direction) component of the vector from the right representative point to the left representative point of the paired representative points on the left and right sides at each time point t of the swing motion. The range of A is -1 or more and 1 or less.

[0054] The angle estimation unit 132 calculates the rotation angle θ using the following formula. t2 <t θ=arccos(A)×180 / π[deg]=P(t) t <t1 θ=arccos(A)×180 / π[deg]=P(t) t1 ≤ t ≤ t2 θ=(-1)×arccos(A)×180 / π[deg]=-P(t)

[0055] Furthermore, the angle estimation unit 132 calculates the rotation angle θ using the following formula when t1=t2. t1 <t θ=(-1)×arccos(A)×180 / π[deg]=-P(t) t≦t1 θ=arccos(A)×180 / π[deg]=P(t)

[0056] Next, we will explain how to calculate the rotation angle θ when subject 1 is imaged from the direction of the ball flight. In the following, t1 is defined as the point in time when the length of the line segment connecting the paired representative points on the left and right sides of the swing motion is maximum, and t3 is defined as the point in time near impact after the backswing when P(t) = arccos(A) × 180 / π is maximum. A is the maximum value of the norm of the front-to-back (X-axis direction) component of the vector from the right representative point to the left representative point of the paired representative points on the left and right sides, divided by the front-to-back (X-axis direction) component of the vector from the right representative point to the left representative point of the paired representative points on the left and right sides, at each point t of the swing motion. The range of A is -1 or greater and 1 or less.

[0057] The angle estimation unit 132 calculates the rotation angle θ using the following formula. t3 <t θ=((-1)×arccos(A)×180 / π)+P(t3)+180 =-P(t)+P(t3)+180[deg] t≦t3 θ=(arccos(A)×180 / π)-90=P(t)-90[deg]

[0058] Furthermore, the angle estimation unit 132 calculates the rotation angle θ using the following formula when t1=t3. t1 <t θ=((-1)×arccos(A)×180 / π]+P(t3)+180 =-P(t)+P(t3)+180[deg] t≦t1 θ = (arccos(A) × 180 / π) - 90 =P(t)-90[deg]

[0059] The angle estimation unit 132 may use the reference length (maximum value) of the line segment connecting the paired representative points on the left and right during the swing motion, and the length of the line segment at each time point t of the swing motion. When subject 1 is imaged from the front, A may be defined as the absolute value of the left-right (Y-axis direction) component of the vector from the right representative point to the left representative point of the paired left and right at each time point t of the swing motion / the maximum value of the norm of the left-right (Y-axis direction) component of the vector from the right representative point to the left representative point of the paired left and right. In this case, A will be between 0 and 1. Furthermore, when subject 1 is imaged from the direction of the ball flight, the angle estimation unit 132 may use A as the absolute value of the front-back (X-axis direction) component of the vector from the right representative point to the left representative point of the paired left and right at each time point t of the swing motion / the maximum value of the norm of the front-back (X-axis direction) component of the vector from the right representative point to the left representative point of the paired left and right. In this case, the range of A is between 0 and 1.

[0060] When using the absolute value of the left-right (Y-axis direction) or front-back (X-axis direction) component of a vector pointing from the right representative point to the left representative point in a pair of left and right representative points, the angle estimation unit 132 can estimate the rotation angle θ by switching the calculation formula used to estimate (calculate) the rotation angle θ according to the positional relationship of the left and right representative points in the Y-axis and X-axis directions, and the timing of the change in the positional relationship including t1 to t3.

[0061] Furthermore, the angle estimation unit 132 may change the maximum value of the norm of the left-right (Y-axis direction) or front-back (X-axis direction) component of the vector from the right representative point to the left representative point, which is the denominator of A, according to the duration of the swing motion. For example, the angle estimation unit 132 may change the above-mentioned maximum value for the period from address to impact and for the period from impact to catch.

[0062] In other words, the angle estimation unit 132 may estimate the rotation angle θ(t) at each time point t based on the ratio of the reference length (maximum value) of the line segment connecting a pair of representative points on the left and right during the first period of the swing motion (for example, the period from address to impact) to the length of the vector or line segment defined from the representative point at each time point t within the first period. Alternatively, the angle estimation unit 132 may estimate the rotation angle θ(t) at each time point t during the second period of the swing motion following the first period (for example, the period from impact to finish) based on the ratio of the reference length (maximum value) of the line segment connecting a pair of representative points on the left and right during the second period to the length of the line segment at each time point t within the second period. By doing so, the accuracy of estimating the rotation angle θ can be improved.

[0063] Figure 9 is a flowchart showing an example of the operation of the image processing apparatus 10 according to this embodiment, and is a diagram for explaining the image processing method by the image processing apparatus 10 according to this embodiment.

[0064] The positioning unit 131 identifies the positions of multiple representative points on the subject 1's body in the video footage of the subject 1 performing the swing motion of the club 2 (step S11). In particular, the positioning unit 131 identifies the positions of paired representative points on the left and right sides of the subject's body. The positioning unit 131 also identifies the positions of paired representative points on the left and right sides of the subject 1's upper body, and / or the positions of paired representative points on the left and right sides of the subject 1's lower body.

[0065] The angle estimation unit 132 estimates the rotation angle θ of the subject 1's body around the vertical axis based on the vector defined from the representative point whose position has been determined by the position determination unit 131, or the length of the line segment connecting the representative points (step S12). Specifically, the angle estimation unit 132 estimates the rotation angle θ based on the equation (1) described above. That is, the angle estimation unit 132 estimates the rotation angle θ at each time point based on the ratio (=A) of the reference length (e.g., maximum value) of the line segment connecting the pair of representative points on the left and right during the swing motion, and the vector defined from the representative point at each time point t of the swing motion.

[0066] Figure 10 shows an example of the configuration of an image processing system 100 to which the image processing apparatus 10 according to this embodiment is applied.

[0067] As shown in Figure 10, the image processing system 100 comprises an image processing device 10 and a terminal 20.

[0068] The image processing device 10 and the terminal 20 can communicate via a mobile communication network or a wireless / wired LAN. The image processing device 10 is, for example, a server device located on the network.

[0069] Terminal 20 is a user terminal used by Subject 1 or an instructor, and is, for example, a smartphone or tablet. Terminal 20 operates as an imaging device equipped with an imaging function and captures images of Subject 1 performing a swing motion. Terminal 20 then transmits the captured video images to the image processing device 10. Preferably, the images to be processed by the image processing device 10 are video images of a predetermined period of the swing motion performed by Subject 1 (for example, the period from address to finish). Terminal 20 may, for example, extract video images from the captured video images for a period specified by Subject 1 or the like (for example, the period from address to finish) and transmit them to the image processing device 10. By doing so, it is possible to reduce the transmission capacity and shorten the transmission time compared to transmitting the entire captured video image.

[0070] The image processing device 10 estimates the rotation angle θ of subject 1's body from the video transmitted from terminal 20, as described above, and transmits the result of the swing motion performed by subject 1 to terminal 20. As an evaluation result, for example, the rotation angle θ estimated by the image processing device 10 (for example, the rotation angle θ of subject 1's left and right shoulders and / or the rotation angle θ of subject 1's left and right hip joints) may be transmitted to terminal 20. Alternatively, as an evaluation result, the difference (torsion difference) between the rotation angle θ of subject 1's left and right shoulders and the rotation angle θ of subject 1's left and right hip joints may be transmitted to terminal 20.

[0071] In this embodiment, the image processing device 10 has been described using an example in which the image processing device 10 includes a position identification unit 131 and an angle estimation unit 132, but the disclosure is not limited thereto. Part or all of the position identification unit 131 and the angle estimation unit 132 may be provided in the terminal 20. Therefore, the terminal 20 may operate as the image processing device 10 according to the disclosure.

[0072] As described above, the image processing apparatus 10 according to this embodiment comprises a position identification unit 131 and an angle estimation unit 132. The position identification unit 131 identifies the positions of multiple representative points on the body of subject 1 in a video image of subject 1 performing a swing motion with a club 2. The angle estimation unit 132 estimates the rotation angle θ of subject 1's body around the vertical axis based on a vector defined from the representative points whose positions are identified by the position identification unit 131, or the length of a line segment connecting the representative points.

[0073] With this configuration, the rotation angle θ of subject 1's body during a swing motion can be estimated from moving images captured by a single imaging device. Furthermore, there is no need to attach markers or sensors to subject 1's body, and no prior calibration is required. Therefore, according to this disclosure, the rotation angle θ of subject 1's body during a swing motion can be estimated more easily.

[0074] Although not specifically mentioned in the embodiments, a program that causes the computer to operate as an image processing device 10 may be provided. Furthermore, the program may be recorded on a computer-readable medium. Using a computer-readable medium allows for installation on the computer. Here, the computer-readable medium on which the program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, but may be, for example, a CD-ROM or DVD-ROM.

[0075] Although the embodiments described above are representative examples, it will be apparent to those skilled in the art that many modifications and substitutions are possible within the spirit and scope of the present invention. Therefore, the present invention should not be interpreted as being limited by the embodiments described above, and various modifications and changes are possible without departing from the scope of the claims. For example, it is possible to combine multiple component blocks shown in the configuration diagram of the embodiments into one, or to divide one component block. [Explanation of symbols]

[0076] 1 Subject 2 clubs 10 Image Processing Device 11 Communications Department 12 Storage section 13 Control Unit 20 devices 100 Image Processing Systems 131 Location identification part 132 Angle estimation part

Claims

1. An image processing device that estimates the rotation angle around the vertical axis of a subject's body from video footage of a subject performing a club swing motion, A position identification unit that identifies the positions of multiple representative points on the subject's body in the aforementioned moving image, The system includes an angle estimation unit that estimates the rotation angle of the subject's body around the vertical axis based on a vector defined from a representative point whose position has been determined by the position determination unit, or the length of a line segment connecting two representative points, The angle estimation unit is an image processing device that estimates the rotation angle at each time point based on the ratio of the reference length of the line segment during the swing motion to the length of a vector or line segment defined from the representative point at each time point of the swing motion.

2. In the image processing apparatus according to claim 1, The image processing apparatus wherein the aforementioned moving image is a moving image of the subject taken from the front, or a moving image of the subject taken from the direction of the ball's flight path.

3. In the image processing apparatus according to claim 1, The position identification unit is an image processing device that identifies the positions of a pair of representative points on the left and right sides of the subject's body as the positions of the plurality of representative points.

4. In the image processing apparatus according to claim 1, The position identification unit identifies the positions of the plurality of representative points, which are pairs of representative points on the left and right sides of the subject's upper body, or pairs of representative points on the left and right sides of the subject's lower body.

5. In the image processing apparatus according to claim 1, The position identification unit identifies the positions of the multiple representative points as a pair of representative points on the left and right sides of the subject's upper body, and the positions of a pair of representative points on the left and right sides of the subject's lower body. The angle estimation unit estimates the rotation angle of the subject's upper body based on a vector defined from a pair of representative points on the left and right sides of the subject's upper body or the length of a line segment connecting the representative points, estimates the rotation angle of the subject's lower body based on a vector defined from a pair of representative points on the left and right sides of the subject's lower body or the length of a line segment connecting the representative points, and calculates the torsion difference, which is the difference between the estimated rotation angle of the subject's upper body and the rotation angle of the subject's lower body.

6. In the image processing apparatus according to claim 1, Image processing apparatus comprising: An angle estimation unit that estimates the rotation angle at each time point based on the ratio of the reference length of the line segment during the first period of the swing motion to the length of a vector or line segment defined from the representative point at each time point within the first period; and an image processing apparatus that estimates the rotation angle at each time point based on the ratio of the reference length of the line segment during the second period following the first period to the length of a vector or line segment defined from the representative point at each time point within the second period.

7. An image processing method using an image processing device that estimates the rotation angle around the vertical axis of a subject's body from video footage of a subject performing a club swing motion, Identifying the positions of multiple representative points on the subject's body in the aforementioned video, This includes estimating the rotation angle around the vertical axis of the subject's body based on a vector defined from a representative point whose position has been identified, or the length of a line segment connecting two representative points, An image processing method for estimating the rotation angle at each point in time based on the ratio of the reference length of the line segment during the swing motion to the length of a vector or line segment defined from the representative point at each point in time of the swing motion.

8. An image processing system comprising a terminal for capturing images of a subject performing a club swing motion, and an image processing device for estimating the rotation angle of the subject's body around its vertical axis from the moving images captured by the terminal, The aforementioned image processing device is A position identification unit that identifies the positions of multiple representative points on the subject's body in the aforementioned moving image, The system includes an angle estimation unit that estimates the rotation angle of the subject's body around the vertical axis based on a vector defined from a representative point whose position has been determined by the position determination unit, or the length of a line segment connecting two representative points, The angle estimation unit is an image processing system that estimates the rotation angle at each point in time based on the ratio of the reference length of the line segment during the swing motion to the length of a vector or line segment defined from the representative point at each point in time of the swing motion.

9. A program that causes a computer to operate as an image processing device as described in claim 1.

Images