Apparatus and method for sensing the motion of a sphere moving on a plane, and apparatus and method for sensing the motion of a golf ball moving on a putting mat

Abstract

The present invention aims to provide a motion sensing device and method for a flat moving sphere that can be used in any location with a simple configuration, and a motion sensing device and method for a golf ball moving on a putting mat, which can detect a sports sphere without dedicated lighting by using a low-cost single camera and can also detect the sports sphere without dedicated lighting, thereby enabling spatial recognition and eliminating the constraint of having to fix the camera's installation position.

Description

[Technical field]

[0001] The present invention relates to a sensing device and a sensing method for calculating information about the movement of a spherical object such as a sports ball, e.g., a golf ball or a bowling ball, by acquiring and analyzing an image of the spherical object moving on a plane. [Background technology]

[0002] 2. Description of the Related Art Generally, a stereoscopic camera device using two cameras is used as a device for acquiring coordinate information of an object in a three-dimensional space.

[0003] In other words, two-dimensional coordinate information of a target object is extracted from a two-dimensional image of the target object captured by one camera and from a two-dimensional image of the same target object captured by another camera, and the extracted two-dimensional coordinate information is applied to a predefined correlation function to calculate three-dimensional coordinate information of the target object.

[0004] Such stereoscopic camera devices are widely used to calculate position information of a target object in three-dimensional space, and the most representative example is their use in a so-called screen golf system.

[0005] That is, in the screen golf system, the ceiling camera and the side camera are linked in a stereoscopic manner, and the position change information in the three-dimensional space caused by the movement of the golf ball hit by the user's golf swing can be calculated.

[0006] In this way, detection of a sports ball and calculation of its motion status are typically performed using multiple cameras, and images captured by each of the multiple cameras are transmitted to a computing device to calculate position information in three-dimensional space, and the motion status of the object is calculated based on the coordinates in real space thus obtained.

[0007] In this regard, prior art documents such as Korean Patent Publication No. 10-1902283, Korean Patent Publication No. 10-1826837, and Korean Patent Publication No. 10-1723432 disclose technologies relating to a virtual golf simulation system using a stereoscopic camera sensing device as mentioned above.

[0008] However, the above-mentioned sports object movement sensing method using the stereoscopic camera sensing device has limitations in that the positions of the two cameras must be fixed and that dedicated lighting must be provided to sense the movement of an object in three-dimensional space, and therefore the method can only be used when the cameras are accurately installed by a professional technician in a specific location. This means that the method cannot be used easily without spatial constraints.

[0009] In addition, since a plurality of cameras connected in a stereoscopic manner is required, a processing device for processing image data transmitted from each camera must have a very high processing capacity. [Prior art documents] [Patent documents]

[0010] [Patent Document 1] Korean Patent Registration No. 10-1902283 [Patent Document 2] Korean Patent Registration No. 10-1826837 [Patent Document 3] Korean Patent Registration No. 10-1723432 [Patent Document 4] Korean Patent Publication No. 10-2019-0014490 [Patent Document 5] Korean Patent Registration No. 10-1141048 Summary of the Invention [Problem to be solved by the invention]

[0011] The present invention provides a motion sensing device and method for a sports ball moving on a plane as a target object, which uses a low-cost single camera to detect the sports ball without dedicated lighting, and which is capable of spatial recognition, thus freeing the restriction that the camera must be installed in a fixed position, and which has a simple configuration and can be used anywhere, as well as a motion sensing device and method for a golf ball moving on a putting mat. [Means for solving the problem]

[0012] A motion sensing device for a planar moving structure according to one embodiment of the present invention is a sensing device for a sphere moving on a plane, and includes a single camera that acquires an image from an arbitrary position at a single viewing angle that includes the motion plane of the sphere, and a sensing processing unit that sets a positional relationship between the reference plane and the single camera from the image acquired by the single camera using the motion plane of the sphere as a reference plane, detects an object corresponding to the sphere in the acquired image, calculates planar position coordinates of the sphere on the reference plane using the positional relationship between the reference plane and the single camera and information on the object, and calculates information regarding the movement of the sphere from changes in the calculated planar position coordinates.

[0013] According to one embodiment of the present invention, a sensing device for a golf ball moving on a putting mat includes a single camera for acquiring an image at a single viewing angle including the putting mat at an arbitrary position, and a sensing processing unit for recognizing a predetermined and displayed feature on the putting mat from the acquired image of the single camera, setting a positional relationship with the single camera using the putting mat as a reference plane, detecting feature points on a contour of an object corresponding to the golf ball in the acquired image, calculating position coordinates of a point where a center point of the golf ball is vertically projected onto the reference plane using the feature points as planar position coordinates of the golf ball, and calculating information regarding the movement characteristics of the golf ball on the putting mat from changes in the calculated planar position coordinates.

[0014] A motion sensing method for a planar moving structure according to one embodiment of the present invention includes the steps of acquiring an image from an arbitrary position using a single camera at a single viewing angle that includes the motion plane of the sphere, setting a positional relationship between the reference plane and the single camera from the acquired image of the single camera using the motion plane of the sphere as a reference plane, detecting feature points on a contour of an object corresponding to the sphere in the acquired image, and calculating, using the positional relationship between the reference plane and the single camera, the position coordinates of a point where the center point of the sphere is projected perpendicularly to the reference plane from the feature points, as the planar position coordinates of the sphere on the reference plane.

[0015] According to an embodiment of the present invention, a method for sensing a golf ball moving on a putting mat includes the steps of: acquiring an image at a single viewing angle including the putting mat from an arbitrary position using a single camera; setting a positional relationship between the single camera and the putting mat as a reference plane from the acquired image of the single camera; detecting an object corresponding to the golf ball from the image acquired by the single camera and preparing to hit the golf ball; detecting an impact of the golf ball from the image acquired by the single camera; and, when the impact is detected, calculating position coordinates of a point where a center point of the golf ball is projected perpendicularly onto the reference plane from a point on a contour of the object using the set positional relationship between the single camera as a planar position coordinate of the golf ball; and calculating the planar position coordinates of the golf ball for each frame of the image acquired by the single camera and calculating information on a movement characteristic of the golf ball from changes in the planar position coordinates. Effect of the Invention

[0016] The motion sensing device and method for a sphere moving on a plane and the motion sensing device and method for a golf ball moving on a putting mat according to the present invention are capable of detecting a sports sphere moving on a plane as a target object using a low-cost single camera without dedicated lighting. Since spatial recognition is possible, the camera installation position is not restricted to be fixed, and the device and method can be used in any location with a simple configuration, and has the effect of being able to calculate information regarding the exact position and motion of a sphere moving on a plane. [Brief description of the drawings]

[0017] [Figure 1] 1 is a diagram showing a motion sensing device for a golf ball moving on a putting mat, as a specific example of a motion sensing device for a planar moving sphere according to the present invention. [Diagram 2] A diagram showing the image captured by a single camera in the positional relationship between the single camera and the putting mat as shown in Figure 1. [Diagram 3] 4 is a flowchart illustrating a method for sensing the motion of a planar moving sphere according to an embodiment of the present invention. [Figure 4] 4 is a diagram showing an example of a method for detecting an object corresponding to a sphere and detecting its contour from an image acquired by a single camera of a sensing device according to an embodiment of the present invention. [Diagram 5] 4 is a diagram showing an example of a method for detecting the upper and lower endpoints of a sphere on an image acquired by a single camera of a sensing device according to an embodiment of the present invention. [Figure 6] 1 is a diagram showing that a sensing device according to an embodiment of the present invention recognizes features in an image, detects an object corresponding to a sphere moving on a plane, and detects feature points of the object. [Figure 7] 7 is a diagram showing calculation of the position coordinates of the point obtained by vertically projecting the center point of a sphere onto a reference plane, based on the feature points on the object shown in FIG. 6. [Figure 8]4 is a flowchart illustrating a method for sensing the movement of a golf ball moving on a putting mat according to an embodiment of the present invention. [Figure 9] 9 is a diagram illustrating an example of a binarized image of an image captured by a single camera for explaining the method according to the flowchart of FIG. 8. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will now be described in detail with reference to the accompanying drawings, in which: FIG. 1 is a diagram showing a motion sensing apparatus and method for a plane moving sphere and a motion sensing apparatus and method for a golf ball moving on a putting mat according to the present invention;

[0019] The present invention can use a camera to obtain images of any spherical object moving on a flat surface, such as a golf ball or a bowling ball, and analyze the images to calculate the movement information of the spherical object. As a specific example, a camera can be used to sense the movement of a golf ball on a putting mat as it is putted by a user with a putter, and the movement information can be calculated.

[0020] The camera described above may be implemented as a "single camera" that captures an image at a single viewing angle including the motion plane of the sphere.

[0021] In conventional camera sensor-based screen golf systems, golf practice or putting practice systems, the camera that detects the movement of the golf ball must be fixed at a preset position, and there are major constraints that multiple cameras must be configured in a stereoscopic manner to detect the three-dimensional movement of the golf ball. However, the present invention breaks away from such constraints and is characterized by the fact that a single camera with a single viewing angle, i.e., a single camera, can detect the movement of the golf ball at any position.

[0022] FIG. 1 shows a motion sensing device for a golf ball moving on a putting mat as a specific example of a motion sensing device for a planar moving sphere according to the present invention.

[0023] A motion sensing device for a planar moving structure according to an embodiment of the present invention includes a single camera 200 and a sensing processing unit 220 as shown in FIG.

[0024] A single camera 200 captures an image at a preset frame rate from an arbitrary position at a single viewing angle and a viewing angle including the plane of motion of a sphere, for example, a putting mat 100. For example, an image is captured at a position as shown in Fig. 1, not in a stereoscopic manner, but at a single viewing angle and including the putting mat 100 where a golf ball 10 is hit by a putter 20 and moves in a plane.

[0025] The sensing processing unit is a component that processes an image captured by a single camera, analyzes the processed image, and calculates information about the position of the sphere on a plane.

[0026] When a single camera is used to capture an image with a field of view viewed from an arbitrary position rather than from a preset fixed position, it is extremely difficult to calculate the three-dimensional position information of a sphere in space due to the sense of perspective in the captured image, and even if three-dimensional position information is calculated, the accuracy will be significantly reduced.

[0027] The present invention uses a single camera to set the plane on which the sphere moves as a reference plane, and calculates the position coordinates of the point where the center point of the sphere is vertically projected onto the reference plane, i.e., the position coordinates on the plane of the sphere, to calculate the movement information of the sphere on the reference plane.

[0028] The sensing processing unit recognizes a "reference plane" for the motion plane of the sphere from the image captured by the single camera, sets the positional relationship between the reference plane and the single camera, and uses that positional relationship to calculate the planar position coordinates of the sphere on the "reference plane" from an object corresponding to the sphere in the image captured by the single camera.

[0029] 1, the sensing processing unit 220 can recognize pre-prepared and displayed feature portions 111, 112, 113, and 114 on the putting mat 100 from an image captured by a single camera 200, and set a reference plane for the plane of motion of the golf ball and position information of the single camera relative to that reference plane, i.e., three-dimensional position information. The sensing processing unit can be configured to detect an object corresponding to the golf ball in the captured image, and calculate the position of the point where the center point of the golf ball 10 is vertically projected onto the reference plane, i.e., the coordinates of the planar position of the golf ball.

[0030] As shown in FIG. 1, characteristic features 111-114 are recognized in an image captured by a single camera 200, and the characteristic features are analyzed to calculate x, y, and z coordinate information of the reference plane and the single camera based on the origin of the xyz coordinate system, thereby setting the positional relationship between the reference plane and the single camera.

[0031] Figure 2 shows images captured by a single camera in the positional relationship between the single camera and the putting mat as shown in Figure 1, where (a) in Figure 2 shows the state before the golf ball is hit, and (b) in Figure 2 shows the movement of the golf ball after it has been hit.

[0032] (a) and (b) in Figure 2 show an example of images captured by a single camera, where i100 represents an image of a putting mat, i111 to i114 represent images of features, i10 represents an image of a stationary golf ball, and ir10 represents an image of a moving golf ball.

[0033] As shown in FIG. 1, a single camera 200 can capture an image of a putting mat 100, which is the plane on which the sphere moves, from an angle of view looking diagonally downward from one side of the upper part. However, as shown in FIGS. 2(a) and (b), depending on the distance from the single camera, the parts of the putting mat that are closer to the single camera appear larger and the parts that are farther away appear smaller, and the size of the sphere varies in the image depending on the distance between the sphere and the single camera.

[0034] The image captured by a single camera is a two-dimensional image, and when analyzing the image content, it is very difficult to accurately determine the coordinates of the position of the sphere on a plane due to the difference in size depending on the distance, as shown in Figure 2 (a) and (b).

[0035] However, as described above, the present invention uses the "feature" to set the positional relationship between the reference plane and a single camera, and uses this to analyze the image to accurately calculate the planar position coordinates of the sphere.

[0036] Therefore, the sensing device according to the embodiment of the present invention can recognize a feature prepared in advance on a plane as shown in FIG. 1 through an image as shown in FIG.

[0037] The aforementioned "characteristic part" may be in the form of markers 111-114 prepared in advance on a flat surface as shown in FIG. 1, or it may be in the form of the motion plane itself, for example, the form of a rectangular putting mat itself as shown in FIG. 1.

[0038] A sensing device according to one embodiment of the present invention presets information regarding the shape and size of a feature, and can grasp information regarding a reference plane and the position of a single camera on a set coordinate system by referring to and analyzing the preset feature information to determine how the shape and size of a feature recognized through an image have changed.

[0039] For example, as shown in FIG. 1, characteristic parts 111, 112, 113, and 114 are provided in advance at the corners of the putting mat 100, and information regarding the shape, size, etc. of the characteristic parts 111, 112, 113, and 114 is set in advance. The sensing processing unit 220 can compare and analyze the changes in the shape and size of parts i111, i112, i113, and i114 corresponding to the characteristic parts through the image acquired by the single camera 200 as shown in (a) and (b) of FIG. 2 with the previously set information, and can then calculate the coordinates at which the single camera is located based on a reference plane on the xyz coordinate system and store them in the setting information.

[0040] In addition, for example, the rectangular shape of the putting mat 100 itself as shown in FIG. 1 can be set in advance as a characteristic part, and by analyzing the changes in shape and size of the part (i100) corresponding to the putting mat in the image as shown in FIG. 2, the positional relationship between the reference plane and the single camera can be calculated and saved in the setting information.

[0041] The feature may be in the form of an artificially applied marker, and if the form of the motion plane can be identified, the form of the motion plane itself can be set as the feature. Even if the form is an artificially applied marker, it is possible as long as the changes in shape and size can be easily analyzed in the image, so any shape of marker can be used as the feature as described above.

[0042] The sensing processing unit recognizes predefined features displayed on the motion plane through the image of a single camera, calculates the positional relationship between the single camera and the motion plane, sets a reference plane as described above, and then calculates where the sphere is located on the reference plane; a specific method for calculating the position of the sphere on the reference plane will be described later.

[0043] Now, a method for sensing the motion of a planar moving sphere according to an embodiment of the present invention will be described with reference to a flow chart of FIG.

[0044] The flowchart shown in FIG. 3 illustrates a method for sensing the motion of a sphere based on the configuration of the sensing device described in FIGS.

[0045] First, a single camera captures an image of the motion plane of a sphere within its viewing angle at an arbitrary position (S110).

[0046] A single camera can be configured to capture images in ambient lighting by adjusting imaging parameters such as brightness, contrast, and gamma, without the need for dedicated lighting.

[0047] The sensing processing unit recognizes a predetermined feature on the acquired image (S120), and then sets a positional relationship between a reference plane and a single camera with respect to the motion plane of the sphere (S130).

[0048] The sensing processing unit recognizes a sphere from an image captured by a single camera (S140), and determines whether the sphere has started to move (S150).

[0049] If the sphere starts moving, the sensing processing unit analyzes the image captured by the single camera to detect an object in the image that corresponds to the sphere (S160).

[0050] Then, the sensing processing unit can detect feature points on the contour of the detected object as described above (S170).

[0051] The sensing processing unit detects points on the contour of an object detected from an image captured by a single camera, i.e., feature points, and calculates the position coordinates of the point where the center point of the sphere is vertically projected onto a reference plane as the planar position coordinates of the sphere using the feature points (S180).

[0052] By connecting the planar position coordinates of the sphere calculated as described above to each of the multiple image frames acquired by a single camera, the motion trajectory of the sphere can be calculated, and the velocity of the sphere can be calculated based on the change in the planar position coordinates of the sphere over time.

[0053] In this manner, the sensing device of the present invention can calculate the motion information of the sphere (S190).

[0054] The specific details of the calculation of the planar position coordinates of the sphere using the feature points of the object in the image as described above, i.e., the specific details of the process from detecting the object in the image through steps S160 to S180 as described above to calculating the coordinate information projected onto the reference plane, will be described with reference to Figures 4 to 6.

[0055] Figures 4 and 5 are diagrams for explaining how to detect an object corresponding to a sphere from an image captured by a single camera of a sensing device according to one embodiment of the present invention, and then calculate feature points for calculating the planar position coordinates of the sphere. Figures 6 and 7 are diagrams for explaining how to calculate the position coordinates of a point obtained by projecting the center point of a sphere vertically onto a reference plane based on the upper and lower points, which are feature points of an object detected by a sensing device according to one embodiment of the present invention.

[0056] First, an example of a method for detecting an object from an image captured by a single camera and detecting its contour will be described with reference to FIG.

[0057] FIG. 4(a) shows an enlarged view of a portion of an image captured by a single camera in the configuration shown in FIG.

[0058] The sensing processing unit recognizes pre-defined features (such as i114) on the motion plane from an image such as that shown in (a) of Figure 4, and uses this to calculate the positional relationship between the reference plane and the single camera, and stores the calculated information in the setting information.

[0059] It was found that the pixels in the area corresponding to the sphere in the image captured by a single camera have brightness values ​​that are significantly different from the surrounding pixels.

[0060] However, the group of pixels that are distinguished from the surroundings on the image are not limited to specific areas. For example, when putting, the area corresponding to the putter can also appear as a group of pixels with brightness values ​​that are distinguished from the surroundings, and the same goes for the user's feet.

[0061] When there are many objects, which are groups of pixels with similar characteristics, in an image, it is necessary to detect which objects correspond to the sphere.

[0062] Conditions such as brightness, roundness, aspect ratio, etc. for pixels on an image are preset according to the characteristics of a sphere, and it is possible to detect which objects appearing on an image correspond to a sphere based on the preset conditions. As shown in Figure 4, objects that correspond to a sphere are displayed as OB.

[0063] However, as can be seen in the image in Figure 4(a), the object OB does not have a clear contour and is not a perfect sphere, so in order to calculate the planar position coordinates of the sphere, it is necessary to detect the accurate contour of the object OB.

[0064] Figure 4(b) shows an enlarged view of object OB. Since the motion plane is located on the bottom side, the upper boundaries of most objects in the image captured under any lighting, whether natural lighting, indoor lighting, or dedicated lighting, appear quite accurately.

[0065] In particular, in the case of a sphere, as shown in (a) and (b) of Figure 4, the upper end portion of the object OB appears in the image quite accurately, so the curved portion of the upper boundary Tb of the object can be easily identified.

[0066] However, because the motion plane is located on the floor, a shadow is cast on the bottom edge of the sphere under natural or artificial lighting. As can be seen in the image in Figure 4(b), the pixel brightness of the object is partially darkened due to the shadow at the bottom S of the object OB, and the boundary of the lower part of the object OB is quite unclear, and the bright pixel part seen in the image is different from the actual outline of the sphere.

[0067] In order to accurately detect the bottom boundary of the object OB, the curve of the top boundary Tb of the object OB can be used. Figure 4(c) shows the result of detecting the contour of an object by using the curve of the top boundary Tb of the object OB to form a circle fc with the curvature of that top boundary.

[0068] First, as shown in (b) of Figure 4, the curve of the upper boundary Tb of an object OB in an image can be detected using the brightness values ​​of the pixels of the object OB. For example, a threshold value for the brightness values ​​of the object's pixels can be set in advance, and the curve of the upper boundary Tb can be detected by identifying the pixels that fall on the boundary of that threshold value.

[0069] As described above, after detecting the curve of the upper boundary Tb of the object, the contour of the object OB can be identified by calculating a circle fc having the curvature of the curve of the upper boundary Tb, as shown in (c) of Figure 4.

[0070] As shown in (c) of Figure 4, by detecting the contour of the object OB using a circle fc that utilizes the curvature of the curve of the upper boundary Tb, it is possible to clearly identify the boundary of the lower part of the object that becomes unclear due to shading caused by the lighting and the loss of pixels in the lower part.

[0071] As described above, the top end point TP of the top boundary Tb of an object can be detected from the object's contour, and the bottom end point BP of the object that is symmetrical to the top end point TP can be detected.

[0072] It has been explained above that the planar position coordinates of a sphere can be calculated using feature points on the contour of an object, and the above-mentioned upper point TP and lower point BP can function as those feature points.

[0073] As described above, after detecting an object OB corresponding to a sphere in the image, the upper boundary Tb of the object OB is detected and a circle is constructed using the curvature of the curve of the upper boundary, thereby identifying the contour of the object, and then the characteristic points, the upper point TP and the lower point BP, can be detected.

[0074] However, the shape of an object OB detected in an image may not only be a circle whose contour is easy to identify, but also a shape whose contour is difficult to identify.

[0075] If the upper boundary of an object as described above does not exhibit the desired curved form, a circle constructed based on that curvature may appear too large or too small.

[0076] Therefore, the present invention can use in parallel the method of detecting feature points (upper and lower points) on the object contour using the circle fitting method that uses the curvature of the curve of the upper boundary of the object as described above, as well as the method of detecting feature points using a detection figure DR as shown in Figure 4(d).

[0077] As shown in (d) of FIG. 4, the upper boundary Tb of the object OB is detected, and the upper end point TP on the upper boundary Tb is detected. A predetermined type of figure, for example a rectangular detected figure DR is generated to match the size and shape of the object OB, and the upper point TP can be detected using the size d of the matched detected figure DR. It is also possible to detect a lower end point BP located symmetrically about the size d from the upper point.

[0078] The above-mentioned detection figure DR is a figure whose type is predetermined, and its shape and size can be generated by rotating or transforming it to match the object OB, and it can appear as a rotated rectangle as shown in Figure 4(d).

[0079] Here, the detection figure DR being matched to the object OB means that the detection figure DR most appropriately includes the object OB by changing the length of the sides of the detection figure DR or by rotating the detection figure DR.

[0080] For example, when the detection figure DR is generated so as to include the object OB, the case where the average brightness value inside the detection figure DR is the largest can be determined as the case where the detection figure DR is matched with the object OB as described above.

[0081] As described above, the matched detected figure DR may appear with different lengths on one side and the length of the side perpendicular to the direction of movement of the sphere can be determined by the diameter d of the object, and the point at the position corresponding to the upper end point TP of the upper boundary of the object by the diameter d can be detected as the lower end point BP. Figure 4(d) shows an example of the detection of the upper point TP and the lower point BP, which are characteristic points of an object, using the method described above.

[0082] When an object is detected in an image captured by a single camera, it may not appear simply in a circular shape, but may appear significantly blurred and spread out in the image, as shown in FIG. 5(a).

[0083] In such a case, if the upper boundary Tb of the object OB is detected as shown in FIG. 5(b), its curvature will be significantly larger, and therefore the method of circle fitting based on the curvature will not be able to detect the object contour accurately.

[0084] In such a case, the method of detecting feature points of an object using the detection figure DR as described above can produce more desirable results.

[0085] As shown in FIG. 5B, the upper boundary Tb of the object OB can be detected, and a detected figure DR, for example a rectangle, can be generated so as to match the size and shape of the object OB.

[0086] At this time, the detected figure DR only has a predetermined type of figure, and its size and rotated posture can be transformed to suit the state of the object.

[0087] The above-mentioned matched detected figure DR is a rectangular figure with long and short sides, and the size of the object in the direction of its spread, i.e., the direction perpendicular to the direction of movement of the golf ball, can be determined from the diameter of the object outline.

[0088] As a result, the size d of the short side of the detected figure DR matched to the object OB shown in FIG. 5(b) is the diameter of the object.

[0089] Therefore, as shown in FIG. 5C, an upper end point TP can be detected from the upper boundary Tb of the object, and a point at a position symmetrical to the upper end point TP by a distance of size d can be detected as a lower end point BP.

[0090] The circle fitting method using the curvature of the upper boundary of an object shown in Figures 4(b) and (c) is a method of detecting the contour of an object and then detecting the feature points, which are the upper and lower points, while the method shown in Figure 4(d) and Figures 5(b) and (c) is a method of calculating the size of an object using a detected figure instead of detecting the contour of the object and then detecting the positions of the upper and lower points using this. However, a sensing device according to one embodiment of the present invention can accurately detect the feature points of an object by using the two methods described above in parallel.

[0091] As described above, the top end point TP and bottom end point BP of the object are detected and used to obtain the coordinates of the point where the center point of the sphere is vertically projected onto the reference plane, i.e., the planar position coordinates of the sphere.

[0092] FIG. 6 shows that feature points i111 to i114 are recognized from the image captured by a single camera at a time as shown in FIG. 2(d), an object OB corresponding to a structure moving on a plane is detected, and the feature points of that object, the upper point TP and the lower point BP, are detected.

[0093] As mentioned above, the direction of the line passing through the detected upper point TP and lower point BP is the line of sight of the single camera, the lower point BP is the position of a point close to the position of the single camera in the line of sight of the single camera, and the upper point TP is the position of a point far from the position of the single camera from the line of sight of the single camera.

[0094] By using the setting information regarding the positional relationship between the reference plane and the single camera and the position information of the upper and lower points of the object described above, it is possible to calculate the "planar position coordinates of the sphere", i.e., the position coordinates of the point where the center point of the sphere is vertically projected onto the reference plane.

[0095] Here, the aforementioned "center point of the sphere" is completely different from the center point of the contour of an object in an image captured by a single camera as shown in Figure 6, and the aforementioned "planar position coordinates of the sphere" cannot be calculated from the coordinates of the center point of the contour of the object.

[0096] When the center point of the contour of an object is designated as A in an image captured by a single camera as shown in FIG. 6, point A is not the center point of the actual sphere, but a point at a different position that is significantly different from the center point of the actual sphere.

[0097] In addition, the top point TP and bottom point BP of the object OB shown in Figure 6 are not points on an actual sphere, but are the positions where the line of sight as seen from a single camera position intersects with the reference surface of the putting mat, so the top end point TP and bottom end point BP of the object shown in Figure 6 are completely different from the top end point and bottom point on a sphere in real space.

[0098] Similarly, since the center point A of the outline of the object OB shown in Figure 6 is not the center point of the actual structure on a plane, it cannot be used to calculate the planar position coordinates of the structure. Instead, the top end point TP and bottom end point BP of the object can be used to calculate the planar position coordinates of the sphere through geometric calculations.

[0099] This will be explained with reference to FIG.

[0100] FIG. 7 shows how to calculate the position coordinates of the point obtained by projecting the center point of the sphere perpendicularly onto the reference plane based on the upper and lower points that are characteristic points of the object as shown in FIG.

[0101] FIG. 7(a) shows a cross section taken along the line of sight of a single camera connecting the upper point TP and the lower point BP of an object on the xy coordinate plane in real space, and FIG. 7(b) shows the xy coordinate plane in real space.

[0102] As described above, a single camera can capture an image and recognize a portion of the captured image that corresponds to a characteristic portion, thereby saving setting information regarding the positional relationship between the reference plane and the single camera.

[0103] That is, the sensing processing unit of the sensing device according to one embodiment of the present invention can recognize features from an image captured by a single camera, and can recognize a reference plane sPL with respect to the motion plane of the sphere as shown in (a) of Figure 7, and set a geometric relationship between the reference plane sPL and the position P200 of the single camera.

[0104] By setting the positional relationship between the reference plane and the single camera as described above, the height H of the single camera position P200 relative to the reference plane sPL can be set as shown in (a) of Figure 7, and the x, y coordinate information of the single camera position P200 can also be set as shown in (b) of Figure 7. Using the height and position information set for the single camera position P200, the angle information of the line of sight direction from the single camera position P200 to the reference plane sPL can also be calculated as shown in (a) of Figure 7.

[0105] The upper point TP and the lower point BP of the object OB shown in Figure 6 are the points (TP, BP) where the line of sight of a single camera passes through the upper and lower parts of the sphere CB in the actual coordinate system and intersects with the reference plane sPL, as shown in (a) and (b) of Figure 7.

[0106] The "planar position coordinates of a sphere" to be calculated by the sensing device of the present invention are the x, y coordinates of a point Pc obtained by vertically projecting the center point C of a sphere CB onto the reference plane sPL, as shown in FIG. 7(a).

[0107] However, since the position of the center point C of the sphere CB is unknown, the coordinates of point Pc can be calculated by utilizing the geometric relationship between the positions of the upper point TP and the lower point BP on the reference plane sPL as described above.

[0108] As shown in FIG. 7(a), the bottom end point BP on the reference plane sPL differs from the point Pc formed when the center point C of the sphere is vertically projected onto the reference plane sPL, and this difference is taken to be an error E1.

[0109] Similarly, as shown in (a) of Figure 7, the upper end point TP on the reference plane sPL differs from the point Pc obtained by vertically projecting the center point C of the sphere onto the reference plane sPL, and this difference is defined as the error E2.

[0110] Furthermore, as shown in (a) of Figure 7, the distance L1 from the origin O to the lower point BP and the distance L2 from the origin O to the upper point TP can be calculated using setting information of the positional relationship between the reference plane sPL and the single camera position P200, and the angle a between the reference plane sPL and the line of sight of the single camera at the lower point BP can be calculated from the height H and distance L1 of the single camera position P200, and the angle b between the reference plane sPL and the line of sight of the single camera at the upper point TP can be calculated from the height H and distance L2 of the single camera position P200.

[0111] As shown in (a) of Figure 7, the radius r of the actual sphere is a known value that is preset in the sensing device, so by using the position and angle of the upper point TP on the reference plane sPL, the position and angle of the lower point BP on the reference plane, and a circle defined by the radius r of the actual sphere, the position coordinates of the point Pc where the center point C of the sphere is projected vertically onto the reference plane sPL can be calculated through geometric calculations.

[0112] In Fig. 7(a), the error E1 between point Pc, which is the vertical projection of the center C of the sphere onto the reference plane sPL, and the lower end point BP, can be calculated by a geometric operation, and the error E2 between point Pc and the upper end point TP can also be calculated by a geometric operation. That is, as shown in Fig. 7(a), the error E1 can be calculated by a trigonometric function by applying angle information a and the radius r of the sphere to the triangle formed by the center point C of the sphere, the lower point BP, and point Pc.

[0113] In addition, as shown in (a) of Figure 7, the error E2 can be calculated using trigonometric functions by applying the b angle information and the radius r of the sphere to the triangle formed by the point VP where the line connecting the single camera position P200 and the upper point TP intersects with a line perpendicular to the center point C of the sphere, the upper point TP, and point Pc.

[0114] As shown in (b) of Figure 7, since the coordinates of points BP and TP are already known on the xy coordinate plane of reference plane sPL, it is possible to calculate the x, y coordinate information of point Pc, which is the position corrected by error E1 or error E2.

[0115] Therefore, as described above, by detecting the upper point TP and the lower point BP, which are the characteristic points of the object, it is possible to use these to calculate the error E1 from the lower point and the error E2 from the upper point by geometric calculations such as those shown in Figure 7(a). These errors can then be used to calculate the coordinates of the point where the center point of the sphere is projected perpendicularly onto the reference plane, i.e., the coordinates on the xy plane (reference plane) of point Pc, which is the planar position coordinate of the sphere.

[0116] In this manner, by calculating the planar position coordinates of the sphere from all frames of images to be analyzed among images captured by a single camera, information regarding the motion of the sphere due to its planar movement can be calculated.

[0117] Now, a method for sensing the movement of a golf ball moving on a putting mat according to an embodiment of the present invention will be described with reference to a flow chart of FIG.

[0118] While the flowchart shown in FIG. 3 illustrates a process for calculating the planar position coordinates of a sphere using images captured by a single camera regarding the planar motion of the sphere, the flowchart shown in FIG. 8 illustrates a process for sensing the motion of a golf ball when the sphere is a golf ball and the golf ball moves on a putting mat, which is its plane of motion.

[0119] Therefore, the methods of detecting an object in an image, detecting the contour of the object, and calculating the coordinates of the point where the center point of a sphere is projected vertically onto a reference plane described in Figures 4 to 7 can also be applied to the method of calculating the planar position coordinates of a golf ball on a putting mat described in Figure 8.

[0120] The flowchart shown in FIG. 8 illustrates a more specific process specialized for golf putting than the sensing method according to the flowchart shown in FIG.

[0121] As shown in FIG. 8, first, an image is captured by a single camera at an arbitrary position (S210), and the sensing processing unit recognizes features on the putting mat from the captured image and sets the positional relationship between the reference surface and the single camera (S220).

[0122] (a) of Figure 9 shows a binarized image of the image captured by a single camera. By recognizing i111, i112, i113, and i114, which correspond to the characteristic parts of the putting mat, the position of the reference plane and the single camera can be recognized and set.

[0123] As described above, in the process of setting the reference plane and the position of the single camera, image parameters such as brightness, contrast, gamma, etc. can be automatically adjusted so that object detection according to the ambient lighting is performed effectively.

[0124] Meanwhile, as shown in FIG. 8, the sensing processing unit sets a reference plane, and then sets a region of interest for recognizing a golf ball on an image of a single camera (S232).

[0125] Fig. 9(a) shows a case where a region of interest ROI is set to a size and shape preset so as to include an object OB corresponding to a golf ball. As shown in Fig. 9(a), when it is determined that an object OB, i.e., an object OB determined to correspond to a golf ball, is present within the region of interest ROI (S234), it is determined that preparation for hitting is complete, and the state becomes "ball ready" (S236).

[0126] If the region of interest ROI for ball-ready determination is set too narrow, the position where the golf ball can be initially placed will be limited, so it is desirable to set the region of interest ROI to be as wide as possible, as shown in Figure 9(a), so that the user can place the golf ball in any position on the putting mat and start putting.

[0127] When the ball is ready, the sensing processing unit sets a trigger sensing area TR for sensing whether the golf ball has been hit (S236). The trigger sensing area TR can be set to a predetermined area included within the object OB, as shown in FIG. 9(b).

[0128] As shown in FIG. 9B, a trigger sensing region TR is set inside an object OB, and it is determined whether or not a change in brightness inside the trigger sensing region TR exceeds a preset reference value (S242).

[0129] As shown in (b) of Figure 9, a trigger detection area TR is set within an object OB, and in the process of detecting the brightness within the trigger detection area TR, if the golf ball does not move slightly, the brightness within the trigger detection area TR does not change significantly, so the system does not move to the next stage for trigger judgment. However, if the golf ball is hit and begins to move, the brightness within the trigger detection area TR changes significantly.

[0130] In this way, when the brightness inside the trigger sensing region TR changes significantly and exceeds a preset reference value, it is determined that the golf ball has moved primarily, and a first trigger signal is generated.

[0131] For example, when the trigger sensing area TR is generated, the internal brightness value is 100 and the reference value of the brightness value change for generating the first trigger signal is 30. If the golf ball moves as shown in Figure 9(b) and the brightness inside the trigger sensing area TR drops from 100 to 60, the brightness value changes by 40, which exceeds the reference value of 30, and the first trigger signal is generated.

[0132] Meanwhile, if the first trigger signal is generated as described above, the sensing processing unit detects objects from multiple frames of images, calculates the amount of movement of the objects, and determines whether the calculated amount of movement of the objects exceeds a preset reference value (S244).

[0133] In actuality, when the golf ball moves from its initial position on the putting mat, it may happen that the user touches the golf ball by mistake with the putter or foot, in addition to hitting the golf ball. Therefore, a reference value for the amount of movement of the golf ball is preset, and the golf ball is judged to have been hit only when it moves more than a certain amount.

[0134] After the first trigger signal is generated as described above, the object is detected and the amount of movement is calculated. If it is determined that the object has moved beyond a preset reference value, the sensing processing unit generates a ball trigger (S246).

[0135] In other words, after the ball is ready (S236), the sensing processing unit detects a change in brightness within the trigger detection area to generate a first trigger, and then detects the amount of movement of the object again to generate a ball trigger, thereby detecting whether or not the golf ball has been hit using a dual trigger method of the first trigger and the ball trigger.

[0136] As described above, when the ball trigger occurs, the sensing processing unit reads and analyzes the images acquired by a single camera around the time of the ball trigger, and detects an object corresponding to a golf ball for each frame of the image to be analyzed (S250). This can be done using the same method as the method for detecting an object corresponding to a sphere described in FIG.

[0137] After detecting objects as described above, feature points on the contour of each detected object are detected, and there are two methods available for detecting the feature points, as described above. As described with reference to Figures 4 and 5, the contour of the object can be calculated using a circle fitting method that uses the curvature of the curve of the upper boundary of the object, and the upper and lower points can be detected from the contour, and the size information of the object can be calculated using the detected figure, and the upper and lower points can be detected using this. After calculating the feature points on the contour of the object in this way, the position coordinates of the point where the center point of the golf ball is projected vertically onto the reference plane are calculated as the planar position coordinates of the golf ball using this (S260).

[0138] The method of calculating the plane position coordinates of the golf ball described above can be performed by calculating the error E1 or E2 from the upper and lower end points of the object, as described with reference to FIG.

[0139] By calculating the planar position coordinates of the golf ball as described above for each video frame to be analyzed, the change in the position coordinates can be calculated, and information regarding the movement characteristics of the golf ball can be calculated from the change in the planar position coordinates of the golf ball (S270).

[0140] After calculating the information regarding the movement characteristics of the golf ball, the sensing processing unit can transmit the above-mentioned golf ball movement characteristics information to a client (300, see FIG. 1), such as a simulator of a putting simulation device or a computer of a golf information providing device, to provide the information to a user.

[0141] Meanwhile, after calculating the information regarding the movement characteristics of the golf ball as described above, the sensing processing unit can check the characteristic portion again to check whether the reference plane has shifted from the initially set state (S280). If the reference plane has shifted, the process returns to step S220 to set the reference plane again, and if there is no change in the reference plane, sensing of the movement of the golf ball is performed based on the previously set reference plane.

[0142] As described above, the device and method for sensing the motion of a sphere moving on a plane and the device and method for sensing the motion of a golf ball moving on a putting mat according to the present invention are capable of detecting a sports sphere moving on a plane as a target object by using a low-cost single camera without dedicated lighting. In addition, since spatial recognition is possible, the camera installation position is not restricted to be fixed, and the device and method can be used in any location with a simple configuration, and has the advantage of being able to calculate information regarding the exact position and motion of a sphere moving on a plane. [Industrial Applicability]

[0143] The motion sensing device and method of a flat-surface moving sphere and the motion sensing device and method of a golf ball moving on a putting mat according to the present invention can be used in technical fields based on sensing of a sports sphere moving on a flat surface, in the field of golf analysis based on analysis of the movement of a golf ball due to a golf swing, and in the field of virtual golf simulation systems.

Claims

1. As a sensing device for a sphere moving on a plane, A single camera that captures an image from an arbitrary position at a single viewing angle that includes the motion plane of the sphere; a sensing processing unit that sets a positional relationship between the reference plane and the single camera using a motion plane of the sphere as a reference plane from the captured image of the single camera, detects an object corresponding to the sphere in the captured image, calculates planar position coordinates of the sphere on the reference plane using the positional relationship between the reference plane and the single camera and information on the object, and calculates information related to the movement of the sphere from changes in the calculated planar position coordinates; Including, The sensing processing unit is detecting an upper boundary of an object detected in the image, detecting an upper end point of the upper boundary, generating a detection figure that matches a size and a shape of the object, and detecting a lower end point of the object corresponding to the upper end point using a size of the generated detection figure; calculating, using the upper end point or the lower end point, a position coordinate of a point where the center point of the sphere is vertically projected onto the reference plane as a planar position coordinate of the sphere; A motion sensing device for a planar moving sphere.

2. The single camera includes: The planar moving spherical motion sensing device of claim 1 , wherein when the positional relationship between the reference surface and the single camera is set without a dedicated light, an image with ambient lighting can be obtained by adjusting image parameters.

3. The sensing processing unit is The motion sensing device for a planar moving sphere as described in claim 1 or 2, which recognizes a feature that is pre-set and displayed on the motion plane of the sphere through the image, and sets a positional relationship between the reference plane and the single camera based on the recognized information.

4. A sensing device for a sphere moving on a plane, A single camera that captures an image from an arbitrary position at a single viewing angle that includes the motion plane of the sphere; a sensing processing unit that sets a positional relationship between the reference plane and the single camera using a motion plane of the sphere as a reference plane from the captured image of the single camera, detects an object corresponding to the sphere in the captured image, calculates planar position coordinates of the sphere on the reference plane using the positional relationship between the reference plane and the single camera and information on the object, and calculates information related to the movement of the sphere from changes in the calculated planar position coordinates; Including, The sensing processing unit is A planar moving sphere motion sensing device which detects a curve of an upper boundary of an object detected in the image, calculates a circle having a curvature of the curve of the upper boundary, detects at least one point on the circle as a feature point, and calculates, using the feature point, the position coordinates of a point where the center point of the sphere is vertically projected onto the reference plane as the planar position coordinates of the sphere.

5. The sensing processing unit is A motion sensing device for a planar moving sphere described in any one of claims 1 to 4, which detects the upper and lower end points of the contour of the object, and calculates the planar position coordinates of the sphere by correcting the error of the point where the upper or lower end point is projected onto the reference surface in the line of sight of the single camera using setting information regarding the positional relationship between the reference surface and the single camera.

6. A sensing device for a golf ball moving on a putting mat, A single camera that captures an image at a single viewing angle that includes the putting mat at an arbitrary position; a sensing processing unit that recognizes a preset and displayed feature on the putting mat from the captured image of the single camera, sets a positional relationship with the single camera using the putting mat as a reference plane, detects feature points on an outline of an object corresponding to the golf ball in the captured image, calculates position coordinates of a point where a center point of the golf ball is vertically projected onto the reference plane using the feature points as planar position coordinates of the golf ball, and calculates information regarding a movement characteristic of the golf ball on the putting mat from changes in the calculated planar position coordinates; Including, The sensing processing unit is detecting an upper boundary of an object detected in the image, detecting an upper end point of the upper boundary, generating a detection figure that matches a size and a shape of the object, and detecting a lower end point of the object corresponding to the upper end point using a size of the generated detection figure; calculating a position coordinate of a point where the center point of the golf ball is vertically projected onto the reference plane using the upper end point or the lower end point as a planar position coordinate of the golf ball; A sensing device for a golf ball moving on a putting mat.

7. As a method for sensing the motion of a flat moving sphere, acquiring an image at a single viewing angle including a motion plane of the sphere from an arbitrary position by a single camera; determining a positional relationship between the reference plane and the single camera based on the image captured by the single camera and the motion plane of the sphere as the reference plane; detecting an upper boundary of an object corresponding to the sphere in the acquired image, detecting an upper end point of the upper boundary, generating a detection figure that matches a size and a shape of the object, and detecting a lower end point of the object corresponding to the upper end point using a size of the generated detection figure; calculating a position coordinate of a point where a center point of the sphere is vertically projected onto the reference surface from the upper end point or the lower end point using a positional relationship between the reference surface and the single camera as a planar position coordinate of the sphere on the reference surface; A method for sensing the motion of a planar moving sphere, comprising:

8. The step of setting a positional relationship with the single camera includes: recognizing the motion plane of the sphere as a reference plane through a feature portion that is preset and displayed on the motion plane of the sphere from the image acquired in the step of acquiring the image; The method for sensing motion of a planar moving sphere according to claim 7, further comprising the step of: setting position information of the single camera based on recognition of the reference surface.

9. The step of calculating the planar position coordinates of the sphere includes: detecting a contour of the object based on an upper boundary of the object detected in the image; detecting a point and angle information of an upper end point or a lower end point of the contour of the object projected onto the reference plane in a line of sight direction of the single camera; The method for sensing the motion of a planar moving sphere as described in claim 7 or 8, comprising a step of calculating the planar position coordinates of the sphere by correcting an error in the point where the line of sight direction of the single camera passing through the upper end point or the lower end point meets the reference surface using setting information regarding the positional relationship between the reference surface and the single camera, size information of the sphere, and the detected angle information.

10. As a sensing method for a golf ball moving on a putting mat, acquiring an image at a single viewing angle including the putting mat from an arbitrary position using a single camera; determining a positional relationship between the single camera and the putting mat as a reference plane from an image captured by the single camera; detecting an object corresponding to the golf ball from the image acquired by the single camera and preparing to strike the golf ball; detecting an impact of a golf ball from an image acquired by the single camera; calculating, when the impact is detected, a position coordinate of a point where a center point of the golf ball is projected perpendicularly to the reference plane from a point on the contour of the object using a positional relationship between the set reference plane and the single camera, as a planar position coordinate of the golf ball; calculating planar position coordinates of the golf ball for each frame of the image captured by the single camera, and calculating information on the movement characteristics of the golf ball from the changes in the planar position coordinates; Including, The step of calculating the plane position coordinates of the golf ball includes: detecting an upper boundary of an object detected in the image; detecting a top endpoint on the top boundary; generating a detection figure that matches a size and a shape of the object, and detecting a bottom end point of the object corresponding to the top end point using a size of the generated detection figure; calculating a position coordinate of a point where a center point of the golf ball is vertically projected onto the reference plane using the upper end point or the lower end point as a planar position coordinate of the golf ball; A sensing method for a golf ball moving on a putting mat.

11. A method for sensing a golf ball moving on a putting mat, comprising: acquiring an image at a single viewing angle including the putting mat from an arbitrary position using a single camera; determining a positional relationship between the single camera and the putting mat as a reference plane from an image captured by the single camera; detecting an object corresponding to the golf ball from the image acquired by the single camera and preparing to strike the golf ball; detecting an impact of a golf ball from an image acquired by the single camera; calculating, when the impact is detected, a position coordinate of a point where a center point of the golf ball is projected perpendicularly to the reference plane from a point on the contour of the object using a positional relationship between the set reference plane and the single camera, as a planar position coordinate of the golf ball; calculating planar position coordinates of the golf ball for each frame of the image captured by the single camera, and calculating information on the movement characteristics of the golf ball from the changes in the planar position coordinates; Including, The step of calculating the plane position coordinates of the golf ball includes: detecting a curve of an upper boundary of an object detected in the image; calculating a circle having a curvature of the curve of the upper boundary and detecting at least one point on the circle as a feature point; a step of calculating a position coordinate of a point where a center point of the golf ball is vertically projected onto the reference plane using the feature points as a planar position coordinate of the golf ball.

Images