Motion sensing device and method for a plane moving sphere using a camera, and motion sensing device and method for a golf ball moving on a putting mat

Abstract

The present invention aims to provide a device and method for sensing the motion of a two-dimensionally moving ball using a camera, and a device and method for sensing the motion of a golf ball moving on a putting mat, using a low-cost single camera, which not only can detect the sports ball even without separate dedicated lighting, but also can perform spatial recognition. This device and method are free from the constraint that the camera setting position must be fixed, and can be used with a simple structure and not be restricted by location.

Description

Technical Field

[0001] The present invention relates to a sensing device and a sensing method thereof for acquiring and analyzing an image of a spherical object such as a golf ball or bowling ball moving on a plane to calculate information about the movement of the spherical object. Background Art

[0002] Typically, a stereoscopic camera device using two cameras is used as a device for acquiring coordinate information of a certain object in a three-dimensional space.

[0003] That is, two-dimensional coordinate information about the object is extracted from a 2D image of the object taken by one camera and a 2D image of the same object taken by another camera, and the three-dimensional coordinate information of the object is calculated using a predefined correlation function for the two-dimensional coordinate information extracted respectively.

[0004] This type of stereoscopic camera device is widely used to calculate the position information of a certain object in three-dimensional space. As the most representative example, in a virtual golf simulation device such as the so-called screen golf, the ceiling camera and the side camera are linked in three-dimensional form to allow the user to calculate the position change information in three-dimensional space generated by the golf ball movement hit by the golf swing.

[0005] Thus, usually, the detection of the moving sphere and the calculation of the motion state are performed by utilizing multiple cameras to transmit the images captured by each of the multiple cameras to a computing device to calculate the position information in the three-dimensional space, and the motion state of the object is calculated based on the coordinates in the actual space obtained in this way.

[0006] In this regard, conventional art documents such as Korean Patent Publication No. 10-1902283, Korean Patent Publication No. 10-1826837, and Korean Patent Publication No. 10-1723432 disclose technologies related to a virtual golf simulation system using a stereoscopic camera sensing device as described above.

[0007] However, in the method of sensing the movement of a moving object using a stereoscopic camera sensing device as described above, the position of each of the two cameras needs to be fixed, and in order to sense the movement of objects in three-dimensional space, special lighting is required. Therefore, there is a limitation that it must be accurately set up by specialized technicians in a specific place, and there is a problem that it cannot be used simply within spatial constraints.

[0008] Furthermore, since a plurality of cameras need to be connected in a stereoscopic manner, there is a problem that a processing device for processing image data transmitted from each camera must have a very high level of processing capability.

[0009] [Prior art literature]

[0010] Korean Patent Gazette No. 10-1902283

[0011] Korean Patent Gazette No. 10-1826837

[0012] Korean Patent Gazette No. 10-1723432

[0013] Korean Patent Publication No. 10-2019-0014490

[0014] Korean Patent Gazette No. 10-1141048 Summary of the Invention

[0015] Technical issues

[0016] The present invention aims to provide a motion sensing device and method for a two-dimensionally moving ball, and a motion sensing device and method for a golf ball moving on a putting mat, using a low-cost single camera, which uses a single camera and uses the camera as a target object. The device can not only detect the sports ball even without separate dedicated lighting, but also perform spatial recognition. This device and method are free from the constraint that the camera setting position must be fixed, and can be used with a simple structure and not be restricted by location.

[0017] Technical Solution

[0018] An embodiment of the present invention provides a motion sensing device for a plane-moving sphere using a camera, including: a camera that acquires images at an arbitrary position with a single-view shooting angle including the plane of motion of the sphere; and a sensing processing unit that sets a positional relationship with the camera from the image acquired by the camera using the plane of motion of the sphere as a reference plane, detects an object corresponding to the sphere on the acquired image, and calculates the plane position coordinates of the sphere on the reference plane using the positional relationship with the camera and information about the object, so as to calculate information about the motion of the sphere from the changes in the calculated plane position coordinates.

[0019] On the other hand, a sensing device for a golf ball moving on a putting mat according to an embodiment of the present invention includes: a camera that acquires an image at an arbitrary position with a single-angle perspective at a shooting angle including the putting mat; and a sensing processing unit that identifies a pre-set and displayed feature portion on the putting mat from the image acquired by the camera and sets a positional relationship with the camera with the putting mat as a reference plane, and detects feature points on the contour of the object corresponding to the golf ball on the acquired image, and uses the feature points to calculate the position coordinates of the point where the center point of the golf ball is projected in the vertical direction onto the reference plane as the plane position coordinates of the golf ball, so as to calculate information about the movement of the golf ball on the putting mat from the changes in the calculated plane position coordinates.

[0020] On the other hand, a motion sensing method for a planar moving sphere using a single camera according to an embodiment of the present invention includes: a step of acquiring an image at a single perspective by a camera at an arbitrary position with a shooting angle including the moving plane of the sphere; a step of setting a positional relationship with the camera from the acquired image of the camera with the moving plane of the sphere as a reference plane; a step of detecting feature points on the contour of the object corresponding to the sphere on the acquired image; and a step of calculating the position coordinates of the point where the center point of the sphere is projected in the vertical direction to the reference plane from the feature points using the positional relationship with the camera as the planar position coordinates of the sphere on the reference plane.

[0021] On the other hand, a method for sensing a golf ball moving on a putting mat according to an embodiment of the present invention includes: a step of acquiring an image by a camera at an arbitrary position in a single-view manner with a shooting angle including the putting mat; a step of setting a positional relationship with the camera using the putting mat as a reference plane from the image acquired by the camera; a step of detecting an object corresponding to the golf ball from the image acquired by the camera to prepare for hitting the golf ball; a step of sensing the hitting of the golf ball from the image acquired by the camera; a step of calculating, when the hitting is sensed, the position coordinates of a point projected vertically onto the reference plane from a point on the outline of the object to the center point of the golf ball using the set positional relationship between the reference plane and the camera; and a step of calculating the plane position coordinates of the golf ball for each frame of the image acquired by the camera to calculate information about the movement of the golf ball from the change.

[0022] Effects of the Invention

[0023] The motion sensing device and method for a plane-moving sphere using a camera, and the motion sensing device and method for a golf ball moving on a putting mat of the present invention have the following effects: a plane-moving sports sphere is used as a target object, and a low-cost single camera is used. Not only can the sports sphere be detected even without separate dedicated lighting, but spatial recognition can also be performed. Therefore, the constraint of a fixed camera setting position is eliminated, and the device and method can be used with a simple structure without being restricted by location, and accurate position and motion information about the sphere moving on the plane can be calculated. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a diagram showing a specific example of a motion sensing device for a ball moving in a plane using a camera according to the present invention, and shows a motion sensing device for a golf ball moving on a putting mat.

[0025] Figure 2 It is from Figure 1 A diagram showing an image captured by a single camera in the positional relationship between the single camera and the putting mat.

[0026] Figure 3 The flowchart is used to illustrate a method for sensing the motion of a planar moving sphere using a single camera according to an embodiment of the present invention.

[0027] Figure 4 and Figure 5 1 is a diagram for explaining an embodiment of the present invention, in which an object corresponding to a sphere is detected in an image captured by a single camera of a sensing device, and feature points for calculating the planar position coordinates of the sphere are calculated.

[0028] Figure 6 and Figure 7 This diagram is a diagram showing calculation of the position coordinates of a point where the center point of a sphere is projected vertically onto a reference plane based on the upper and lower endpoints of a feature point of an object detected by a sensing device according to an embodiment of the present invention.

[0029] Figure 8 The flowchart is used to illustrate a method for sensing motion of a golf ball by moving a putting mat according to an embodiment of the present invention.

[0030] Figure 9 To illustrate the corresponding Figure 8 The flowchart method is a diagram showing an example of a binary image of an image acquired by a single camera. DETAILED DESCRIPTION

[0031] The following describes in detail the motion sensing device and method for a planar moving sphere using a camera and the motion sensing device and method for a golf ball moving on a putting mat according to the present invention with reference to the appended drawings.

[0032] In the present invention, any spherical object, whether a golf ball or a bowling ball, moving on a plane can be captured and analyzed using a single camera to calculate information about the spherical object's motion. For example, a camera can be used to sense the movement of a golf ball on a putting mat as it is pushed by a user with a putter to calculate motion information. Here, the camera can be implemented as a "single camera" that captures images from a single perspective at a shooting angle that includes the plane of motion of the spherical object.

[0033] A major constraint in previous camera-based screen golf systems or golf practice or putting practice systems is that the camera that senses the movement of the golf ball must be fixed at a predetermined position, and in order to sense the three-dimensional movement of the golf ball, multiple cameras must be installed in a three-dimensional manner. The present invention is characterized by breaking away from this constraint and being able to sense the movement of the golf ball at any position using a single camera.

[0034] Figure 1 This is a specific example of the motion sensing device for a ball moving in a plane using a camera according to the present invention, and shows the motion sensing device for a golf ball moving on a putting mat.

[0035] like Figure 1 As shown, a motion sensing device for a planar moving sphere using a camera according to an embodiment of the present invention includes a single camera 200 and a sensing processing unit 220 .

[0036] The single camera 200 can capture images at a predetermined frame / second at a single viewing angle at any position and including the motion plane of the sphere (eg, the putting mat 100). Figure 1 The position shown is a monoscopic image, not a stereoscopic image, obtained from a shooting angle including the putting mat 100 in which the golf ball 10 is hit by the putter 20 and moves in a plane.

[0037] The sensing processing unit receives and processes the image acquired by the single camera, and analyzes the processed image to calculate components of information about the position of the sphere on a plane.

[0038] When a camera is used to obtain an image from an arbitrary position rather than a pre-set fixed position, it is difficult to obtain the three-dimensional position information of the sphere in space due to the sense of perspective in the obtained image. Even if the three-dimensional position information is obtained, its accuracy is bound to be very low. Therefore, the present invention is characterized in that a single camera is used to set the plane of movement of the sphere as a reference plane, and the position coordinates of the point on the reference plane projected in the vertical direction by the center point of the sphere, that is, the position coordinates on the plane of the sphere, are calculated to calculate the movement information of the sphere on the reference plane.

[0039] The sensing processing unit identifies a "reference plane" relative to the motion plane of the sphere from the image captured by the single camera, sets a positional relationship with the single camera, and uses the positional relationship to calculate the planar position coordinates of the sphere on the "reference plane" from the object corresponding to the sphere in the image captured by the single camera.

[0040] For example, Figure 1 As shown, the sensing processing unit 220 can be configured to identify the pre-set and displayed feature portions 111, 112, 113, and 114 on the putting mat 100 from the image captured by the single camera 200, to set a reference plane as the movement plane of the golf ball 10 and position information of the single camera relative to the reference plane, i.e., three-dimensional position information, and detect an object corresponding to the golf ball in the captured image to calculate the position of the point where the center point of the golf ball 10 is projected in the vertical direction onto the reference plane, i.e., the coordinates of the plane position of the golf ball.

[0041] like Figure 1 As shown, the single camera 200 identifies the feature parts 111 to 114 on the acquired image, analyzes the feature parts on the image, and calculates the x, y, and z coordinate information of the reference plane and the single camera based on the origin of the xyz coordinate system, thereby setting the positional relationship between the reference plane and the single camera.

[0042] Figure 2 Shown based on Figure 1 The position relationship between the single camera and the putter pad is shown in the image captured by the single camera. Figure 2 (a) shows the state of the golf ball before being hit, Figure 2 (b) shows the state in which the golf ball moves after being hit.

[0043] Figure 2 (a) and (b) show the contents on the image, i100 represents the image of the putting mat, i111 to i114 represent the images of the characteristic parts, i10 represents the image of the golf ball in a stationary state, and ir10 represents the image of the golf ball in a moving state.

[0044] like Figure 1 As shown, when an image of a motion plane of a ball such as a putting mat 100 is acquired at a shooting angle obliquely viewed downward from the position of the single camera 200 on the upper side, as shown in FIG. Figure 2 As shown in (a) and (b), depending on the distance from the single camera, the closer the part is to the single camera of the putting mat, the larger it appears; the farther the part is from the single camera of the putting mat, the smaller it appears, and the size of the sphere on the image is displayed differently depending on the distance between the sphere and the single camera.

[0045] The image captured by a single camera is a two-dimensional image. In order to analyze the content of the image, such as Figure 2 As shown in (a) and (b), it is difficult to accurately grasp the coordinates of the position of the sphere on the plane due to the difference in size corresponding to the distance.

[0046] However, as described above, the present invention can accurately calculate the plane position coordinates of the sphere by using the “feature portion” to set the positional relationship between the reference plane and the single camera and using it to analyze the image.

[0047] To this end, the sensing device of one embodiment of the present invention can be Figure 2 The image recognition shown in Figure 1 Pre-prepared features on the plane shown.

[0048] The above-mentioned “feature part” can be as follows Figure 1 The form of the pre-prepared marks 111 to 114 shown in FIG. 1 may also be the form of the motion plane itself (for example, Figure 1 The shape of the putting mat itself is a quadrilateral shape as shown).

[0049] The sensing device of one embodiment of the present invention can pre-set information about the shape and size of the feature part, and analyze how the shape and size of the feature part recognized through the image change by referring to the pre-set information of the feature part, thereby obtaining information about the position of the reference plane and the single camera in the set coordinate system.

[0050] For example, Figure 1 As shown, when the feature parts 111, 112, 113, and 114 are respectively prepared in advance at the corner parts of the putting mat 100, and information about the shapes and sizes of the feature parts 111, 112, 113, and 114 is pre-set, as shown in FIG. Figure 2 As shown in (a) and (b), the sensing processing unit 220 can compare and analyze the changes in shape and size of the parts i111, i112, i113, and i114 corresponding to the characteristic parts through the image obtained by the single camera 200 with the pre-set information, and thereby calculate the position coordinates of the single camera based on the reference plane in the xyz coordinate system and store it as setting information.

[0051] In addition, for example, Figure 1 The quadrilateral shape of the putting mat 100 shown is inherently predetermined as a feature and can be modified by Figure 2 In the image shown, the changes in the shape and size of the portion i100 corresponding to the putting pad are analyzed to calculate the positional relationship between the reference plane and the single camera, and the calculated value is stored as setting information.

[0052] The characteristic part can be in the same form as the artificially assigned mark. If the form of a specific motion plane can be used, the form of the motion plane itself can be set as the characteristic part. Even in the case of the form of an artificially assigned mark, as long as the changes in its shape and size can be easily analyzed from the overall image, a mark of any shape can be used as the characteristic part as described above.

[0053] The sensing processing unit can use the image recognition of a single camera to display a predefined feature on the moving plane, and calculate the positional relationship between the single camera and the moving plane. After setting the reference plane as described above, the position of the sphere on the reference plane is calculated. The specific method for calculating the position of the sphere on the reference plane will be described later.

[0054] On the other hand, refer to Figure 3 The flowchart of FIG. 1 illustrates a motion sensing method for a planar moving sphere using a single camera according to an embodiment of the present invention.

[0055] Figure 3 The flowchart shown shows the Figure 1 and Figure 2 A method for sensing the motion of a sphere using a configuration of a sensing device has been described.

[0056] First, a single camera captures an image of the motion plane of a sphere within its shooting angle at an arbitrary position ( S110 ).

[0057] Preferably, the single camera is configured to be able to capture images using ambient lighting even in the absence of dedicated lighting by adjusting image parameters such as brightness, contrast, and gamma.

[0058] The sensing processing unit identifies a predetermined feature portion on the acquired image ( S120 ), and thereby sets a positional relationship between a reference plane for the motion plane of the sphere and the single camera ( S130 ).

[0059] The sensing processing unit recognizes the sphere from the image acquired by the single camera ( S140 ), and determines whether the movement of the sphere has started ( S150 ).

[0060] If the movement of the sphere has started, the sensing processing unit analyzes the image captured by the single camera to detect an object on the image corresponding to the actual sphere ( S160 ).

[0061] Then, the sensing processing unit may detect feature points on the detected outline of the object ( S170 ).

[0062] The sensing processing unit uses the object detected on the image obtained from the single camera to detect the point (feature point) on the contour of the object, and uses the feature point to calculate the position coordinates of the point where the center point of the sphere is projected in the vertical direction onto the reference plane as the plane position coordinates of the sphere (S180).

[0063] When the planar position coordinates of the sphere calculated as described above are connected to each frame in the multiple frames acquired by a single camera, the motion trajectory of the sphere can be calculated, and the motion information of the sphere can be calculated by the change of the planar position coordinates of the sphere corresponding to time, such as the speed of the sphere (S190).

[0064] Refer to the following Figures 4 to 6 The specific contents of calculating the plane position coordinates of the sphere using the feature points of the object on the image as described above, that is, the specific contents of the process from detecting the object on the image to calculating the coordinate information projected onto the reference surface through the above steps S160 to S180 are explained.

[0065] Figure 4 and Figure 5 1 is a diagram for explaining how to detect an object corresponding to a sphere in an image captured by a single camera of a sensing device according to an embodiment of the present invention and calculate feature points for calculating the planar position coordinates of the sphere. Figure 6 and Figure 7 This diagram is a diagram showing calculation of the position coordinates of a point where the center point of a sphere is projected vertically onto a reference plane based on the upper and lower endpoints of a feature point of an object detected by a sensing device according to an embodiment of the present invention.

[0066] First, refer to Figure 4 An example of a method for detecting an object from an image captured by a single camera and detecting its contour will be described.

[0067] Figure 4 (a) shows an enlarged view of Figure 1 A portion of an image acquired by a single camera in the configuration shown.

[0068] The sensor processing unit can be Figure 4 In the image shown in (a), a pre-set feature part (i114, etc.) set on the motion plane is identified, and the positional relationship between the reference plane and the single camera is calculated using it, and the calculated information is stored for setting.

[0069] The pixels corresponding to the portion of the sphere on the image acquired by the single camera appear to have brightness values ​​that are significantly different from those of the surrounding pixels.

[0070] However, as for the group of pixels distinguished from the surroundings, not only the sphere, but also the part corresponding to the putter in the case of a putter may be displayed as a group of pixels having a brightness value different from the surroundings, and the same applies to the user's feet.

[0071] Thus, when there are multiple groups of pixels having similar characteristics, ie, objects, on an image, it is necessary to detect which object corresponds to the sphere.

[0072] According to the characteristics of the sphere, the brightness, roundness, aspect ratio and other information of the pixels on the image can be pre-set, and based on the pre-set conditions for multiple objects appearing on the image, it can be detected which object corresponds to the sphere. Figure 4 As shown, the object corresponding to the sphere will be represented by OB.

[0073] However, if Figure 4 As shown in the image (a), the outline of the object OB is not clear and is not in the shape of a complete sphere. Therefore, in order to calculate the plane position coordinates of the sphere, it is necessary to detect the accurate outline of the object OB.

[0074] Figure 4 (b) shows an enlarged view of the object OB. Since the motion plane is located on the bottom side, most of the upper boundary of the object in the image taken by a single camera under lighting (whether natural lighting, indoor lighting, or special lighting) is displayed quite accurately.

[0075] In particular, in the case of a sphere, Figure 4 As shown in (a) and (b), since the upper end portion of the object OB appears quite accurately on the image, the curved portion of the upper end boundary Tb of the object can be easily identified.

[0076] However, since the motion plane is located on the bottom side, a shadow will appear at the bottom end of the sphere under either natural or artificial lighting. Figure 4 As shown in the image (b), it can be seen that the lower part S of the object OB is affected by the shadow, and the pixel brightness of the object becomes darker, making the boundary of the lower end of the object OB quite blurred, and the bright pixel part displayed in the image is different from the outline of the actual sphere.

[0077] In order to accurately determine the lower boundary of the object OB, the curve of the upper boundary Tb of the object OB can be used. Figure 4 (c) shows the result of detecting the contour of the object by using the curve of the upper boundary Tb of the object OB to find a circle fc having the curvature of the curve of the upper boundary.

[0078] First, if Figure 4 As shown in (b), the brightness values ​​of the pixels of the object OB can be used to calculate the curve of the upper boundary Tb of the object OB in the image. For example, the curve of the upper boundary Tb can be detected by pre-setting a threshold value of the pixel brightness value of the object and specifically corresponding to the pixels of the boundary of the threshold.

[0079] After the curve of the upper boundary Tb of the object is detected as described above, Figure 4 As shown in (c), the outline of the object OB can be specified by calculating a circle fc having the curvature of the curve of the upper boundary Tb.

[0080] like Figure 4 As shown in (c), the contour of the object OB can be detected using a circle fc having the curvature of the curve of the upper boundary Tb, thereby clearly identifying the boundary of the lower end portion of the object where pixels disappear due to dimming of the lighting and the boundary becomes unclear.

[0081] As described above, the upper end point TP of the upper boundary Tb of the object can be obtained from the contour of the object, and the lower end point BP of the object symmetrical to the upper end point TP can be detected.

[0082] As mentioned above, the plane position coordinates of the sphere can be calculated using the feature points on the outline of the object, and the upper end point TP and the lower end point BP can function as the feature points.

[0083] As described above, after detecting the object OB corresponding to the sphere in the image, the upper boundary Tb of the object OB can be calculated to form a circle using the curvature of the curve to determine the outline of the object, and thereby the upper endpoint TP and the lower endpoint BP as feature points can be detected.

[0084] However, since there are situations where the shape of the object OB detected from the image is not suitable for a specific circular contour, when the upper boundary of the object as described above does not present an ideal curve shape, the circle formed based on the curvature may appear too large.

[0085] Therefore, the present invention can use the method of detecting the feature points (upper and lower end points) on the object contour by the circle fitting method of forming a circle using the curvature of the curve of the upper end boundary of the object as described above in parallel with the method of detecting the feature points (upper and lower end points) on the object contour as described above. Figure 4 (d) shows a method for detecting feature points using detection graphics DR.

[0086] like Figure 4As shown in (d), the upper boundary Tb of the object OB can be detected, and the upper endpoint TP on its upper boundary Tb can be detected, and a predetermined type of graphic, such as a rectangular detection graphic DR, can be generated to match the size and shape of the object OB, so as to utilize the size d of the matched detection graphic DR to detect the lower endpoint BP at a position that matches the upper endpoint TP to a degree equivalent to the size d.

[0087] For the detection pattern DR, only the type of the pattern is predetermined, and its shape and size can be generated according to the rotation or deformation of the object OB, such as Figure 4 As shown in (d), it can be presented as a rotated rectangle.

[0088] Here, matching the detection pattern DR with the object OB means that the detection pattern DR is made to most appropriately include the object OB by changing or rotating the length of the side of the detection pattern DR or the like.

[0089] For example, when the detection pattern DR is generated to include the object OB, the case where the average value of the brightness value inside the detection pattern DR is the largest can be judged as the above-mentioned case of "the detection pattern DR matches the object OB".

[0090] As described above, the length of the side of one side of the matched detection figure DR may be different from the length of the side on the other side. The length of the side perpendicular to the moving direction of the sphere can be defined as the diameter d of the object, and the point at the position corresponding to the upper end point TP relative to the upper boundary of the object at a degree equivalent to the diameter d can be detected as the lower end point BP. Figure 4 (d) shows an example of detecting the upper end point TP and the lower end point BP as feature points of the object in the above-described manner.

[0091] When detecting objects in images acquired by a single camera, they are not only round, but also have Figure 5 The detected object OB in the image shown in (a) appears to be quite blurred and diffuse.

[0092] In this case, if Figure 5 As shown in FIG. 5( b ), when the upper boundary Tb of the object OB is detected, its curvature appears to be quite large, so it is impossible to detect the accurate outline of the object by performing circle fitting based on the curvature.

[0093] Therefore, in this case, using Figure 4 The method of detecting feature points of an object using the detection pattern DR described in (d) may produce more ideal results.

[0094] like Figure 5As shown in (b), the upper end boundary Tb of the object OB is detected, and a rectangular detection pattern DR is generated to match the size and shape of the object OB, for example.

[0095] In this case, the characteristic is that, for the detection pattern DR, only the type of the pattern is predetermined, while its size, rotational posture, etc. can be deformed according to the state of the object.

[0096] The detection pattern DR is a rectangular pattern with long and short sides. Since the diffusion direction of the object, that is, the size of the direction perpendicular to the moving direction, can be regarded as the diameter of the object outline, Figure 5 The size d of the short side of the detection pattern DR shown in (b) that matches the object OB is the diameter of the object.

[0097] Therefore, if Figure 5 As shown in (c), the upper end point TP can be detected at the upper boundary Tb of the object, and a point symmetrically located with respect to the upper end point TP by a size d can be detected as the lower end point BP.

[0098] Figure 4 The circle fitting method using the curvature of the upper boundary of the object shown in (b) and (c) is a method for detecting the outline of the object and obtaining the upper and lower endpoints as feature points. Figure 4 (d) and Figure 5 The methods shown in (b) and (c) are methods of using a detection pattern to determine the size of an object instead of determining the outline of the object, and using the size of the object to detect the positions of the upper and lower endpoints. The sensing device of one embodiment of the present invention can detect the feature points of an object by using the above two methods in parallel.

[0099] The upper endpoint TP and the lower endpoint BP of the object described above can be calculated and used to obtain the coordinates of the point where the center point of the sphere is projected onto the reference plane in the vertical direction, that is, the plane position coordinates of the sphere.

[0100] Figure 6 Shown in Figure 2 In the image observed by the single camera shown in (d), by identifying the parts i111~i114 corresponding to the characteristic parts, information about the positional relationship between the reference plane and the single camera can be set, and the object OB corresponding to the sphere moving on the plane can be detected to detect the upper end point TP and the lower end point BP as the characteristic points on the object.

[0101] The direction of the line passing through the upper endpoint TP and the lower endpoint BP detected as described above is the line of sight direction of the single camera, the lower endpoint BP is the position of the point close to the position of the single camera in the line of sight direction of the single camera, and the upper endpoint TP is the position of the point away from the position of the single camera.

[0102] Using the above-mentioned setting information about the positional relationship between the reference plane and the single camera, as well as the information on the positions of the upper and lower end points of the above-mentioned object, the "plane position coordinates of the sphere" can be calculated, that is, the position coordinates of the point where the center point of the sphere is projected onto the reference plane in the vertical direction.

[0103] Here, the above-mentioned "center point of the sphere" is the same as Figure 6 The center point of the object's outline on the image captured by the single camera shown is completely different. Even if the coordinates of the center point of the object's outline are obtained, the "plane position coordinates of the sphere" cannot be obtained from them.

[0104] When Figure 6 When the center point of the outline of the object in the image captured by the single camera is A, the point A is not the center point of the actual sphere, but is displayed to be greatly different from the center point of the actual sphere.

[0105] Figure 6 The upper endpoint TP and the lower endpoint BP of the object OB shown are not points on the actual sphere, but are position coordinates on the pad in the direction of sight observed from the position of a single camera, so Figure 6 The upper end point TP and the lower end point BP in φ are completely different from the upper end point and the lower end point of the sphere in real space.

[0106] Likewise, due to Figure 6 The center point A of the outline of the object OB shown is not the center point of the sphere on the actual plane, so it cannot be used to find the plane position coordinates of the sphere. The plane position coordinates of the sphere can be found by geometric calculation using the upper end point TP and the lower end point BP of the object. For this, refer to Figure 7 Provide a description.

[0107] Figure 7 Shown based on Figure 6 The upper and lower endpoints of the detected object as feature points are the position coordinates of the points on the reference plane projected in the vertical direction from the center point of the sphere. Figure 7 (a) shows a cross section taken along the line of sight of a single camera connecting the upper endpoint TP and the lower endpoint BP of the object in the xy coordinate plane of the real space, Figure 7 (b) shows the xy coordinate plane of the real space.

[0108] As described above, an image may be acquired by a single camera, and a portion corresponding to a feature (preset) on the acquired image may be identified to store setting information regarding the positional relationship between the reference plane and the single camera.

[0109] That is, the sensing processing unit of the sensing device of one embodiment of the present invention can identify the feature part from the image obtained by the single camera, and Figure 7 As shown in (a), a reference plane sPL relative to the motion plane of the sphere is identified, and a geometric relationship between the reference plane sPL and the position P200 of the single camera is set.

[0110] According to the above-mentioned setting of the positional relationship between the reference plane and the single camera, Figure 7 The height H of the single camera position P200 relative to the reference plane sPL is set as shown in (a), and the x and y coordinate information of the single camera position P200 can also be set as shown in (b), and the height and position information set for the single camera position P200 can also be used as shown in (b). Figure 7 As shown in (a), the angle information of the sight line direction from the single camera position P200 to the reference plane sPL is calculated.

[0111] exist Figure 6 In the figure, the upper endpoint TP and the lower endpoint BP of the object OB are as follows: Figure 7 (a) and (b) show the projection of the upper and lower ends of the sphere CB onto the points TP and BP on the reference plane sPL in the actual coordinate system in the line of sight of the single camera.

[0112] The plane position coordinates of the sphere required by the sensing device of the present invention are as follows: Figure 7 (a) shows the x and y coordinates of the point Pc on the reference plane sPL where the center point C of the sphere CB is projected in the vertical direction.

[0113] However, since the position of the center point C of the sphere CB is unknown, the coordinates of the point Pc can be calculated using the geometric relationship between the positions of the upper end point TP and the lower end point BP on the reference plane sPL as described above.

[0114] like Figure 7 As shown in (a), the lower end point BP on the reference plane sPL is different from the point Pc where the center point C of the sphere is projected onto the reference plane sPL in the vertical direction. This difference is referred to as an error E1.

[0115] Likewise, if Figure 7 As shown in (a), the upper end point TP on the reference plane sPL is different from the point Pc where the center point C of the sphere is projected onto the reference plane sPL in the vertical direction. This difference is referred to as an error E2.

[0116] In addition, if Figure 7As shown in (a), the distance L1 from the origin O to the lower end point BP and the distance L2 from the origin O to the upper end point TP can be calculated using the setting information of the positional relationship between the reference plane sPL and the single camera position P200. The angle a formed by the reference plane sPL and the line of sight direction of the single camera at the lower end point BP can be calculated from the height H of the single camera position P200 and the distance L1, and the angle b formed by the reference plane sPL and the line of sight direction of the single camera at the upper end point TP can be calculated from the height H of the single camera position P200 and the distance L2.

[0117] like Figure 7 As shown in (a), since the radius r of the actual sphere is a known value that is pre-set in the sensing device, the position coordinates of the point Pc projected onto the reference plane sPL in the vertical direction relative to the center C of the sphere can be calculated by geometric calculation using the position and angle of the upper end point TP, the position and angle of the lower end point BP on the reference plane sPL, and the circle based on the radius r of the actual sphere.

[0118] The point Pc where the center C of the sphere is projected onto the reference plane SPL in the vertical direction can be expressed as follows: Figure 7 As shown in (a), E1 is calculated as the error from the lower end point BP by geometric calculation, and E2 is calculated as the error from the upper end point TP by geometric calculation. Figure 7 As shown in (a), E1 can be calculated by trigonometric function using the angle information a and the radius r of the sphere for the triangle formed by the center point C, the lower end point BP, and Pc of the sphere.

[0119] In addition, if Figure 7 As shown in (a), E2 can be calculated by trigonometric function using the b angle information and the radius r of the sphere for the triangle formed by the point VP, the upper endpoint TP, and Pc that intersects the line connecting the single camera position P200 and the upper endpoint TP perpendicularly from the center point C of the sphere.

[0120] like Figure 7 As shown in (b), since the coordinates of the BP point and the TP point on the x and y coordinate plane of the reference plane sPL are known, the x and y coordinate information of Pc, which is the position of the correction E1 or E2, can be calculated.

[0121] Therefore, as described above, when the upper end point TP and the lower end point BP as the feature points of the object are detected, it is possible to utilize them by Figure 7 The geometric operation shown in (a) calculates the error E1 from the lower endpoint and the error E2 from the upper endpoint, and the errors can be used to determine the coordinates of the center point of the sphere projected onto the reference plane in the vertical direction, that is, the coordinates of the point Pc on the xy plane (reference plane) which serves as the plane position coordinates of the sphere.

[0122] When the planar position coordinates of the sphere in all frames of the image to be analyzed in the image acquired by a single camera are respectively obtained by this method, information on the motion caused by the planar movement of the sphere can be calculated.

[0123] On the other hand, refer to Figure 8 The flowchart of FIG. 1 illustrates a method for sensing motion of a golf ball moving on a putting mat according to an embodiment of the present invention.

[0124] Figure 3 The flowchart shown in FIG shows a process of obtaining the planar position coordinates of a sphere using images obtained by a single camera in relation to the planar motion of the sphere. Figure 8 The flowchart shown relates to a method of sensing the motion of a golf ball generated when the spherical object is a golf ball and the golf ball moves on a putting mat as a motion plane.

[0125] Therefore, the above Figures 4 to 7 The detection of objects on the image and the detection of the outline of the object, as well as the calculation method of the coordinates of the point where the center point of the sphere is projected onto the reference surface in the vertical direction can also be applied to the object to be Figure 8 The method described in Calculate the planar position coordinates of a golf ball on a putting mat.

[0126] Figure 8 The flowchart shown shows the ratio corresponding to Figure 3 The sensing method of the flowchart shown is more specific and specialized to the process of golf putting.

[0127] like Figure 8 As shown, first, an image is acquired by a single camera at an arbitrary position (S210), and a sensing processing unit recognizes a feature on the putter pad from the acquired image to set a positional relationship between a reference plane and the single camera (S220).

[0128] Figure 9 (a) shows a binary image of an image acquired by a single camera. By identifying the characteristic portions i111, i112, i113, and i114 corresponding to the putting pad, the position of the reference plane and the single camera can be identified and set.

[0129] In the process of setting the position of the reference plane and the single camera as described above, image parameters such as brightness, contrast, and gamma can be automatically adjusted to effectively detect objects based on ambient lighting.

[0130] On the other hand, Figure 8 As shown, after setting the reference plane, the sensing processing unit sets a region of interest for recognizing a golf ball on the image of the single camera ( S232 ).

[0131] exist Figure 9 In (a), a case where a region of interest ROI of a predetermined size and shape is set so as to include an object OB corresponding to a golf ball is shown. Figure 9 As shown in (a), when it is determined that the object OB, ie, the golf ball, exists in the region of interest ROI (S234), it is determined that the shot preparation is completed and the "ball ready" state is reached (S236).

[0132] When the region of interest ROI for determining that the ball is ready is set too narrow, the position where the golf ball is initially placed is limited. Therefore, it is preferable to set the region of interest ROI as Figure 9 The area shown in (a) is set to be as wide as possible so that the user can place the golf ball at any position on the putting mat to putt.

[0133] When the ball is in the ready state, the sensing processing unit sets a trigger sensing area TR for sensing whether the golf ball has been hit (S236). Figure 9 As shown in (b), the trigger sensing area TR can be set to an area within the area of ​​the object OB.

[0134] like Figure 9 As shown in (b) of FIG. 2 , a trigger sensing region TR is set within the object OB, and it is determined whether the brightness change within the trigger sensing region TR exceeds a preset reference value ( S242 ).

[0135] like Figure 9 As shown in (b), in the process of setting the trigger sensing area TR inside the object OB and sensing the brightness inside the trigger sensing area TR, if the golf ball moves slightly, the brightness inside the trigger sensing area TR does not change much, so there is no transition to the next step for determining the start. However, if the golf ball is hit and moves, the brightness inside the trigger sensing area TR changes greatly.

[0136] In this manner, when the brightness inside the trigger sensing region TR changes significantly and exceeds a preset reference value, it is determined that the golf ball has moved, and the first trigger signal can be generated.

[0137] For example, if the brightness value inside the trigger sensing area TR is 100 when the trigger sensing area TR is generated, and the reference value for generating a trigger signal is 30, when the golf ball moves, the trigger sensing area TR is generated. Figure 9 In the state shown in (b), when an object moves and the brightness inside the trigger sensing area TR drops from 100 to 60, a change of 40 occurs, which exceeds the reference value of 30, and thus a first trigger signal is generated.

[0138] On the other hand, as described above, when the first trigger signal is generated, the sensor processing unit detects the object in the multiple frames of image to calculate the movement amount of the object, and determines whether the calculated movement amount of the object exceeds a preset reference value ( S244 ).

[0139] In reality, when a golf ball moves from its initial position on the putting mat, in addition to the user hitting the golf ball, there are also cases where the user accidentally touches the golf ball with the putter or foot. Therefore, a reference value for the amount of movement of the golf ball is pre-set, and the golf ball is only determined to be hit when it moves beyond a certain amount.

[0140] After the first trigger signal is generated, the object is detected to calculate the degree of movement. When it is determined that the degree of movement of the object exceeds a preset reference value, the sensing processing unit generates a ball trigger (S246).

[0141] That is, after the ball is ready (S236), the sensing processing unit senses the brightness change inside the trigger sensing area to perform the first trigger, and then senses the movement of the object again to generate a double trigger to sense whether the golf ball is hit.

[0142] When the ball trigger is generated as described above, the sensing processing unit calls the images captured by the single camera before and after the ball trigger time point and analyzes them, and detects an object corresponding to the golf ball for each frame of the analysis target image (S250). Figure 4 The same method as described in the method for detecting objects corresponding to spheres.

[0143] After detecting the objects as described above, the feature points on the contour of each detected object are detected, as described above by Figure 4 and Figure 5 As described above, the object's contour can be determined by circle fitting using the curvature of the curve of the object's upper boundary, thereby detecting the upper and lower endpoints, or the object's size information can be calculated using a detection pattern and used to detect the upper and lower endpoints. After determining the feature points on the object's contour as described above, the coordinates of the point where the center point of the golf ball is projected in the vertical direction onto the reference plane are calculated as the plane coordinates of the golf ball (S260).

[0144] The calculation method of the plane position coordinates of the golf ball can be as described above. Figure 7 The calculation is described by finding the error E1 or E2 from the upper and lower end points of the object.

[0145] When the planar position coordinates of the golf ball calculated as described above are obtained for each frame of the image to be analyzed, changes in the position coordinates can be calculated, and information on the movement of the golf ball can be calculated from the changes in the planar position coordinates of the golf ball ( S270 ).

[0146] After calculating the information about the movement of the golf ball, the sensing processing unit transmits the above movement information to the client 300 (refer to Figure 1 ), such as a simulator of a putting simulation device, a computer of a golf information providing device, etc. to provide to users.

[0147] On the other hand, after calculating the information about the movement of the golf ball as described above, the sensing processing unit may recheck the characteristic portion to check whether the reference plane has deviated from the initially set state (S280). If the reference plane has deviated, the process returns to step S220 to reset the reference plane. If the reference plane has not changed, the movement of the golf ball is sensed based on the previously set reference plane.

[0148] As described above, the motion sensing device and method for a plane-moving sphere using a camera, and the motion sensing device and method for a golf ball moving on a putting mat of the present invention have the following advantages, namely, a plane-moving sports sphere is used as a target object, an inexpensive single camera is used, and not only can the sports sphere be detected even without separate dedicated lighting, but spatial recognition can also be performed, thereby getting rid of the constraint that the camera setting position must be fixed, and can be used with a simple structure without being restricted by location, and can calculate accurate position and motion information about the sphere moving on the plane.

[0149] Industrial applicability

[0150] The motion sensing device and method for a plane-moving sphere using a camera, and the motion sensing device and method for a golf ball moving on a putting mat of the present invention can be used in the technical field based on sensing of a plane-moving sports sphere, the field of golf analysis based on analysis of the movement of a golf ball during a golf swing, the field of virtual golf simulation systems, etc.

Claims

1. A motion sensing device for a sphere moving on a plane using a camera, which is a sensing device for a sphere moving on a plane. The motion sensing device for a sphere moving on a plane using a camera is characterized by comprising: A camera, which acquires images at a single perspective at an arbitrary position and at a shooting angle including the motion plane of the sphere; as well as a sensing processing unit that sets a positional relationship with the camera from an image captured by the camera using the motion plane of the sphere as a reference plane, detects an object corresponding to the sphere on the captured image, and calculates plane position coordinates of the sphere on the reference plane using the positional relationship with the camera and information about the object, thereby calculating information about the motion of the sphere from changes in the calculated plane position coordinates; The sensing processing unit is configured to detect the upper boundary of the object detected from the image, detect the upper endpoint on the upper boundary, generate a detection pattern that matches the size and shape of the object, and use the size of the generated detection pattern to detect the lower endpoint of the object corresponding to the upper endpoint, so as to use the upper endpoint or the lower endpoint to calculate the position coordinates of the point where the center point of the sphere is projected in the vertical direction onto the reference plane as the plane position coordinates of the sphere.

2. The motion sensing device for a planar moving sphere using a camera according to claim 1, characterized in that: The camera is configured to acquire an image corresponding to the ambient lighting by adjusting image parameters when the positional relationship between the reference surface and the camera is set without dedicated lighting.

3. The motion sensing device for a planar moving sphere using a camera according to claim 1, characterized in that: The sensing processing unit is configured to recognize, through the image, a feature portion that is preset and displayed on the motion plane of the sphere, and set a positional relationship between the reference plane and the camera based on the recognized information.

4. A motion sensing device for a sphere moving on a plane using a camera, which is a sensing device for a sphere moving on a plane, characterized by comprising: A camera, which acquires images at a single perspective at an arbitrary position and at a shooting angle including the motion plane of the sphere; as well as a sensing processing unit that sets a positional relationship with the camera from an image captured by the camera using the motion plane of the sphere as a reference plane, detects an object corresponding to the sphere on the captured image, and calculates plane position coordinates of the sphere on the reference plane using the positional relationship with the camera and information about the object, thereby calculating information about the motion of the sphere from changes in the calculated plane position coordinates; The sensing processing unit is configured to detect a curve of an upper boundary of an object detected from the image, calculate a circle having a curvature of the curve of the upper boundary to detect at least one point on the circle as a feature point, and use the feature point to calculate the position coordinates of a point where the center point of the sphere is projected in the vertical direction onto the reference plane as the plane position coordinates of the sphere.

5. The motion sensing device for a planar moving sphere using a camera according to claim 1 or 4, characterized in that: The sensing processing unit is configured to detect the upper endpoint and the lower endpoint of the contour of the object, and calculate the plane position coordinates of the sphere by correcting the error of projecting the upper endpoint or the lower endpoint onto the point of the reference plane in the line of sight of the camera using setting information about the positional relationship between the reference plane and the camera.

6. A sensing device for a golf ball moving on a putting mat, characterized in that: include: a camera that acquires images at a single viewing angle at any position and including the putting pad; as well as a sensing processing unit that identifies a pre-set and displayed feature portion on the putting mat from an image captured by the camera, sets a positional relationship with the camera using the putting mat as a reference plane, detects feature points on a contour of an object corresponding to the golf ball in the captured image, calculates position coordinates of a point where the center point of the golf ball is projected in a vertical direction onto the reference plane using the feature points as plane position coordinates of the golf ball, and calculates information about the movement of the golf ball on the putting mat from changes in the calculated plane position coordinates. The sensing processing unit is configured to detect an upper boundary of the object detected from the image, detect an upper endpoint on the upper boundary, generate a detection pattern matching the size and shape of the object, and detect a lower endpoint of the object corresponding to the upper endpoint using the size of the generated detection pattern, so as to calculate the position coordinates of a point where the center point of the golf ball is projected in the vertical direction onto the reference plane using the upper endpoint or the lower endpoint as the planar position coordinates of the golf ball.

7. A method for sensing the motion of a planar moving sphere using a camera, characterized in that: include: The step of acquiring an image by the camera at an arbitrary position and a single perspective at a shooting angle including the motion plane of the sphere; The step of setting a positional relationship with the camera from an image acquired by the camera using the motion plane of the sphere as a reference plane; The steps of detecting an upper boundary of the object corresponding to the sphere on the acquired image, detecting an upper endpoint on the upper boundary, generating a detection pattern matching the size and shape of the object, and detecting a lower endpoint of the object corresponding to the upper endpoint using the size of the generated detection pattern; as well as The step of calculating the position coordinates of a point where the center point of the sphere is projected vertically onto the reference plane from the upper end point or the lower end point using the positional relationship with the camera as the plane position coordinates of the sphere on the reference plane.

8. The method for sensing the motion of a planar moving sphere using a camera according to claim 7, wherein: The step of setting the position relationship with the camera includes: a step of identifying the motion plane of the sphere as a reference plane from the image acquired in the step of acquiring the image by using a feature portion pre-set and displayed on the motion plane of the sphere; and The step of setting the position information of the camera based on the identification of the reference surface.

9. The method for sensing the motion of a planar moving sphere using a camera according to claim 7, wherein: The steps of calculating the plane position coordinates of the sphere include: a step of detecting the outline of the object based on the upper boundary of the object detected on the image; A step of detecting point and angle information of projecting the upper endpoint or the lower endpoint of the outline of the object onto the reference surface in the direction of sight of the camera; The step of calculating the plane position coordinates of the sphere by correcting the error of projecting the upper end point or the lower end point onto the point on the reference plane in the line of sight of the camera using the setting information about the positional relationship between the reference plane and the camera, the size information of the sphere and the detected angle information.

10. A method for sensing a golf ball moving on a putting mat, characterized in that: include: The step of acquiring an image by a camera at a single viewing angle at an arbitrary position and at a shooting angle including the putting pad; The step of setting a positional relationship between the putting pad and the camera based on the image acquired by the camera and taking the putting pad as a reference plane; detecting an object corresponding to the golf ball from the image captured by the camera to prepare for hitting the golf ball; a step of sensing a golf ball hit from an image captured by the camera; When the impact is sensed, the step of calculating, using the set positional relationship between the reference plane and the camera, position coordinates of a point projected in the vertical direction from a point on the outline of the object to the center point of the golf ball onto the reference plane as the plane position coordinates of the golf ball; as well as a step of calculating the plane position coordinates of the golf ball for each frame of the image captured by the camera, so as to calculate information about the movement of the golf ball from the change; The step of calculating the plane position coordinates of the golf ball includes: a step of detecting an upper boundary of the object detected from the image; a step of detecting an upper endpoint on said upper boundary; generating a detection pattern that matches the size and shape of the object, and detecting the lower endpoint of the object corresponding to the upper endpoint using the size of the generated detection pattern; and The step of calculating the position coordinates of a point where the center point of the golf ball is projected onto the reference plane in the vertical direction using the upper end point or the lower end point as the planar position coordinates of the golf ball.

11. A method for sensing a golf ball moving on a putting mat, characterized in that: include: The step of acquiring an image by a camera at a single viewing angle at an arbitrary position and at a shooting angle including the putting pad; The step of setting a positional relationship between the putting pad and the camera based on the image acquired by the camera and taking the putting pad as a reference plane; detecting an object corresponding to the golf ball from the image captured by the camera to prepare for hitting the golf ball; a step of sensing a golf ball hit from an image captured by the camera; When the impact is sensed, the step of calculating, using the set positional relationship between the reference plane and the camera, position coordinates of a point projected in the vertical direction from a point on the outline of the object to the center point of the golf ball onto the reference plane as the plane position coordinates of the golf ball; as well as a step of calculating the plane position coordinates of the golf ball for each frame of the image captured by the camera, so as to calculate information about the movement of the golf ball from the change; The step of calculating the plane position coordinates of the golf ball includes: a step of detecting a curve of an upper boundary of the object detected from the image; a step of calculating a circle having a curvature of the curve of the upper end boundary to detect at least one point on the circle as a feature point; and The step of calculating the position coordinates of a point where the center point of the golf ball is projected in the vertical direction onto the reference plane as the planar position coordinates of the golf ball using the feature point.

Images