Display control device, method, and program

Abstract

To provide a display control device, method, and program that can easily recognize the inclination of a face surface with respect to a horizontal plane.SOLUTION: A display control device includes a captured image acquisition unit 21 that acquires a captured image of a club head 41a of a golf club 41, a contour extraction unit 22 that extracts the contour of the club head 41a on the basis of the captured image, a line segment detection unit 23 that detects a line segment representing a predetermined part of the club head 41a on the basis of the captured image, and a control unit 24 that displays the contour of the club head 41a extracted by the contour extraction unit 22 together with the line segment detected by the line segment detection unit 23.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a display control device, method, and program for displaying information about a glove head based on an image of a club head captured during a golf club swing. [Background technology]

[0002] Conventionally, various devices have been proposed that photograph a golfer's swing and provide the photographed information. By providing the golfer with photographed information of the swing, it is possible to encourage the golfer to improve his / her golf game or to select a golf club that is suitable for the golfer.

[0003] As an example of the above-mentioned device, Patent Document 1 proposes a device that photographs a golf club and a golf ball from above when the golf club is swung, detects the face surface of the club head contained in each photographed image, and displays line segments representing the detected multiple face surfaces. By displaying the line segments representing the face surfaces in this way, it is possible to recognize the opening of the face surfaces, etc. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2021-103483 A Summary of the Invention [Problem to be solved by the invention]

[0005] Here, an important element in a golfer's swing is the inclination of the face of the club head during the swing. The inclination of the face here refers to the inclination of the face of the club with respect to the horizontal plane when the face is viewed from the side. The inclination of the face affects the flight distance of a golf ball, the height to which the golf ball rises, and the amount of spin of the golf ball, and by recognizing the inclination of the face of the club head during the swing, the characteristics of the golfer's swing can be understood, and a golf club having a loft angle and lie angle that match the characteristics can be selected.

[0006] In Patent Document 1, the face of the club head when viewed from above is simply displayed as a line segment, so although it is possible to recognize the opening of the face as described above, it is not possible to recognize the inclination of the face.

[0007] In view of the above problems, an object of the present invention is to provide a display control device, method, and program that can easily recognize the inclination of a face surface relative to a horizontal plane. [Means for solving the problem]

[0008] The display control device of the present invention includes an image acquisition unit that acquires an image of a club head of a golf club, a contour extraction unit that extracts the contour of the club head based on the captured image, a line detection unit that detects lines representing specified parts of the club head based on the captured image, and a control unit that displays the contour of the club head extracted by the contour extraction unit together with the lines detected by the line detection unit. Effect of the Invention

[0009] According to the display control device of the present invention, the contour of the club head is extracted based on a photographed image of the club head, and line segments representing specific parts of the club head are detected and displayed together with the extracted contour of the club head and the detected line segments, thereby making it possible to represent the loft angle of the club head in three dimensions and making it easy to recognize the inclination of the face surface relative to the horizontal plane. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a golf impact analysis system using an embodiment of a display control device of the present invention; [Diagram 2] A diagram showing an example of a display in which the outline of a driver's club head and the boundary line between the face and crown of the club head are aligned and displayed. [Diagram 3] A diagram showing an example of a gradation pattern displayed within the outline of the driver club head shown in FIG. 2. [Figure 4] A diagram showing an example of display when the golf club is an iron [Diagram 5] 2 is a flowchart illustrating the process flow of the golf impact analysis system shown in FIG. 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] A golf impact analysis system using an embodiment of the display control device of the present invention will be described in detail below with reference to the drawings. The golf impact analysis system of this embodiment is characterized by a method of displaying information about the club head contained in the captured image, but first, the golf impact analysis system as a whole will be described. Figure 1 is a schematic diagram of the golf impact analysis system 1 of this embodiment.

[0012] 1, the golf impact analysis system 1 of this embodiment includes a photographing device 10, a control device 20, and a projection device 30. The photographing device 10 and the control device 20, and the control device 20 and the projection device 30 are connected to each other via wire or wirelessly so as to be able to communicate with each other, and are configured to be able to exchange various signals.

[0013] The photographing device 10 photographs the club head 41a of the golf club 41 from above when the golfer 40 swings the golf club 41. The photographing device 10 is installed above the golfer 40 who swings the golf club 41, and is installed directly above the range where the club head 41a passes near the floor surface so that the photographing device 10 can photograph the range. Specifically, the photographing device 10 is installed directly above a predetermined photographing range R including a position P of the golf ball 42 preset on the floor surface near the center. The photographing device 10 may be installed on a support member such as a stand, or may be installed on the ceiling. The photographing device 10 photographs the club head 41a from above.

[0014] The photographing device 10 includes an illumination unit 11 and a camera unit 12. The illumination unit 11 in this embodiment has an infrared light source and irradiates the infrared light emitted from the infrared light source onto the photographing range R. The camera unit 12 also has an imaging element such as a CMOS (Complementary metal-oxide-semiconductor) image sensor or a CCD (Charge-coupled device) image sensor, and an IR filter arranged in front of the imaging element. The IR filter is an optical filter that absorbs visible light and transmits infrared light.

[0015] Based on a control signal output from a control unit 24 (described later) of the control device 20, the photographing device 10 emits infrared light from the illumination unit 11 and photographs the above-mentioned photographing range R with the camera unit 12. Specifically, the photographing device 10 emits infrared light from the illumination unit 11 at a predetermined frame rate and photographs the club head 41a passing through the above-mentioned photographing range R. The photographed images photographed by the photographing device 10 at the predetermined frame rate are output to the control device 20.

[0016] The control device 20 is composed of a computer or the like, and includes hardware such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), semiconductor memories such as a ROM (Read Only Memory) and a RAM (Random access memory), storage such as a hard disk, and a communication I / F. In this embodiment, the control device 20 corresponds to the display control device of the present invention.

[0017] The control device 20 includes a photographed image acquisition unit 21, a contour extraction unit 22, a line segment detection unit 23, a control unit 24, a display unit 25, and an input unit .

[0018] A golf impact analysis program is installed in the semiconductor memory or hard disk of the control device 20. This program is executed by the CPU and GPU, causing the above-mentioned photographed image acquisition unit 21, contour extraction unit 22, line segment detection unit 23, and control unit 24 to function. Note that in this embodiment, all of the functions of the above-mentioned units are executed by the golf impact analysis program, but this is not limiting, and some or all of the functions may be configured from hardware such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or other electric circuits.

[0019] Each part of the control device 20 will be described in detail below.

[0020] The photographed image acquisition section 21 acquires a plurality of photographed images of the club head 41a photographed in time series by the photographing device 10 while the golf club 41 is being swung.

[0021] The contour extraction unit 22 extracts the contour of the club head 41a included in each captured image based on the captured image acquired by the captured image acquisition unit 21. The contour extraction unit 22 of this embodiment extracts the contour of the club head 41a by segmenting the captured image using an object recognition model based on machine learning.

[0022] Segmentation techniques that can be used include CNN (Convolutional Neural Network), FCN (Full Convolutional Network), SegNet, FPN (Feature Pyramid Network), R-CNN, and RNN (Recurrent Neural Network).

[0023] The method for extracting the contour of the club head 41a from each captured image is not limited to segmentation using an object recognition model as described above, and other known methods such as pattern recognition can be used.

[0024] The line segment detection unit 23 detects a line segment representing a predetermined portion of the club head 41a included in each photographed image based on the photographed images acquired by the photographed image acquisition unit 21. For example, when the golf club 41 is a driver, the line segment detection unit 23 detects a boundary line between the face surface and the crown portion of the driver as a line segment representing a predetermined portion of the club head 41a. Note that the term "face surface" is used throughout this specification, and this face surface means the surface of the club head 41a that impacts with a golf ball, but may also be expressed as the "face surface." In other words, the terms "face surface" and "face surface" used in this specification have the same meaning.

[0025] Furthermore, for example, if the golf club 41 is an iron, the line segment detection unit 23 detects the leading edge and top line of the iron as line segments that represent predetermined parts of the club head.

[0026] The line detection unit 23 has a trained model obtained by prior machine learning of the relationship between a plurality of arbitrary captured images of the club head 41a and the boundary line between the face surface and the crown portion of the club head 41a contained in each of the captured images.

[0027] Then, the line segment detection unit 23 detects the boundary line between the face surface and the crown portion of the club head 41a contained in the input photographed image by inputting the photographed image acquired by the photographed image acquisition unit 21 to the trained model. The trained model is generated using, for example, a CNN (convolutional neural network), but is not limited to this, and other known machine learning models may be used for machine learning.

[0028] The trained model is generated by machine learning an image that includes annotations for the boundary between the driver's face and the crown of the car, based on semantic segmentation and key point machine learning techniques that take heat maps into account.

[0029] In addition, the trained model is preferably obtained by machine learning the relationship between a plurality of captured images with different brightness and the above-mentioned boundary line included in each captured image. In other words, the plurality of captured images used in machine learning the trained model are preferably captured images with different brightness. This makes it possible to detect the boundary line between the face surface and the crown part with the same accuracy even if the lighting environment of the captured images input to the trained model is different and the brightness is different. In addition, when inputting the captured image of the detection target to the trained model, it is more preferable that the line segment detection unit 23 performs grayscale averaging (flattening) processing on the captured image.

[0030] In addition, in this embodiment, the line segment detection unit 23 has a trained model, but this is not limited to the above, and it is also possible to use a trained model that is pre-stored in an external server device other than the control device 20, for example.

[0031] In addition, when detecting the leading edge and top line of an iron, the line detection unit 23 can detect them using a method similar to the method for detecting the boundary line of a driver. That is, the line detection unit 23 detects the leading edge and top line of an iron included in a predetermined photographed image by using a trained model generated by machine learning a photographed image on which annotations are added to the leading edge and top line of an iron included in the photographed image.

[0032] Furthermore, the line segment representing the predetermined part of the club head 41a is not limited to the above example, and for example, the line segment detection unit 23 may detect a line segment connecting the toe and heel of the club head 41a. In this case, too, a trained model generated by machine learning a photographed image on which an annotation is added to the line segment connecting the toe and heel of the club head included in the photographed image is used to detect the line segment connecting the toe and heel of the club head included in the predetermined photographed image.

[0033] Next, the control unit 24 controls the entire golf impact analysis system 1. Specifically, the control unit 24 communicates with the photographing device 10 via the communication I / F, and outputs a control signal to the photographing device 10 to instruct the photographing start timing, the photographing frame rate, etc. The control unit 24 also communicates with the projection device 30 via the communication I / F, and causes the projection device 30 to display various information on the floor surface.

[0034] In particular, the control unit 24 of this embodiment combines the contour of the club head 41a contained in each captured image extracted by the contour extraction unit 22 with the line segments representing specific parts of the club head 41a detected by the line segment detection unit 23, and displays them on the floor surface using the projection device 30.

[0035] 2 is a diagram showing a display example in which the contour L of the driver's club head 41a and the boundary line B between the face surface and the crown of the club head 41a are displayed together. As shown in Fig. 2, the control unit 24 displays the contour L extracted from each of the photographed images photographed in time series, for example, side by side with the boundary line B detected from each photographed image.

[0036] In this way, by displaying the boundary line B between the face surface and the crown portion of the club head 41a in accordance with the contour L of the club head 41a, the loft angle of the club head can be expressed three-dimensionally, thereby making it easy to recognize the inclination of the face surface relative to the horizontal plane.

[0037] Furthermore, the control unit 24 may cause the contour L and the boundary line B to be displayed in sequence in the order in which the photographed images were taken.

[0038] The control unit 24 may also display on the floor images taken in chronological order during the swing of the golf club 41, and sequentially display the contour L and boundary line B on each of the sequentially displayed images. This allows the user to recognize the change in the inclination of the face surface during the swing of the golf club 41, and to check whether the swing is good or bad, and whether the golf club 41 is suitable.

[0039] Furthermore, when the control unit 24 displays the contour L and the boundary line B on the floor surface as described above, it can display within the contour L a pattern or design of a predetermined color different from that of the photographed image of the club head 41a. FIG. 3 is a diagram showing an example in which a gradation pattern is displayed within the contour L of the driver's club head 41a shown in FIG. 2. By displaying a gradation pattern within the contour L as shown in FIG. 3, the contour L can be displayed more three-dimensionally. Furthermore, the display within the contour L is not limited to the gradation display as shown in FIG. 3, but may be displayed three-dimensionally by displaying a point cloud or a mesh.

[0040] 3, the entire inside of the contour L is displayed in gradation, but the present invention is not limited to this. For example, a pattern or design of a predetermined color may be displayed only inside the contour L on the side opposite the face surface side with respect to the boundary line B. This allows the contour L to look more three-dimensional and the range of the face surface to be displayed more clearly.

[0041] FIG. 4 is a diagram showing a display example when the golf club 41 is an iron. When the golf club 41 is an iron, the contour L of the club head 41a, the leading edge LE, and the top line TL are displayed on the floor surface together as described above. Although FIG. 4 shows only the contour L, the leading edge LE, and the top line TL extracted and detected from one photographed image, the control unit 24 can display the contour L, the leading edge LE, and the top line TL extracted and detected from each photographed image photographed in time series, as in the case of a driver golf club. Also, the contour L, the leading edge LE, and the top line TL may be displayed sequentially in the order in which the photographed images were photographed.

[0042] In this way, by displaying the contour L, leading edge LE and top line TL of the iron club head 41a in alignment with the floor surface, the three-dimensional loft angle of the golf club 41 can be expressed and the change in the inclination of the face surface during the swing can also be recognized.

[0043] In this embodiment, when the golf club 41 is an iron, the leading edge LE and the top line TL are displayed, but only one of them may be displayed.

[0044] In addition, in this embodiment, when the golf club 41 is an iron, the leading edge LE and the top line TL are detected and displayed, but the trailing edge may be detected and displayed. For example, when the face surface is excessively inclined and the top line advances ahead of the leading edge, the leading edge cannot be seen from above, so the top line and the trailing edge may be displayed. In this case, only the top line may be displayed.

[0045] Furthermore, when the golf club 41 is an iron, the control unit 24 may display within the contour L a pattern or design of a predetermined color different from that in the photographed image of the club head 41a, in the same manner as in the case of a driver.

[0046] The control unit 24 can also cause the display unit 25 to display the outline L of the club head 41a projected onto the floor by the projection device 30, the lines representing specific parts of the club head 41a, and each captured image.

[0047] The display unit 25 includes a display device such as a liquid crystal display, and the input unit 26 includes an input device such as a mouse and a keyboard. The control device 20 may be configured as a tablet terminal, and the display unit 25 and the input unit 26 may be configured as a touch panel.

[0048] Next, returning to FIG. 1, the projection device 30 is composed of a projector, and as described above, under the control of the control unit 24 of the control device 20, projects and displays the contour L of the club head 41a and lines representing specific parts of the club head 41a (such as the boundary line B, leading edge LE and top line TL) onto the floor surface.

[0049] The projection device 30 is installed so as to be able to project and display the outline of the club head 41a and lines representing predetermined parts of the club head 41a (such as the boundary line B, leading edge LE, and top line TL) within a predetermined projection range that includes the placement position P of the golf ball 42 on the floor surface near the center. The projection device 30 may be installed on a support member such as a stand together with the image capture device 10, or may be installed on the ceiling.

[0050] Next, the flow of processing in the golf impact analysis system 1 will be described with reference to the flowchart shown in Fig. 5. Note that the case where the golf club 41 is a driver will be described here.

[0051] First, each device of the golf impact analysis system 1 is started (S10). Next, a golfer stands near a photographing range R and places a golf ball 42 at a golf ball placement position P within the photographing range R as shown in FIG.

[0052] Then, the user inputs an instruction to start shooting in the control device 20, and shooting by the shooting device 10 starts (S12).

[0053] After the image capturing by the image capturing device 10 is started, the golfer swings the golf club 41 and hits the golf ball 42 (S14).

[0054] While the golfer swings the golf club 41, the photographing device 10 photographs images within the photographing range R at a predetermined frame rate (S16), and the photographed images photographed in chronological order are sequentially output to the control device 20. The photographing device 10 ends photographing when a preset time has elapsed since the start of photographing (S18).

[0055] The captured image output from the photographing device 10 is input to the control device 20 and acquired by the captured image acquisition unit 21 (S20). The contour extraction unit 22 inputs each captured image acquired by the captured image acquisition unit 21 to the above-mentioned object recognition model, thereby extracting the contour L of the club head 41a (S22). In addition, the line segment detection unit 23 inputs each captured image acquired by the captured image acquisition unit 21 to the above-mentioned trained model, thereby detecting the boundary line B between the face surface and the crown portion of the club head 41a (S24).

[0056] Then, the control unit 24 causes the projection device 30 to project and display the contour L extracted from each captured image and the boundary line B detected from each captured image (S26).

[0057] In the above embodiment, an example is described in which the club head 41a is photographed from directly above. However, the photographing angle is not limited to this as long as at least the entire club head 41a and the above-mentioned specified parts of the club head 41a can be photographed.

[0058] Furthermore, the present invention is not limited to the above-described embodiment, and the components can be modified and embodied in the implementation stage without departing from the gist of the invention. Furthermore, various inventions can be formed by appropriately combining the multiple components disclosed in the above-described embodiment. For example, all the components shown in the embodiment may be appropriately combined. Of course, various modifications and applications are possible without departing from the spirit of the invention.

[0059] The present invention further discloses the following supplementary notes.

[0060] (Appendix 1) The display control device of the present invention includes an image acquisition unit that acquires an image of a club head of a golf club, a contour extraction unit that extracts the contour of the club head based on the captured image, a line detection unit that detects lines representing specified parts of the club head based on the captured image, and a control unit that displays the contour of the club head extracted by the contour extraction unit together with the lines detected by the line detection unit.

[0061] (Appendix 2) In the display control device described in Supplementary Note 1, when the golf club is a driver, the line detection unit can detect a boundary line between the face surface and the crown portion of the driver as a line segment representing a predetermined part of the club head.

[0062] (Appendix 3) In the display control device described in Supplementary Note 1 or Supplementary Note 2, when the golf club is an iron, the line detection unit can detect the leading edge and top line of the iron as line segments representing a predetermined part of the club head.

[0063] (Appendix 4) In the display control device according to any one of Supplementary Note 1 to Supplementary Note 3, the control unit can display within the outline a predetermined color or pattern different from the photographed image of the club head.

[0064] (Appendix 5) In the display control device described in Appendix 2, the control unit can display a pattern or design of a predetermined color different from the photographed image of the club head within the above outline only on the opposite side to the face side of the boundary line.

[0065] (Appendix 6) The display control method of the present invention obtains a photographed image of a club head of a golf club, extracts the contour of the club head based on the photographed image, detects lines representing specified parts of the club head based on the photographed image, and displays the extracted contour of the club head together with the detected lines.

[0066] (Appendix 7) The display control program of the present invention causes a computer to execute the steps of acquiring a photographed image of a club head of a golf club, extracting the contour of the club head based on the photographed image, detecting lines representing specified parts of the club head based on the photographed image, and displaying the extracted contour of the club head together with the detected lines. [Explanation of symbols]

[0067] 1. Golf Impact Analysis System 10. Imaging Device 11 Lighting Section 12 Camera section 20 Control device 21 Image acquisition unit 22 Contour Extraction Unit 23 Line detector 24 Control Unit 25 Display section 26 Input section 30 Projection device 40 Golfer 41 Golf Clubs 41a Club head 42 Golf balls B Border L Contour LE Leading Edge P Installation position R Shooting range TL Topline

Claims

1. an image acquisition unit for acquiring an image of a club head of a golf club; a contour extraction unit that extracts a contour of the club head based on the captured image; a line segment detection unit that detects a line segment representing a predetermined portion of the club head based on the captured image; a control unit that displays the contour of the club head extracted by the contour extraction unit together with the line segments detected by the line segment detection unit.

2. 2. The display control device according to claim 1, wherein, when the golf club is a driver, the line segment detection section detects a boundary line between a face surface and a crown portion of the driver as the line segment representing the predetermined portion of the club head.

3. 2. The display control device according to claim 1, wherein, when the golf club is an iron, the line segment detection section detects a leading edge and a top line of the iron as the line segments representing the predetermined portion of the club head.

4. The display control device according to claim 1 , wherein the control unit displays within the outline a predetermined color or pattern different from the photographed image of the club head.

5. 3. The display control device according to claim 2, wherein the control unit displays a pattern or design of a predetermined color different from that of the photographed image of the club head within the outline only on the side opposite the face side of the boundary line.

6. Acquire a photographed image of a club head of a golf club; Extracting a contour of the club head based on the captured image; Detecting a line segment representing a predetermined portion of the club head based on the captured image; A display control method for displaying the extracted club head contour together with the detected line segments.

7. acquiring an image of a club head of a golf club; extracting a contour of the club head based on the captured image; detecting a line segment representing a predetermined portion of the club head based on the captured image; and displaying the extracted club head contour together with the detected line segments.

Images