Information-processing method, program, and information processing apparatus

Abstract

To provide an information processing method, device, and program for appropriately specifying a golf ball before being hit by a swing motion from an image in which the swing motion of golf of a person of a subject is reflected.SOLUTION: In an information-processing system, an information processing apparatus 300 includes: a moving image data obtaining unit 3001 that obtains an image showing a golf swing motion of a person as a subject captured by a camera 100 and transmitted by a user terminal 200; a feature-point extracting unit 3002 that recognizes a golf club and a golf ball shown in the image, and a golf ball identifying unit 3003 that identifies whether the golf ball recognized by the feature-point extracting unit 3002 is a target to be launched by the swing motion, based on a positional relationship between the golf club and the golf ball recognized by the feature-point extracting unit 3002 in the image.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to an information processing method and the like. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for identifying the position of a golf ball that has been hit by a golf swing motion in an image showing a subject's golf swing motion (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7083333 Summary of the Invention [Problem to be solved by the invention]

[0004] By the way, there are cases where the golf swing motion of a person who is a subject is evaluated, and swing diagnosis and a golf club suitable for that person are proposed. In this case, for example, in order to evaluate the swing motion, it may be necessary to know the positional relationship between the golf ball and the golf club or a part of the person's body before it is hit by the swing motion.

[0005] In view of the above problem, an object of the present invention is to provide a technology that can appropriately identify a golf ball before it is hit by a golf swing from an image showing a subject's golf swing. [Means for solving the problem]

[0006] In order to achieve the above object, in one embodiment of the present disclosure, An acquisition step in which an information processing device acquires an image showing a golf swing motion of a subject person; a recognition step in which an information processing device recognizes a golf ball and a person or a golf club appearing in the image; Based on the positional relationship in the image between the golf ball recognized in the recognition step and a person or a golf club, a specifying step of specifying whether the golf ball recognized in the recognition step is a target to be hit by the swing operation is included. An information processing method is provided.

[0007] Also, in another embodiment of the present disclosure, an information processing apparatus is caused to execute an acquisition step of acquiring an image in which a golf swing motion of a person as a subject is shown, a recognition step of recognizing a golf ball and a person or a golf club shown in the image, and a specifying step of specifying whether the golf ball recognized in the recognition step is a target to be hit by the swing operation based on the positional relationship in the image between the golf ball and the person or the golf club recognized in the recognition step. A program is provided.

[0008] Also, in still another embodiment of the present disclosure, an acquisition unit that acquires an image in which a golf swing motion of a person as a subject is shown, a recognition unit that recognizes a golf ball and a person or a golf club shown in the image, and a specifying unit that specifies whether the golf ball recognized by the recognition unit is a target to be hit by the swing operation based on the positional relationship in the image between the golf ball and the person or the golf club recognized by the recognition unit are provided. An information processing apparatus is provided.

Advantages of the Invention

[0009] According to the above-described embodiment, it is possible to appropriately specify a golf ball before being hit by a swing operation from an image in which a golf swing motion of a person as a subject is shown.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

[0011] Hereinafter, embodiments will be described with reference to the drawings.

[0012] [Overview of the Information Processing System] With reference to FIGS. 1 to 3, an overview of the information processing system 1 according to the present embodiment will be described.

[0013] FIG. 1 is a diagram showing an example of the information processing system 1. FIG. 2 is a diagram showing an example of a moving image (moving image 20) representing the state of a user's golf swing motion. FIG. 3 is a diagram showing another example of a moving image (moving image 30) representing the state of a user's golf swing motion.

[0014] Note that in FIG. 2, frames 21 to 28 out of all the frames of the moving image 20 are shown in an extracted manner. Frame 21 represents the address state in the golf swing motion by the user. Frame 22 represents the take-back state in the golf swing motion by the user. Frame 23 represents the backswing state in the golf swing motion by the user. Frame 24 represents the top state in the golf swing motion by the user. Frame 25 represents the halfway down state in the golf swing motion by the user. Frame 26 represents the impact state in the golf swing motion by the user. Frame 27 represents the follow state in the golf swing motion by the user. Frame 28 represents the finish state in the golf swing motion by the user. Similarly, in FIG. 3, frame 31 out of all the frames of the moving image 30 is shown in an extracted manner. Frame 31 represents the halfway down state in the golf swing motion by the user.

[0015] As shown in FIG. 1, the information processing system 1 includes a camera 100, a user terminal 200, and an information processing device 300.

[0016] The information processing system 1 performs an analysis of the user's golf swing motion (hereinafter, "motion analysis") in the information processing device 300 based on moving image data in which the state of the user's golf swing motion acquired by the camera 100 is stored. Then, the information processing system 1 notifies the user of the analysis result through the user terminal 200.

[0017] For example, the information processing system 1 analyzes the user's golf swing motion to perform a diagnosis regarding the user's golf swing motion (hereinafter, "swing diagnosis"), and as a result of the motion analysis, notifies the user of the result of the swing diagnosis. The result of the swing diagnosis includes various index values obtained by the motion analysis and proposals for improvement points of the swing based on the various index values. Further, the information processing system 1 analyzes the user's golf swing motion to perform selection of a tool such as a golf club suitable for the user's golf swing motion (hereinafter, "tool selection"), and as a result of the motion analysis, may notify the user of the result of the tool selection. The result of the tool selection includes various index values obtained by the motion analysis and information regarding the specifications of a recommended tool for the user's golf swing motion derived based on the various index values and information on tools conforming to the specifications of the recommended tool.

[0018] The camera 100 images the user's golf swing motion and acquires a moving image representing the state of the motion. The camera 100 may acquire a moving image representing the state of the user's golf swing motion in response to an operation by a photographer different from the user performing the golf swing motion, or may acquire a moving image representing the state of the user's golf swing motion in response to the user's operation by means of a self-timer function or the like.

[0019] The camera 100 may acquire a moving image representing the user's golf swing motion imaged from one viewpoint, or may acquire a moving image representing the user's golf swing motion imaged from a plurality of viewpoints. In the latter case, for example, moving images representing the user's golf swing motion viewed from different viewpoints at the same timing are acquired by a plurality of cameras 100. Further, by changing the position of one camera 100, images representing the user's motion with different viewpoints may be sequentially acquired.

[0020] For example, as shown in FIG. 2, the camera 100 images the golf swing motion from the front of the user in a face-to-face manner with the user. Also, as shown in FIG. 3, the camera 100 may image the golf swing motion from behind the user. The rear of the user means the rear of the user when the direction of the virtual fly ball line (hereinafter, "virtual fly ball line") assumed by the user is defined as "front". Further, the camera 100 may acquire both a moving image representing the golf swing motion from the front of the user and a moving image of the golf swing motion from behind the user.

[0021] In FIG. 1, the camera 100 and the user terminal 200 are drawn separately, but the camera 100 may be built into the user terminal 200 or may be provided separately from the user terminal 200. In the latter case, the output of the camera 100 (moving image data) may be taken into the user terminal 200 by communication through the communication interface 206 described later, or may be taken into the user terminal 200 through the recording medium 201A described later.

[0022] The user terminal 200 is a terminal device used by the user. The user terminal 200 may be, for example, a terminal device disposed in a golf lesson facility, a shop, or the like, or may be a terminal device possessed by the user.

[0023] The user terminal 200 is, for example, a portable terminal device, that is, a mobile terminal. The mobile terminal is, for example, a smartphone, a tablet terminal, a laptop-type PC (Personal Computer), or the like. Also, the user terminal 200 may be a stationary terminal device. The stationary terminal device is, for example, a desktop-type PC.

[0024] The user terminal 200 is communicably connected to the information processing apparatus 300 through a predetermined communication line. The predetermined communication line includes, for example, a local area network (LAN). The predetermined communication line may also include a wide area network (WAN). The wide area network includes, for example, the Internet. The wide area network may also include a mobile communication network with a base station at the terminal or a satellite communication network using communication satellites. The predetermined communication line may also include a short-range communication line using a predetermined wireless communication standard such as WiFi, Bluetooth (registered trademark), or local 5G (5 th Generation).

[0025] The user terminal 200 captures a moving image representing the state of the user's golf swing operation from the camera 100 and transmits it to the information processing apparatus 300. Then, the user terminal 200 presents the information regarding the result of the motion analysis, which is returned from the information processing apparatus 300, to the user through the display device 208 described later. The information regarding the result of the motion analysis is, for example, information representing the diagnosis result of the golf swing based on the motion analysis. The information regarding the result of the motion analysis may also be information representing recommended golf clubs, balls, and other recommended tools based on the motion analysis.

[0026] Based on the moving image representing the state of the user's golf swing operation received from the user terminal 200, the information processing apparatus 300 performs motion analysis of the golf swing operation and returns information regarding the result of the motion analysis to the user terminal 200.

[0027] The information processing apparatus 300 is, for example, a server apparatus with relatively high processing power. The server apparatus may be a cloud server, an on-premises server, or an edge server. Depending on the required processing power, the information processing apparatus may also be a terminal apparatus with lower processing power than the server apparatus. The terminal apparatus may be a stationary terminal apparatus or a portable terminal apparatus (mobile terminal).

[0028] [Configuration of the Information Processing System] Next, in addition to FIG. 1, with reference to FIGS. 4 and 5, the configuration of the information processing system 1 will be described.

[0029] <Configuration of the User Terminal> FIG. 4 is a block diagram showing an example of the configuration of the user terminal 200.

[0030] The functions of the user terminal 200 may be realized by any hardware or any combination of hardware and software, etc. For example, as shown in FIG. 4, the user terminal 200 includes an external interface 201, an auxiliary storage device 202, a memory device 203, a CPU 204, a communication interface 206, an input device 207, a display device 208, and a sound output device 209. These components are connected by a bus B2. Also, as described above, when the camera 100 is built into the user terminal 200, the camera 100 may be connected to the bus B2 in the same manner as the other components.

[0031] The external interface 201 functions as an interface for reading data from the recording medium 201A and writing data to the recording medium 201A. The recording medium 201A includes, for example, a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disc), a BD (Blu-ray (registered trademark) Disc), an SD memory card, a USB (Universal Serial Bus) memory, etc. Thereby, the user terminal 200 can read various data used in processing through the recording medium 201A and store them in the auxiliary storage device 202, or install programs for realizing various functions.

[0032] Note that the user terminal 200 may acquire various data and programs used in processing from an external device (for example, the information processing device 300) through the communication interface 206.

[0033] The auxiliary storage device 202 stores various installed programs and also stores files, data, etc. necessary for various processes. The auxiliary storage device 202 includes, for example, an HDD (Hard Disc Drive), an SSD (Solid State Disc), a flash memory, and the like.

[0034] When there is an instruction to start a program, the memory device 203 reads and stores the program from the auxiliary storage device 202. The memory device 203 includes, for example, a DRAM (Dynamic Random Access Memory) and an SRAM (Static Random Access Memory).

[0035] The CPU 204 executes various programs loaded from the auxiliary storage device 202 to the memory device 203, and realizes various functions related to the user terminal 200 according to the programs.

[0036] The communication interface 206 is used as an interface for communicably connecting to an external device. Thereby, the user terminal 200 can acquire moving image data from the camera 100 through the communication interface 206. Further, the user terminal 200 can communicate with an external device such as the information processing device 300 through the communication interface 206. Further, the communication interface 206 may have a plurality of types of communication interfaces according to the communication method and the like with the connected device.

[0037] The input device 207 receives various inputs from the user.

[0038] The input device 207 includes, for example, an input device (hereinafter, "mechanical input device") that receives mechanical input from the user. The mechanical input device includes, for example, buttons, toggles, levers, a touch panel mounted on the display device 208, a touch pad provided separately from the display device 208, a keyboard, a mouse, and the like.

[0039] In addition, the input device 207 may include a voice input device capable of receiving voice input from the user. The voice input device includes, for example, a microphone capable of collecting the user's voice.

[0040] In addition, the input device 207 may include a gesture input device capable of receiving gesture input from the user. The gesture input device includes, for example, a camera capable of imaging the user's gestures.

[0041] In addition, the input device 207 may include a biological input device capable of receiving biological input from the user. The biological input device includes, for example, a camera capable of acquiring image data containing information about the user's fingerprint or iris.

[0042] The display device 208 displays an information screen, an operation screen, etc. toward the user. The display device 208 is, for example, a liquid crystal display, an organic EL (Electroluminescence) display, or the like.

[0043] The sound output device 209 conveys various information to the user of the user terminal 200 by sound. The sound output device 209 is, for example, a buzzer, an alarm, a speaker, or the like.

[0044] <Configuration of the information processing device> FIG. 5 is a block diagram showing an example of the configuration of the information processing device 300.

[0045] The functions of the information processing device 300 may be realized by any hardware, or any combination of hardware and software, etc. For example, as shown in FIG. 5, the information processing device 300 includes an external interface 301, an auxiliary storage device 302, a memory device 303, a CPU 304, a high-speed arithmetic device 305, a communication interface 306, an input device 307, a display device 308, and a sound output device 309. These components are connected by a bus B3.

[0046] The external interface 301 functions as an interface for reading data from the recording medium 301A and writing data to the recording medium 301A. The recording medium 301A includes, for example, a flexible disk, CD, DVD, BD, SD memory card, USB memory, etc. Thereby, the information processing apparatus 300 can read various data used in processing through the recording medium 301A, store it in the auxiliary storage device 302, or install a program for realizing various functions.

[0047] In addition, the information processing apparatus 300 may acquire various data and programs used in processing from an external device through the communication interface 306.

[0048] The auxiliary storage device 302 stores installed various programs and also stores files, data, etc. necessary for various processes. The auxiliary storage device 302 includes, for example, an HDD, SSD, flash memory, etc.

[0049] When there is an instruction to start a program, the memory device 303 reads the program from the auxiliary storage device 302 and stores it. The memory device 303 includes, for example, DRAM and SRAM.

[0050] The CPU 304 executes various programs loaded from the auxiliary storage device 302 to the memory device 303 and realizes various functions related to the information processing apparatus 300 according to the programs.

[0051] The high-speed arithmetic unit 305 operates in conjunction with the CPU 304 and performs arithmetic processing at a relatively high speed compared to the CPU 304. The high-speed arithmetic unit 305 includes, for example, a GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), etc.

[0052] In addition, the high-speed arithmetic unit 305 may be omitted depending on the speed of necessary arithmetic processing.

[0053] The communication interface 306 is used as an interface for communicably connecting to an external device. Thereby, the information processing apparatus 300 can communicate with an external device such as the user terminal 200 through the communication interface 306. Further, the communication interface 306 may have a plurality of types of communication interfaces according to the communication method or the like with the connected device.

[0054] The input device 307 receives various inputs from the user.

[0055] The input device 307 includes, for example, a mechanical input device that receives a mechanical operation input from the user. The mechanical input device includes, for example, buttons, toggles, levers, a touch panel mounted on the display device 308, a touch pad provided separately from the display device 308, a keyboard, a mouse, and the like.

[0056]

[0057]

[0058]

[0059]

[0060] The display device 308 displays an information screen or an operation screen for the user. The display device 308 is, for example, a liquid crystal display, an organic EL display, or the like.

[0060] The sound output device 309 conveys various types of information to the user of the information processing device 300 by sound. The sound output device 309 is, for example, a buzzer, an alarm, a speaker, or the like.

[0061] [Functional Configuration of Information Processing System] Next, with reference to FIGS. 6 to 9, the functional configuration of the information processing system 1 will be described.

[0062] FIG. 6 is a functional block diagram showing an example of the functional configuration of the information processing system. FIG. 7 is a diagram showing an example of feature points extracted from a moving image representing the state of a golf swing operation by a user. Specifically, FIG. 7 is a diagram showing a specific example of feature points representing the user's operation for each of a plurality of frames included in a moving image representing the state of a golf swing operation by the user. FIG. 8 is a diagram showing a specific example in which a plurality of golf balls are reflected in an image representing the state of a golf swing operation by a user. FIG. 8 includes FIG. 8A representing an image of one frame at the address time among moving images representing a golf swing operation in which the user is imaged from the front, and FIG. 8B representing one frame at the address time among moving images representing a golf swing operation in which the user is imaged from the rear. FIG. 9 is a diagram showing a specific example in which an object misdetected as a golf ball is reflected in an image representing the state of a golf swing operation by a user. Specifically, FIG. 9 represents an image of one frame at the address time among moving images representing the state of a golf swing operation by the user.

[0063] Further, in FIG. 7, data 70 of feature points (black-filled circles and white circles in the figure) representing the user's actions for each of a plurality of frames of a moving image representing the user's golf swing motion is schematically visualized and shown. In FIG. 7, data 71 to 78 of feature points for each frame of a part (frames 21 to 28) of all the frames of the moving image 20 in FIG. 2 are excerpted and shown. Data 71 represents the data of the feature points corresponding to frame 21 in the address state in the golf swing motion by the user. Data 72 represents the data of the feature points corresponding to frame 22 in the take-back state in the golf swing motion by the user. Data 73 represents the data of the feature points corresponding to frame 23 in the backswing state in the golf swing motion by the user. Data 74 represents the data of the feature points corresponding to frame 24 in the top state in the golf swing motion by the user. Data 75 represents the data of the feature points corresponding to frame 25 in the halfway-down state in the golf swing motion by the user. Data 76 represents the data of the feature points corresponding to frame 26 in the impact state in the golf swing motion by the user. Data 77 represents the data of the feature points corresponding to frame 27 in the follow state in the golf swing motion by the user. Data 78 represents the data of the feature points corresponding to frame 28 in the finish state in the golf swing motion by the user.

[0064] As shown in FIG. 6, the user terminal 200 includes an application screen display processing unit 2001, a moving image data acquisition unit 2002, a moving image data transmission unit 2003, and an analysis result data acquisition unit 2004. These functions are realized, for example, by loading an application program (hereinafter simply referred to as "app") installed in the auxiliary storage device 202 into the memory device 203 and executing it with the CPU 204.

[0065] The application screen display processing unit 2001 causes the display device 208 to display a screen related to the app (hereinafter referred to as "app screen").

[0066] The moving image data acquisition unit 2002 acquires data of a moving image representing the state of the user's operation from the camera 100. For example, the moving image data acquisition unit 2002 acquires data of a moving image representing the state of the user's operation from the camera 100 in response to an input from the user on a predetermined application screen using the input device 207. At this time, the moving image data acquisition unit 2002 may acquire a moving image that has already been captured by the camera 100, or may acquire in real time the latest moving image acquired by the camera 100.

[0067] The moving image data transmission unit 2003 transmits data of a moving image representing the state of the user's operation, acquired by the moving image data acquisition unit 2002, to the information processing device 300 through the communication interface 206. For example, the moving image data transmission unit 2003 transmits the moving image data acquired from the camera 100 to the information processing device 300 in response to an input from the user on a predetermined application screen using the input device 207.

[0068] The analysis result data acquisition unit 2004 acquires reply data including information regarding the result of the motion analysis, received from the information processing device 300. The content of the reply data including information representing the result of the motion analysis is displayed on the display device 208 by the application screen display processing unit 2001. Thereby, the user can confirm the content of the reply data including information regarding the result of the motion analysis.

[0069] As shown in FIG. 6, the information processing device 300 includes a moving image data acquisition unit 3001, a feature point extraction unit 3002, a golf ball identification unit 3003, a motion analysis unit 3004, and an analysis result transmission unit 3005. These functions are realized, for example, by a program installed in the auxiliary storage device 302 being loaded into the memory device 303 and executed by the CPU 304.

[0070] The moving image data acquisition unit 3001 acquires data of a moving image representing the state of the user's operation, received from the user terminal 200.

[0071] For each frame of the moving image data acquired by the moving image data acquisition unit 3001, the feature point extraction unit 3002 extracts a plurality of feature points related to the swing motion of the user, the subject (see FIG. 7).

[0072] Also, as described above, there may be a case where the moving image data acquisition unit 3001 acquires moving image data from a plurality of different viewpoints. In this case, for each of the moving image data corresponding to the plurality of viewpoints, the feature point extraction unit 3002 extracts a plurality of feature points related to the swing motion of the user, the subject, for each frame of the motion image.

[0073] For example, the plurality of feature points include feature points representing the body parts of the user, the subject, on the image of the frame (hereinafter, "physical feature points"). The physical feature points represent the joint positions in the user's skeleton, and the physical feature points included in the plurality of feature points include the points on the image corresponding to the head, shoulders, elbows, wrists, waist, knees, ankles, etc. (see the black-filled circles in the figure). Thereby, for example, the motion analysis unit 3004 can analyze the position and posture of the body parts in the golf swing motion by the user, or analyze the movement trajectory and posture change of the body parts.

[0074] For example, the feature point extraction unit 3002 utilizes an image recognition engine to which a known bone detection technique or the like is applied, recognizes a person from each frame of the moving image data, and extracts the physical feature points of the recognized person.

[0075] Also, for example, the plurality of feature points include feature points representing the parts of the golf club held by the user. For example, as shown in FIG. 7, the plurality of feature points include feature points corresponding to the head of the golf club held by the user (see the black-filled circles in the figure). Thereby, for example, the motion analysis unit 3004 can analyze the position of the head of the golf club and the posture of the shaft in the golf swing motion by the user, or analyze the movement trajectory of the head and the posture change of the shaft.

[0076] For example, the feature point extraction unit 3002 utilizes an image recognition engine to which known bone detection techniques or the like are applied, recognizes a golf club from each frame of the moving image data, and extracts the feature points of the recognized golf club.

[0077] Also, for example, as shown in FIG. 7, the plurality of feature points include feature points representing a golf ball (see the white circles in the figure). Thereby, for example, the motion analysis unit 3004 can analyze the positional relationship and its change between the golf ball and the human body part until it is hit by the swing motion.

[0078] For example, the feature point extraction unit 3002 utilizes a known image recognition engine to recognize a golf ball from each frame of the moving image data. Then, the feature point extraction unit 3002 may extract the center of the golf ball identified as the target golf ball (hereinafter, for convenience, referred to as the "golf ball to be shot") to be hit by the user's swing motion by the golf ball identification unit 3003 described later as a feature point.

[0079] Note that the feature point extraction unit 3002 does not necessarily perform the process of extracting the feature points corresponding to the golf ball for the frames after the impact timing in the golf swing motion.

[0080] The golf ball identification unit 3003 identifies whether the golf ball recognized by the feature point extraction unit 3002 from the frames before the impact of the swing motion is the target (shot target) golf ball to be hit by the user's swing motion.

[0081] Further, the feature point extraction unit 3002 extracts physical feature points and golf club feature points for all frames of the moving image, and obtains data representing the swing motion of each frame represented by the physical feature points and the golf club feature points. The data representing the swing motion includes the speed, acceleration, angular velocity, angular acceleration, etc. of a predetermined body part of a person or a predetermined part of the golf club. Thereby, the information processing apparatus 300 determines, for each frame of the moving image, whether the operating conditions corresponding to the impact of the swing motion are satisfied based on the data representing the swing motion, thereby specifying the timing of the impact from among the frames of the moving image. Therefore, the golf ball specifying unit 3003 can specify the golf ball to be shot with respect to the frames before the frame corresponding to the specified impact timing.

[0082] For example, the golf ball specifying unit 3003 specifies the golf ball to be shot from among the plurality of golf balls recognized by the feature point extraction unit 3002.

[0083] For example, as shown in FIGS. 8A and 8B, an image representing the golf swing motion by the user may be taken during golf practice. In images 80 and 85, in addition to the golf balls 81 and 86 to be shot, a group of spare golf balls 82 and 87 to be used in the next and subsequent shots are reflected. Therefore, the feature point extraction unit 3002 recognizes not only the golf balls 81 and 86 but also the golf balls included in the group of golf balls 82 and 87 from each of the images 80 and 85, and as a result, may recognize a plurality of golf balls.

[0084] Also, for example, as shown in FIG. 9, an object similar to a golf ball may be reflected in a moving image representing the golf swing motion. In the image 90, in addition to the golf ball 91 to be shot, a tee marker 92 whose shape and color are similar to those of a golf ball is reflected. Therefore, the feature point extraction unit 3002 may erroneously recognize not only the golf ball 91 but also the tee marker 92 as a golf ball.

[0085] The details of the golf ball specifying unit 3003 will be described later.

[0086] Based on a plurality of feature points extracted by the feature point extraction unit 3002, the motion analysis unit 3004 performs motion analysis regarding the golf swing motion by the user.

[0087] The analysis result transmission unit 3005 transmits information regarding the result of the motion analysis by the motion analysis unit 3004 to the user terminal 200. As a result, the user can confirm information regarding the result of the motion analysis through the application screen of the user terminal 200.

[0088] [Operation of the Information Processing System] Next, with reference to FIG. 10, the operation of the information processing system 1 will be described.

[0089] FIG. 10 is a sequence diagram schematically showing an example of the operation of the information processing system 1.

[0090] As shown in FIG. 10, the user terminal 200 activates an application in response to a predetermined input from the user using the input device 207 (step S102).

[0091] After completion of the process of step S102, the moving image data acquisition unit 2002 acquires moving image data representing the state of the golf swing motion by the user from the camera 100 in response to a predetermined input from the user on a predetermined application screen using the input device 207 (step S104).

[0092] After completion of the process of step S104, the moving image data transmission unit 2003 transmits the moving image data to the information processing apparatus 300 through the communication interface 206 (step S106).

[0093] The moving image data acquisition unit 3001 acquires the moving image data transmitted (uploaded) from the user terminal 200 in the process of step S106 (step S108).

[0094] After the completion of the process in step S108, the feature point extraction unit 3002 extracts a plurality of feature points related to the user's swing motion for each frame of the moving image data (step S110).

[0095] After the completion of the process in step S110, the golf ball identification unit 3003 identifies the golf ball to be shot for the frames of the moving image before the timing of the impact of the swing motion (step S112).

[0096] Thereby, the feature points corresponding to the golf ball are determined, and a plurality of feature points used for motion analysis can be determined.

[0097] After the completion of the process in step S112, the motion analysis unit 3004 performs motion analysis related to the golf swing motion based on the determined plurality of feature points (step S114).

[0098] After the completion of the process in step S114, the analysis result transmission unit 3005 transmits reply data including information related to the result of the motion analysis to the user terminal 200 through the communication interface 306 (step S116).

[0099] The analysis result data acquisition unit 2004 acquires the reply data transmitted from the information processing device 300 in the process of step S116 (step S118).

[0100] After the completion of the process in step S118, the application screen display processing unit 2001 causes the display device 208 to display the information related to the result of the motion analysis included in the reply data (step S120).

[0101] [Method for Identifying the Golf Ball to be Shot] Next, with reference to FIGS. 11 and 12, a method for identifying the golf ball to be shot on the image will be described.

[0102] [Overview] The golf ball identification unit 3003 identifies the golf ball to be shot from among the plurality of recognized golf balls based on the positional relationship between the golf club and the golf ball in the image recognized by the feature point extraction unit 3002.

[0103] For example, as shown in FIG. 8A, in the case of an image 80 obtained by imaging the subject user from the front, the actual golf ball 81 to be shot is closer to the golf club in the vertical direction (hereinafter, for convenience, referred to as the "Y-axis direction") of the image 80 than the balls in the preliminary golf ball group 82.

[0104] Also, as shown in FIG. 8B, in the case of an image 85 obtained by imaging the subject user from the rear, the actual golf ball 86 to be shot is closer to the golf club in the horizontal direction (hereinafter, for convenience, referred to as the "X-axis direction") of the image 80 than the balls in the preliminary golf ball group 87.

[0105] Also, as shown in FIG. 9, for example, an object such as a tee marker 92 that is erroneously detected is often at a greater absolute distance from the head of the golf club than the golf ball 91 to be shot.

[0106] Therefore, the golf ball identification unit 3003 calculates the distance D in the X-axis direction x and the distance D in the Y-axis direction y between the feature points of the golf ball and the feature points (e.g., the head) of the golf club recognized by the feature point extraction unit 3002. Then, as shown in Equation (1), the golf ball identification unit 3003 calculates the calculated distance D in the X-axis direction x and the distance D in the Y-axis direction y and the coefficients α x , α y (both positive numbers) to calculate an index value EI1 representing the positional relationship between the golf ball and the golf club.

[0107]

Equation

[0108] When a video image is captured from the front of the subject, the coefficient α y is the coefficient α x The coefficient α is set to be larger. x ,α y is set.

[0109]

number

[0110] As a result, the distance D in the X-axis direction at the index value EI1 x Distance D in the Y-axis direction y Therefore, for an image captured from the front of the subject user, the difference in the position of the golf ball in the Y-axis direction relative to the golf club can be expressed by the magnitude of the index value EI1. Therefore, the golf ball identification unit 3003 can reliably distinguish, for example, the golf ball 81 in the image 80 from the golf balls in the group of spare golf balls 82.

[0111] In addition, when the moving image is captured from behind the subject user, the coefficient α x is the coefficient α y The coefficient α is set to be larger. x ,α y is set.

[0112]

number

[0113] As a result, the distance D in the Y-axis direction at the index value EI1 y Distance D in the X-axis direction xcan be increased. Therefore, regarding an image captured from behind the user of the subject, the difference in the X-axis direction position of the golf ball with respect to the golf club can be expressed by the magnitude of the index value EI1. The golf ball specifying unit 3003 can surely discriminate, for example, the golf ball 86 in the image 85 and the balls in the preliminary golf ball group 87 based on the magnitude of the index value EI1.

[0114] Also, by using the distance D in the X-axis direction x and the distance D in the Y-axis direction y the distance between the golf club and the golf ball can be expressed by the magnitude of the index value EI1. Therefore, the golf ball specifying unit 3003 can surely discriminate, for example, the golf ball 91 in the image 90 and the tee marker 92 based on the magnitude of the index value EI1.

[0115] For example, the golf ball specifying unit 3003 determines the orientation of the person (user) shown in the moving image data acquired by the moving image data acquisition unit 3001. Then, when the user is facing the front with respect to the viewpoint of the moving image, the coefficient α x , α y prescribed in advance so as to satisfy the condition of Equation (2) is set, and the index value EI1 is calculated for each frame of the moving image using Equation (1). On the other hand, when the user is facing sideways with respect to the viewpoint of the moving image, the coefficient α x , α y prescribed in advance so as to satisfy the condition of Equation (3) is set, and the index value EI1 is calculated for each frame of the moving image using Equation (1). Then, when there are a plurality of golf balls extracted by the feature point extraction unit 3002 for each frame of the moving image, the golf ball having the minimum index value EI1 among them is specified as the golf ball to be shot.

[0116] Further, the golf ball identification unit 3003 may identify the golf ball to be shot based on the index value EI2 representing the reliability of the golf ball recognized by the feature point extraction unit 3002 in addition to the index value EI1 representing the positional relationship between the golf ball and the golf club.

[0117] The index value EI2 representing the reliability represents how accurately the golf ball recognized by the feature point extraction unit 3002 represents the golf ball, and is defined by a value between 0 and 1 (0 ≤ EI2 ≤ 1), for example. The index value EI2 representing the reliability is output together with the feature point extraction unit 3002 when the golf ball is recognized by the feature point extraction unit 3002.

[0118] For example, when the image recognition engine uses a machine learning-based algorithm, the index value EI2 representing the reliability is output as an index value related to the accuracy of the rectangular frame surrounding the object corresponding to the golf ball and the prediction probability of being classified into the class of the golf ball. The index value EI2 representing the reliability is, for example, the confidence score when the YOLO (You Only Look Once) algorithm is used.

[0119] Further, when the image recognition engine uses a rule-based algorithm, the index value EI2 representing the reliability is output as an index value representing the degree to which the golf ball exceeds the standard recognized as a golf ball defined by the rule.

[0120] For example, when there are multiple golf balls extracted by the feature point extraction unit 3002 for each frame of the moving image, the golf ball identification unit 3003 identifies, among the golf balls with relatively high index values EI2 representing reliability, the golf ball with the minimum index value EI1 as the golf ball to be shot. The relatively high index value EI2 representing reliability may mean that the index value EI2 is equal to or higher than a predetermined standard, or higher than the predetermined standard. Thereby, for example, even if an object with a relatively low index value EI2 representing the reliability of the tee marker 92 in the image 90 or the like is recognized as a golf ball, the possibility of being identified as the golf ball hit by the swing operation can be reduced. Therefore, even when there is an object misrecognized as a golf ball at a relatively close position to the golf club, a situation where the misrecognized object is identified as the golf ball to be shot can be suppressed.

[0121] For each frame of the moving image, the golf ball identification unit 3003 calculates the distances D in the X-axis direction and Y-axis direction between the golf ball and the golf club x , D y using, for example, the position of the golf club in the target frame. Also, for each frame of the moving image, the golf ball identification unit 3003 calculates the distances D in the X-axis direction and Y-axis direction between the golf ball and the golf club x , D y using the position of the golf club recognized in a specific frame. For example, the golf ball identification unit 3003 uses the feature points of the golf club extracted in the frame corresponding to the address timing of the swing operation among all the frames of the moving image to calculate the distances D x , D y for each frame.

[0122] <First Example of Processing> FIG. 11 is a flowchart schematically showing a first example of a process for identifying a golf ball hit by a golf swing operation by a user.

[0123] The flowchart of FIG. 11 corresponds to the process of step S112 in FIG. 10.

[0124] In step S202, the golf ball identification unit 3003 determines the orientation of the user with respect to the data of the moving image acquired by the moving image data acquisition unit 3001. The orientation of the user in the moving image is input, for example, on the application screen through the input device 207 in the user terminal 200, and the information is transmitted to the information processing device 300 together with the moving image data. Also, the orientation of the user in the moving image may be automatically determined based on the positional relationship of a plurality of feature points extracted by the feature point extraction unit 3002.

[0125] When the process of step S202 is completed, the information processing device 300 proceeds to step S204.

[0126] In step S204, the golf ball identification unit 3003 calculates the distances Dx and Dy in the X-axis direction and the Y-axis direction between each golf ball recognized by the feature point extraction unit 3002 and the golf club for the frames in a specific range among all the frames of the moving image. The frames in the specific range are the frames in the range before the timing of the impact of the golf swing motion by the user shown in the moving image.

[0127] When the process of step S204 is completed, the information processing device 300 proceeds to step S206.

[0128] In step S206, the golf ball identification unit 3003 calculates an index value EI1 representing the positional relationship between each golf ball recognized by the feature point extraction unit 3002 and the golf club for each frame in the specific range.

[0129] When the process of step S206 is completed, the information processing device 300 proceeds to step S208.

[0130] In step S208, the golf ball identification unit 3003 identifies, as the golf ball to be shot, the golf ball with the minimum index value EI1 for each frame in the specific range.

[0131] When the process of step S208 is completed, the information processing apparatus 300 ends the process of this flowchart.

[0132] In addition, when there is only one frame in which a golf ball recognized in a specific range of frames exists, the golf ball specifying unit 3003 automatically specifies that one golf ball as the golf ball to be shot for that frame. Therefore, the processes of steps S204 and S206 may be performed only on the frames in which a plurality of golf balls are recognized among the frames in the specific range.

[0133] <Second Example of Process> FIG. 12 is a flowchart schematically showing a second example of a process for specifying a golf ball hit by a golf swing operation by a user.

[0134] The flowchart of FIG. 12 corresponds to the process of step S112 in FIG. 10.

[0135] As shown in FIG. 12, since the processes of steps S302, S304, and S306 are the same as the processes of steps S202, S204, and S206 in FIG. 11 described above, the description thereof is omitted.

[0136] When the process of step S306 is completed, the information processing apparatus 300 proceeds to step S308.

[0137] In step S308, for each frame in a specific range, the golf ball specifying unit 3003 determines that the golf ball with the smallest index value EI1 among the golf balls with an index value EI2 greater than or equal to the threshold value EI2 th is the golf ball to be shot. The threshold value EI2 th is defined in advance as a value in the range between 0 and 1.

[0138] When the process of step S308 is completed, the information processing apparatus 300 ends the process of this flowchart.

[0139] Further, when there is no golf ball whose index value EI2 is equal to or greater than the threshold value EI2 th in step S308, the golf ball to be shot is not specified. In this case, for example, for that frame, the feature point extraction unit 3002 may interpolate the feature points of the golf ball using the feature points of the golf ball extracted in frames within a predetermined adjacent range.

[0140] [Other Embodiments] Next, other embodiments will be described.

[0141] Appropriate modifications and changes may be made to the above-described embodiments as appropriate.

[0142] For example, in the above-described embodiments, the functions of the user terminal 200 and the information processing apparatus 300 may be realized by one information processing apparatus, or may be realized in a distributed manner by three or more information processing apparatuses. For example, the functions of the information processing apparatus 300 may be transferred to the user terminal 200.

[0143] Also, in the above-described embodiments and examples of their modifications and changes, as a method of making the influence degrees on the index value EI1 of the distances D in the X-axis direction and the Y-axis direction between the golf ball and the golf club different x , D y a method different from the method using the coefficient α x , α y may be adopted. For example, the distance measurement criteria between the X-axis direction and the Y-axis direction on the image are made different. Specifically, when making the influence degree on the index value EI1 of the distance D in the X-axis direction higher than that of the distance D in the Y-axis direction x , for example, the distance D in the X-axis direction y is calculated with 1 pixel as one unit, and the distance D in the Y-axis direction x is calculated with 2 pixels as one unit. On the other hand, when making the influence degree on the index value EI1 of the distance D in the Y-axis direction higher than that of the distance D in the X-axis direction y , for example, the distance D in the X-axis direction y is calculated with 2 pixels as one unit, and the distance D in the Y-axis direction x is calculated with 1 pixel as one unit. On the other hand, when making the influence degree on the index value EI1 of the distance D in the Y-axis direction higher than that of the distance D in the X-axis direction x is calculated with 2 pixels as one unit, and the distance D in the Y-axis directiony is calculated with one pixel as one unit.

[0144] In addition, in the above-described embodiments and examples of their modifications and changes, a standard for the index value EI1 representing the positional relationship between the golf club and the golf ball may be provided.

[0145] For example, in step S208 of FIG. 11 described above, for each frame in a specific range, as the golf ball to be shot, the index value EI1 is the threshold value EI1 th Among the following golf balls, the golf ball with the minimum index value EI1 is specified. Threshold value EI1 th is preset as a value greater than 0 (EI1 th > 0).

[0146] Similarly, in step S308 of FIG. 12 described above, for each frame in a specific range, as the golf ball to be shot, the index value EI1 is the threshold value EI1 th Below and the index value EI2 is the threshold value EI2 th Among the above golf balls where EI1 is the minimum, the golf ball with the minimum index value EI1 is specified.

[0147] However, if there is no golf ball whose index value EI1 is less than or equal to the threshold value EI1th, in steps S208 and S308, the golf ball to be shot is not specified. In this case, for example, for that frame, the feature point extraction unit 3002 may interpolate the feature points of the golf ball using the feature points of the golf ball extracted in the frames within a predetermined adjacent range.

[0148] In addition, in the above-described embodiments and examples of their modifications and changes, on the image, instead of the positional relationship between the golf ball and the golf club, based on the positional relationship between the golf ball and the person, it may be specified whether it is the golf ball to be shot. Specifically, the golf ball specifying unit 3003 may specify the golf ball to be shot based on the positional relationship between the recognized feature points of the golf ball and predetermined physical feature points of the person (for example, either the left or right ankle, etc.).

[0149] [Function] Next, the functions of the information processing method, program, and information processing apparatus according to the present embodiment will be described.

[0150] In the first aspect of the present embodiment, the information processing method is executed by an information processing apparatus and includes an acquisition step, a recognition step, and a specification step. The information processing apparatus is, for example, the above-described information processing apparatus 300 or the user terminal 200. The acquisition step is, for example, the above-described step S108. The recognition step is, for example, the above-described step S110. The specification step is, for example, the above-described step S112. Specifically, in the acquisition step, an image in which the golf swing motion of the subject person is reflected is acquired. In the recognition step, a golf ball and a person or a golf club reflected in the image are recognized. Then, in the specification step, based on the positional relationship in the image between the golf ball and the person or the golf club recognized in the recognition step, it is specified whether the golf ball recognized in the recognition step is the target to be hit by the swing motion.

[0151] Also, in the first aspect of the present embodiment, the program may cause the information processing apparatus to execute the acquisition step, the recognition step, and the specification step.

[0152] Also, in the first aspect of the present embodiment, the information processing apparatus may include an acquisition unit, a recognition unit, and a specification unit. The acquisition unit is, for example, the above-described moving image data acquisition unit 3001. The recognition unit is, for example, the above-described feature point extraction unit 3002. The specification unit is, for example, the above-described golf ball specification unit 3003. Specifically, the acquisition unit acquires an image in which the golf swing motion of the subject person is reflected. The recognition unit recognizes a golf ball and a person or a golf club reflected in the image. Then, the specification unit specifies whether the golf ball recognized by the recognition unit is the target to be hit by the swing motion based on the positional relationship in the image between the golf ball and the person or the golf ball recognized by the recognition unit.

[0153] As a result, the information processing apparatus can identify a golf ball placed at a position suitable for the person of the subject or the golf club held by the person as a target to be hit in a swinging motion. Therefore, the information processing apparatus can appropriately identify a golf ball before being hit by the swinging motion from an image in which the golf swing motion of the person of the subject is reflected.

[0154] Further, in the second aspect of the present embodiment, on the premise of the above-described first aspect, in the specifying step, when a plurality of golf balls are recognized in the recognizing step, based on the positional relationship in the image between each of the plurality of golf balls recognized in the recognizing step and the person or the golf club, the golf ball to be hit by the swinging motion may be specified from among the plurality of golf balls.

[0155] As a result, the information processing apparatus can appropriately identify a golf ball before being hit by the swinging motion from among a plurality of golf balls recognized from an image in which the golf swing motion of the person of the subject is reflected.

[0156] Further, in the third aspect of the present embodiment, on the premise of the above-described first or second aspect, in the acquiring step, a moving image including a series of the images in which the swinging motion is reflected may be acquired. And in the specifying step, based on the positional relationship between the golf ball recognized in the recognizing step and the reference golf club for each of the images corresponding to the timing before impact in the swinging motion among the series of images, with reference to the position of the golf club recognized in the recognizing step from a specific one of the images corresponding to the timing of the address in the swinging motion among the series of images, it may be specified whether the golf ball recognized in the recognizing step is a target to be hit by the swinging motion.

[0157] As a result, based on the position of the person or golf club recognized in the image corresponding to the timing of the address of the swinging motion of the subject person, the information processing apparatus can specify whether the recognized golf ball included in the moving image is the target to be hit by the swinging motion.

[0158] Also, in the fourth aspect of the present embodiment, on the premise of any one of the above-described first to third aspects, in the specifying step, based on the first distance in the horizontal direction and the second distance in the vertical direction in the image between the golf ball recognized in the recognizing step and the person or golf club, it may be specified whether the golf ball recognized in the recognizing step is the target to be hit by the swinging motion. The first distance is, for example, the above-described distance D x That is. The second distance is, for example, the above-described distance D y That is.

[0159] As a result, the information processing apparatus can evaluate their positional relationship including information on the distance between the golf ball recognized from the image and the person or golf club. Therefore, the information processing apparatus can more appropriately specify the golf ball before being hit by the swinging motion from the image showing the golf swinging motion of the subject person.

[0160] Also, in the fifth aspect of the present embodiment, on the premise of the above-described fourth aspect, in the specifying step, based on the first distance and the second distance, a first index value regarding the positional relationship between the golf club recognized in the recognizing step and the person or golf ball is obtained such that the influence degree of either one of the first distance and the second distance becomes higher than the influence degree of the other, and based on the obtained first index value, it may be specified whether the golf ball recognized in the recognizing step is the target to be hit by the swinging motion.

[0161] As a result, the information processing apparatus can obtain a first index value representing the positional relationship between the golf ball and the person or golf club on the image, using the first distance in the horizontal direction and the second distance in the vertical direction. Therefore, the information processing apparatus can more appropriately identify the golf ball before it is hit by the swing motion from the image in which the golf swing motion of the subject person is reflected, based on the first index value.

[0162] Also, in the sixth aspect of the present embodiment, on the premise of the above-described fifth aspect, the first index value may be the sum of a value obtained by multiplying a positive coefficient that is relatively large with respect to the one with a higher degree of influence among the first distance and the second distance, and a value obtained by multiplying a positive coefficient that is relatively small with respect to the other with a lower degree of influence. The coefficients are, for example, the above-described coefficients α x , α y and so on.

[0163] As a result, the information processing apparatus can obtain a first index value representing the positional relationship between the golf ball and the person or golf club on the image, using the first distance in the horizontal direction and the second distance in the vertical direction.

[0164] Also, in the seventh aspect of the present embodiment, on the premise of the above-described fifth or sixth aspect, in the specifying step, according to the orientation of the subject person in the image, it may be determined which of the first distance and the second distance has a higher degree of influence in the first index value.

[0165] As a result, the information processing apparatus can appropriately set the first index value according to the orientation of the subject person in the image.

[0166] Also, in the eighth aspect of the present embodiment, on the premise of the above-described seventh aspect, in the specifying step, when the subject person is facing the viewpoint of the image directly, the degree of influence of the second distance may be made higher than that of the first distance, and when the subject person is sideways with respect to the viewpoint of the image, the degree of influence of the first distance may be made higher than that of the second distance.

[0167] As a result, the information processing apparatus can appropriately set the first index value according to the orientation of the person who is the subject of the image.

[0168] Further, in the ninth aspect of the present embodiment, on the premise of the above-described fifth to eighth aspects, in the specific step, when a plurality of golf balls are recognized in the recognition step, as the target to be hit by the swing operation, the golf ball with the smallest first index value may be specified from among the plurality of golf balls.

[0169] As a result, the information processing apparatus can specify, based on the first index value, the golf ball before being hit by the swing operation from among the plurality of golf balls recognized from the image in which the golf swing operation of the person who is the subject is reflected.

[0170] Further, in the tenth aspect of the present embodiment, on the premise of any one of the above-described first to ninth aspects, in the recognition step, a second index value representing the reliability regarding the recognized golf ball may be output. And in the specific step, based on the second index value output in the recognition step, in addition to the positional relationship in the image between the golf ball recognized in the recognition step and the person or the golf club, it may be specified whether the golf ball recognized in the recognition step is the target to be hit by the swing operation.

[0171] As a result, the information processing apparatus can, for example, specify that a golf ball recognized with a relatively high reliability in the recognition step is the target to be hit by the swing operation of the person who is the subject. Therefore, the information processing apparatus can more appropriately specify the golf ball before being hit by the swing operation from the image in which the golf swing operation of the person who is the subject is reflected.

[0172] Although the embodiments have been described in detail above, the present disclosure is not limited to such specific embodiments, and various modifications and changes are possible within the scope of the gist described in the claims.

Explanation of Symbols

[0173] 1 Information Processing System 100 Camera 200 User Terminal 300 Information Processing Device 2001 App Screen Display Processing Unit 2002 Moving Image Data Acquisition Unit 2003 Moving Image Data Transmission Unit 2004 Analysis Result Data Acquisition Unit 3001 Moving Image Data Acquisition Unit 3002 Feature Point Extraction Unit 3003 Golf Ball Identification Unit 3004 Motion Analysis Unit 3005 Analysis Result Transmission Unit

Claims

1. An acquisition step in which an information processing apparatus acquires an image in which a golf swing motion of a person as a subject is reflected; A recognition step in which the information processing apparatus recognizes a golf ball and a person or a golf club reflected in the image; and A specifying step in which the information processing apparatus specifies whether the golf ball recognized in the recognition step is a target to be hit by the swing motion based on a positional relationship in the image between the golf ball and the person or the golf club recognized in the recognition step, the information processing method comprising: An information processing method.

2. In the specifying step, when a plurality of golf balls are recognized in the recognition step, based on a positional relationship in the image between each of the plurality of golf balls recognized in the recognition step and a person or a golf club, a target golf ball to be hit by the swing motion is specified from among the plurality of golf balls. The information processing method according to claim 1.

3. In the acquisition step, a moving image including a series of the images in which the swing motion is reflected is acquired; In the specifying step, based on a positional relationship between the golf ball recognized in the recognition step and the reference person or golf club in each of the images corresponding to a timing before impact in the swing motion among the series of images, with reference to a position of the person or golf club recognized in the recognition step from a specific one of the images corresponding to the timing of the address in the swing motion among the series of images, it is specified whether the golf ball recognized in the recognition step is a target to be hit by the swing motion. The information processing method according to claim 1 or 2.

4. In the specifying step, based on a first distance in the horizontal direction and a second distance in the vertical direction in the image between the golf ball recognized in the recognition step and the person or the golf club, it is specified whether the golf ball recognized in the recognition step is a target to be hit by the swing motion. The information processing method according to claim 1 or 2.

5. In the specific step, based on the first distance and the second distance, a first index value representing the positional relationship between the golf club recognized in the recognition step and a person or a golf ball is obtained such that the influence degree of either one of the first distance and the second distance becomes higher than that of the other, and based on the obtained first index value, it is specified whether the golf ball recognized in the recognition step is a target to be hit by the swing operation. The information processing method according to claim 4.

6. The first index value is a total value of a value obtained by multiplying a relatively large positive coefficient by the one with the higher influence degree among the first distance and the second distance, and a value obtained by multiplying a relatively small positive coefficient by the other with the lower influence degree. The information processing method according to claim 5.

7. In the specific step, according to the orientation of the person as the subject in the image, it is determined which of the influence degrees of the first distance and the second distance in the first index value is to be made higher. The information processing method according to claim 5.

8. In the specific step, when the person as the subject is facing forward with respect to the viewpoint of the image, the influence degree of the second distance is made higher than that of the first distance, and when the person as the subject is sideways with respect to the viewpoint of the image, the influence degree of the first distance is made higher than that of the second distance. The information processing method according to claim 7.

9. In the specific step, when a plurality of golf balls are recognized in the recognition step, as the target to be hit by the swing operation, the golf ball with the smallest first index value is specified from among the plurality of golf balls. The information processing method according to claim 5.

10. In the recognition step, a second index value representing the reliability regarding the recognized golf ball is output. In the specific step, in addition to the positional relationship in the image between the golf ball recognized in the recognition step and a person or a golf club, based on the second index value output in the recognition step, it is specified whether the golf ball recognized in the recognition step is a target to be hit by the swing operation. The information processing method according to claim 1 or 2.

11. In an information processing apparatus, an acquisition step of acquiring an image in which a golf swing operation of a person as the subject is reflected; a recognition step of recognizing a golf ball and a person or a golf club reflected in the image Based on the positional relationship in the image of the golf ball and the person or golf club recognized in the recognition step, a specifying step of specifying whether the golf ball recognized in the recognition step is the target to be hit by the swinging operation is executed. Program.

12. An acquisition unit that acquires an image showing the golf swing motion of the subject person, A recognition unit that recognizes a golf ball and a person or a golf club shown in the image, and Based on the positional relationship in the image of the golf ball and the person or golf club recognized by the recognition unit, a specifying unit that specifies whether the golf ball recognized by the recognition unit is the target to be hit by the swinging operation is provided. Information processing apparatus.

Images