Information processing methods, programs, and information processing devices.

Abstract

This technology provides information that can easily help in understanding the characteristics of a golf swing motion performed by a subject. [Solution] An information processing device 300 according to one embodiment of the present disclosure includes an energy information acquisition unit 3005C that acquires information representing the energy of a predetermined body part during a golf club swing motion performed by a subject, and an evaluation index extraction unit 3007 that extracts evaluation indices representing the golf club swing motion performed by a subject based on the information representing the energy of a predetermined body part during a golf club swing motion performed by a subject.

Description

Technical Field

[0001] This disclosure relates to an information processing method and the like.

Background Art

[0002] Conventionally, a technique for extracting energy generated (exerted) from a body part of a subject during a golf swing operation and analyzing the swing operation of the subject is known (for example, see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in Patent Document 1, only a block (graph) corresponding to the temporal change of the energy exerted from the body part of the subject is presented. Therefore, for example, the subject or other persons may not be able to grasp the characteristics of the subject's golf swing operation without specific advice.

[0005] [[ID=三十九]]

Means for Solving the Problems

[0006] To achieve the above object, in one embodiment of the present disclosure, an information processing apparatus includes a first acquisition step of acquiring information representing energy of a predetermined body part in a swing operation of a golf club by a subject, The information processing device includes a first extraction step of extracting an index representing the swing motion of the subject based on information representing the energy of a predetermined body part in the swing motion, Information processing methods are provided.

[0007] In other embodiments of this disclosure, In an information processing device, A first acquisition step involves obtaining information representing the energy of a specific body part during the golf club swing motion performed by the subject, A first extraction step is performed, in which an index representing the subject's swing motion is extracted based on information representing the energy of the predetermined body part in the swing motion. A program will be provided.

[0008] Furthermore, in yet another embodiment of this disclosure, A first acquisition unit acquires information representing the energy of a specific body part during the swing motion of a golf club by the subject, The system includes a first extraction unit that extracts an index representing the swing motion of the subject based on information representing the energy of a predetermined body part in the swing motion, An information processing device is provided. [Effects of the Invention]

[0009] According to the above-described embodiment, it is possible to generate information that supports easy understanding of the characteristics of a golf swing motion performed by a subject. [Brief explanation of the drawing]

[0010] [Figure 1] This is a diagram showing an example of an information processing system. [Figure 2] This figure shows an example of a video representing a user's golf swing motion. [Figure 3] This figure shows another example of a video representing a user's golf swing motion. [Figure 4]This is a diagram showing an example of the configuration of a user terminal. [Figure 5] This is a diagram showing an example of the configuration of an information processing apparatus. [Figure 6] This is a functional block diagram showing an example of the functional configuration of an information processing system. [Figure 7] This is a diagram for explaining an example of two-dimensional motion data. [Figure 8] This is a diagram for explaining an example of an evaluation index. [Figure 9] This is a diagram for explaining an example of an app screen displayed on a user terminal. [Figure 10] This is a sequence diagram schematically showing an example of the operation of an information processing system.

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments will be described with reference to the drawings.

[0012] In this specification, a person who uses the functions of the information processing system 1 through the user terminal 200 is referred to as a "user", and a person who is engaged in operations such as providing the functions of the information processing system 1 to the user or management through the information processing apparatus 300 may be conveniently referred to as an "operator, etc.".

[0013] [Overview of the Information Processing System] Referring to FIGS. 1 to 3, the overview of the information processing system 1 according to this embodiment will be described.

[0014] FIG. 1 is a diagram showing an example of the information processing system 1. FIG. 2 is a diagram showing an example (moving image 20) of a moving image representing the state of a golf swing operation by a user. FIG. 3 is a diagram showing another example (moving image 30) of a moving image representing the state of a golf swing operation by a user. Specifically, FIGS. 2 and 3 are specific examples of moving images representing the state of a golf swing operation by a user.

[0015] Figure 2 shows frames 21-28 of the entire video 20. Frame 21 represents the address position in the user's golf swing. Frame 22 represents the take-back position in the user's golf swing. Frame 23 represents the backswing position in the user's golf swing. Frame 24 represents the top position in the user's golf swing. Frame 25 represents the halfway down position in the user's golf swing. Frame 26 represents the impact position in the user's golf swing. Frame 27 represents the follow-through position in the user's golf swing. Frame 28 represents the finish position in the user's golf swing.

[0016] Similarly, Figure 3 shows frame 31, an excerpt from all the frames of the video 30. Frame 31 represents the halfway down position in the user's golf swing motion.

[0017] As shown in Figure 1, the information processing system 1 includes a camera 100, a user terminal 200, and an information processing device 300.

[0018] The information processing system 1, using the information processing device 300, acquires information representing the energy of a predetermined body part of the subject during the swing motion (hereinafter referred to as "energy information") based on video data captured by the camera 100, which shows the subject swinging a striking tool. The information processing system 1 also extracts evaluation indicators representing the swing motion based on the information representing the energy of a predetermined body part of the subject during the swing motion using the striking tool. The information processing system 1 then presents the evaluation indicators representing the subject's swing motion using the striking tool to the user at the user terminal 200, and presents support information for improving the user's swing motion based on these evaluation indicators.

[0019] The subject of the photograph is, for example, a user of Information Processing System 1. Alternatively, the subject may be a customer of Information Processing System 1 who is receiving sales assistance for equipment (e.g., hitting equipment or balls). Furthermore, the subject may be a student receiving lessons from Information Processing System 1, who is providing advice on their swing. The following explanation will focus primarily on the case where the subject is the user of Information Processing System 1 themselves.

[0020] A swing motion using a striking tool is, for example, a golf swing. In this case, the striking tool is a golf club. Alternatively, a swing motion using a striking tool may be a baseball swing or a tennis swing. In this case, the striking tool is a baseball bat or a tennis racket. In this embodiment, the explanation will focus on the case where the swing motion using a striking tool is a golf swing, and "swing motion using a striking tool" will be simply referred to as "swing motion."

[0021] The specified body part is a specific body part that represents the characteristics of the user's swing motion, and may be one or more. The specified body part may be, for example, the upper body part excluding the arms (hereinafter simply referred to as "upper body part"), the lower body part, and the arms. Furthermore, the specified body part may be one or two of the arms, upper body part, and lower body part. The energy of the specified body part refers to the energy exerted (generated) by the specified body part and corresponds to the amount of work done by the specified body part.

[0022] Camera 100 captures the user's swing motion and acquires a video image representing that swing motion. The video image is composed of a series of still images (hereinafter referred to as "frames").

[0023] Camera 100 is, for example, a so-called two-dimensional camera and acquires two-dimensional moving images. Alternatively, camera 100 may be a three-dimensional camera capable of acquiring depth information of the two-dimensional moving images in addition to the two-dimensional moving images. Depth information of a two-dimensional moving image refers to, for example, information representing the depth position of the object depicted in each pixel of each frame that makes up the two-dimensional moving image.

[0024] Camera 100 may, for example, acquire video footage representing the user's swing motion in response to the operation of a photographer other than the user performing the swing motion. Alternatively, camera 100 may acquire video footage representing the user's swing motion in response to the user's operation using a self-timer function or the like.

[0025] Camera 100 may acquire video images representing the user's swing motion captured from a single viewpoint, or it may acquire video images representing the user's swing motion captured from multiple viewpoints. In the latter case, for example, multiple cameras 100 may acquire video images representing the user's swing motion from different viewpoints at the same time. Alternatively, by changing the position of one camera 100, images representing the user's swing motion from different viewpoints may be acquired sequentially.

[0026] For example, as shown in Figure 2, camera 100 faces the user directly and captures the golf swing motion from the front of the user. Alternatively, as shown in Figure 3, camera 100 may capture the golf swing motion from behind the user. "Behind the user" means behind the user when the direction of the ball flight path assumed by the user (hereinafter referred to as the "virtual ball flight path") is considered "forward". Camera 100 may also acquire both video footage representing the golf swing motion from the front of the user and video footage representing the golf swing motion from behind the user. In other words, camera 100 may acquire video footage representing the user's actions from one viewpoint, or it may acquire video footage representing the user's actions from multiple viewpoints.

[0027] In Figure 1, the camera 100 and the user terminal 200 are drawn separately, but the camera 100 may be built into the user terminal 200 or it may be provided separately from the user terminal 200. In the latter case, the output of the camera 100 (i.e., video data) may be received by the user terminal 200 via communication through the communication interface 206 described later, or it may be received by the user terminal 200 via the recording medium 201A described later.

[0028] The user terminal 200 is a terminal device used by the user. The user terminal 200 may be a terminal device placed in, for example, a golf lesson facility or shop, or it may be a terminal device owned by the user.

[0029] The user terminal 200 is, for example, a portable terminal device, i.e., a mobile terminal. A mobile terminal is, for example, a smartphone, a tablet device, or a laptop PC (Personal Computer). Alternatively, the user terminal 200 may be a stationary terminal device. A stationary terminal device is, for example, a desktop PC.

[0030] The user terminal 200 is connected to the information processing device 300 in a communicative manner via a predetermined communication line. The predetermined communication line includes, for example, a local area network (LAN). The predetermined communication line may also include a wide area network (WAN). The wide area network includes, for example, the internet. The wide area network may also include a mobile communication network with base stations as its endpoints or a satellite communication network utilizing communication satellites. The predetermined communication line may also include, for example, WiFi, Bluetooth®, or local 5G®. th This may include short-range communication lines using specified wireless communication standards such as Generation.

[0031] The user terminal 200 captures video images representing the user's swing motion from the camera 100 and transmits them to the information processing device 300. The user terminal 200 then presents the user with evaluation metrics representing the user's swing motion, which are returned by the information processing device 300.

[0032] The information processing device 300 acquires information representing the movement state of a predetermined body part during the user's swing, based on a video image of the user's swing received from the user terminal 200. The information processing device 300 then returns the acquired information representing the movement state of the predetermined body part during the user's swing to the user terminal 200.

[0033] The information processing device 300 is, for example, a server device with relatively high processing power. The server device may be a cloud server, an on-premise server, or an edge server. Depending on the required processing power, the information processing device may also be a terminal device with lower processing power than the server device. The terminal device may be a stationary terminal device or a portable terminal device (mobile terminal).

[0034] [Configuration of the Information Processing System] In addition to Figure 1, the configuration of the information processing system 1 will be explained with reference to Figures 4 and 5.

[0035] <User terminal configuration> Figure 4 is a block diagram showing an example of the configuration of user terminal 200.

[0036] The functions of the user terminal 200 can be realized by any hardware or any combination of hardware and software. For example, as shown in Figure 4, the user terminal 200 includes an external interface 201, an auxiliary storage device 202, a memory device 203, a CPU 204, a communication interface 206, an input device 207, a display device 208, and an audio output device 209. These components are connected by bus B2. Also, as described above, if the camera 100 is built into the user terminal 200, for example, the camera 100 is connected to bus B2, just like the other components.

[0037] The external interface 201 functions as an interface for reading data from and writing data to the recording medium 201A. The recording medium 201A includes, for example, flexible disks, CDs (Compact Discs), DVDs (Digital Versatile Discs), BDs (Blu-ray® Discs), SD memory cards, and USB (Universal Serial Bus) memory. This allows the user terminal 200 to read various data used in processing through the recording medium 201A, store it in the auxiliary storage device 202, and install programs that implement various functions.

[0038] Furthermore, the user terminal 200 may acquire various data and programs for processing from an external device (for example, an information processing device 300) via the communication interface 206.

[0039] The auxiliary storage device 202 stores various installed programs, as well as files and data necessary for various processes. The auxiliary storage device 202 includes, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or flash memory.

[0040] When a program startup command is received, the memory device 203 reads the program from the auxiliary storage device 202 and stores it. The memory device 203 includes, for example, DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory).

[0041] The CPU 204 executes various programs loaded from the auxiliary storage device 202 into the memory device 203, and implements various functions related to the user terminal 200 according to the programs.

[0042] The communication interface 206 is used as an interface for communicating with external devices. This allows the user terminal 200 to acquire video data from the camera 100 through the communication interface 206. Furthermore, the user terminal 200 can communicate with external devices, such as an information processing device 300, through the communication interface 206. The communication interface 206 may also have multiple types of communication interfaces to suit the communication method between the connected devices.

[0043] The input device 207 receives various inputs from the user.

[0044] The input device 207 includes, for example, an input device that accepts mechanical input from a user (hereinafter referred to as "mechanical input device"). The mechanical input device includes, for example, buttons, toggles, levers, a touch panel implemented on the display device 208, a touchpad provided separately from the display device 208, a keyboard, a mouse, etc.

[0045] Furthermore, the input device 207 may include a voice input device capable of receiving voice input from the user. The voice input device may include, for example, a microphone capable of picking up the user's voice.

[0046] Furthermore, the input device 207 may include a gesture input device capable of receiving gesture input from the user. The gesture input device may include, for example, a camera capable of capturing images of the user's gestures.

[0047] Furthermore, the input device 207 may include a biometric input device capable of receiving biometric input from the user. The biometric input device may include, for example, a camera capable of acquiring image data containing information about the user's fingerprints or iris.

[0048] The display device 208 visually conveys information to the user by displaying information screens, operation screens, etc. The display device 208 is, for example, a liquid crystal display or an organic EL (electroluminescence) display.

[0049] The sound output device 209 conveys various information audibly to the user of the user terminal 200 by outputting a predetermined sound or voice. The sound output device 209 may be, for example, a buzzer, alarm, speaker, etc.

[0050] <Configuration of the information processing device> Figure 5 is a block diagram showing an example of the configuration of the information processing device 300.

[0051] The functions of the information processing device 300 may be realized by any hardware or any combination of hardware and software. For example, as shown in Figure 5, the information processing device 300 includes an external interface 301, an auxiliary storage device 302, a memory device 303, a CPU 304, a high-speed arithmetic unit 305, a communication interface 306, an input device 307, a display device 308, and a sound output device 309. These components are connected by bus B3.

[0052] The external interface 301 functions as an interface for reading data from and writing data to the recording medium 301A. The recording medium 301A includes, for example, a flexible disk, CD, DVD, BD, SD memory card, USB memory, etc. This allows the information processing device 300 to read various data used in processing through the recording medium 301A, store it in the auxiliary storage device 302, and install programs that realize various functions.

[0053] Furthermore, the information processing device 300 may acquire various data and programs for processing from external devices via the communication interface 306.

[0054] The auxiliary storage device 302 stores various installed programs, as well as files and data necessary for various processes. The auxiliary storage device 302 includes, for example, an HDD, SSD, or flash memory.

[0055] When a program startup command is received, the memory device 303 reads the program from the auxiliary storage device 302 and stores it. The memory device 303 includes, for example, DRAM or SRAM.

[0056] The CPU 304 executes various programs loaded from the auxiliary storage device 302 into the memory device 303, and implements various functions related to the information processing device 300 according to the programs.

[0057] The high-speed computing unit 305 works in conjunction with the CPU 304 and performs calculations on the CPU 304 at a relatively high speed. The high-speed computing unit 305 includes, for example, a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array).

[0058] Furthermore, the high-speed arithmetic unit 305 may be omitted depending on the required processing speed for the information processing device 300.

[0059] The communication interface 306 is used as an interface for connecting to external devices in a communicative manner. This allows the information processing device 300 to communicate with external devices, such as a user terminal 200, through the communication interface 306. Furthermore, the communication interface 306 may have multiple types of communication interfaces to suit the communication method between the connected devices.

[0060] The input device 307 receives various inputs from workers, etc.

[0061] The input device 307 includes, for example, a mechanical input device that accepts mechanical operation input from an operator or the like. The mechanical input device includes, for example, buttons, toggles, levers, a touch panel mounted on the display device 308, a touchpad, keyboard, mouse, etc., provided separately from the display device 308.

[0062] Furthermore, the input device 307 includes, for example, a voice input device capable of receiving voice input from a worker or the like. The voice input device includes, for example, a microphone capable of collecting the voice of a worker or the like.

[0063] Furthermore, the input device 307 includes, for example, a gesture input device capable of receiving gesture input from a worker or the like. The gesture input device includes, for example, a camera capable of capturing images of the worker's or the like's gestures.

[0064] Furthermore, the input device 307 includes, for example, a biometric input device capable of receiving biometric input from a worker or the like. The biometric input device includes, for example, a camera capable of acquiring image data containing information about the worker's fingerprints or iris.

[0065] The display device 308 visually conveys various information to workers by displaying information screens and operation screens. The display device 308 is, for example, a liquid crystal display or an organic EL display.

[0066] The sound output device 309 transmits various information by sound to operators of the information processing device 300. The sound output device 309 may be, for example, a buzzer, alarm, or speaker.

[0067] [Functional Configuration of Information Processing Systems] Referring to Figures 6 to 8, a first example of the functional configuration of the information processing system 1 will be described.

[0068] Figure 6 is a functional block diagram showing an example of the functional configuration of information processing system 1. Figure 7 is a diagram illustrating an example of 2D motion data. Figure 8 is a diagram illustrating an example of evaluation indicators.

[0069] Specifically, Figure 7 is a diagram showing feature points representing the user's movements for each of multiple frames included in a video showing the user's golf swing motion. In Figure 7, the data 70 of feature points (black circles in the figure) representing the user's movements for each of multiple frames in a video showing the user's golf swing motion are schematically visualized. In Figure 7, the feature point data 71 to 78 for each frame of a portion of all frames (specifically, frames 21 to 28) of the video 20 in Figure 2 are shown as an excerpt. Data 71 represents the feature point data corresponding to frame 21, which shows the address state in the user's golf swing motion. Data 72 represents the feature point data corresponding to frame 22, which shows the take-back state in the user's golf swing motion. Data 73 represents the feature point data corresponding to frame 23, which shows the backswing state in the user's golf swing motion. Data 74 represents the feature point data corresponding to frame 24, which shows the top state in the user's golf swing motion. Data 75 represents the feature point data corresponding to frame 25, which shows the halfway down state in the user's golf swing motion. Data 76 represents feature point data corresponding to frame 26, which shows the impact state in the user's golf swing. Data 77 represents feature point data corresponding to frame 27, which shows the follow-through state in the user's golf swing. Data 78 represents feature point data corresponding to frame 28, which shows the finish state in the user's golf swing.

[0070] Figure 8 is a graph that shows the cumulative energy of multiple predetermined body parts during the user's swing, from the top of the swing (time t0) to the impact (time t1). In this example, the multiple predetermined body parts are the arms, upper body, and lower body, and the energy is cumulative from bottom to top in the order of lower body, upper body, and arms. The dashed line in Figure 8 represents the time change in the acceleration of the golf club head from the top of the swing to the impact.

[0071] As shown in Figure 6, the user terminal 200 includes an application screen display processing unit 2001, a video data acquisition unit 2002, a video data transmission unit 2003, and a reply data acquisition unit 2004. These functions are realized, for example, when an application program (hereinafter simply referred to as "app") installed in the auxiliary storage device 202 is loaded into the memory device 203 and executed by the CPU 204.

[0072] The application screen display processing unit 2001 displays the application-related screen (hereinafter referred to as the "application screen") on the display device 208.

[0073] The video data acquisition unit 2002 acquires video data representing the user's swing motion from the camera 100. For example, the video data acquisition unit 2002 acquires video data representing the user's swing motion from the camera 100 in response to user input on a predetermined application screen using the input device 207. In this case, the video data acquisition unit 2002 may acquire video images that have already been captured by the camera 100, or it may acquire the latest video images being acquired (captured) by the camera 100 in real time. The video data includes two-dimensional video data. In addition to the two-dimensional video data, the video data may also include information about the depth direction of the two-dimensional video.

[0074] The video data transmission unit 2003 transmits video data representing the user's swinging motion, acquired by the video data acquisition unit 2002, to the information processing device 300 via the communication interface 206. For example, the video data transmission unit 2003 transmits video data acquired from the camera 100 to the information processing device 300 in response to user input on a predetermined application screen using the input device 207.

[0075] The reply data acquisition unit 2004 acquires reply data received from the information processing device 300, which includes evaluation indicators representing the user's swing motion.

[0076] The evaluation index representing the user's swing motion included in the reply data is the evaluation index representing the user's swing motion as seen in the video transmitted to the information processing device 300 by the video image data transmission unit 2003.

[0077] The application screen display processing unit 2001 displays an application screen on the display device 208 that includes evaluation indicators representing the user's swing motion, which are included in the reply data acquired by the reply data acquisition unit 2004. This allows the user to easily understand the characteristics of their own swing motion by checking the evaluation indicators that represent their own swing motion.

[0078] Furthermore, the application screen display processing unit 2001 may display an application screen on the display device 208 that includes support information for improving the user's swing motion, based on evaluation indicators representing the user's swing motion included in the reply data acquired by the reply data acquisition unit 2004.

[0079] As shown in Figure 6, the information processing device 300 includes a motion image data acquisition unit 3001, a two-dimensional motion data acquisition unit 3002, an event timing acquisition unit 3003, a three-dimensional motion data acquisition unit 3004, a motion information acquisition unit 3005, an evaluation timing acquisition unit 3006, an evaluation index extraction unit 3007, and a reply data transmission unit 3008. These functions are realized, for example, by loading a program installed in the auxiliary storage device 302 into the memory device 303 and executing it on the CPU 304.

[0080] The video data acquisition unit 3001 acquires video data representing the user's swinging motion, which is received from the user terminal 200.

[0081] The 2D motion data acquisition unit 3002 acquires observational data (hereinafter referred to as "2D motion data") that represents the user's swing motion in two dimensions, based on the motion data (specifically, 2D motion data) acquired by the motion image data acquisition unit 3001. The 2D motion data is a time-series collection of data that represents the user's motion state in two dimensions at each point in time corresponding to each frame of the motion image data. Hereinafter, the data included in the 2D motion data that represents the user's motion state in two dimensions at each point in time may be conveniently referred to as "2D time point data". The data that represents the user's motion state in two dimensions at each point in time (2D time point data) includes, for example, data that represents the position of the user's body parts or parts of the tools (striking implements) held by the user in two dimensions, and data that represents the posture angle in two dimensions.

[0082] Furthermore, if the video data acquisition unit 3001 acquires video data from multiple different viewpoints, the 2D motion data acquisition unit 3002 acquires 2D motion data for each of the multiple video images corresponding to the multiple viewpoints.

[0083] For example, as shown in Figure 7, the 2D motion data acquisition unit 3002 uses known image processing techniques to extract multiple feature points related to the swinging motion of the subject (user) for each frame of the video. Known image processing techniques include, for example, image recognition techniques for detecting (estimating) skeletal information of people and objects in an image, such as so-called bone detection and bone estimation.

[0084] The multiple feature points include, for example, feature points representing the user's body parts as a subject in the frame image (hereinafter referred to as "physical feature points"). Physical feature points represent, for example, the joint positions in the user's skeleton, and the physical feature points included in the multiple feature points include points on the image corresponding to the head, shoulders, elbows, wrists, hips, knees, ankles, etc. As a result, the 2D motion data acquisition unit 3002 can acquire 2D motion data that represents the position and posture angle of the user's body parts in two dimensions.

[0085] Furthermore, the multiple feature points may include feature points representing parts of the tool held by the user. For example, as shown in Figure 7, feature points corresponding to the head of the club held by the user are extracted. This allows the 2D motion data acquisition unit 3002 to acquire 2D motion data that represents the position and orientation angle of the parts of the tool held by the user in two dimensions.

[0086] Furthermore, multiple feature points may include feature points that represent the ball being struck by the striking tool.

[0087] Furthermore, the 2D motion data acquisition unit 3002 may adjust the coordinate system and scale of the coordinate system for the 2D motion data. For example, the 2D motion data acquisition unit 3002 acquires 2D motion data based on the person being photographed (i.e., the user) and assuming a coordinate system that is in line with the scale of the real world.

[0088] The event timing acquisition unit 3003 acquires the timing of a predetermined event within the time series of 2D motion data based on the 2D motion data acquired by the 2D motion data acquisition unit 3002. If the 2D motion data acquisition unit 3002 acquires 2D motion data from multiple viewpoints, the event timing acquisition unit 3003 acquires the timing of a predetermined event within the time series of the target 2D motion data for each of the multiple 2D motion data.

[0089] A specified event refers to a specific event in a swing motion using a striking tool. For example, specified events in a golf swing motion include address, take-back, backswing, top, halfway down, impact, and follow-through. The specified event for which timing is acquired within the time series of 2D motion data may be one or multiple events.

[0090] The event timing acquisition unit 3003 acquires the timing of a predetermined event, for example, by estimating a 2D time point data corresponding to a predetermined event from among the 2D time point data for each time point included in the 2D motion data. In this case, the event timing acquisition unit 3003 selectively estimates the timing of a predetermined event from a discrete time system defined by each 2D time point data included in the 2D motion data. Alternatively, the event timing acquisition unit 3003 may treat the time series of the 2D motion data as a continuous time system and estimate the timing of a predetermined event within that system. In this case, the timing of a predetermined event may be estimated as the timing between each time point corresponding to adjacent 2D time point data.

[0091] The event timing acquisition unit 3003 can acquire the timing of a predetermined event within the time series of two-dimensional motion data using a known method.

[0092] For example, the event timing acquisition unit 3003 acquires the timing of a predetermined event by estimating it based on the similarity between the two-dimensional motion data at each point in time and reference two-dimensional motion data (reference motion data) corresponding to a predetermined event (see Japanese Patent Publication No. 2024-108340).

[0093] Furthermore, the event timing acquisition unit 3003 may estimate the timing of a predetermined event based on whether a physical quantity representing the operating state of a specific feature point, obtained from each 2D time point data included in the 2D motion data, satisfies the conditions corresponding to a predetermined event. For example, at the top of a golf swing, the speed of the golf club head becomes zero. Therefore, the event timing acquisition unit 3003 can acquire the event timing of the top of the swing by extracting 2D time point data from each 2D time point data included in the 2D motion data that satisfies the conditions under which it can be determined that the speed of the feature point corresponding to the club head is zero.

[0094] Furthermore, the operator may use the input device 307 to input a frame corresponding to a predetermined event from among all the frames of the video while viewing the video displayed on the display device 308. In this case, the event timing acquisition unit 3003 can acquire the timing of a predetermined event in response to the predetermined input from the operator via the input device 307.

[0095] Furthermore, the user may use the input device 207 to input a frame corresponding to a predetermined event from among all the frames of the video while viewing the video displayed on the display device 208. In this case, the event timing acquisition unit 3003 can acquire the timing of a predetermined event in response to the predetermined input from the user via the input device 207.

[0096] The 3D motion data acquisition unit 3004 acquires observational data (hereinafter referred to as "3D motion data") that represents the user's swing motion in three dimensions, corresponding to the video data acquired by the video data acquisition unit 3001.

[0097] Three-dimensional motion data is a time-series collection of data that represents the user's motion state in three dimensions at each point in time. Hereafter, the data representing the user's motion state at each point in time included in the three-dimensional motion data may be conveniently referred to as "three-dimensional time point data." Three-dimensional motion data, for example, as with two-dimensional motion data, represents the three-dimensional positions of multiple feature points related to the user's swing motion as seen in the video at multiple point in time.

[0098] For example, the 3D motion data acquisition unit 3004 acquires 3D motion data representing the user's swing motion based on multiple 2D motion data corresponding to multiple viewpoints. Specifically, the 3D motion data acquisition unit 3004 can acquire 3D motion data by estimating the 3D position of multiple feature points by matching the same type of feature points in multiple 2D time point data at the same timing with different viewpoints.

[0099] The motion information acquisition unit 3005 acquires information (motion information) representing the motion state of a predetermined body part in the user's swing motion, as captured in the video image data acquired by the video image data acquisition unit 3001, based on the 3D motion data acquired by the 3D motion data acquisition unit 3004, which is the 3D motion data of the user as the subject of the motion image. Specifically, the motion information acquisition unit 3005 acquires motion information of a predetermined body part in the user's swing motion for each of the multiple 3D time point data included in the 3D motion data. The motion information acquisition unit 3005 includes a velocity / angular velocity information acquisition unit 3005A, a mass / moment of inertia information acquisition unit 3005B, and an energy information acquisition unit 3005C.

[0100] The velocity / angular velocity information acquisition unit 3005A acquires information representing the velocity (velocity information) and information representing the angular velocity (angular velocity information) of a predetermined body part for each of the multiple 3D time point data included in the 3D motion data. For example, the velocity / angular velocity information acquisition unit 3005A calculates the velocity and angular velocity of a predetermined body part based on the changes in the 3D position and orientation angle of the predetermined body part between the target 3D time point data and other 3D time point data adjacent in the time series. In this way, the velocity / angular velocity information acquisition unit 3005A can acquire the velocity information and angular velocity information of a predetermined body part.

[0101] Furthermore, the speed / angular velocity information acquisition unit 3005A may acquire information representing the speed of the striking tool (speed information) and information representing the angular velocity (angular velocity information) for each of the multiple 3D time point data included in the 3D motion data. For example, the speed / angular velocity information acquisition unit 3005A calculates the speed and angular velocity of the golf club head based on the changes in the 3D position and orientation angle of the golf club head between the target 3D time point data and other 3D time point data adjacent in the time series. In this way, the speed / angular velocity information acquisition unit 3005A can acquire the speed information and angular velocity information of the golf club head.

[0102] The mass and moment of inertia information acquisition unit 3005B acquires information representing the mass of a predetermined body part (mass information) and information representing the moment of inertia (moment of inertia information).

[0103] For example, the mass of a given body part is a predetermined fixed value. In this case, the mass / moment of inertia information acquisition unit 3005B can acquire mass information by reading the mass value that has been pre-registered in, for example, the auxiliary storage device 302.

[0104] Similarly, for example, the moment of inertia of a given body part is a predetermined fixed value. In this case, the mass / moment of inertia information acquisition unit 3005B can acquire moment of inertia information by reading the value of the moment of inertia that is pre-registered in, for example, the auxiliary storage device 302.

[0105] Furthermore, the mass / moment of inertia information acquisition unit 3005B may acquire mass information of a predetermined body part by calculating the size of a predetermined body part based on the positional information of multiple feature points corresponding to 3D motion data, and estimating the mass of the predetermined body part based on the calculated size. The size of a predetermined body part is, for example, the dimension (link length) between adjacent feature points, and the mass / moment of inertia information acquisition unit 3005B may acquire mass information of a predetermined body part by multiplying the link length by a certain coefficient predetermined in a manner corresponding to the predetermined body part.

[0106] Similarly, the mass / moment of inertia information acquisition unit 3005B may acquire moment of inertia information for a predetermined body part by calculating the size of a predetermined body part based on the positional information of multiple feature points corresponding to 3D motion data, and estimating the moment of inertia of the predetermined body part based on the calculated size. For example, the mass / moment of inertia information acquisition unit 3005B may acquire moment of inertia information for a predetermined body part by multiplying the link length by a predetermined coefficient that corresponds to the predetermined body part.

[0107] Furthermore, the mass / moment of inertia information acquisition unit 3005B may acquire mass information of a predetermined body part using a predetermined conversion formula or the like, based on information representing height and weight input from the user through the input device 207 of the user terminal 200.

[0108] Similarly, the mass / moment of inertia information acquisition unit 3005B may acquire moment of inertia information for a predetermined body part using a predetermined conversion formula or the like, based on information representing height and weight input from the user through the input device 207 of the user terminal 200.

[0109] The energy information acquisition unit 3005C acquires information (energy information) representing the energy of a predetermined body part, specifically the energy generated (exerted) by the predetermined body part, for each of the multiple three-dimensional time point data included in the three-dimensional motion data.

[0110] The energy information acquisition unit 3005C can acquire energy information for a predetermined body part by calculating the energy of that body part using a known method.

[0111] For example, the energy information acquisition unit 3005C acquires information representing the energy of a given body part (energy information) by calculating and summing the translational kinetic energy Etrans and rotational kinetic energy Erot of the given body part.

[0112] The energy information acquisition unit 3005C calculates the translational kinetic energy Etrans using the following equation (1).

[0113]

number

[0114] Mass m is the mass of a given body part. Information representing mass m is acquired by the mass / moment of inertia information acquisition unit 3005B. Velocity vx, velocity vy, and velocity vz are the velocity components of the given body part in the x-axis, y-axis, and z-axis directions of the global coordinate system, respectively. Information representing velocity vx, velocity vy, and velocity vz is acquired by the velocity / angular velocity information acquisition unit 3005A.

[0115] Furthermore, the energy information acquisition unit 3005C calculates the rotational kinetic energy Erot using the following equations (2) and (3).

[0116]

number

[0117] The moment of inertia I is a tensor of the moments of inertia around the principal axes of inertia. Moments of inertia Ixx, Iyy, and Izz are the principal moments of inertia with respect to the x, y, and z axes of the global coordinate system. Information representing moments of inertia Ixx, Iyy, and Izz is acquired by the mass / moment of inertia information acquisition unit 3005B. Angular velocities ωx, ωy, and ωz are the angular velocity components of a given body part around the x, y, and z axes of the global coordinate system, respectively. Angular velocities ωx, ωy, and ωz are acquired by the mass / moment of inertia information acquisition unit 3005B.

[0118] The evaluation timing acquisition unit 3006 acquires (extracts) predetermined timings (hereinafter referred to as "evaluation timings") for evaluating the user's swing motion within the time series of three-dimensional motion data. Specifically, the evaluation timing acquisition unit 3006 extracts evaluation timings corresponding to predetermined operating states of the golf club.

[0119] For example, the evaluation timing acquisition unit 3006 extracts the evaluation timing within a discrete time system defined by each 3D time point data included in the 3D motion data. Alternatively, the evaluation timing acquisition unit 3006 may extract the evaluation timing within a continuous time system that includes the time between the 3D time point data.

[0120] For example, the evaluation timing acquisition unit 3006 acquires (extracts) an evaluation timing TM1 in which the degree of increase in the acceleration of the golf club head reaches a predetermined standard, starting from the top state of the user's swing motion.

[0121] The evaluation timing TM1 corresponds to the timing when the golf club head begins to accelerate significantly. The evaluation timing acquisition unit 3006 acquires the timing when the acceleration of the golf club head reaches a predetermined ratio based on its maximum value, starting from the top of the user's swing motion, as the evaluation timing TM1. The predetermined ratio is set slightly lower than the ratio based on the maximum value of the acceleration of the golf club head when the golf club head begins to accelerate significantly, starting from the top of the swing motion. Alternatively, the evaluation timing acquisition unit 3006 may calculate the rate of increase in acceleration (i.e., jerk) of each 3D time point data included in the 3D motion data, and extract the timing when the rate of increase in acceleration reaches a predetermined standard as the evaluation timing TM1.

[0122] Furthermore, for example, the evaluation timing acquisition unit 3006 acquires (extracts) the evaluation timing TM2 in which the acceleration of the golf club head is maximum during the motion interval between the top of the user's swing motion and impact.

[0123] The evaluation timing acquisition unit 3006 calculates the golf club head position for each 3D time point in the 3D motion data based on the velocity and angular velocity information acquisition unit 3005A. This allows the evaluation timing acquisition unit 3006 to extract evaluation timings TM1 and TM2 within the time series of 3D motion data corresponding to the motion interval from the top of the swing to the impact. Hereinafter, the period from the top of the swing to evaluation timing TM2 may be conveniently referred to as the "acceleration phase," and the period from evaluation timing TM2 to the impact of the swing may be conveniently referred to as the "release phase."

[0124] The evaluation index extraction unit 3007 extracts evaluation indexes representing the user's swing motion based on energy information corresponding to each 3D time point data acquired by the energy information acquisition unit 3005C.

[0125] For example, as shown in Figure 8, the evaluation index extraction unit 3007 extracts the maximum value EG1 of the sum of energy of the arm, upper body, and lower body parts during the period from time t0 to time t1 as the evaluation index EI1. The evaluation index EI1 is an evaluation index that represents the energy of the entire body during the user's swing motion.

[0126] Furthermore, as shown in Figure 8, for example, the evaluation index extraction unit 3007 extracts the maximum value EG2 of the energy of the body parts excluding the arms during the period from time t0 to time t2, which corresponds to the evaluation timing TM1, as the evaluation index EI2. The energy of the body parts excluding the arms corresponds to the sum of the energy of the upper body and the lower body. The evaluation index EI2 is an evaluation index that represents how the user uses the body parts excluding the arms (i.e., the upper body and the lower body) during the acceleration phase of the downswing.

[0127] Furthermore, the evaluation index extraction unit 3007 may, in place of or in addition to the evaluation index EI2, extract the energy of body parts other than the arms at time t2 corresponding to the evaluation timing TM1.

[0128] Furthermore, as shown in Figure 8, for example, the evaluation index extraction unit 3007 extracts the energy value EG3 of the arm part at time t3, which corresponds to the evaluation timing TM2, as the evaluation index EI3. The evaluation index EI3 is an evaluation index that represents how the user uses the arm part during the acceleration phase of the downswing.

[0129] Furthermore, the evaluation index extraction unit 3007 may, in place of or in addition to the evaluation index EI3, extract the maximum energy value of the arm region during the period from time t0 to time t3, which corresponds to the evaluation timing TM2.

[0130] Furthermore, as shown in Figure 8, for example, the evaluation index extraction unit 3007 extracts the degree of decrease from the maximum value EG2 of the energy of the body parts excluding the arms during the period from time t0 to time t1 to the value EGBimp at time t1 as the evaluation index EI4. The evaluation index EI4 is an evaluation index that represents how the user uses the body parts excluding the arms (i.e., the upper body and lower body) during the release phase of the downswing. For example, the evaluation index EI4 is the value obtained by dividing the energy value EGBimp of the body parts excluding the arms at time t1 by the maximum value EG2 (i.e., EI4 = EGBimp / EG2).

[0131] Furthermore, as shown in Figure 8, for example, the evaluation index extraction unit 3007 extracts the degree of decrease from the maximum energy value EGAmax of the arm region during the period from time t0 to time t1 to the value EGAimp at time t1 as the evaluation index EI5. The evaluation index EI5 is an evaluation index that represents how the user uses the arm region during the release phase of the downswing.

[0132] The evaluation index extraction unit 3007 may extract all of the evaluation indices EI1 to EI5, or it may extract only some of them.

[0133] Furthermore, the evaluation index extraction unit 3007 may extract other types of evaluation indexes in place of or in addition to the evaluation indexes EI1 to EI5.

[0134] For example, as shown in Figure 8, the evaluation index extraction unit 3007 may extract other types of evaluation indices, such as the timing when the energy of body parts excluding the arms is at its maximum (time t4) and the timing when the energy of the arms is at its maximum (time t5) during the period from time t0 to time t1. This is because, in a golf downswing, it is desirable that energy peaks occur sequentially from the parts of the human skeleton furthest from the golf club, ensuring that energy is properly transferred to the golf club. In other words, it is desirable that the timing when the energy of the arms is at its maximum is later than the timing when the energy of body parts excluding the arms is at its maximum, and the example in Figure 8 represents a relatively desirable swing motion.

[0135] The reply data transmission unit 3008 transmits reply data containing information on evaluation indicators representing the user's swing motion, extracted by the evaluation indicator extraction unit, to the user terminal 200 via the communication interface 206. This allows the user terminal 200 to display the evaluation indicators representing the user's swing motion on the display device 208, or to display support information based on the evaluation indicators (see Figure 9).

[0136] [Specific examples of app screens] Referring to Figure 9, a specific example of an application screen displayed on the display device 208 of the user terminal 200 will be explained.

[0137] Figure 9 shows an example of an application screen (application screen 90).

[0138] As shown in Figure 9, the app screen 90 displays a spider chart representing the evaluation results of the user's swing motion based on evaluation indices EI1 to EI5.

[0139] The spider chart shows the evaluation results for each of the evaluation axes from 91 to 95.

[0140] Evaluation axis 91 is labeled with the text information "Whole Body Energy" and corresponds to the evaluation index EI1 mentioned above. Evaluation axis 92 is labeled with the text information "Arm Usage (Acceleration Phase)" and corresponds to the evaluation index EI3 mentioned above. Evaluation axis 93 is labeled with the text information "Body Usage (Acceleration Phase)" and corresponds to the evaluation index EI2 mentioned above. Evaluation axis 94 is labeled with the text information "Body Usage (Release Phase)" and corresponds to the evaluation index EI4 mentioned above. Evaluation axis 95 is labeled with the text information "Arm Usage (Release Phase)" and corresponds to the evaluation index EI5 mentioned above.

[0141] In this way, the application screen display processing unit 2001 displays an application screen 90 that shows the evaluation result of the swing motion based on the evaluation index, based on the information of the evaluation index included in the reply data received from the information processing device 300. This allows the user to relatively easily understand the characteristics of their own swing motion and to relatively easily consider areas for improvement in their swing motion.

[0142] [Specific examples of how information processing systems work] Referring to Figure 10, a specific example of the operation of the information processing system 1 according to this embodiment will be described.

[0143] Figure 10 is a sequence diagram schematically showing an example of the operation of the information processing system 1.

[0144] As shown in Figure 10, the user terminal 200 launches an application in response to a predetermined input from the user using the input device 207 (step S102).

[0145] After the completion of the process in step S102, the motion image data acquisition unit 2002 acquires motion image data from the camera 100 that represents the user's swinging motion in response to a predetermined input from the user on a predetermined application screen using the input device 207 (step S104).

[0146] After the completion of the process in step S104, the video data transmission unit 2003 transmits the video data to the information processing device 300 via the communication interface 206 (step S106).

[0147] The video data acquisition unit 3001 acquires the video data transmitted (uploaded) from the user terminal 200 in the process of step S106 (step S108).

[0148] After the completion of the process in step S108, the 2D motion data acquisition unit 3002 acquires 2D motion data by extracting feature points representing the user's swinging motion from each frame of the motion data (step S110).

[0149] After the completion of the process in step S110, the event timing acquisition unit 3003 acquires the timing of events for each of the two viewpoints' 2D motion data (step S112).

[0150] After the completion of the process in step S112, the 3D motion data acquisition unit 3004 compares the timing of events acquired for the 2D motion data from the two viewpoints and aligns the time-series 2D time point data of the 2D motion data from the two viewpoints (step S114). After the completion of the process in step S114, the 3D motion data acquisition unit 3004 acquires 3D motion data based on the 2D motion data of the two viewpoints that have been aligned in the process of step S114 (step S116).

[0151] After the processing in step S116 is completed, the velocity / angular velocity information acquisition unit 3005A acquires velocity information and angular velocity information for a predetermined body part for each 3D time point data included in the 3D motion data, based on the 3D motion data acquired in the processing in step S116 (step S118).

[0152] After the completion of the process in step S118, the mass and moment of inertia information acquisition unit 3005B acquires mass information and moment of inertia information of a predetermined body part of the user that is captured in the moving image data (step S120).

[0153] After the completion of the process in step S120, the energy information acquisition unit 3005C acquires energy information for a predetermined body part for each 3D time point data included in the 3D motion data (step S122).

[0154] After the completion of the process in step S122, the evaluation timing acquisition unit 3006 acquires (extracts) the evaluation timing from the time series of the 3D motion data (step S124).

[0155] After the completion of the process in step S124, the evaluation index extraction unit 3007 extracts evaluation indices representing the user's swing motion based on the energy information corresponding to each 3D time point data acquired in step S122 and the evaluation timing information acquired in step S124 (step S126).

[0156] After the processing in step S126 is completed, the reply data transmission unit 3008 sends reply data containing the evaluation indicator information extracted in the processing in step S126 to the user terminal 200 (step S128).

[0157] The reply data acquisition unit 2004 acquires the reply data sent from the information processing device 300 in the process of step S128 (step S130).

[0158] After the completion of the process in step S130, the application screen display processing unit 2001 displays a predetermined application screen (for example, see Figure 9) on the display device 208 based on the evaluation index information contained in the reply data (step S132).

[0159] [Other embodiments] Other embodiments will be described.

[0160] The embodiments described above may be modified or altered as appropriate. Hereinafter, examples of modifications or alterations to the embodiments described above will be referred to as "modified versions" for convenience.

[0161] For example, in the above-described embodiment, the functions of the user terminal 200 and the information processing device 300 may be implemented by a single information processing device, or they may be implemented in a distributed manner by three or more information processing devices. For example, the functions of the information processing device 300 are integrated into the user terminal 200. Alternatively, for example, the functions of the information processing device 300 may be implemented by a first information processing device having the functions of a moving image data acquisition unit 3001, a two-dimensional motion data acquisition unit 3002, an event timing acquisition unit 3003, and a three-dimensional motion data acquisition unit 3004; a second information processing device having the function of an motion information acquisition unit 3005; and a third information processing device having the functions of an evaluation timing acquisition unit 3006, an evaluation index extraction unit 3007, and a reply data transmission unit 3008.

[0162] Furthermore, in the embodiments and their modifications described above, the event timing acquisition unit 3003 may estimate the timing of a predetermined event based on measurement data from a motion sensor attached to the user's body when the video data was acquired. For example, the event timing acquisition unit 3003 estimates the timing of a predetermined event based on whether a physical quantity (e.g., velocity or acceleration) representing the operating state of a specific part of the user's body, obtained based on the output of the motion sensor, satisfies the conditions corresponding to the predetermined event.

[0163] Furthermore, in the embodiments and their modifications described above, the 3D motion data acquisition unit 3004 acquires 3D motion data of the user's swing motion as seen in the video footage based on the video footage. However, 3D motion data may be acquired by other methods. For example, the 3D motion data acquisition unit 3004 may acquire 3D motion data of the user's swing motion as seen in the video footage using motion capture technology. In this case, the user performs the swing motion with multiple motion capture markers attached to their body. Alternatively, the 3D motion data acquisition unit 3004 may acquire 3D motion data representing the user's swing motion using depth information from a 2D video.

[0164] Furthermore, in the embodiments and their modifications described above, the motion information acquisition unit 3005 acquires motion information of a predetermined body part in the user's swing motion based on three-dimensional motion data, but motion information of a predetermined body part may be acquired by other methods. For example, the motion information acquisition unit 3005 acquires motion information of a predetermined body part in the user's swing motion based on measurement data from a motion sensor attached to a predetermined body part of the user when the motion image data was acquired.

[0165] [Effect] The operation of the information processing method, program, and information processing device according to this embodiment will be described.

[0166] In a first aspect of this embodiment, an information processing method is provided, which includes a first acquisition step and a second extraction step. The first acquisition step is, for example, step S122 described above. The first extraction step is, for example, step S126 described above. Specifically, in the first acquisition step, the information processing device acquires information representing the energy of a predetermined body part in the swing motion of a golf club performed by a subject. Then, in the first extraction step, the information processing device extracts an index representing the subject's swing motion based on the information representing the energy of the predetermined body part in the swing motion. The information processing device is, for example, the information processing device 300 described above.

[0167] Furthermore, in the first aspect of this embodiment, a program may be provided to cause the information processing device to perform the first acquisition step and the first extraction step.

[0168] Furthermore, in a first aspect of this embodiment, an information processing device is provided that includes a first acquisition unit and a first extraction unit. Specifically, the first acquisition unit is, for example, the energy information acquisition unit 3005C described above. The first extraction unit is, for example, the evaluation index extraction unit 3007 described above. Specifically, the first acquisition unit acquires information representing the energy of a predetermined body part during a golf club swing motion performed by a subject. The first extraction unit then extracts an index representing the subject's swing motion based on the energy of the predetermined body part during the swing motion.

[0169] This allows the user of the concession processing device to understand the characteristics of the subject's swing motion based on indicators representing the subject's swing motion. Therefore, the information processing device can generate information that supports the easy understanding of the characteristics of the subject's golf swing motion.

[0170] Furthermore, in a second aspect of this embodiment, based on the first aspect described above, the information processing method may include a second extraction step. The second extraction step is, for example, the evaluation timing acquisition unit 3006 described above. Specifically, in the second extraction step, the information processing device extracts a predetermined timing in the swing motion. Then, in the first extraction step, the index may be extracted based on the information representing the energy of the predetermined body part in the swing motion and the predetermined timing.

[0171] This allows the information processing device to extract indicators that represent the subject's swing motion.

[0172] Furthermore, in a third aspect of this embodiment, based on the second aspect described above, the second extraction step may extract a plurality of predetermined timings, including a first timing and a second timing that occurs after the first timing. In the first extraction step, the degree of energy reduction from the first timing to the second timing in the swing motion may be extracted as the index.

[0173] This allows the information processing device to focus on the trend of energy decrease in a predetermined body part from the first timing to the second timing in the swing motion and extract an index representing the subject's swing motion.

[0174] Furthermore, in a fourth aspect of this embodiment, based on the third aspect described above, the first timing may be the timing when the energy of the predetermined body part is at its maximum. Also, the second timing may be the timing of impact in the swing motion. Also, the predetermined body part may be at least one of the body part excluding the arm and the body part of the arm. In the first extraction step, the degree of energy decrease in at least one of the body part excluding the arm and the body part of the arm from the first timing to the second timing may be extracted as the index.

[0175] This allows the information processing device to focus on the trend of energy decrease in parts of the subject's body that do not include the arms, or in parts of the arms, from the first timing to the second timing in the swing motion, and extract indicators that represent the subject's swing motion.

[0176] Furthermore, in the fifth aspect of this embodiment, assuming any one of the second to fourth aspects described above, the predetermined body part may be a body part that does not include the arms. Also, the predetermined timing may be a third timing in which the degree of increase in the acceleration of the golf club reaches a predetermined standard, starting from the top position in the swing motion. In the first extraction step, the energy in the body part that does not include the arms during the period from the top position in the swing motion to the third timing may be extracted as the index.

[0177] This allows the information processing device to focus on the energy of the subject's body parts, excluding the arms, during the range from the top of the swing motion until the golf club begins to accelerate significantly, and extract an index representing the subject's swing motion.

[0178] Furthermore, in the sixth aspect of this embodiment, assuming any one of the second to fifth aspects described above, the predetermined body part may be the arm. Also, the predetermined timing may be the fourth timing, where the acceleration of the golf club in the swing motion is at its maximum. In the first extraction step, the energy in the arm from the top position in the swing motion to the fourth timing may be extracted as the index.

[0179] This allows the information processing device to focus on the energy of the subject's arm region when the acceleration of the golf club during the subject's swing is at its maximum, and extract an index that represents the subject's swing motion.

[0180] Furthermore, in a seventh aspect of this embodiment, based on the fifth or sixth aspect described above, the first extraction step may obtain the energy at the predetermined part at the predetermined timing in the swing motion, or the maximum value of the energy at the predetermined part from the top position to the predetermined timing in the swing motion.

[0181] This allows the information processing device to extract an index representing the subject's swing motion by focusing on the energy at a predetermined timing during the swing motion, or the maximum energy from the top of the swing to a predetermined timing.

[0182] Furthermore, in the eighth aspect of this embodiment, assuming any one of the first to seventh aspects described above, the first acquisition step may involve acquiring information representing the energy of each predetermined body part during the swing motion for each of the multiple predetermined body parts. Then, in the first extraction step, the maximum value of the sum of the energies for each of the multiple predetermined body parts during the swing motion may be extracted as the index.

[0183] This allows the information processing device to focus on the maximum value of the sum of energy from multiple predetermined body parts during the subject's swing motion and extract an index representing the subject's swing motion.

[0184] Furthermore, in the ninth aspect of this embodiment, assuming any one of the first to eighth aspects described above, the first acquisition step may acquire information representing the energy of the predetermined body part in the swing motion for each of the plurality of predetermined body parts, including a first body part and a second body part that is further away on the human skeleton than the first body part relative to the golf club held in the hand. The first body part is, for example, the arm, and the second body part is the upper body or lower body. Then, in the first extraction step, the timing at which the energy of the first body part in the swing motion is maximized and the timing at which the energy of the second body part is maximized may be extracted as indicators.

[0185] This allows the information processing device to extract an index representing the subject's swing motion by focusing on the difference in timing between the first body part and the second body part, which is further away from the first body part than the first body part in relation to the golf club held in the hand on the person's skeleton, where the energy is maximized.

[0186] Furthermore, in the tenth aspect of this embodiment, the information processing method may include a second acquisition step, based on any one of the first to ninth aspects described above. The second acquisition step is, for example, step S118 described above. Specifically, in the second acquisition step, the information processing device may acquire information representing the velocity and angular velocity of the predetermined body part in the swing motion. Then, in the first acquisition step, based on the information representing the velocity and angular velocity of the predetermined body part in the swing motion, information representing the energy of the predetermined body part in the swing motion may be acquired.

[0187] As a result, the information processing device can acquire information representing the energy of a predetermined body part during a subject's swing motion, based on information representing the velocity and angular velocity of that body part during the swing motion.

[0188] Furthermore, in the eleventh aspect of this embodiment, the information processing method may include a third acquisition step, based on any one of the first to tenth aspects described above. The third acquisition step is, for example, step S110 described above. Specifically, in the third acquisition step, the information processing device may acquire information representing the position of the predetermined body part based on the video image showing the swinging motion performed by the subject. Then, in the first acquisition step, information representing the energy of the predetermined body part in the swinging motion may be acquired based on the information representing the position of the predetermined body part.

[0189] As a result, the information processing device can acquire information representing the energy of a predetermined body part in the subject's swing motion, based on the positional information of that body part in the video footage showing the subject's swing motion.

[0190] Although embodiments have been described in detail above, this disclosure is not limited to these specific embodiments, and various modifications and changes are possible within the scope of the gist described in the claims. [Explanation of symbols]

[0191] 1. Information Processing System 100 Cameras 200 user terminals 300 Information Processing Devices 2001 Application screen display processing unit 2002 Moving Image Data Acquisition Unit 2003 Motion Image Data Transmission Unit 2004 Reply Data Acquisition Unit 3001 Moving Image Data Acquisition Unit 3002 2D motion data acquisition unit 3003 Event Timing Acquisition Unit 3004 3D motion data acquisition unit 3005 Operation information acquisition unit 3005A Velocity / angular velocity information acquisition section 3005B Mass / Moment of Inertia Information Acquisition Unit 3005C Energy Information Acquisition Unit 3006 Evaluation timing acquisition unit 3007 Evaluation Metric Extraction Unit 3008 Reply Data Transmission Section EI1 Evaluation Metrics EI2 Evaluation Metrics EI3 Evaluation Metrics EI4 Evaluation Metrics EI5 Evaluation Metrics

Claims

1. The information processing device acquires information representing the energy of a predetermined body part during the swing motion of a golf club by the subject in the first acquisition step, The information processing device includes a first extraction step of extracting an index representing the swing motion of the subject based on information representing the energy of a predetermined body part in the swing motion, Information processing methods.

2. The information processing device includes a second extraction step of extracting a predetermined timing in the swing motion, In the first extraction step, the index is extracted based on information representing the energy of the predetermined body part in the swing motion and the predetermined timing. The information processing method according to claim 1.

3. In the second extraction step, a plurality of predetermined timings are extracted, including a first timing and a second timing that is later than the first timing. In the first extraction step, the degree of energy decrease from the first timing to the second timing in the swing motion is extracted as the index. The information processing method according to claim 2.

4. The first timing is the timing when the energy of the predetermined body part is at its maximum. The second timing is the timing of impact in the swing motion, The aforementioned predetermined body part is at least one of the part that does not include the arm and the part that includes the arm. In the first extraction step, the degree of energy reduction in at least one of the parts of the body that do not include the arm, and the parts of the arm, from the first timing to the second timing is extracted as the indicator. The information processing method according to claim 3.

5. The aforementioned specified body part is a body part that does not include the arm. The predetermined timing is a third timing in which the degree of increase in the acceleration of the golf club reaches a predetermined standard, starting from the top position in the swing motion. In the first extraction step, the energy in a body part excluding the arms during the swing motion from the top position to the third timing is extracted as the index. The information processing method according to claim 2.

6. The aforementioned designated body part is the arm, The predetermined timing is the fourth timing in which the acceleration of the golf club during the swing motion is at its maximum. In the first extraction step, the energy in the arm region during the swing motion from the top position to the fourth timing is extracted as the index. The information processing method according to claim 2.

7. In the first extraction step, the energy at the predetermined part at the predetermined timing in the swing motion, or the maximum value of the energy at the predetermined part from the top position to the predetermined timing in the swing motion is obtained. The information processing method according to claim 5 or 6.

8. In the first acquisition step, information representing the energy of the predetermined body part in the swing motion is acquired for each of the plurality of predetermined body parts. In the first extraction step, the maximum value of the sum of the energies for each of the predetermined body parts in the swing motion is extracted as the index. The information processing method according to any one of claims 1 to 6.

9. In the first acquisition step, for each of the predetermined body parts, including a first body part and a second body part that is further away from the first body part on the human skeleton than the first body part, information representing the energy of the predetermined body part during the swing motion is acquired for each predetermined body part, which includes the first body part and the golf club held in the hand. In the first extraction step, the timing at which the energy of the first body part is maximized during the swing motion and the timing at which the energy of the second body part is maximized are extracted as indicators. The information processing method according to any one of claims 1 to 6.

10. The information processing device includes a second acquisition step of acquiring information representing the velocity and angular velocity of the predetermined body part in the swing motion, In the first acquisition step, information representing the energy of the predetermined body part in the swing motion is acquired based on information representing the velocity and angular velocity of the predetermined body part in the swing motion. The information processing method according to any one of claims 1 to 6.

11. The information processing device includes a third acquisition step of acquiring information representing the position of a predetermined body part based on a video image showing the swinging motion performed by the subject, In the first acquisition step, information representing the energy of the predetermined body part in the swing motion is acquired based on information representing the position of the predetermined body part. The information processing method according to any one of claims 1 to 6.

12. In an information processing device, A first acquisition step involves obtaining information representing the energy of a specific body part during the golf club swing motion performed by the subject, A first extraction step is performed, in which an index representing the swing motion of the subject is extracted based on information representing the energy of the predetermined body part in the swing motion. program.

13. A first acquisition unit that acquires information representing the energy of a predetermined body part during the swing motion of a golf club by the subject, The system includes a first extraction unit that extracts an index representing the swing motion of the subject based on information representing the energy of a predetermined body part in the swing motion, Information processing device.

Images