Method and system for providing information about the sections that make up a golf swing

Abstract

To provide a technique for making it possible to provide a user with adequate feedback on a golf swing.SOLUTION: According to one aspect of the invention, a method for providing information on segments constituting a golf swing is provided which includes the steps of: acquiring two or more images of a golf swing; and generating information on at least one segment constituting the golf swing with reference to a result of comparing the two or more images.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a method and a system for providing information regarding sections constituting a golf swing.

Background Art

[0002] Recently, with the increase in leisure time, the number of people enjoying golf has been increasing, and various industries related to golf have been growing.

[0003] Golf is a sport in which a user (e.g., a golfer) uses a golf club to send a golf ball to a target position by making a golf swing. Therefore, when analyzing a golf swing and providing appropriate feedback based on the golf swing, it can be helpful for improving golf skills.

[0004] As an example of a conventional method for analyzing a golf swing, there is a method of acquiring an image related to the swing posture of a user making a golf swing using a camera and analyzing the acquired image.

[0005] However, such a method had two major problems. The first is that in the analysis of a golf swing, although information regarding the time intervals occupied by the partial motions constituting the golf swing (hereinafter referred to as "sections constituting the golf swing") (e.g., the length of the backswing section or the length of the downswing section) is important, no specific analysis regarding the sections constituting the golf swing was performed. The second is that appropriate feedback regarding the results of analyzing the sections constituting the golf swing was not provided for the user, and thus it was not very helpful for improving the user's golf skills.

[0006] Therefore, the present inventors propose a technique that enables specific analysis of the sections constituting a golf swing by generating information on the sections constituting a golf swing by analyzing two or more images related to the golf swing, and a technique that enables appropriate feedback regarding the golf swing to be provided to the user by comparing the information on at least one section constituting the golf swing with reference information.

Prior Art Documents

Patent Documents

[0007]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0008] An object of the present invention is to solve all of the problems of the prior art described above.

[0009] Another object of the present invention is to acquire two or more images related to a golf swing and generate information on at least one section constituting the golf swing with reference to the comparison result of the two or more images.

[0010] Another object of the present invention is to compare two or more images related to a golf swing and determine the sections constituting the golf swing corresponding to a specific image with reference to the comparison result, so as to enable accurate analysis of the sections constituting the golf swing.

[0011] Another object of the present invention is to enable precise classification of the sections constituting a golf swing even when the frame rate of a sensor (e.g., a camera) is low.

Means for Solving the Problems

[0012] A representative configuration of the present invention for achieving the above object is as follows.

[0013] According to one aspect of the present invention, there is provided a method including the steps of obtaining two or more images related to a golf swing, and generating information related to at least one section constituting the golf swing with reference to a comparison result of the two or more images.

[0014] According to another aspect of the present invention, there is provided a system including an image acquisition unit that acquires two or more images related to a golf swing, and a section information generation unit that generates information related to at least one section constituting the golf swing with reference to a comparison result of the two or more images.

[0015] In addition, there are further provided other methods for embodying the present invention, other systems, and a non-transitory computer-readable recording medium recording a computer program for executing the above method.

Advantages of the Invention

[0016] According to the present invention, it becomes possible to obtain two or more images related to a golf swing and generate information related to at least one section constituting the golf swing with reference to a comparison result of the two or more images.

[0017] Also, according to the present invention, by comparing two or more images related to a golf swing and determining a section constituting the golf swing corresponding to a specific image with reference to a comparison result, an accurate analysis of the section constituting the golf swing can be made.

[0018] Also, according to the present invention, even when the frame rate of a sensor (for example, a camera) is low, the section constituting the golf swing can be precisely divided.

Brief Description of the Drawings

[0019]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Mode for Carrying Out the Invention

[0020] The detailed description of the present invention described below refers to the accompanying drawings that illustrate specific embodiments in which the present invention can be implemented as examples. Such embodiments are described in detail so that those skilled in the art can fully implement the present invention. It should be understood that various embodiments of the present invention are different from each other but do not necessarily have to be mutually exclusive. For example, the specific shapes, structures, and characteristics described in this specification can be changed and embodied from one embodiment to another without departing from the spirit and scope of the present invention. Also, it should be understood that the positions or arrangements of individual components within each embodiment can be changed without departing from the spirit and scope of the present invention. Therefore, the detailed description given below is not intended to be limiting, and the scope of the present invention should be understood to include the scope claimed by the claims of the claims and all ranges equivalent thereto. Similar reference numerals in the drawings indicate the same or similar components across various aspects.

[0021] Hereinafter, in order to enable those with ordinary knowledge in the technical field to which the present invention pertains to easily implement the present invention, a detailed description will be given with reference to the accompanying drawings regarding many preferred embodiments of the present invention.

[0022] Configuration of the entire system

[0023] FIG. 1 is a drawing showing a schematic configuration of an overall system according to an embodiment of the present invention.

[0024] As shown in FIG. 1, the overall system according to an embodiment of the present invention can include a communication network 100, an information providing system 200, and a device 300.

[0025] First, the communication network 100 according to an embodiment of the present invention can be configured regardless of the mode of communication such as wired communication or wireless communication, and can be composed of various communication networks such as a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN). Preferably, the communication network 100 referred to in this specification can be a known Internet or World Wide Web (WWW). However, the communication network 100 is not limited thereto, and may include at least a part of a known wired / wireless data communication network, a known telephone network, or a known wired / wireless television communication network.

[0026] For example, the communication network 100 may be a wireless data communication network that implements at least a part of conventional communication methods such as Wi-Fi (WiFi) communication, Wi-Fi Direct communication, Long Term Evolution (LTE) communication, 5G communication, Bluetooth communication (including Bluetooth Low Energy (BLE) communication) (Bluetooth is a registered trademark), infrared communication, ultrasonic communication, and the like. As another example, the communication network 100 may be an optical communication network that implements at least a part of conventional communication methods such as LiFi (Light Fidelity).

[0027] Next, the information providing system 200 according to an embodiment of the present invention can perform a function of acquiring two or more images related to a golf swing and generating information related to at least one section constituting the golf swing with reference to a comparison result of the two or more images.

[0028] Further, the information providing system 200 according to an embodiment of the present invention may perform a function of acquiring information related to at least one section constituting the golf swing and generating feedback information related to the golf swing with reference to a comparison result between preset reference information and the information related to the at least one section.

[0029] Regarding the configuration and function of the information providing system 200 according to the present invention, it will be examined in detail through the following detailed description.

[0030] Next, the device 300 according to an embodiment of the present invention is a digital device including a function capable of communicating after being connected to the information providing system 200, and any digital device having a memory means and equipped with a microprocessor to have computing power, such as a smartphone, a tablet, a smartwatch, a smartband, smart glasses, a desktop computer, a notebook computer, a workstation, a PDA, a web pad, a mobile phone, etc., can be adopted as the device 300 according to the present invention.

[0031] In particular, the device 300 can include an application (not shown) that assists a user in receiving the provision of the functions according to the present invention from the information providing system 200. Such an application can be downloaded from the information providing system 200 or an external application distribution server (not shown). On the other hand, the nature of such an application can be generally similar to the image acquisition unit 210, the section information generation unit 220, the section information acquisition unit 230, the feedback information generation unit 240, the information providing unit 250, the communication unit 260, and the control unit 270 of the information providing system 200 as described later. Here, at least a part of the application may be replaced by a hardware device or a firmware device that can perform functions that are substantially the same or equivalent thereto as needed.

[0032] Configuration of the information providing system

[0033] Hereinafter, the internal configuration of the information providing system 200 that performs important functions for the implementation of the present invention and the functions of each component will be examined in detail.

[0034] FIG. 2 is a drawing showing in detail the internal configuration of the information providing system 200 according to an embodiment of the present invention.

[0035] As shown in FIG. 2, an information providing system 200 according to an embodiment of the present invention may include an image acquisition unit 210, a section information generation unit 220, a section information acquisition unit 230, a feedback information generation unit 240, an information providing unit 250, a communication unit 260, and a control unit 270. According to an embodiment of the present invention, at least a part of the image acquisition unit 210, the section information generation unit 220, the section information acquisition unit 230, the feedback information generation unit 240, the information providing unit 250, the communication unit 260, and the control unit 270 may be program modules that communicate with an external system (not shown). Such program modules may be included in the information providing system 200 in the form of an operation system, an application program module, or other program modules, and may be physically stored in various known storage devices. Further, such program modules may be stored in a remote storage device that can communicate with the information providing system 200. On the other hand, such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform specific operations described later according to the present invention or execute specific types of abstract data.

[0036] In this specification, a golf swing may mean a swinging motion for sending a golf ball to a target point. Here, the golf swing may be divided into eight stages of motion: address, take away, back swing, top of swing, down swing, impact, follow through, and finish. However, the above eight-stage divided motion is merely an example, and it is obvious to those skilled in the art that the golf swing motion can be divided in various ways.

[0037] FIG. 3 is a drawing exemplarily showing sections constituting a golf swing according to an embodiment of the present invention.

[0038] For example, referring to Division Example 1 in FIG. 3, a golf swing according to an embodiment of the present invention can be divided into 10 stages of intervals: address, takeaway, takeback, backswing, top, transition, downswing, impact, follow-through, and finish. Also, referring to Division Example 2 in FIG. 3, a golf swing according to an embodiment of the present invention may be divided into 7 stages of intervals: address, takeback, backswing, top, downswing, follow-through, and finish. Further, referring to Division Example 3 in FIG. 3, a golf swing according to an embodiment of the present invention may be divided into 3 stages of intervals: backswing, top (Pause interval), and downswing.

[0039] On the other hand, although the information providing system 200 has been described as above, such description is merely exemplary, and it is obvious to those skilled in the art that at least some of the components or functions of the information providing system 200 may be realized within the device 300 or a server (not shown) as required, or may be included in an external system (not shown).

[0040] First, the image acquisition unit 210 according to an embodiment of the present invention can perform a function of acquiring two or more images related to a golf swing.

[0041] Specifically, two or more images related to a golf swing according to an embodiment of the present invention can be acquired from a sensor (for example, an image sensor such as a camera). For example, the image acquisition unit 210 according to an embodiment of the present invention can acquire a plurality of frame images included in a user's golf swing video from the sensor as two or more images related to the golf swing.

[0042] For example, two or more images according to an embodiment of the present invention can include a first image and a second image that are acquired continuously. To give a specific example, if the first image according to an embodiment of the present invention is the 90th frame of a user's golf swing video, the second image according to an embodiment of the present invention may mean the 89th or 91st frame of the user's golf swing video. However, according to the present invention, it is obvious to those skilled in the art that the two or more images acquired by the image acquisition unit 210 according to an embodiment of the present invention do not necessarily have to be images acquired continuously, and may be images separated by a predetermined interval in time (for example, not continuous) (for example, the first image is the 50th frame image and the second image is the 90th frame image).

[0043] In addition, the image acquisition unit 210 according to an embodiment of the present invention can acquire two or more images related to a golf swing from at least one sensor. For example, at least one sensor according to an embodiment of the present invention may include various sensors such as a ground pressure sensor and a video sensor.

[0044] Also, the images that can be acquired by the image acquisition unit 210 according to an embodiment of the present invention may include at least one of a 2D image and a 3D image. For example, according to an embodiment of the present invention, the image acquisition unit 210 can acquire a 2D image from one video sensor (for example, a camera), and generate a 3D image by using two or more 2D images acquired from two or more video sensors, so as to acquire two or more 3D images.

[0045] Next, the section information generation unit 220 according to an embodiment of the present invention can perform a function of generating information related to at least one section constituting a golf swing with reference to the comparison result of two or more images.

[0046] Specifically, the section information generation unit 220 according to an embodiment of the present invention can compare two or more images.

[0047] FIG. 4 is a drawing exemplarily illustrating a process of comparing images related to a golf swing according to an embodiment of the present invention.

[0048] For example, referring to FIG. 4, an interval information generation unit 220 according to an embodiment of the present invention can compare a first image 410 and a second image 420. Specifically, it can compare first joint information 411 of the first image 410 and second joint information 421 of the second image 420. Here, joint information (i.e., first joint information, second joint information) according to an embodiment of the present invention can be extracted from an image (i.e., the first image, the second image, etc.) by a joint information estimation model learned using an artificial neural network. Also, joint information according to an embodiment of the present invention may include information regarding line segments formed by extracting the positions of the joints of the user in the image and connecting the joints at different positions thus extracted.

[0049] As another example, referring to FIG. 4, an interval information generation unit 220 according to an embodiment of the present invention can compare first shaft information of the first image 410 and second shaft information of the second image 420. Specifically, referring to FIG. 4, an interval information generation unit 220 according to an embodiment of the present invention can compare first inclination information included in the first shaft information and second inclination information included in the second shaft information with each other.

[0050] However, the method of comparing two or more images according to an embodiment of the present invention does not necessarily have to be performed in a manner of comparing joint information or shaft information, and can be variously changed within a range that can achieve the object of the present invention. For example, the method of comparing two or more images according to an embodiment of the present invention can also be performed in a manner of comparing pixel values of two or more images, or comparing specific numerical values derived by filtering two or more images according to preset conditions respectively.

[0051] Referring to FIG. 3, the sections constituting a golf swing according to an embodiment of the present invention can be variously divided. However, in order to calculate the rhythm of the golf swing (for example, the ratio of the length of the first section related to the backswing and the length of the second section related to the downswing), it may be necessary to accurately analyze the information related to the first section related to the backswing and the information related to the second section related to the downswing. Therefore, at least one section constituting a golf swing according to an embodiment of the present invention can include the first section related to the backswing and the second section related to the downswing, and the information related to at least one section generated by the section information generation unit 220 according to an embodiment of the present invention can include information related to the ratio between the first section and the second section.

[0052] Also, the section information generation unit 220 according to an embodiment of the present invention can generate information related to at least one section constituting a golf swing.

[0053] For example, the information related to at least one section generated by the section information generation unit 220 according to an embodiment of the present invention can include at least one of information related to the length of at least one section constituting a golf swing and section ratio information related to the ratio between two or more sections constituting a golf swing.

[0054] Taking a specific example, the information related to at least one section generated by the section information generation unit 220 according to an embodiment of the present invention can include at least one of the first section information related to the length of the first section, the second section information related to the length of the second section, and the section ratio information related to the ratio between the first section and the second section.

[0055] Continuing the explanation, the section information generation unit 220 according to an embodiment of the present invention can generate information related to at least one section by determining the presence or absence of identity between the section corresponding to the first image and the section corresponding to the second image with reference to the comparison result of the first image and the second image.

[0056] For example, referring to FIG. 4, when the section information generation unit 220 according to an embodiment of the present invention determines that the second image 420 corresponds to a section related to the top of the swing (hereinafter referred to as the top section), it determines the similarity between the first joint information 421 extracted from the first image 410 and the second joint information 421 extracted from the second image 420. When the similarity between the first joint information 411 and the second joint information 421 is higher than a preset similarity (for example, when the similarity between the swing postures in the first image 410 and the second image 420 is high), it can be determined that the section corresponding to the first image 410 and the section corresponding to the second image 420 are the same (that is, it is determined that both the first image 410 and the second image 420 correspond to the top section). Here, if the similarity between the first joint information 411 and the second joint information 421 is lower than the preset similarity (for example, when the similarity between the swing postures in the first image 410 and the second image 420 is low), it can be determined that the section corresponding to the first image 410 and the section corresponding to the second image 420 are not the same (that is, it is determined that the second image 420 corresponds to the top section, but the first image corresponds to the backswing section, which is the previous section).

[0057] In addition, at least one of the first image and the second image according to an embodiment of the present invention may be a reference image determined according to a condition preset for dividing the swing section.

[0058] For example, the reference image determined according to the condition preset for dividing the swing section according to an embodiment of the present invention may mean an image in which the degree of change in posture during the user's golf swing is below a predetermined standard (for example, an image corresponding to the time point when it is determined that the user has stopped during the golf swing). Here, the reference image according to an embodiment of the present invention may mean an image corresponding to the time point (for example, the time point when the degree of change in posture is below a predetermined standard) when it is determined that the user has momentarily stopped at the top section in the golf swing section.

[0059] Therefore, the section information generation unit 220 according to an embodiment of the present invention identifies a reference image determined by preset conditions from two or more images related to a golf swing (for example, a plurality of image frames included in a swing video), and compares the reference image with other images, thereby determining the identity of the section corresponding to the reference image (for example, the top section) and the section corresponding to other images (for example, the backswing section, the top section, or the downswing section), and can generate information regarding at least one section.

[0060] Taking a specific example, the section information generation unit 220 according to an embodiment of the present invention acquires a plurality of image frames related to a golf swing, and identifies, as a reference image, a frame in which the change of the user among consecutive frames of the plurality of image frames is below a preset reference (for example, when comparing joint information extracted from the image, a frame in which the movement of the joint is below a preset reference, or a frame in which the angular change of the shaft is below a preset reference), compares other images acquired before or after the reference image with the reference image, and determines the identity of the section corresponding to the reference image and the section corresponding to other images, thereby generating information regarding at least one section.

[0061] Referring to FIG. 4 and giving another specific example, when the section information generation unit 220 according to an embodiment of the present invention specifies the third image 430 as a reference image (for example, an image corresponding to the top section), the section information generation unit 220 can compare the third image 430 with the second image 420 to determine whether the second image 420 is an image corresponding to the top section. Here, according to an embodiment of the present invention, since the similarity between the second image 420 and the third image 430 is equal to or greater than a preset similarity, it can be determined that the second image 420 is an image corresponding to the top section. Next, the section information generation unit 220 according to an embodiment of the present invention compares the second image 420 with the first image 410 and can determine whether the section corresponding to the first image 410 and the section corresponding to the second image 420 are the same section. Here, when it is determined that the section corresponding to the second image 420 and the section corresponding to the first image 410 are not the same, the section information generation unit 220 according to an embodiment of the present invention determines that the second image 420 and the third image 430 are images corresponding to the same section (for example, the top section), and the first image 410 is an image corresponding to the backswing section, so that information about at least one section can be generated.

[0062] Therefore, according to the present invention, by sequentially analyzing adjacent images (for example, comparing the first image 410 with the second image 420, comparing the second image 420 with the third image 430, and continuously analyzing in a manner of comparing the third image 430 with a fourth image (not shown)), it becomes possible to determine which section of the golf swing the two or more images related to the golf swing correspond to.

[0063] On the one hand, it is obvious to those skilled in the art that the reference image according to an embodiment of the present invention does not necessarily have to be an image corresponding to the top section, and it may be an image corresponding to an address section or an impact time point. For example, the section information generation unit 220 according to an embodiment of the present invention extracts the position information of the golf ball from an image, and identifies, as a reference image determined by a preset condition, an image corresponding to the time point (for example, the impact time point) when the position of the golf ball starts to change in two or more images, and may compare two or more images using the corresponding reference image.

[0064] Since the frame rate of an image sensor (for example, a camera) (for example, a frame rate of 30 frames per second) is finite, there may be a time gap between the continuously acquired images even if two or more images are continuously acquired. By the way, for a precise analysis of the section constituting the golf swing, it is important to accurately calculate the length of a specific section of the golf swing, so it may be necessary to determine the start time point and the end time point of the specific section of the golf swing.

[0065] Therefore, the section information generation unit 220 according to an embodiment of the present invention can determine the time point when the section corresponding to the first image and the section corresponding to the second image are divided in response to the determination that the section corresponding to the first image and the section corresponding to the second image are not the same.

[0066] For example, referring to FIG. 4, assume that the section information generation unit 220 according to an embodiment of the present invention determines that the section corresponding to the first image 410 is a backswing section, and the sections corresponding to the second image 420 and the third image 430 are top sections. Here, according to an embodiment of the present invention, the section information generation unit 220 refers to the information at the time corresponding to the first image 410 (for example, t = 3.00 [sec]) and the information at the time corresponding to the second image 420 (for example, t = 3.03 [sec]), and determines that the time point at which two different sections related to the golf swing (that is, the backswing section and the top section) are divided corresponds to the intermediate value of the time point corresponding to the first image 410 and the time point corresponding to the second image (for example, t = 3.15 [sec]), so that the time point corresponding to the end point of the backswing section (that is, the start time point of the top section) (for example, t = 3.15 [sec]) can be determined.

[0067] On the other hand, according to an embodiment of the present invention, as described above, the intermediate value of the time point corresponding to the first image and the time point corresponding to the second image can be determined as the time point at which two different sections are divided. However, in order to generate more accurate information regarding the sections constituting the golf swing, it may be necessary to more precisely determine the time point at which the section corresponding to the first image and the section corresponding to the second image are divided based on the similarity between the first image and the second image.

[0068] Therefore, the section information generation unit 220 according to an embodiment of the present invention determines the weighted value information regarding the second image based on the similarity between the first image and the second image, and can determine the length of the time incorporated into the section corresponding to the first image among the time corresponding to the second image with reference to the weighted value information.

[0069] For example, referring to FIG. 4, when the first image 410 corresponds to the backswing interval and the second image 420 corresponds to the top interval in an embodiment of the present invention, the similarity between the first image 410 and the second image 420 can be calculated. Here, if the similarity between the first image 410 and the second image 420 in an embodiment of the present invention is high, then the time point when the interval corresponding to the first image 410 and the interval corresponding to the second image 420 are divided can be determined to be closer to the time point corresponding to the first image 410 among the time point corresponding to the first image 410 (t = 3.00 [sec]) and the time point corresponding to the second image 420 (t = 3.03 [sec]).

[0070] In other words, according to an embodiment of the present invention, even though the golf swing intervals corresponding to the first image and the second image are different from each other, if the similarity between the first image and the second image is high, the time point corresponding to the first image can be regarded as close to the start time point of the interval corresponding to the second image. Therefore, among the time point corresponding to the first image and the time point when the second image is acquired, the time point closer to the time point corresponding to the first image can be determined as the time point when the interval corresponding to the first image and the interval corresponding to the second image are divided.

[0071] As another example, referring to FIG. 4, when the first image 410 belongs to an interval related to the backswing and the second image 420 belongs to an interval related to the top in an embodiment of the present invention, the similarity between the first image 410 and the second image 420 can be calculated, and the weighted value information related to the first image 410 can be determined based on the similarity.

[0072] Taking a specific example with reference to FIG. 4, the interval information generation unit 220 according to an embodiment of the present invention can determine the weighted value (for example, when the similarity is high, the weighted value can be 2, and when the similarity is low, the weighted value can be 0.5) information related to the first image 410 that is applied to the first image 410 corresponding to the corresponding similarity based on the similarity between the first image 410 and the second image 420.

[0073] Here, referring to FIG. 4, according to an embodiment of the present invention, when the similarity between the first image 410 and the second image 420 is high (for example, when the similarity satisfies a preset condition), the section information generation unit 220 determines weighted value information regarding the first image 410 (for example, information that the weighted value is 2), and refers to the weighted value information to determine the length of the time (for example, 0.01 [sec]) incorporated into the section corresponding to the second image 420 among the time corresponding to the first image 410. In other words, since the weighted value regarding the first image 410 is 2, by reflecting the time point corresponding to the first image 410 twice more than the time point corresponding to the second image 420, the time point 410 corresponding to the first image 410 (for example, t = 3.00 [sec]) and the time point 420 corresponding to the second image 420 (for example, t = 3.03 [sec]) can be determined as the time point (for example, t = 3.01 [sec]) at which the two sections are divided by internal division in a ratio of 1:2.

[0074] Conversely, referring to FIG. 4, as another example, according to an embodiment of the present invention, when the similarity between the first image 410 and the second image 420 is low (for example, when the similarity does not satisfy a preset condition), the section information generation unit 220 determines weighted value information regarding the first image 410 (for example, information that the weighted value is 0.5), and refers to the weighted value information to determine the length of the time (for example, 0.02 [sec]) incorporated into the section corresponding to the second image 420 among the time corresponding to the first image 410. In other words, since the weighted value regarding the first image 410 is 0.5, by reflecting the time point corresponding to the first image 410 0.5 times more than the time point corresponding to the second image 420, the time point 410 corresponding to the first image 410 (for example, t = 3.00 [sec]) and the time point 420 corresponding to the second image 420 (for example, t = 3.03 [sec]) can be determined as the time point (for example, t = 3.02 [sec]) at which the two sections are divided by internal division in a ratio of 0.5:1 (that is, 2:1).

[0075] However, it is obvious to those skilled in the art that the weighted value information according to an embodiment of the present invention does not necessarily have to relate to a constant value set in advance, and may be determined in real time by an artificial neural network model. Further, the similarity according to an embodiment of the present invention can be calculated by comparing the similarity between the first joint information 411 and the second joint information 421 using a similarity estimation model learned using an artificial neural network, but it is not necessarily calculated based on joint information, and other information related to the swing posture (for example, information related to the angle of the shaft, the displacement of the golf club head, etc.) may be calculated by comparison.

[0076] Therefore, according to an embodiment of the present invention, when the weighted value is a specific constant value, by applying the weighted value to the first image by only the corresponding constant value, at a time point closer (or farther) to the time point corresponding to the first image, the division time point of the section can be determined.

[0077] Further, according to the present invention, when the weighted value information is determined, if the similarity between the first image and the second image is high, the length of the time incorporated into the section corresponding to the first image among the time corresponding to the second image is increased, and if the similarity between the first image and the second image is low, the length of the time incorporated into the section corresponding to the first image among the time corresponding to the second image is shortened, so that the time point at which two or more sections are divided can be determined more accurately.

[0078] Hereinafter, with respect to the content of generating information regarding at least one section constituting a golf swing according to an embodiment of the present invention, it will be described in detail with reference to FIGS. 3 and 4.

[0079] First Embodiment

[0080] According to an embodiment of the present invention, the section information generation unit 220 can determine at least one image determined to be a backswing section among two or more images as a reference image (for example, among the images corresponding to before the impact time, an image determined that the golf club rises as time passes can be determined as the reference image).

[0081] Next, the section information generation unit 220 can determine the presence or absence of identity between the section corresponding to the reference image and the sections corresponding to other images acquired before the acquisition of the reference image, with reference to the comparison result between the reference image and other images continuously acquired immediately before the reference image. If the sections corresponding to the two images are identical to each other, the section information generation unit 220 can determine the presence or absence of identity of the sections by comparing still other images acquired even earlier than other images acquired before the acquisition of the reference image. That is, the section information generation unit 220 can determine that when the comparison result between the first image and the second image and the analysis result of the second image indicate that the user is determined to have stopped in the second image, the second image corresponds to the address section and the first image corresponds to the backswing section.

[0082] Next, corresponding to the determination that the section corresponding to the first image (for example, the section related to the backswing) and the section corresponding to the second image (for example, the address section) are not identical, the section information generation unit 220 can determine the time point at which the two sections are divided (that is, the time point at which the address section ends and the backswing section starts). Here, the section information generation unit 220 can determine the weighted value information regarding the second image based on the similarity between the first image and the second image, and with reference to the weighted value information, can determine the time point at which the address section ends (in other words, the time point at which the backswing section starts).

[0083] Second Embodiment

[0084] According to an embodiment of the present invention, the section information generation unit 220 can determine at least one image determined to be the top section among two or more images as a reference image (for example, among the images acquired before the impact time, an image in which the golf club head is at a predetermined distance or more from the ground and the degree of change in the user's posture is below a predetermined standard can be determined as the reference image).

[0085] Next, the section information generation unit 220 can determine the start time of the top section by sequentially comparing the reference image with the images acquired before the reference image. Conversely, the section information generation unit 220 may determine the end time of the top section by sequentially comparing the reference image with the images acquired after the reference image.

[0086] Third Embodiment

[0087] According to an embodiment of the present invention, the section information generation unit 220 can determine at least one image determined to be the downswing section among two or more images as a reference image (for example, among the images acquired before the impact time, an image in which the golf club head is determined to approach the ground can be determined as the reference image).

[0088] Next, the section information generation unit 220 can determine the end time of the top section by sequentially comparing the reference image with the images acquired before the reference image. Conversely, the section information generation unit 220 may determine the end time of the downswing section by sequentially comparing the reference image with the images acquired after the reference image. Here, instead of determining the end time of the downswing section based on the similarity of two or more images, the section information generation unit 220 may acquire the position information of the golf ball and determine it in a manner of specifying the impact time with reference to the change in the position of the ball.

[0089] Fourth Embodiment

[0090] According to one embodiment of the present invention, the section information generation unit 220 can determine at least one of the start time of the backswing section, the end time of the backswing section (i.e., the start time of the top section), the end time of the top section (i.e., the start time of the downswing section), and the end time of the downswing section (i.e., the impact time) according to the various embodiments described above, and generate information regarding at least one of the length of the backswing section (i.e., the length from the start time of the backswing section to the end time of the backswing section), the length of the top section (i.e., the length from the start time of the top section to the end time of the top section), and the length of the downswing section (i.e., the length from the start time of the downswing section to the end time of the downswing section).

[0091] Also, according to one embodiment of the present invention, the section information generation unit 220 can generate information regarding the ratio of the length of the backswing section and the length of the downswing section as information regarding at least one section.

[0092] Hereinafter, a method for providing feedback regarding a golf swing will be specifically described.

[0093] Next, the section information acquisition unit 230 according to one embodiment of the present invention can perform a function of acquiring information regarding at least one section constituting a golf swing.

[0094] Specifically, at least one section constituting a golf swing can include, as described above, a first section regarding a backswing and a second section regarding a downswing. Here, the information regarding at least one section can include at least one of first section information regarding the length of the first section, second section information regarding the length of the second section, and section ratio information regarding the ratio between the first section and the second section.

[0095] In addition, the section information acquisition unit 230 according to an embodiment of the present invention can acquire information regarding at least one section constituting a golf swing from the section information generation unit 220 described above. However, the section information acquisition unit 230 according to an embodiment of the present invention may acquire information regarding at least one section constituting a golf swing from a system (not shown) inside or outside the information providing system 200.

[0096] For example, when the information providing system 200 according to an embodiment of the present invention is linked with a virtual golf simulator system (not shown), the section information acquisition unit 230 according to an embodiment of the present invention may acquire at least one of first section information regarding the length of the first section, second section information regarding the length of the second section, and section ratio information regarding the ratio between the first section and the second section from the virtual golf simulator system.

[0097] As another example, the section information generation unit 220 according to an embodiment of the present invention may acquire information regarding at least one section constituting a golf swing from the device 300. Here, the device 300 may include a mobile device including a video sensor, but is not limited thereto.

[0098] Next, the feedback information generation unit 240 according to an embodiment of the present invention can perform a function of generating feedback information regarding a golf swing by referring to a comparison result between preset reference information and information regarding at least one section.

[0099] Specifically, the preset reference information according to an embodiment of the present invention may include at least one of first reference section information corresponding to the first section information, second reference section information corresponding to the second section information, and reference section ratio information corresponding to the section ratio information.

[0100] For example, the preset reference information according to an embodiment of the present invention can include at least one of first reference section information corresponding to a preferred golf swing posture (for example, information regarding the length of the backswing section of a professional golfer), second reference section information (for example, information regarding the length of the downswing section of a professional golfer), and reference section ratio information (for example, information regarding the length of the backswing section and the length of the downswing section of a professional golfer).

[0101] Taking a specific example, the preset reference information according to an embodiment of the present invention can include at least one of first reference section information (for example, information indicating that the length of the backswing section is preferably 1.2 seconds), second reference section information (for example, information indicating that the length of the downswing section is preferably 0.4 seconds), and section ratio information (for example, information indicating that the ratio of the length of the backswing section to the length of the downswing section is preferably 3:1).

[0102] As another example, the preset reference information according to an embodiment of the present invention can include at least one reference reference information.

[0103] Taking a specific example, when the first reference section information according to an embodiment of the present invention includes information indicating that the length of the first section is preferably 0.9 seconds, the reference reference information included in the first reference section information according to an embodiment of the present invention can include at least one of the following: information indicating that when the length of the first section is 0.8 seconds or more and less than 1 second, it is a preferred swing; information indicating that when the length of the first section is 0.6 seconds or more and less than 0.8 seconds, it is a swing that is not preferred because the length of the first section is slightly short; information indicating that when the length of the first section is 1 second or more and less than 1.2 seconds, it is a swing that is not preferred because the length of the first section is slightly long; information indicating that when the length of the first section is less than 0.6 seconds, it is a swing that is not preferred because the length of the first section is very short; and information indicating that when the length of the first section is 1.2 seconds or more, it is a swing that is not preferred because the length of the first section is very long.

[0104] Therefore, since the reference information according to the present invention includes the reference reference information, at least one of the length and ratio of the section can correspond to the reference reference information that is subdivided (for example, the reference information can include two or more reference reference information), and by comparing the subdivided reference reference information with the information regarding at least one section, various feedback information can be generated.

[0105] Continuing the explanation, the feedback information generation unit 240 according to an embodiment of the present invention can compare the reference information with the information regarding at least one section.

[0106] For example, when the section information acquisition unit 230 according to an embodiment of the present invention acquires section ratio information (for example, information that the ratio of the length of the first section to the length of the second section is 4.8:1) in the user's golf swing, the feedback information generation unit 240 can compare the preset reference information (for example, information that the ratio of the length of the first section to the length of the second section of a professional golfer is 4:1) with the section ratio information. Here, the feedback information generation unit 240 according to an embodiment of the present invention can generate information for suggesting to make the backswing a little faster (that is, shorten the length of the backswing section) as the feedback information, so as to guide the user's section ratio information to be close to the reference section ratio information.

[0107] Also, the feedback information generation unit 240 according to an embodiment of the present invention can compare at least one reference reference information included in the reference information with the information regarding at least one section.

[0108] For example, when the length of the second interval is 0.25 seconds or more and less than 0.35 seconds, the reference reference information according to an embodiment of the present invention can include information that it is a preferable swing, when the length of the second interval is 0.2 seconds or more and less than 0.25 seconds, information that it is an unfavorable swing because the length of the second interval is slightly short, when the length of the second interval is 0.35 seconds or more and less than 0.4 seconds, information that it is an unfavorable swing because the length of the second interval is slightly long, when the length of the second interval is less than 0.2 seconds, information that it is an unfavorable swing because the length of the second interval is very short, and when the length of the second interval is 0.4 seconds or more, information that it is an unfavorable swing because the length of the second interval is very long. Here, when the information regarding at least one interval according to an embodiment of the present invention is that the length of the second interval is 0.3 seconds, the feedback information generation unit 240 according to an embodiment of the present invention can compare the reference reference information with the information regarding at least one interval and generate information that the user's downswing is a preferable swing as feedback information. Here, if the information regarding at least one interval according to an embodiment of the present invention is that the length of the second interval is 0.6 seconds, the feedback information generation unit 240 according to an embodiment of the present invention may generate information that the user's downswing is an unfavorable swing because it is very long as feedback information.

[0109] In other words, the feedback information generation unit 240 according to an embodiment of the present invention compares the reference reference information including at least one interval information with the information regarding at least one interval, determines the interval information to which the length of the specific interval constituting the user's golf swing belongs, and can generate preset feedback information corresponding to the relevant interval information.

[0110] On the one hand, it should be regarded that at least one section compared by the feedback information generation unit 240 according to an embodiment of the present invention may include a top section in addition to the first section and the second section. For example, the feedback information generation unit 240 according to an embodiment of the present invention compares the reference reference information with the information regarding the top section, and when the length of the top section in the user's golf swing is compared with the reference reference information, determines whether it is an appropriate length, and may generate feedback information regarding the top section.

[0111] Continuing the explanation, according to an embodiment of the present invention, the information regarding at least one section may include first section information regarding the length of the first section and second section information regarding the length of the second section, and the feedback information generation unit 240 can determine the section for which a correction is proposed among the first section and the second section by referring to the first comparison result corresponding to the first section information and the second comparison result corresponding to the second section information. Here, the feedback information generated by the feedback information generation unit 240 may include information regarding the section for which a correction is proposed.

[0112] For example, the feedback information generation unit 240 according to an embodiment of the present invention refers to the first comparison result of comparing the first section information (information that the length of the backswing is 1.5 seconds) with the first reference section information (information that it is ideal when the length of the backswing is 0.8 seconds or more and 1.0 seconds or less), and the second comparison result of comparing the second section information (information that the length of the downswing is 0.3 seconds) with the second reference section information (information that it is ideal when the length of the downswing is 0.25 seconds or more and less than 0.35 seconds), and determines that the length of the first section must be made shorter, but the length of the second section is ideal, so the section for which a correction is proposed among the first section and the second section can be determined to be the first section. Here, the feedback information generation unit 240 according to an embodiment of the present invention can generate feedback information stating that the length of the first section must be corrected to be shorter.

[0113] As another example, the feedback information generation unit 240 according to an embodiment of the present invention compares the first interval information (information indicating that the length of the backswing is 1.5 seconds) with the first reference interval information (information indicating that it is ideal if the length of the backswing is 0.8 seconds or more and 1.0 seconds or less), and compares the second interval information (information indicating that the length of the downswing is 0.7 seconds) with the second reference interval information (information indicating that it is ideal if the length of the downswing is more than 0.25 seconds and less than 0.35 seconds). With reference to the first comparison result and the second comparison result, it is possible to determine that the section for which correction is proposed among the first section and the second section is the first section. In other words, the feedback information generation unit 240 according to an embodiment of the present invention preferably corrects the backswing first and then corrects the downswing when both the length of the backswing and the length of the downswing are long compared to the reference standard information. Therefore, the section for which correction is proposed can be determined as the first section.

[0114] According to the present invention, when there is a difference between the length of the first section and the length of the second section from the lengths corresponding to the reference information, by specifying the section that requires correction first among the first section and the second section, the user can easily correct their posture using the feedback information.

[0115] On the other hand, the reference information according to an embodiment of the present invention may be directly determined by the user. For example, the user according to an embodiment of the present invention can determine at least one of the first reference interval information, the second reference interval information, and the reference interval ratio information as specific information (for example, information regarding the length or ratio of the section targeted by the user). As a specific example, the user according to an embodiment of the present invention can provide specific reference interval ratio information (for example, information indicating that the ratio of the length of the first section to the length of the second section is 4:1) for the information providing system 200, and the feedback information generation unit 40 can generate feedback information by comparing the corresponding reference interval ratio information with the interval ratio information.

[0116] FIG. 5 is a drawing exemplarily illustrating a user interface provided through a display according to an embodiment of the present invention.

[0117] Referring to FIG. 5(b), the section information acquisition unit 240 according to an embodiment of the present invention can acquire at least one of information regarding the direction in which the user's swing posture is photographed (for example, the front direction of the user or the side direction of the user), information regarding the type of club used by the user (for example, iron or driver), and target section ratio information of the user (for example, reference section ratio information). It is obvious to those skilled in the art that this may be acquired by the image acquisition unit 210 or by other components inside the information providing system 200. Here, the target section ratio information of the user to be acquired is preset reference section ratio information, and the feedback information generation unit 240 can generate feedback information by comparing the corresponding preset reference section ratio information with the user's section ratio information.

[0118] On the other hand, the information providing system 200 according to an embodiment of the present invention can further include an information providing unit 250. The information providing unit 250 according to an embodiment of the present invention can perform a function of providing at least one of preset reference information, information regarding at least one section, and feedback information through a display.

[0119] Specifically, the information providing unit 250 according to an embodiment of the present invention can visually provide at least one of preset reference information, information regarding at least one section, and feedback information through a display of the device 300 or an external system (not shown). Here, the information providing unit 250 according to an embodiment of the present invention can be associated with UI (User Interface) / UX (User eXperience).

[0120] For example, referring to FIG. 5(a), the information providing unit 250 according to an embodiment of the present invention can exemplarily provide a swing video corresponding to specific section ratio information (for example, golf swing rhythm information).

[0121] As another example, referring to FIG. 5(c), the information providing unit 250 according to an embodiment of the present invention may provide information regarding a method in which the length of a specific section (for example, the tempo of the backswing, the tempo of the downswing) and the ratio between two sections (for example, the rhythm of the golf swing) are calculated.

[0122] FIG. 6 is a drawing exemplarily showing a situation in which feedback information according to an embodiment of the present invention is provided.

[0123] Referring to FIG. 6(a), the information providing unit 250 according to an embodiment of the present invention can provide, through a display, information regarding the length of at least one section (for example, when the user makes a golf swing, the length of the backswing section is 1.65 seconds, the length of the top section is 0.03 seconds, the length of the downswing is 0.28 seconds, and the length from the start of the address section to the impact point is 1.95 seconds) and section ratio information (for example, the ratio of the length of the backswing section to the length of the downswing section is 5.8:1).

[0124] Referring to FIG. 6(b), according to an embodiment of the present invention, when the feedback information generated by the feedback information generation unit 240 includes information indicating that the length of the first section (i.e., the length of the backswing section) in the user's golf swing is determined to be somewhat longer than the first reference section information and the length of the backswing needs to be further shortened (i.e., the backswing needs to be made earlier), the information providing unit 250 can provide information saying "shorten the backswing further" through the display. If it is determined that the length of the first section in the user's golf swing is excessively long compared to the first reference section information, the feedback information generated by the feedback information generation unit 240 can include information indicating that the length of the backswing needs to be made very short (i.e., the backswing needs to be made very early), and the information providing unit 250 may provide information saying "make the backswing very early" through the display.

[0125] Therefore, according to the present invention, by comparing at least one reference reference information included in the reference information with information regarding at least one section, the feedback that can be provided to the user can be made diverse.

[0126] The information providing unit 250 according to an embodiment of the present invention can cumulatively provide information regarding at least one section in response to the golf swing being made two or more times.

[0127] For example, referring to Fig. 6(a), the information providing unit 250 according to an embodiment of the present invention can compare the length of a specific swing section constituting the user's golf swing with the length of the immediately preceding swing section in response to the golf swing being made two or more times, and provide information regarding an increase or decrease in the length of the section compared with the immediately preceding swing. Taking a specific example, referring to Fig. 6(a), when the user makes a golf swing two or more times, the information providing unit 250 can provide, through the display, information that the length of the user's backswing section is shorter than that of the immediately preceding swing, the length of the top section is longer than that of the immediately preceding swing, and the length of the downswing section is shorter than that of the immediately preceding swing.

[0128] As another example, referring to Fig. 6(a), according to an embodiment of the present invention, the information providing unit 250 can display preset reference information (information indicated by a solid line in the graph of Fig. 6(a)) and information regarding at least one section (information indicated by a dotted line in the graph of Fig. 6(a)).

[0129] Fig. 6 is a drawing exemplarily illustrating a situation in which feedback information according to an embodiment of the present invention is provided.

[0130] Referring to Fig. 6(a), as another example, the information providing unit 250 according to an embodiment of the present invention can cumulatively provide, on the screen, the results of the golf swing in response to the user's golf swing being made two or more times (for example, 5 times). Here, the user according to an embodiment of the present invention can easily confirm his or her own golf swing results, and can be assisted in improving strength.

[0131] As another example, the information providing unit 250 according to an embodiment of the present invention can provide statistical information corresponding to at least one of the preset reference information, the information regarding the at least one section, and the feedback information.

[0132] Referring to FIGS. 6(b) and 6(c), the information providing unit 250 according to an embodiment of the present invention can provide, as statistical information, information regarding the average length of the backswing section, information regarding the average length of the downswing section, and average section ratio information corresponding to section ratio information, in response to the user making a golf swing two or more times.

[0133] As yet another example, the information providing unit 250 according to an embodiment of the present invention can provide at least one of a coordinate system element corresponding to the first section information and the second section information and a slope element corresponding to the section ratio information through a display.

[0134] Referring to FIG. 7(a), the information providing unit 250 according to an embodiment of the present invention can visually provide the result of a golf swing on a two-dimensional coordinate composed of an axis corresponding to a first section (i.e., the backswing section) (e.g., the horizontal axis) and an axis corresponding to a second section (e.g., the vertical axis). Here, according to an embodiment of the present invention, since the section ratio information can be represented by the slope elements of the two axes, the information providing unit 250 can provide the slope element corresponding to the section ratio information through a display.

[0135] Therefore, according to the present invention, the user can be assisted in improving their golf skills by being able to easily understand information regarding the length of the first section, the length of the second section, and the section ratio in their own swing through the display.

[0136] Also, referring to FIG. 7, the feedback information generation unit 240 according to an embodiment of the present invention can further generate at least one of information regarding the consistency of a golf swing and information regarding the accuracy calculated based on the similarity between the golf swing and reference information, in response to the user making a golf swing two or more times. Here, the similarity between the golf swing and the reference information can be calculated based on a linear regression analysis model or a similarity model using an artificial neural network, but is not limited thereto.

[0137] Hereinafter, with reference to FIGS. 5 to 7, details will be described regarding content that provides feedback related to a golf swing according to an embodiment of the present invention.

[0138] Fifth Embodiment Referring to FIG. 5(b), according to an embodiment of the present invention, information regarding at least one of a swing posture, a club used, the number of practice times, and a target rhythm can be acquired.

[0139] Next, the feedback information generation unit 240 can preset reference information with reference to information regarding at least one of the club used (for example, an iron or a driver) and the target rhythm.

[0140] Next, the section information acquisition unit 230 can acquire information regarding at least one section constituting the golf swing.

[0141] Next, the feedback information generation unit 240 can generate feedback information regarding the golf swing with reference to the preset reference information and the comparison result of the information regarding the at least one section.

[0142] Therefore, according to the present invention, the reference information can be preset according to the type of specific club selected by the user, and the user can receive the provision of feedback information customized for the specific club.

[0143] Next, the communication unit 260 according to an embodiment of the present invention can perform a function that enables data transmission and reception from / to the image acquisition unit 210, the section information generation unit 220, the section information acquisition unit 230, the feedback information generation unit 240, and the information provision unit 250.

[0144] Finally, the control unit 270 according to an embodiment of the present invention can perform a function of controlling the data flow among the image acquisition unit 210, the section information generation unit 220, the section information acquisition unit 230, the feedback information generation unit 240, the information providing unit 250, and the communication unit 260. That is, the control unit 270 according to an embodiment of the present invention controls the data flow from / to the outside of the information providing system 200 or the data flow among the components of the information providing system 200, so as to control the image acquisition unit 210, the section information generation unit 220, the section information acquisition unit 230, the feedback information generation unit 240, the information providing unit 250, and the communication unit 260 to perform their respective unique functions.

[0145] The embodiments according to the present invention described above can be embodied in the form of program instruction words executable through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium can include program instruction words, data files, data structures, etc. alone or in combination. The program instruction words recorded on the computer-readable recording medium can be those specially designed and configured for the present invention or those publicly known and usable by those skilled in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instruction words, such as ROM, RAM, flash memory, etc. Examples of program instruction words include not only machine language codes made by compilers but also high-level language codes executable by a computer using an interpreter or the like. The hardware device can be changed into one or more software modules to perform the processing according to the present invention, and vice versa.

[0146] As described above, the present invention has been explained by way of specific examples such as specific components and drawings, but this is merely provided to assist in a more general understanding of the present invention. The present invention is not limited to the above examples, and those with ordinary knowledge in the technical field to which the present invention pertains can attempt various modifications and changes from such descriptions.

[0147] Therefore, the idea of the present invention should not be defined as being limited to the described examples, and not only the scope of the following claims, but also all scopes equivalent to or equivalently changed from this scope of claims belong to the category of the idea of the present invention.

Explanation of Reference Numerals

[0148] 100: Communication network 200: Information providing system 210: Image acquisition unit 220: Section information generation unit 230: Section information acquisition unit 240: Feedback information generation unit 250: Information providing unit 260: Communication unit 270: Control unit 300: Device 410: First image 411: First joint information 420: Second image 421: Second joint information 430: Third image 431: Third joint information

Claims

【Claim 1】 A method for providing information regarding time intervals occupied by partial movements constituting a golf swing, comprising: continuously acquiring a first image and a second image regarding the golf swing; generating information regarding two or more time intervals regarding two or more partial movements constituting the golf swing by determining whether there is identity between a time interval corresponding to the first image and a time interval corresponding to the second image with reference to a result of comparing first joint information of the first image and second joint information of the second image; [[ID= ​ ​ ​ ​ ​ ​ ​ ​ ​ With reference to the result of comparing the first joint information of the first image and the second joint information of the second image, by determining whether the time intervals corresponding to the first image and the time intervals corresponding to the second image are the same, it includes an interval information generation unit that generates information regarding two or more time intervals related to two or more partial operations constituting the golf swing. Corresponding to the determination that the time interval corresponding to the first image and the time interval corresponding to the second image are not the same, the interval information generation unit determines a point in time at which the time interval corresponding to the first image and the time interval corresponding to the second image are divided between the first point in time when the first image is acquired and the second point in time when the second image is acquired. The point in time at which the time interval corresponding to the second image and the time division corresponding to the first image are divided by the interval information generation unit is a system that determines that the higher the similarity between the first image and the second image, the closer the point in time is to the first point in time than the second point in time. **Claim 6**: At least the two or more time intervals include a first time interval related to a first partial operation and a second time interval related to a second partial operation. The system according to claim 5, wherein the information regarding the two or more time intervals includes information regarding the ratio of the lengths of the first time interval and the second time interval. **Claim 7**: The system according to claim 5, wherein at least one of the first image and the second image is a reference image determined by a preset condition for dividing a time interval. Above

Images