Marker-free movement capture system and a method for measuring movement of a human body

Abstract

A markerless movement capture system measures human body motion during sports activities in natural field environments. It includes a fixed first camera that records images of a person with sports equipment within a designated area. One or more second cameras are positioned at fixed points to capture simultaneous footage of the person during activity. A computer system processes the data from both cameras. The system also employs mobile carriers to transport the cameras into the field environment. A control unit directs the cameras and carriers. When the system detects a human action, it analyzes image data captured over a set time window before and after the action to assess movement.

Description

FIELD OF THE INVENTION

[0001] The present invention relates to a marker-free movement capture system and a method for measuring movement of a human body while performing a sport activity in natural field space characteristic for the sport.BACKGROUND OF THE INVENTION

[0002] Golf training booths are commonly used to improve golfers'swing mechanics and overall performance through technologies like cameras and simulators. However, these booths have key limitations.

[0003] Current booths lack mobility, as they are typically large, fixed installations requiring dedicated spaces, limiting convenience and accessibility. High accuracy in performance analysis often depends on intrusive markers or sensors attached to the user, which can disrupt natural movements. Additionally, these booths fail to realistically simulate golf course conditions, offering only limited replication of environmental factors such as wind or terrain.

[0004] There is a clear need for a portable, marker-free training solution that provides accurate analysis and a more immersive golfing experience, addressing the shortcomings of existing technologies.SUMMARY OF THE INVENTION

[0005] It is an object of the invention to provide a more versatile golf training system, enabling golfers to practice under real golf course conditions.

[0006] In a first aspect of the invention, a marker-free movement capture system is provided for measuring movement of a human body while performing a sport activity in a natural field space characteristic for the sport, comprising:

[0007] a first camera positioned at a first fixed location relative to an image capturing space, configured to capture image data of the human body carrying the sport equipment during the sport activity within the image capturing space,

[0008] at least one second camera positioned at one or more second fixed locations relative to the image capturing space, configured to capture image data of the human body carrying the sport equipment during the sport activity simultaneous to the first camera,

[0009] a computer system comprising a receiver and a processor for receiving and processing the image data collected by the first camera and the at least one second camera,

[0010] wherein the system further comprises:

[0011] at least one mobile carrier for carrying the first and the at least one second camera,

[0012] a control unit,

[0013] wherein the control unit is configured to instruct the at least one mobile carrier together with the cameras to move to the first natural field space,

[0014] wherein, in response to detecting an action executed by the human within the natural field space, the image data captured by the first and at least one second camera, collected over a predefined time window encompassing both before and after the detected action, is provided as input to the computer system to calculate the human's movement.

[0015] In an embodiment, the sport is golf and where the human is a golfer executing a golf swing with a golf club, where the first natural field space includes a first space within a golf course, and where the detected action corresponds to the moment of the golfer strikes the golf ball to a second natural field space, and where the calculated movement includes the movement of the human body and / or or the golf club.

[0016] Accordingly, a portable and marker-free system is provided that overcomes the key limitations of traditional golf training booths and that allows golfers to practice under real golf course conditions. This includes scenarios such as practicing driving, using irons, or putting, catering to players of all skill levels.

[0017] To ensure the accuracy of the detection of the action, high-speed digital camera(s) may be employed capable of capturing a rapid sequence of images, allowing for precise tracking of the golf ball's movement before, during, and immediately after the strike. By leveraging this high frame rate, the system can reliably identify the exact moment when the ball leaves the field of view, ensuring that the detected action aligns closely with the golfer's strike. This approach is particularly effective in scenarios requiring high precision, such as analyzing the dynamics of a golf swing or optimizing performance in a simulated or natural environment.

[0018] The computer system may be an external computer system such as any type of a cloud server, or a mobile computer system such as a laptop, or any type of a “home-based” computer.

[0019] By utilizing only the captured image data collected over the predefined time window it is ensured that the system processes only the essential image data captured within a predefined time window, ensuring efficient use of processing power for movement analysis. The predefined time window may include time t1 before the action detected and t2 after the detected action, where the predefined time window is Δt=t1+t2. The time t1 and t2 may depend on the sport activity, where as an example for golf, t1 and t2 may be equal and be around 1 sec., whereas for other sport activities, t1 and t2 may differ and be less or more than 1 sec. Preferably, irrespective of the sport activity, t1 and t2 may be adjustable, even by the human exercising the sport activity. As an example, in case the sport activity is golf, t1 and t2 may be selected such that t1≠t2.

[0020] Further, there is considerable advantage in recording a golf swing on a golf course or in a training area, where players encounter challenges that differ significantly from those in a booth setting. These challenges include varying surfaces where the ball lies (such as fairway, green, bunker, sand, roads, high grass, or rough), sloped terrain affecting the ball's position or the player's stance, and diverse weather conditions. Additionally, the player gains a clear view of the target, which can influence the shot. For instance, the shot might need to be very high to stop the ball on the green or curved left or right to avoid obstacles like trees between the ball and the target. Thus, the system according to the present invention enables swing measurements that are reliable, accurate, and fast, ideally matching the performance and precision of similar hardware in a booth setting under optimal conditions.

[0021] In an embodiment, the system further comprises a display configured to display the results of the analyzed movement of the human body. In that way, in case the sport activity is golf, the golfer may be provided with immediate feedback based on the analyzed movements of the golfer's body and / or the golf club. The players and / or coaches may also access the video and the data offline when connected to a computer device which makes the system not dependent on wireless communication. This of course also applies for other types of sports.

[0022] In an embodiment, the marker-free movement capture system further comprises a location triggering device configured to trigger the first location data of the first natural field space utilized by the control unit. The location triggering device may comprise any type of a portable computer, such as a mobile phone, e.g. carried by the golfer where the triggering may be performed by the golfer.

[0023] In an embodiment, the marker-free movement capture system further comprises a direction triggering device for triggering direction data, in response to a command from the user or the golfer, indicating the intended trajectory of the golf ball. Both the location triggering device and the direction triggering device may be built into one and the same device, e.g. a mobile phone, where the direction triggering device may be a built in compass on the mobile phone where e. g via pointing the mobile phone to the intended direction, and via any type of triggering command from e.g. the golfer, the direction data may be triggered.

[0024] In an embodiment, the at least one mobile carrier comprises an Unmanned Aerial Vehicle (UAV) such as a drone. The mobile carrier can be a single unit carrying a camera or multiple carriers, each equipped to hold one or more cameras. Each mobile carrier, when paired with a camera, functions as a single operational unit, for instance, a single drone equipped with one or more cameras.

[0025] An advantage of utilizing drones as mobile carriers is their ability to move dynamically and adapt to different positions and angles, ensuring optimal camera placement for capturing detailed and precise data. For example, multiple drones working in coordination can provide comprehensive coverage of an event or action, such as a golf swing, from various perspectives.

[0026] Furthermore, equipping a single drone with multiple cameras can unlock the potential for advanced imaging capabilities, such as creating high-resolution 3D images or videos. This setup mimics the approach used by Apple for the VisionPro, where synchronized camera systems are used to capture depth and spatial information. The ability to generate 3D images in real time provides a significant advantage for applications such as motion analysis, enabling more detailed and accurate feedback to the user. This could be particularly beneficial in sports like golf, where precise biomechanical feedback can enhance performance and reduce the risk of injury by identifying and correcting improper techniques.

[0027] A specific implementation of this concept could involve outfitting a drone with three camera lenses, each oriented differently and equipped with varying focal lengths. The camera hardware could be configured to use only one lens at a time, akin to the setup commonly found on modern mobile phones. To ensure stability and consistency during recording, a preferred configuration would keep the drone positioned horizontally in the air, as this orientation provides the most stable platform for capturing high-quality footage.

[0028] In an embodiment, the at least one more mobile carrier includes a first and at least one second mobile carrier, where the control unit is further configured to move the mobile carriers to position the cameras at the first and the at least one second fixed location relative to the human, and to instruct the mobile carriers to position the cameras at the corresponding fixed location relative to the human, ensuring that the relative positioning of the cameras is preserved. The position of the cameras and internal calibration may as an example be optimized in a golf training booth, where e. g a first camera in the training booth is located in front of the golf player at a position (x1,y1,z1), another camera is located sidewise relative to the golf player at second position (x2,y2,z2), and a third camera is located at a position (x3,y3,z3) etc., where under this fixed location the cameras are calibrated. These internal coordinates are then maintained at the sports field, e.g. at the golf course. The number of cameras may in an alternative embodiment be two or more. Accordingly, referring to the example where the sport is golf, by remaining this positioning of the cameras is fixed, a reliable and accurate movement measurement of the golfer and / or the golf club is provided. These positions may be stored within the control unit.

[0029] The control unit can be integrated into one or all of the mobile carriers and / or can be part of an external control computer.

[0030] The control unit is in an alternative embodiment further configured to, subsequent to striking the golf ball to the second natural field space, to instruct the first and the at least one second mobile carriers together with the cameras to move to the second natural field space in response to a second location data within the golf course. This may be repeated throughout the golf course exercise while the golfer exercises with different golf clubs, for example driving, using irons, chipping or putting.

[0031] In an embodiment, each of the cameras comprises a communication module for sending the captured image data to the computer system where it is processed.

[0032] In an embodiment, the image data is stored in a storage medium, e.g. in the cameras, where a selected number of swings could then be uploaded to the computer system at some later time point.

[0033] According to a second aspect of the invention, a method is provided for measuring movement of a human body while performing a sport activity in natural field space characteristic for the sport, comprising:

[0034] positioning a first camera at a first fixed location relative to an image capturing space, configured to capture image data of the human body during the sport activity within the image capturing space,

[0035] positioning at least one second camera at one or more second fixed locations relative to the image capturing space, configured to capture image data of the human body during the sport activity simultaneous to the first camera,

[0036] receiving and processing the image data collected by the first camera and the at least one second camera by a computer system,

[0037] wherein the method further comprises:

[0038] providing at least one mobile carrier for carrying the first and the at least one second camera,

[0039] instructing the at least one mobile carrier together with the cameras to move to the first natural field space,

[0040] wherein, in response to detecting an action executed by the human within the natural field space, the image data captured by the first and at least one second cameras, collected over a predefined time window encompassing both before and after the detected action, is provided as input to the computer system to calculate the human's movement.

[0041] The sport, may according to the present invention include, but is not limited to, tennis, skiing, ball sport, just to mention a few, where the sport equipment is a tennis racket, skis and ski poles, and football.

[0042] In an embodiment, according to the method, the sport is golf and where the human is a golfer executing a golf swing with a golf club, where the first natural field space includes a first space within a golf course, and where the detected action corresponds to the moment of the golfer strikes the golf ball to a second natural field space, and where the calculated movement includes the movement of the golfer body and / or or the golf club.

[0043] In an embodiment, the at least one more mobile carrier includes a first and at least one second mobile carrier, where the method further comprises:

[0044] moving the first mobile carrier to position the first camera at the first fixed location relative to the human,

[0045] moving the at least one second mobile carrier to position the at least one second camera to the corresponding to at least one fixed location relative to the human, ensuring that the relative positioning of the cameras is maintained.

[0046] In a preferred embodiment, the method further comprises performing a calibration, such as intrinsic calibration, on the first camera and the at least one second camera while positioned at the first and at least one second fixed locations, prior to capturing image data capture. This may include using charuco board, which is commonly used when calibrating cameras used to obtain accurate 3D measurements of objects and as well in golf booths. Similar calibration may be repeated on the first camera and the at least one second camera within the first natural field space, such as using again such charuco boards or similar means that are useful in calibrating the camera. This ensured high accuracy and quality in the movement measurement.

[0047] In an embodiment, a drone with calibration object that may be used for extrinsic calibration and to be flown into the natural field space such that a calibration can be formed fully automatically ensuring that measurements performed in that natural field space with the set location of the cameras will be more accurate.

[0048] In an embodiment, the human, e.g. the golfer, may wear clothing, e.g. a shirt, having specific patterns that may be utilized to make the necessary calibration at the golf field.

[0049] Further, the first and the at least one second mobile carriers may in an embodiment comprise a positioning module configured to internally position the mobile carriers within the first natural field space before performing the calibration as mentioned above.

[0050] Accordingly, the improved method provides valuable feedback to the golfer, which can significantly enhance the golfer's performance. This feedback may include detailed analysis of the golf club's motion, such as its speed, angle, and trajectory, enabling the golfer to refine their swing technique. Additionally, the system offers insights into the biomechanics of the golfer's movements by tracking and displaying the rotations and positions of key body parts, including the pelvis, upper torso, arms, and hands.

[0051] For instance, the feedback can highlight the degree of rotation of the pelvis during the backswing and follow-through, which is critical for generating power and maintaining balance. It can also show how the upper torso aligns and rotates in relation to the pelvis, providing cues on improving swing mechanics and reducing strain on the body. Furthermore, by monitoring the movements of the arms and hands, the system can identify inefficiencies, such as improper wrist angles or timing issues during the downswing, and suggest corrections to achieve better accuracy and consistency.

[0052] This comprehensive feedback not only helps golfers make incremental improvements to their form but also facilitates a deeper understanding of how each element of their body's motion contributes to the overall swing. By leveraging this data, golfers can achieve greater control, optimize their performance, and reduce the risk of injury, ultimately enhancing their overall experience on the course or in the simulator.

[0053] In general, the various aspects of the invention may be combined and coupled in any way possible within the scope of the invention. These and other aspects, features and / or advantages of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Embodiments of the invention will be described, by way of example only, with reference to the drawings, in which

[0055] FIG. 1 illustrates graphically a marker-free motion capture booth system comprising five cameras,

[0056] FIG. 2 illustrates a system according to the present invention, which replicates the setup of the golf booth shown in FIG. 1 in a first natural field space,

[0057] FIG. 3 provides a detailed illustration of a scenario where the detected action is associated with the moment the golfer (not shown) strikes the golf ball using a golf club, and

[0058] FIG. 4 shows a flowchart of a method according to the present invention for measuring movement of a human body while performing a sport activity in natural field space characteristic for the sport.DESCRIPTION OF EMBODIMENTS

[0059] The present invention relates to a marker-free movement capture system for measuring movement of a human body while performing a sport activity.

[0060] In the following, for simplicity, it will be assumed that the sport is golf and where the human is a golfer executing a golf swing with a golf club.

[0061] FIG. 1 illustrates graphically a marker-free motion capture booth system 100 comprising five cameras 102-106 that have been calibrated e.g. via intrinsic calibration, where the cameras are positioned at fixed locations relative to each other and the golfer 101 while the golfer is practicing a golf swing and strikes the golf ball 107 toward a durable projection screen 108, which forms part of a simulator setup displaying a virtual golf course or practice environment.

[0062] FIG. 2 illustrates a system 200 according to the present invention, which replicates the setup of the golf booth shown in FIG. 1. In this configuration, the golfer practices driving in a first natural field space 208. The system comprises five cameras 202-206 mounted on separate drones 210-214. The number of cameras in FIGS. 1 and 2 can vary, being fewer or greater than five. Additionally, the cameras in FIG. 2 may be integrated directly into the drones.

[0063] The system 200 further comprises a control unit and a computer system 215, where the control unit is configured, among other functions, to manage the drones 210-214, and the computer system is configured to receive and process the image data captured by the cameras 202-206. The control unit may either be a processor integrated into the computer system 215 or be distributed as part of the drones and cameras.

[0064] The image data processed by the computer system 215 includes image data collected over a predefined time window encompassing both before and after a detected action, which in this scenario is where the golfer strikes the golf ball 207 from the first natural field space 208 to the second natural field space 209.

[0065] In an alternative embodiment, the cameras 202-206 may be calibrated with some regular intervals or upon a request by the golfer, e.g. using charuco boards (not shown) or similar means that are useful in calibrating the camera.

[0066] In another alternative embodiment, the cameras 202-206 rely on the calibration performed e.g. in a golf booth, such as the one shown in FIG. 1, where it ensured that the internal position between the cameras 202-206 and the golfer 201 is essentially the same as shown in FIG. 1.

[0067] The golfer might even wear clothing, e.g. a shirt, having a specific pattern that may be utilized to make the necessary calibration at the golf field.

[0068] FIG. 3 provides a detailed illustration of a scenario where the detected action is associated with the moment the golfer (not shown) strikes the golf ball 307 using a golf club 301. The golf club utilized in this context could be one of several types, including a driver, an iron, or a putter, depending on the specific shot being practiced or executed. The detection of this action may be achieved by monitoring the motion of the golf ball, with the triggering event occurring when the ball is no longer visible within the captured image frames. This disappearance serves as a clear indication that the ball has been struck and is in motion.

[0069] To ensure the accuracy of this detection, high-speed digital cameras may be employed. These cameras are capable of capturing a rapid sequence of images, allowing for precise tracking of the golf ball's movement before, during, and immediately after the strike. By leveraging this high frame rate, the system can reliably identify the exact moment when the ball leaves the field of view, ensuring that the detected action aligns closely with the golfer's strike. This approach is particularly effective in scenarios requiring high precision, such as analyzing the dynamics of a golf swing or optimizing performance in a simulated or natural environment.

[0070] FIG. 4 shows a flowchart of a method according to the present invention for measuring movement of a human body while performing a sport activity in natural field space characteristic for the sport.

[0071] In a first step (S1) 401, a first camera is positioned at a first fixed location relative to an image capturing space, configured to capture image data of the human body during the sport activity within the image capturing space.

[0072] In a second step (S2) 402, at least one second camera is positioned at one or more second fixed locations relative to the image capturing space, configured to capture image data of the human body during the sport activity simultaneous to the first camera. The cameras are calibrated such as by performing an intrinsic calibration, on the first camera and the at least one second camera while positioned at the first and at least one second fixed locations, prior to capturing image data capture

[0073] In a third step (S3) 403, the image data is received and processing the image data collected by the first camera and the at least one second camera by a computer system which may as an example include any type of computer system platform.

[0074] In a fourth step (S4) 404, at least one mobile carrier is provided, e.g. one carrier for one camera, for carrying the cameras.

[0075] In a fifth step (S5) 405, the at least one mobile carrier together with the cameras is instructed to move to the first natural field space. In response to detecting an action executed by the human within the natural field space, the image data captured by the first and at least one second camera, collected over a predefined time window encompassing both before and after the detected action, is provided as input to the computer system to calculate the human's movement.

[0076] As discussed in relation to previous figures, the sport may include any type of sport, tennis, skiing, or golf the sport as discussed in relation to FIGS. 1 to 3.

[0077] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Examples

Embodiment Construction

[0059]The present invention relates to a marker-free movement capture system for measuring movement of a human body while performing a sport activity.

[0060]In the following, for simplicity, it will be assumed that the sport is golf and where the human is a golfer executing a golf swing with a golf club.

[0061]FIG. 1 illustrates graphically a marker-free motion capture booth system 100 comprising five cameras 102-106 that have been calibrated e.g. via intrinsic calibration, where the cameras are positioned at fixed locations relative to each other and the golfer 101 while the golfer is practicing a golf swing and strikes the golf ball 107 toward a durable projection screen 108, which forms part of a simulator setup displaying a virtual golf course or practice environment.

[0062]FIG. 2 illustrates a system 200 according to the present invention, which replicates the setup of the golf booth shown in FIG. 1. In this configuration, the golfer practices driving in a first natural field spac...

FIELD OF THE INVENTION

[0001] The present invention relates to a marker-free movement capture system and a method for measuring movement of a human body while performing a sport activity in natural field space characteristic for the sport.BACKGROUND OF THE INVENTION

[0002] Golf training booths are commonly used to improve golfers'swing mechanics and overall performance through technologies like cameras and simulators. However, these booths have key limitations.

[0003] Current booths lack mobility, as they are typically large, fixed installations requiring dedicated spaces, limiting convenience and accessibility. High accuracy in performance analysis often depends on intrusive markers or sensors attached to the user, which can disrupt natural movements. Additionally, these booths fail to realistically simulate golf course conditions, offering only limited replication of environmental factors such as wind or terrain.

[0004] There is a clear need for a portable, marker-free training solution that provides accurate analysis and a more immersive golfing experience, addressing the shortcomings of existing technologies.SUMMARY OF THE INVENTION

[0005] It is an object of the invention to provide a more versatile golf training system, enabling golfers to practice under real golf course conditions.

[0006] In a first aspect of the invention, a marker-free movement capture system is provided for measuring movement of a human body while performing a sport activity in a natural field space characteristic for the sport, comprising:

[0007] a first camera positioned at a first fixed location relative to an image capturing space, configured to capture image data of the human body carrying the sport equipment during the sport activity within the image capturing space,

[0008] at least one second camera positioned at one or more second fixed locations relative to the image capturing space, configured to capture image data of the human body carrying the sport equipment during the sport activity simultaneous to the first camera,

[0009] a computer system comprising a receiver and a processor for receiving and processing the image data collected by the first camera and the at least one second camera,

[0010] wherein the system further comprises:

[0011] at least one mobile carrier for carrying the first and the at least one second camera,

[0012] a control unit,

[0013] wherein the control unit is configured to instruct the at least one mobile carrier together with the cameras to move to the first natural field space,

[0014] wherein, in response to detecting an action executed by the human within the natural field space, the image data captured by the first and at least one second camera, collected over a predefined time window encompassing both before and after the detected action, is provided as input to the computer system to calculate the human's movement.

[0015] In an embodiment, the sport is golf and where the human is a golfer executing a golf swing with a golf club, where the first natural field space includes a first space within a golf course, and where the detected action corresponds to the moment of the golfer strikes the golf ball to a second natural field space, and where the calculated movement includes the movement of the human body and / or or the golf club.

[0016] Accordingly, a portable and marker-free system is provided that overcomes the key limitations of traditional golf training booths and that allows golfers to practice under real golf course conditions. This includes scenarios such as practicing driving, using irons, or putting, catering to players of all skill levels.

[0017] To ensure the accuracy of the detection of the action, high-speed digital camera(s) may be employed capable of capturing a rapid sequence of images, allowing for precise tracking of the golf ball's movement before, during, and immediately after the strike. By leveraging this high frame rate, the system can reliably identify the exact moment when the ball leaves the field of view, ensuring that the detected action aligns closely with the golfer's strike. This approach is particularly effective in scenarios requiring high precision, such as analyzing the dynamics of a golf swing or optimizing performance in a simulated or natural environment.

[0018] The computer system may be an external computer system such as any type of a cloud server, or a mobile computer system such as a laptop, or any type of a “home-based” computer.

[0019] By utilizing only the captured image data collected over the predefined time window it is ensured that the system processes only the essential image data captured within a predefined time window, ensuring efficient use of processing power for movement analysis. The predefined time window may include time t1 before the action detected and t2 after the detected action, where the predefined time window is Δt=t1+t2. The time t1 and t2 may depend on the sport activity, where as an example for golf, t1 and t2 may be equal and be around 1 sec., whereas for other sport activities, t1 and t2 may differ and be less or more than 1 sec. Preferably, irrespective of the sport activity, t1 and t2 may be adjustable, even by the human exercising the sport activity. As an example, in case the sport activity is golf, t1 and t2 may be selected such that t1≠t2.

[0020] Further, there is considerable advantage in recording a golf swing on a golf course or in a training area, where players encounter challenges that differ significantly from those in a booth setting. These challenges include varying surfaces where the ball lies (such as fairway, green, bunker, sand, roads, high grass, or rough), sloped terrain affecting the ball's position or the player's stance, and diverse weather conditions. Additionally, the player gains a clear view of the target, which can influence the shot. For instance, the shot might need to be very high to stop the ball on the green or curved left or right to avoid obstacles like trees between the ball and the target. Thus, the system according to the present invention enables swing measurements that are reliable, accurate, and fast, ideally matching the performance and precision of similar hardware in a booth setting under optimal conditions.

[0021] In an embodiment, the system further comprises a display configured to display the results of the analyzed movement of the human body. In that way, in case the sport activity is golf, the golfer may be provided with immediate feedback based on the analyzed movements of the golfer's body and / or the golf club. The players and / or coaches may also access the video and the data offline when connected to a computer device which makes the system not dependent on wireless communication. This of course also applies for other types of sports.

[0022] In an embodiment, the marker-free movement capture system further comprises a location triggering device configured to trigger the first location data of the first natural field space utilized by the control unit. The location triggering device may comprise any type of a portable computer, such as a mobile phone, e.g. carried by the golfer where the triggering may be performed by the golfer.

[0023] In an embodiment, the marker-free movement capture system further comprises a direction triggering device for triggering direction data, in response to a command from the user or the golfer, indicating the intended trajectory of the golf ball. Both the location triggering device and the direction triggering device may be built into one and the same device, e.g. a mobile phone, where the direction triggering device may be a built in compass on the mobile phone where e. g via pointing the mobile phone to the intended direction, and via any type of triggering command from e.g. the golfer, the direction data may be triggered.

[0024] In an embodiment, the at least one mobile carrier comprises an Unmanned Aerial Vehicle (UAV) such as a drone. The mobile carrier can be a single unit carrying a camera or multiple carriers, each equipped to hold one or more cameras. Each mobile carrier, when paired with a camera, functions as a single operational unit, for instance, a single drone equipped with one or more cameras.

[0025] An advantage of utilizing drones as mobile carriers is their ability to move dynamically and adapt to different positions and angles, ensuring optimal camera placement for capturing detailed and precise data. For example, multiple drones working in coordination can provide comprehensive coverage of an event or action, such as a golf swing, from various perspectives.

[0026] Furthermore, equipping a single drone with multiple cameras can unlock the potential for advanced imaging capabilities, such as creating high-resolution 3D images or videos. This setup mimics the approach used by Apple for the VisionPro, where synchronized camera systems are used to capture depth and spatial information. The ability to generate 3D images in real time provides a significant advantage for applications such as motion analysis, enabling more detailed and accurate feedback to the user. This could be particularly beneficial in sports like golf, where precise biomechanical feedback can enhance performance and reduce the risk of injury by identifying and correcting improper techniques.

[0027] A specific implementation of this concept could involve outfitting a drone with three camera lenses, each oriented differently and equipped with varying focal lengths. The camera hardware could be configured to use only one lens at a time, akin to the setup commonly found on modern mobile phones. To ensure stability and consistency during recording, a preferred configuration would keep the drone positioned horizontally in the air, as this orientation provides the most stable platform for capturing high-quality footage.

[0028] In an embodiment, the at least one more mobile carrier includes a first and at least one second mobile carrier, where the control unit is further configured to move the mobile carriers to position the cameras at the first and the at least one second fixed location relative to the human, and to instruct the mobile carriers to position the cameras at the corresponding fixed location relative to the human, ensuring that the relative positioning of the cameras is preserved. The position of the cameras and internal calibration may as an example be optimized in a golf training booth, where e. g a first camera in the training booth is located in front of the golf player at a position (x1,y1,z1), another camera is located sidewise relative to the golf player at second position (x2,y2,z2), and a third camera is located at a position (x3,y3,z3) etc., where under this fixed location the cameras are calibrated. These internal coordinates are then maintained at the sports field, e.g. at the golf course. The number of cameras may in an alternative embodiment be two or more. Accordingly, referring to the example where the sport is golf, by remaining this positioning of the cameras is fixed, a reliable and accurate movement measurement of the golfer and / or the golf club is provided. These positions may be stored within the control unit.

[0029] The control unit can be integrated into one or all of the mobile carriers and / or can be part of an external control computer.

[0030] The control unit is in an alternative embodiment further configured to, subsequent to striking the golf ball to the second natural field space, to instruct the first and the at least one second mobile carriers together with the cameras to move to the second natural field space in response to a second location data within the golf course. This may be repeated throughout the golf course exercise while the golfer exercises with different golf clubs, for example driving, using irons, chipping or putting.

[0031] In an embodiment, each of the cameras comprises a communication module for sending the captured image data to the computer system where it is processed.

[0032] In an embodiment, the image data is stored in a storage medium, e.g. in the cameras, where a selected number of swings could then be uploaded to the computer system at some later time point.

[0033] According to a second aspect of the invention, a method is provided for measuring movement of a human body while performing a sport activity in natural field space characteristic for the sport, comprising:

[0034] positioning a first camera at a first fixed location relative to an image capturing space, configured to capture image data of the human body during the sport activity within the image capturing space,

[0035] positioning at least one second camera at one or more second fixed locations relative to the image capturing space, configured to capture image data of the human body during the sport activity simultaneous to the first camera,

[0036] receiving and processing the image data collected by the first camera and the at least one second camera by a computer system,

[0037] wherein the method further comprises:

[0038] providing at least one mobile carrier for carrying the first and the at least one second camera,

[0039] instructing the at least one mobile carrier together with the cameras to move to the first natural field space,

[0040] wherein, in response to detecting an action executed by the human within the natural field space, the image data captured by the first and at least one second cameras, collected over a predefined time window encompassing both before and after the detected action, is provided as input to the computer system to calculate the human's movement.

[0041] The sport, may according to the present invention include, but is not limited to, tennis, skiing, ball sport, just to mention a few, where the sport equipment is a tennis racket, skis and ski poles, and football.

[0042] In an embodiment, according to the method, the sport is golf and where the human is a golfer executing a golf swing with a golf club, where the first natural field space includes a first space within a golf course, and where the detected action corresponds to the moment of the golfer strikes the golf ball to a second natural field space, and where the calculated movement includes the movement of the golfer body and / or or the golf club.

[0043] In an embodiment, the at least one more mobile carrier includes a first and at least one second mobile carrier, where the method further comprises:

[0044] moving the first mobile carrier to position the first camera at the first fixed location relative to the human,

[0045] moving the at least one second mobile carrier to position the at least one second camera to the corresponding to at least one fixed location relative to the human, ensuring that the relative positioning of the cameras is maintained.

[0046] In a preferred embodiment, the method further comprises performing a calibration, such as intrinsic calibration, on the first camera and the at least one second camera while positioned at the first and at least one second fixed locations, prior to capturing image data capture. This may include using charuco board, which is commonly used when calibrating cameras used to obtain accurate 3D measurements of objects and as well in golf booths. Similar calibration may be repeated on the first camera and the at least one second camera within the first natural field space, such as using again such charuco boards or similar means that are useful in calibrating the camera. This ensured high accuracy and quality in the movement measurement.

[0047] In an embodiment, a drone with calibration object that may be used for extrinsic calibration and to be flown into the natural field space such that a calibration can be formed fully automatically ensuring that measurements performed in that natural field space with the set location of the cameras will be more accurate.

[0048] In an embodiment, the human, e.g. the golfer, may wear clothing, e.g. a shirt, having specific patterns that may be utilized to make the necessary calibration at the golf field.

[0049] Further, the first and the at least one second mobile carriers may in an embodiment comprise a positioning module configured to internally position the mobile carriers within the first natural field space before performing the calibration as mentioned above.

[0050] Accordingly, the improved method provides valuable feedback to the golfer, which can significantly enhance the golfer's performance. This feedback may include detailed analysis of the golf club's motion, such as its speed, angle, and trajectory, enabling the golfer to refine their swing technique. Additionally, the system offers insights into the biomechanics of the golfer's movements by tracking and displaying the rotations and positions of key body parts, including the pelvis, upper torso, arms, and hands.

[0051] For instance, the feedback can highlight the degree of rotation of the pelvis during the backswing and follow-through, which is critical for generating power and maintaining balance. It can also show how the upper torso aligns and rotates in relation to the pelvis, providing cues on improving swing mechanics and reducing strain on the body. Furthermore, by monitoring the movements of the arms and hands, the system can identify inefficiencies, such as improper wrist angles or timing issues during the downswing, and suggest corrections to achieve better accuracy and consistency.

[0052] This comprehensive feedback not only helps golfers make incremental improvements to their form but also facilitates a deeper understanding of how each element of their body's motion contributes to the overall swing. By leveraging this data, golfers can achieve greater control, optimize their performance, and reduce the risk of injury, ultimately enhancing their overall experience on the course or in the simulator.

[0053] In general, the various aspects of the invention may be combined and coupled in any way possible within the scope of the invention. These and other aspects, features and / or advantages of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Embodiments of the invention will be described, by way of example only, with reference to the drawings, in which

[0055] FIG. 1 illustrates graphically a marker-free motion capture booth system comprising five cameras,

[0056] FIG. 2 illustrates a system according to the present invention, which replicates the setup of the golf booth shown in FIG. 1 in a first natural field space,

[0057] FIG. 3 provides a detailed illustration of a scenario where the detected action is associated with the moment the golfer (not shown) strikes the golf ball using a golf club, and

[0058] FIG. 4 shows a flowchart of a method according to the present invention for measuring movement of a human body while performing a sport activity in natural field space characteristic for the sport.DESCRIPTION OF EMBODIMENTS

[0059] The present invention relates to a marker-free movement capture system for measuring movement of a human body while performing a sport activity.

[0060] In the following, for simplicity, it will be assumed that the sport is golf and where the human is a golfer executing a golf swing with a golf club.

[0061] FIG. 1 illustrates graphically a marker-free motion capture booth system 100 comprising five cameras 102-106 that have been calibrated e.g. via intrinsic calibration, where the cameras are positioned at fixed locations relative to each other and the golfer 101 while the golfer is practicing a golf swing and strikes the golf ball 107 toward a durable projection screen 108, which forms part of a simulator setup displaying a virtual golf course or practice environment.

[0062] FIG. 2 illustrates a system 200 according to the present invention, which replicates the setup of the golf booth shown in FIG. 1. In this configuration, the golfer practices driving in a first natural field space 208. The system comprises five cameras 202-206 mounted on separate drones 210-214. The number of cameras in FIGS. 1 and 2 can vary, being fewer or greater than five. Additionally, the cameras in FIG. 2 may be integrated directly into the drones.

[0063] The system 200 further comprises a control unit and a computer system 215, where the control unit is configured, among other functions, to manage the drones 210-214, and the computer system is configured to receive and process the image data captured by the cameras 202-206. The control unit may either be a processor integrated into the computer system 215 or be distributed as part of the drones and cameras.

[0064] The image data processed by the computer system 215 includes image data collected over a predefined time window encompassing both before and after a detected action, which in this scenario is where the golfer strikes the golf ball 207 from the first natural field space 208 to the second natural field space 209.

[0065] In an alternative embodiment, the cameras 202-206 may be calibrated with some regular intervals or upon a request by the golfer, e.g. using charuco boards (not shown) or similar means that are useful in calibrating the camera.

[0066] In another alternative embodiment, the cameras 202-206 rely on the calibration performed e.g. in a golf booth, such as the one shown in FIG. 1, where it ensured that the internal position between the cameras 202-206 and the golfer 201 is essentially the same as shown in FIG. 1.

[0067] The golfer might even wear clothing, e.g. a shirt, having a specific pattern that may be utilized to make the necessary calibration at the golf field.

[0068] FIG. 3 provides a detailed illustration of a scenario where the detected action is associated with the moment the golfer (not shown) strikes the golf ball 307 using a golf club 301. The golf club utilized in this context could be one of several types, including a driver, an iron, or a putter, depending on the specific shot being practiced or executed. The detection of this action may be achieved by monitoring the motion of the golf ball, with the triggering event occurring when the ball is no longer visible within the captured image frames. This disappearance serves as a clear indication that the ball has been struck and is in motion.

[0069] To ensure the accuracy of this detection, high-speed digital cameras may be employed. These cameras are capable of capturing a rapid sequence of images, allowing for precise tracking of the golf ball's movement before, during, and immediately after the strike. By leveraging this high frame rate, the system can reliably identify the exact moment when the ball leaves the field of view, ensuring that the detected action aligns closely with the golfer's strike. This approach is particularly effective in scenarios requiring high precision, such as analyzing the dynamics of a golf swing or optimizing performance in a simulated or natural environment.

[0070] FIG. 4 shows a flowchart of a method according to the present invention for measuring movement of a human body while performing a sport activity in natural field space characteristic for the sport.

[0071] In a first step (S1) 401, a first camera is positioned at a first fixed location relative to an image capturing space, configured to capture image data of the human body during the sport activity within the image capturing space.

[0072] In a second step (S2) 402, at least one second camera is positioned at one or more second fixed locations relative to the image capturing space, configured to capture image data of the human body during the sport activity simultaneous to the first camera. The cameras are calibrated such as by performing an intrinsic calibration, on the first camera and the at least one second camera while positioned at the first and at least one second fixed locations, prior to capturing image data capture

[0073] In a third step (S3) 403, the image data is received and processing the image data collected by the first camera and the at least one second camera by a computer system which may as an example include any type of computer system platform.

[0074] In a fourth step (S4) 404, at least one mobile carrier is provided, e.g. one carrier for one camera, for carrying the cameras.

[0075] In a fifth step (S5) 405, the at least one mobile carrier together with the cameras is instructed to move to the first natural field space. In response to detecting an action executed by the human within the natural field space, the image data captured by the first and at least one second camera, collected over a predefined time window encompassing both before and after the detected action, is provided as input to the computer system to calculate the human's movement.

[0076] As discussed in relation to previous figures, the sport may include any type of sport, tennis, skiing, or golf the sport as discussed in relation to FIGS. 1 to 3.

[0077] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Examples

Embodiment Construction

[0059]The present invention relates to a marker-free movement capture system for measuring movement of a human body while performing a sport activity.

[0060]In the following, for simplicity, it will be assumed that the sport is golf and where the human is a golfer executing a golf swing with a golf club.

[0061]FIG. 1 illustrates graphically a marker-free motion capture booth system 100 comprising five cameras 102-106 that have been calibrated e.g. via intrinsic calibration, where the cameras are positioned at fixed locations relative to each other and the golfer 101 while the golfer is practicing a golf swing and strikes the golf ball 107 toward a durable projection screen 108, which forms part of a simulator setup displaying a virtual golf course or practice environment.

[0062]FIG. 2 illustrates a system 200 according to the present invention, which replicates the setup of the golf booth shown in FIG. 1. In this configuration, the golfer practices driving in a first natural field spac...

Claims

1. A marker-free movement capture system for measuring movement of a human body while performing a sport activity in natural field space characteristic for the sport, comprising:a first camera positioned at a first fixed location relative to an image capturing space, configured to capture image data of the human body carrying the sport equipment during the sport activity within the image capturing space,at least one second camera positioned at one or more second fixed locations relative to the image capturing space, configured to capture image data of the human body carrying the sport equipment during the sport activity simultaneous to the first camera,a computer system comprising a receiver and a processor for receiving and processing the image data collected by the first camera and the at least one second camera,wherein the system further comprises:at least one mobile carrier for carrying the first and the at least one second camera,a control unit,wherein the control unit is configured to instruct the at least one mobile carrier together with the cameras to move to the first natural field space,wherein, in response to detecting an action executed by the human within the natural field space, the image data captured by the first and at least one second camera, collected over a predefined time window encompassing both before and after the detected action, is provided as input to the computer system to calculate the human's movement.

2. The marker-free movement capture system according to claim 1, wherein the sport is golf and where the human is a golfer executing a golf swing with a golf club, where the first natural field space includes a first space within a golf course, and where the detected action corresponds to the moment of the golfer strikes the golf ball to a second natural field space, and where the calculated movement includes the movement of the golfer body and / or or the golf club.

3. The marker-free movement capture system according to claim 1, further comprising a location triggering device configured to trigger the first location data of the first natural field space.

4. The marker-free movement capture system according to claims 2, further comprising a direction triggering device for triggering direction data, in response to a command from the user, indicating the intended trajectory of the golf ball.

5. The marker-free movement capture system according to claim 1, where the at least one more mobile carrier includes a first and at least one second mobile carrier, where the control unit is further configured to move the mobile carriers to position the cameras at the first and the at least one second fixed location relative to the human, and to instruct the mobile carriers to position the cameras at the corresponding fixed location relative to the human, ensuring that the relative positioning of the cameras is preserved.

6. The marker-free movement capture system according to claim 1, wherein the first and the at least one second mobile carriers comprise Unmanned Aerial Vehicle (UAV) such as a drone.

7. The marker-free movement capture system according to claim 2, wherein subsequent to striking the golf ball to the second natural field space, the control unit instruct the at least one mobile carrier together with the cameras to move to the second natural field space in response to a second location data within the golf course.

8. The marker-free movement capture system according to claim 1, further comprising a display configured to display the results of the analyzed movement of the human body.

9. A method for measuring movement of a human body while performing a sport activity in natural field space characteristic for the sport, comprising:positioning a first camera at a first fixed location relative to an image capturing space, configured to capture image data of the human body during the sport activity within the image capturing space,positioning at least one second camera at one or more second fixed locations relative to the image capturing space, configured to capture image data of the human body during the sport activity simultaneous to the first camera,receiving and processing the image data collected by the first camera and the at least one second camera by a computer system,wherein the method further comprises:providing at least one mobile carrier for carrying the cameras,instructing the at least one mobile carrier together with the cameras to move to the first natural field space,wherein, in response to detecting an action executed by the human within the natural field space, the image data captured by the first and at least one second camera, collected over a predefined time window encompassing both before and after the detected action, is provided as input to the computer system to calculate the human's movement.

10. The method according to claim 9, wherein the sport is sport is golf and where the human is a golfer executing a golf swing with a golf club, where the first natural field space includes a first space within a golf course, and where the detected action corresponds to the moment of the golfer strikes the golf ball to a second natural field space, and where the calculated movement includes the movement of the golfer body and / or or the golf club.

11. The method according to claim 10, wherein the at least one more mobile carrier includes a first and at least one second mobile carrier, where the method further comprises:moving the first mobile carrier to position the first camera at the first fixed location relative to the human,moving the at least one second mobile carrier to position the at least one second camera to the corresponding to at least one fixed location relative to the human, ensuring that the relative positioning of the cameras is maintained.

12. The method of any of claims 9, further comprising performing a calibration, such as intrinsic calibration, on the first camera and the at least one second camera while positioned at the first and at least one second fixed locations, prior to capturing image data capture.

13. The method of any of claims 9, further comprising performing a calibration, such as intrinsic calibration, on the first camera and the at least one second camera within the first natural field space.

14. The method of claim 13, wherein the first and the at least one second mobile carriers include a positioning module configured to internally position the mobile carriers within the first natural field space before performing the calibration.

15. The method according to claim 1, wherein the sport is selected from tennis, ball sport such as socker, rugby, baseball, skiing, sled sports.

Images