Unmarked kinetic analysis method and system for real-time tracking shooting of unmanned aerial vehicle

Abstract

The invention provides an unmarked kinetic analysis method and system for real-time tracking and shooting of an unmanned aerial vehicle, and the method comprises the steps: carrying out the flight control of the unmanned aerial vehicle, and obtaining the dynamic image data, tracked and shot by the unmanned aerial vehicle, of a sporter running on a playground curve or a curve path; synchronizing and preprocessing the dynamic image data recorded by different cameras; extracting 2D key points of a human body from the preprocessed image, and reconstructing a three-dimensional motion track of the human body in a local coordinate system of the camera; carrying out posture compensation on the three-dimensional motion trail, ensuring the consistency of human body joint point trails under a ground world reference system, carrying out human body model scaling and muscle model scaling calculation, and carrying out inverse kinematics analysis; according to the inverse kinematics data, the centripetal acceleration of the sporter in the curve motion is calculated; and performing inverse dynamics calculation based on the centripetal acceleration, and calculating the ground support force. According to the invention, unmarked dynamic analysis of real-time tracking shooting of the unmanned aerial vehicle can be carried out.

Description

Technical Field

[0001] The present invention relates to the fields of motion biomechanics analysis, computer vision and intelligent mobile shooting technology, and in particular to a marker-free dynamics analysis method and system for real-time tracking and shooting of unmanned aerial vehicles. Background Art

[0002] In recent years, with the rapid development of sports science, rehabilitation medicine and intelligent health monitoring, markerless motion capture technology, as a non-invasive method of human motion capture, has been widely studied and applied in many fields due to its convenience, high efficiency and strong adaptability to the environment.

[0003] Typical markerless kinematic / dynamic analysis solutions rely on fixed multi-camera arrays or depth cameras, combined with human pose estimation algorithms (such as OpenPose, MediaPipe, VIBE, and OpenCap). These systems estimate motion trajectories and biomechanical parameters through 2D / 3D keypoint reconstruction. These systems are often deployed in laboratory environments and require a fixed viewpoint, stable lighting, and precise calibration. Consequently, they are less adaptable to outdoor environments or large open spaces.

[0004] At the same time, most existing systems are statically arranged and cannot move freely with the athletes. They are only suitable for limited space motion analysis within controlled areas and cannot meet the needs of actual scenarios such as running training, track and field testing, and field gait analysis. Summary of the Invention

[0005] In order to solve the technical problems existing in the above-mentioned prior art, the present invention provides a marker-free dynamic analysis method and system for real-time tracking and shooting of drones. The technical solution is as follows:

[0006] In one aspect, a marker-free dynamic analysis method for real-time tracking and shooting of a drone is provided, the method comprising:

[0007] S1. By controlling the flight of the UAV, dynamic image data of an athlete running on a curved path or a curved path in a playground is obtained by real-time tracking of the UAV;

[0008] S2, synchronizing and preprocessing the dynamic image data recorded by different cameras;

[0009] S3. Use the human posture recognition algorithm to extract the 2D key points of the human body from the preprocessed image, and combine binocular or multi-camera visual triangulation and drone flight altitude data to reconstruct the 3D motion trajectory of the human body in the local coordinate system of the camera;

[0010] S4. Based on the heading angle data recorded by the IMU onboard the drone, the posture of the three-dimensional motion trajectory is compensated to ensure the consistency of the human joint trajectory in the ground world reference system, and the human body model scaling and muscle model scaling calculation are performed, and inverse kinematics analysis is performed to obtain inverse kinematics data;

[0011] S5. Calculating the centripetal acceleration of the athlete during the curve motion based on the inverse kinematics data;

[0012] S6. Based on the centripetal acceleration, perform inverse dynamics calculation to deduce the ground support force.

[0013] Optionally, the flight control of the UAV in S1 specifically includes three parts:

[0014] Part 1: Speed ​​Control;

[0015] The control objectives for the speed control of the curve motion are: keeping the relative distance d(t) between the UAV and the athlete constant, and adjusting the linear velocity v(t) of the UAV to offset the centripetal acceleration during the turn. The control equation is:

[0016] v d (t) = v h (t)+K d ·e d (t)+K c ·(a c，traget -a c (t))

[0017] Where: v h (t) is the speed of the athlete; K d is the distance control gain, which is adjusted by experience or PID control; e d (t) is the distance error, e d (t) = d target -d(t),K target : the set distance between the UAV and the athlete, d(t) the actual distance between the UAV and the athlete at the current moment; K c The centripetal acceleration compensation gain of the UAV is adjusted based on experience; a c，traget : UAV target centripetal acceleration; a c (t): actual centripetal acceleration of the UAV;

[0018] Part 2: trajectory prediction layer;

[0019] Predict the athlete's position, velocity, acceleration, and turning radius within the next N frames, as well as the drone's heading angle, providing basic data for drone path optimization, including:

[0020] Trajectory data collection: collect the trajectory points (xh ,y h );

[0021] Use the least squares method to fit historical trajectory points:

[0022] y h (t) = a·x h (t) 2 +b·x h (t)+c

[0023] Where a, b, c are the quadratic curve fitting coefficients;

[0024] Then the predicted position at time t+k is:

[0025]

[0026] y h (t+k)=a·x h (t+k) 2 +b·x h (t+k)+c

[0027] where x h (t): the x-coordinate of the athlete at the current moment; The x-direction velocity of the athlete at the current moment; The x-direction acceleration of the athlete at the current moment; (x h (t+k),y h (t+k)) is the predicted position of the player after k frames; k is the number of predicted frames; Δt is the time interval between each frame;

[0028] Prediction speed:

[0029]

[0030] in: Predict the x-direction velocity; Predict the velocity in the y direction;

[0031] Predicted acceleration:

[0032]

[0033] in is the acceleration in the x direction; is the acceleration in the y direction; a h (t+k) is the total acceleration;

[0034] Turning radius: Predict the turning radius of the athlete through arc fitting:

[0035]

[0036] Heading angle estimation:

[0037]

[0038] Where: (x h (t), y h (t)) is the current position of the UAV; (x c ,y c ) is the position of the previous frame; θ h (t) is the estimated value of the current heading angle of the UAV;

[0039] Part III: Path Optimization Layer;

[0040] Taking into account path error, energy consumption, and centripetal acceleration control, the optimal control path of the UAV in the curve is generated. The goals are: minimize path error and keep the relative distance between the UAV and the athlete unchanged; avoid coordinating paths between the two UAVs to avoid mutual occlusion; reduce energy consumption, optimize propulsion and steering energy consumption, and control battery loss. Therefore, the optimization objective function is defined as:

[0041] J=α·J path +β·J collision +γ·J energy

[0042] The optimal path control strategy is obtained through gradient descent, iterative optimization or Nash equilibrium solution. The minimum value of J and its corresponding parameters are solved to obtain the optimal control path. Among them, α, β, and γ are weight coefficients, which are adjusted according to the actual task.

[0043] J path is the path error term,

[0044] where Q d , Q R 、 : Weight coefficient, which adjusts the influence of path error, turning radius error, and centripetal acceleration error on the control strategy; e d，k is the distance error, e d，k =d target -d(t+k), d target is the set distance between the UAV and the athlete, which is a preset value; d(t+k) is the actual distance between the UAV and the athlete at the time frame k; e R,k is the turning radius error, e R,k =R target -R(t+k), R target The expected turning radius of the athlete is the predicted value, R(t+k): the actual turning radius of the athlete at time k of the frame; Centripetal acceleration error, actarget : The expected centripetal acceleration of the athlete, which is the predicted value, a c (t+k): the actual centripetal acceleration of the athlete at frame k;

[0045] J collision To avoid obstacles,

[0046] where d ij，k : The distance between two UAVs i and j at the kth frame;

[0047] J energy is the energy consumption item,

[0048] Where α and β are energy consumption weights, α is related to air resistance and propulsion efficiency, and β is related to rotational inertia and friction torque; v d (t) is the linear propulsion speed of the UAV; ω d (t) UAV turning rotation speed.

[0049] Optionally, the S3 specifically includes:

[0050] The deep learning model is used to extract the 2D key points of each camera and perform 3D reconstruction using triangulation. The least squares method is used to optimize the reconstruction of the 3D points. The calculation method is:

[0051]

[0052] Where: X = [X, Y, Z] T : coordinates of the three-dimensional point to be found; X i : 2D projection point in the i-th camera; K i : is the camera internal parameter matrix, camera internal parameters; [R i |T i ]: camera extrinsic matrix; R i : rotation matrix; T i is the translation vector, which is calculated according to the camera calibration process;

[0053] Adjust the three-dimensional trajectory based on the drone's flight altitude data:

[0054] Assuming that the height of the drone at time t is H(t), the three-dimensional point coordinates become:

[0055]

[0056] Among them H ref : The set altitude of the drone;

[0057] After solving, the three-dimensional motion trajectory data in the camera's local coordinate system is obtained.

[0058] Optionally, the S4 specifically comprises:

[0059] The attitude compensation of the three-dimensional motion trajectory, the data input of which is the three-dimensional motion trajectory and the IMU heading angle data, the attitude compensation method is: performing IMU data fusion to construct a rotation matrix:

[0060] R IMU (ψ) z ·R y (θ) x (φ)

[0061] R z (ψ) is the heading angle rotation matrix of the unmanned aerial vehicle, which adjusts the left and right rotation of the heading angle of the unmanned aerial vehicle:

[0062]

[0063] R y (θ) is the pitch angle rotation matrix of the unmanned aerial vehicle, which adjusts the up and down inclination of the pitch angle of the unmanned aerial vehicle:

[0064]

[0065] R x (φ) is the roll angle rotation matrix of the unmanned aerial vehicle, which adjusts the left and right roll of the roll angle of the unmanned aerial vehicle:

[0066]

[0067] The three-dimensional points in the local coordinate system obtained in the local coordinate system are collected as:

[0068] P local = [X local , Y local , Z local ] T

[0069] The formula for converting the three-dimensional points in the world coordinate system is:

[0070] P world = R IMU ·P locat

[0071] Then, human body model scaling and muscle model scaling calculation are performed, inverse kinematics analysis is performed, and inverse kinematics data is obtained, the inverse kinematics data including: human body center of mass motion trajectory, joint motion angle, bone motion trajectory, and length of part of muscle ligaments.

[0072] Optionally, the S5 specifically comprises:

[0073] Combining the camera parameters, drone attitude changes, and data sampling errors in the drone system, the formula for centripetal acceleration is expressed as:

[0074]

[0075] Where Δv is the error in the estimated speed of the athlete due to camera distortion correction and frame rate fluctuation, which is calculated based on the optical flow method, frame difference method, and fused IMU data; ΔR is the error in the estimated radius of curvature, which is caused by insufficient camera distortion correction and lens perspective changes, and is corrected based on the radial distortion model; k1, k2, k3 are weight coefficients used to adjust the contribution of attitude angle and perspective error to centripetal acceleration; φ, θ are the roll and pitch angles of the drone. During rapid turns, the change in the drone's attitude will affect the camera's perspective; δ view : Perspective error, the camera field of view offset caused by the change in the drone's yaw angle, which reflects the impact of the camera field of view change on displacement estimation. The calculation formula is: Δx is the viewing angle offset, f is the focal length, is the change in field of view angle.

[0076] Optionally, the S6 specifically includes:

[0077] The ground support force GRF is the result of the acceleration of the human body's center of mass and is expressed as:

[0078] F GRF =[F GRF,x , F GRF，y , F GRF,z ]

[0079] Among them: F GRF,z Indicates the force in the vertical direction supporting the body weight; F GRF,x Indicates the horizontal force in the direction of the curve / centripetal force; F GRF,y Force in the front-to-back direction indicating acceleration / deceleration direction;

[0080] F in the vertical direction GRF,z :

[0081] The vertical GRF is used to support the body weight, F GRF,z =m·(a z +g), where m is the mass of the human body; g is the acceleration due to gravity; a z is the vertical acceleration, which is calculated by taking the second-order derivative of the human body's center of mass trajectory data: Z x (t) is the trajectory data of the center of mass of the human body in the vertical direction;

[0082] F in the horizontal direction GRF，x:

[0083] The GRF in the horizontal direction provides the centripetal force during the movement, F GRF，x =m·(a c +a x ), where a c : centripetal acceleration; a x : The translational acceleration in the horizontal direction is calculated by obtaining the human body's center of mass motion trajectory data and performing a second-order derivative: Z x (t) is the trajectory data of the center of mass of the human body in the horizontal direction;

[0084] F in the front-to-back direction GRF，y :

[0085] GRF in the front and rear directions is used for propulsion or braking, F GRF，y =m·a y , where a y : The acceleration in the forward and backward directions is calculated by obtaining the human body's center of mass motion trajectory data and performing a second-order derivative: Z y (t) is the trajectory data of the center of mass of the human body in the front-back direction;

[0086] Then the ground support force GRF is:

[0087]

[0088] In another aspect, a marker-free dynamic analysis system for real-time tracking and shooting of a drone is provided, the system comprising:

[0089] An acquisition module is used to obtain dynamic image data of an athlete running on a curved path or a curved path in a playground by controlling the flight of the drone;

[0090] Synchronous preprocessing module, used to synchronize and preprocess dynamic image data recorded by different cameras;

[0091] The extraction and reconstruction module is used to extract the 2D key points of the human body from the preprocessed image using the human posture recognition algorithm, and reconstruct the 3D motion trajectory of the human body in the local coordinate system of the camera by combining binocular or multi-camera visual triangulation and drone flight altitude data;

[0092] A compensation module is used to compensate the posture of the three-dimensional motion trajectory based on the heading angle data recorded by the IMU onboard the drone, ensure the consistency of the human joint trajectory in the ground world reference system, perform human body model scaling calculations and muscle model scaling calculations, perform inverse kinematics analysis, and obtain inverse kinematics data;

[0093] a calculation module, configured to calculate the centripetal acceleration of the athlete during the curve motion based on the inverse kinematics data;

[0094] The calculation module is used to perform inverse dynamics calculation based on the centripetal acceleration to calculate the ground support force.

[0095] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the above-mentioned marker-free dynamic analysis method for real-time tracking and shooting of a drone.

[0096] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored. The at least one instruction is loaded and executed by a processor to implement the above-mentioned marker-free dynamic analysis method for real-time tracking and shooting of a drone.

[0097] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0098] 1) Wide-area outdoor applicability: The test site is no longer limited to a specific location. Dynamic analysis can be performed without the need for pre-deployment of multiple fixed cameras, adapting to a wide range of real-world motion scenarios. This system overcomes the limitations of existing systems on site and space, making it suitable for motion data collection and analysis in open outdoor environments (especially on curved paths or playgrounds).

[0099] 2) Flexible deployment and easy expansion: One or more drones can be used to collect data from different angles as needed, making the collection more comprehensive. This makes it easy to apply to application scenarios such as outdoor sports training, daily athlete practice monitoring, and rehabilitation follow-up, enabling the collection and analysis of sports dynamics data anytime, anywhere. The calibration process is simpler than fixed-position collection, reducing the complexity of hardware deployment and calibration, and providing greater flexibility.

[0100] 3) No need to label the athlete for analysis: The artificial intelligence algorithm is integrated into the label-free kinetic parameter estimation process, which improves the accuracy and robustness of parameter estimation.

[0101] 4) Strong adaptability to dynamic motion: By introducing the flight speed of the drone as an auxiliary modeling parameter, the adaptability of dynamic parameter estimation to dynamic situations such as changes in motion speed is improved. Even if the tester's speed suddenly speeds up or slows down, the system can adaptively collect data according to the tester's speed.

[0102] 5) Centripetal acceleration modeling: It can adapt to movements in different venues, paths, and scenarios, accurately capture and compensate for the inertial force applied to the test subject, improve the computational accuracy of various output results, and enhance the versatility and robustness of the sports biomechanics analysis system. BRIEF DESCRIPTION OF THE DRAWINGS

[0103] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0104] Figure 1 This is a flow chart of a marker-free dynamic analysis method for real-time tracking and shooting of a drone provided by an embodiment of the present invention;

[0105] Figure 2 This is a schematic diagram of the relative flight of a drone and an athlete provided by an embodiment of the present invention;

[0106] Figure 3 This is an acceleration analysis diagram of an athlete running on a curved or curved path on a playground, provided by an embodiment of the present invention;

[0107] Figure 4 This is a block diagram of a markerless dynamic analysis system for real-time tracking and shooting of a drone provided by an embodiment of the present invention;

[0108] Figure 5 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0109] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0110] An embodiment of the present invention provides a marker-free dynamic analysis method for real-time tracking and shooting of a drone. The method can be implemented by an electronic device, which can be a terminal or a server. Figure 1 The flowchart of the method is shown, and the processing flow may include the following steps:

[0111] S1. By controlling the flight of the UAV, dynamic image data of an athlete running on a curved path or a curved path in a playground is obtained by real-time tracking of the UAV;

[0112] The embodiment of the present invention mainly includes at least one multi-rotor drone with stable flight capability, and each drone includes three cameras, such as Figure 2As shown, one camera is located below the drone, while the other two cameras are located at either end of a dedicated mounting bracket. When operating with only one drone, the mounting bracket is greater than 1.5m long, with two high-definition cameras mounted at either end. The cameras are arranged horizontally, with a spacing of greater than 2.5m between them, forming a wide-baseline binocular vision system for capturing real-time image sequences of human motion. When operating with multiple drones, the mounting bracket is greater than 0.5m long, with two high-definition cameras mounted at either end. The cameras are arranged horizontally, with a spacing of greater than 0.8m between them. The drone is equipped with a GPS module, an inertial measurement unit (IMU), and a visual navigation system to obtain real-time position information and determine its flight trajectory. Vision / fusion target detection and tracking technology, flight path control algorithms, and drone attitude and velocity control algorithms are used to achieve a human-following feature, maintaining a fixed relative distance from the subject. The cameras below the fuselage are connected to the drone's flight control system to follow the subject's motion trajectory, maintaining a fixed relative distance from the subject.

[0113] Optionally, the flight control of the UAV in S1 specifically includes three parts:

[0114] Part 1: Speed ​​Control;

[0115] The control objectives for the curve motion speed control are: keeping the relative distance d(t) between the UAV and the athlete constant, and adjusting the UAV linear speed v d (t) To offset the centripetal acceleration during turning, the control equation is:

[0116] v d (t) = v h (t)+K d ·e d (t)+K c ·(a c，traget -a c (t))

[0117] Where: v h (t) is the speed of the athlete; K d is the distance control gain, which is adjusted by experience or PID control; e d (t) is the distance error, e d (t) = d target -d(t), d target : the set distance between the UAV and the athlete, d(t) the actual distance between the UAV and the athlete at the current moment; K c The centripetal acceleration compensation gain of the UAV is adjusted based on experience; a c，traget : UAV target centripetal acceleration; a c (t): actual centripetal acceleration of the UAV;

[0118] Part 2: trajectory prediction layer;

[0119] Predict the athlete's position, velocity, acceleration, and turning radius within the next N frames, as well as the drone's heading angle, providing basic data for drone path optimization, including:

[0120] Trajectory data collection: collect the trajectory points (x h ,y h );

[0121] Use the least squares method to fit historical trajectory points:

[0122] y h (t) = a·x h (t) 2 +b·x h (t)+c

[0123] Where a, b, c are the quadratic curve fitting coefficients;

[0124] Then the predicted position at time t+k is:

[0125]

[0126] y h (t+k)=a·x h (t+k) 2 +b·x h (t+k)+c

[0127] where x h (t): the x-coordinate of the athlete at the current moment; The x-direction velocity of the athlete at the current moment; The x-direction acceleration of the athlete at the current moment; (x h (t+k),y h (t+k)) is the predicted position of the player after k frames; k is the number of predicted frames; Δt is the time interval between each frame;

[0128] Prediction speed:

[0129]

[0130] in: Predict the x-direction velocity; Predict the velocity in the y direction;

[0131] Predicted acceleration:

[0132]

[0133] in is the acceleration in the x direction; is the acceleration in the y direction; a h (t+k) is the total acceleration;

[0134] Turning radius: Predict the turning radius of the athlete through arc fitting:

[0135]

[0136] Heading angle estimation:

[0137]

[0138] Where: (x h (t), y h (t)) is the current position of the UAV; (x c ,y c ) is the position of the previous frame; θ h (t) is the estimated value of the current heading angle of the UAV;

[0139] Part III: Path Optimization Layer;

[0140] Taking into account path error, energy consumption, and centripetal acceleration control, the optimal control path of the UAV in the curve is generated. The goals are: minimize path error and keep the relative distance between the UAV and the athlete unchanged; avoid coordinating paths between the two UAVs to avoid mutual occlusion; reduce energy consumption, optimize propulsion and steering energy consumption, and control battery loss. Therefore, the optimization objective function is defined as:

[0141] J=α·J path +β·J collision +γ·J energy

[0142] The optimal path control strategy is obtained through gradient descent, iterative optimization or Nash equilibrium solution. The minimum value of J and its corresponding parameters are solved to obtain the optimal control path. Among them, α, β, and γ are weight coefficients, which are adjusted according to the actual task.

[0143] J path is the path error term,

[0144] where Q d , Q R 、 Weight coefficient, which adjusts the impact of path error, turning radius error, and centripetal acceleration error on the control strategy; e d,k is the distance error, e d，k =d target -d(t+k), d target is the set distance between the UAV and the athlete, which is a preset value; d(t+k) is the actual distance between the UAV and the athlete at the time frame k; e R，kis the turning radius error, e R,k =R target -R(t+k), R target The expected turning radius of the athlete is the predicted value, R(t+k): the actual turning radius of the athlete at time k of the frame; Centripetal acceleration error, a ctarget : The expected centripetal acceleration of the athlete, which is the predicted value, a c (t+k): the actual centripetal acceleration of the athlete at frame k;

[0145] J collision To avoid obstacles,

[0146] where d ij,k : The distance between two UAVs i and j at the kth frame;

[0147] J energy is the energy consumption item,

[0148] Where α and β are energy consumption weights, α is related to air resistance and propulsion efficiency, and β is related to rotational inertia and friction torque; v d (t) is the linear propulsion speed of the UAV; ω d (t) UAV turning rotation speed.

[0149] S2, synchronizing and preprocessing the dynamic image data recorded by different cameras;

[0150] The embodiment of the present invention needs to synchronize the dynamic image data recorded by different cameras. The specific synchronization method is existing in the art and will not be described in detail here.

[0151] The data preprocessing in the embodiment of the present invention includes distortion correction of data recorded by different cameras; and image smoothing and noise reduction using Gaussian filtering.

[0152] S3. Use the human posture recognition algorithm to extract the 2D key points of the human body from the preprocessed image, and combine binocular or multi-camera visual triangulation and drone flight altitude data to reconstruct the 3D motion trajectory of the human body in the local coordinate system of the camera;

[0153] Optionally, the S3 specifically includes:

[0154] The deep learning model is used to extract the 2D key points of each camera and perform 3D reconstruction using triangulation. The least squares method is used to optimize the reconstruction of the 3D points. The calculation method is:

[0155]

[0156] Where: X = [X, Y, Z] T : coordinates of the three-dimensional point to be found; X i : 2D projection point in the i-th camera; K i : is the camera internal parameter matrix, camera internal parameters; [R i |T i ]: camera extrinsic matrix; R i : rotation matrix; T i is the translation vector, which is calculated according to the camera calibration process;

[0157] Adjust the three-dimensional trajectory based on the drone's flight altitude data:

[0158] Assuming that the height of the drone at time t is H(t), the three-dimensional point coordinates become:

[0159]

[0160] Among them H ref : The set altitude of the drone;

[0161] After solving, the three-dimensional motion trajectory data in the camera's local coordinate system is obtained.

[0162] S4. Based on the heading angle data recorded by the IMU onboard the drone, the posture of the three-dimensional motion trajectory is compensated to ensure the consistency of the human joint trajectory in the ground world reference system, and the human body model scaling and muscle model scaling calculation are performed, and inverse kinematics analysis is performed to obtain inverse kinematics data;

[0163] Optionally, the S4 specifically includes:

[0164] The three-dimensional motion trajectory is subjected to attitude compensation, and its data input is the three-dimensional motion trajectory and IMU heading angle data. The attitude compensation method is: performing IMU data fusion and constructing a rotation matrix:

[0165] R IMU =R z (ψ)·R y (θ)·R x (φ)

[0166] Where: R z (ψ) is the heading angle rotation matrix of the UAV, which adjusts the heading angle of the UAV to rotate left and right:

[0167]

[0168] R y (θ): The pitch angle rotation matrix of the drone, which adjusts the pitch angle of the drone up and down;

[0169]

[0170] R x (φ): The roll angle rotation matrix of the drone, which adjusts the roll angle of the drone to roll left and right;

[0171]

[0172] The set of three-dimensional points in the local coordinate system obtained from the local coordinate system is:

[0173] P locot =[X rocot ,Y locat ,Z tocal ] T

[0174] The formula for converting to a three-dimensional point in the world coordinate system is:

[0175] P world =R IMU ·P local

[0176] Then, the human body model scaling and muscle model scaling calculations are performed, and inverse kinematics analysis is performed to obtain inverse kinematics data (the specific method is existing technology and will not be repeated here). The inverse kinematics data includes: the motion trajectory of the human body center of mass, the joint motion angle, the bone motion trajectory, and the length of some muscles and ligaments.

[0177] S5. Calculate the centripetal acceleration of the athlete in the curve motion according to the inverse kinematics data, such as Figure 3 As shown;

[0178] Optionally, the S5 specifically includes:

[0179] Combining the camera parameters, drone attitude changes, and data sampling errors in the drone system, the formula for centripetal acceleration is expressed as:

[0180]

[0181] Where Δv is the error in the estimated speed of the athlete due to camera distortion correction and frame rate fluctuation, which is calculated based on the optical flow method, frame difference method, and fused IMU data; ΔR is the error in the estimated radius of curvature, which is caused by insufficient camera distortion correction and lens perspective changes, and is corrected based on the radial distortion model; k1, k2, k3 are weight coefficients used to adjust the contribution of attitude angle and perspective error to centripetal acceleration; φ, θ are the roll and pitch angles of the drone. During rapid turns, the change in the drone's attitude will affect the camera's perspective; δ view: Perspective error, the camera field of view offset caused by the change in the drone's yaw angle, which reflects the impact of the camera field of view change on displacement estimation. The calculation formula is: Δx is the viewing angle offset, f is the focal length, is the change in field of view angle.

[0182] S6. Based on the centripetal acceleration, perform inverse dynamics calculation to deduce the ground support force.

[0183] Optionally, the S6 specifically includes:

[0184] The ground support force GRF is the result of the acceleration of the human body's center of mass and is expressed as:

[0185] F GRF =[F GRF，x , F GRF,y ,F GRF,z ]

[0186] Among them: F GRF，z Indicates the force in the vertical direction supporting the body weight; F GRF，x Indicates the horizontal force in the direction of the curve / centripetal force; F CRF，y Force in the front-to-back direction indicating acceleration / deceleration direction;

[0187] F in the vertical direction CRF，z :

[0188] The vertical GRF is used to support the body weight, F GRF，z =m·(a z +g), where m is the mass of the human body; g is the acceleration due to gravity; a z is the vertical acceleration, which is calculated by obtaining the human body's center of mass motion trajectory data (including data in three directions) and performing second-order derivative: Z z (t) is the trajectory data of the center of mass of the human body in the vertical direction;

[0189] F in the horizontal direction GRF，x :

[0190] The GRF in the horizontal direction provides the centripetal force during the movement, F GRF，x =m·(a c +a x ), where a c : centripetal acceleration; a x : The translational acceleration in the horizontal direction is calculated by obtaining the human body's center of mass motion trajectory data and performing a second-order derivative: Z x (t) is the trajectory data of the center of mass of the human body in the horizontal direction;

[0191] F in the front-to-back direction GRF，y :

[0192] GRF in the front and rear directions is used for propulsion or braking, F GRF，y =m·a y , where a y : The acceleration in the forward and backward directions is calculated by obtaining the human body's center of mass motion trajectory data and performing a second-order derivative: z y (t) is the trajectory data of the center of mass of the human body in the front-back direction;

[0193] Then the ground support force GRF is:

[0194]

[0195] like Figure 4 As shown, an embodiment of the present invention further provides a marker-free dynamic analysis system for real-time tracking and shooting of a drone, the system comprising:

[0196] An acquisition module 410 is configured to acquire dynamic image data of an athlete running on a curved path or a curved path in a playground, captured by the drone in real time by controlling the drone's flight.

[0197] A synchronization preprocessing module 420 is used to synchronize and preprocess the dynamic image data recorded by different cameras;

[0198] The extraction and reconstruction module 430 is used to extract the 2D key points of the human body from the pre-processed image using a human posture recognition algorithm, and reconstruct the 3D motion trajectory of the human body in the local coordinate system of the camera by combining binocular or multi-camera visual triangulation and drone flight altitude data;

[0199] The compensation module 440 is used to perform posture compensation on the three-dimensional motion trajectory based on the heading angle data recorded by the IMU onboard the UAV, ensure the consistency of the human joint trajectory in the ground world reference system, perform human body model scaling and muscle model scaling calculations, and perform inverse kinematics analysis to obtain inverse kinematics data;

[0200] A calculation module 450 is used to calculate the centripetal acceleration of the athlete in the curve motion according to the inverse kinematics data;

[0201] The calculation module 460 is used to perform inverse dynamics calculation based on the centripetal acceleration to calculate the ground support force.

[0202] The functional structure of a markerless dynamic analysis system for real-time tracking and shooting of a drone provided by an embodiment of the present invention corresponds to the markerless dynamic analysis method for real-time tracking and shooting of a drone provided by an embodiment of the present invention, which will not be repeated here.

[0203] Figure 5 : is a structural diagram of an electronic device 500 provided in an embodiment of the present invention. The electronic device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 501 and one or more memories 502, wherein the memory 502 stores at least one instruction, and the at least one instruction is executed by the processor 5

[0204] 01 Load and execute the steps to implement the above-mentioned label-free dynamic analysis method for real-time tracking and shooting of drones.

[0205] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory device containing instructions. The instructions are executable by a processor in a terminal to implement the aforementioned markerless dynamic analysis method for real-time tracking and filming of a drone. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0206] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0207] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A label-free dynamic analysis method for real-time tracking and shooting of unmanned aerial vehicles, characterized in that: The method comprises: S1. By controlling the flight of the UAV, dynamic image data of an athlete running on a curved path or a curved path in a playground is obtained by real-time tracking of the UAV; S2, synchronizing and preprocessing the dynamic image data recorded by different cameras; S3. Use the human posture recognition algorithm to extract the 2D key points of the human body from the preprocessed image, and combine binocular or multi-camera visual triangulation and drone flight altitude data to reconstruct the 3D motion trajectory of the human body in the local coordinate system of the camera; S4. Based on the heading angle data recorded by the IMU onboard the drone, the posture of the three-dimensional motion trajectory is compensated to ensure the consistency of the human joint trajectory in the ground world reference system, and the human body model scaling and muscle model scaling calculation are performed, and inverse kinematics analysis is performed to obtain inverse kinematics data; S5. Calculating the centripetal acceleration of the athlete during the curve motion based on the inverse kinematics data; S6. Based on the centripetal acceleration, perform inverse dynamics calculation to deduce the ground support force.

2. The method according to claim 1, characterized in that The flight control of the UAV in S1 specifically includes three parts: Part 1: Speed ​​Control; The control objectives for the curve motion speed control are: keeping the relative distance d(t) between the UAV and the athlete constant, and adjusting the UAV linear speed v d (t) To offset the centripetal acceleration during turning, the control equation is: v d (t)=v h (t)+K d ·e d (t)+K c ·(a c，traget -a c (t)) Where: v h (t) is the speed of the athlete; K d is the distance control gain, which is adjusted by experience or PID control; e d (t) is the distance error, e d (t) = d target -d(t), d target : the set distance between the UAV and the athlete, d(t) the actual distance between the UAV and the athlete at the current moment; K c The centripetal acceleration compensation gain of the UAV is adjusted based on experience; a c，traget : UAV target centripetal acceleration; a c (t): actual centripetal acceleration of the UAV; Part 2: trajectory prediction layer; Predict the athlete's position, velocity, acceleration, and turning radius within the next N frames, as well as the drone's heading angle, providing basic data for drone path optimization, including: Trajectory data collection: collect the trajectory points (x h ,y h ); Use the least squares method to fit historical trajectory points: y h (t)=a·x h (t) 2 +b·x h (t)+c Where a, b, c are the quadratic curve fitting coefficients; Then the predicted position at time t+k is: y h (t+k)=a·x h (t+k) 2 +b·x h (t+k)+c where x h (t): the x-coordinate of the athlete at the current moment; The x-direction velocity of the athlete at the current moment; The x-direction acceleration of the athlete at the current moment; (x h (t+k),y h (t+k)) is the predicted position of the player after k frames; k is the number of predicted frames; Δt is the time interval between each frame; Prediction speed: in: Predict the x-direction velocity; Predict the velocity in the y direction; Predicted acceleration: in is the acceleration in the x direction; is the acceleration in the y direction; a h (t+k) is the total acceleration; Turning radius: Predict the turning radius of the athlete through arc fitting: Heading angle estimation: Where: (x h (t), y h (t)) is the current position of the UAV; (x c ,y c ) is the position of the previous frame; θ h (t) is the estimated value of the current heading angle of the UAV; Part III: Path Optimization Layer; Taking into account path error, energy consumption, and centripetal acceleration control, the optimal control path of the UAV in the curve is generated. The goals are: minimize path error and keep the relative distance between the UAV and the athlete unchanged; avoid coordinating paths between the two UAVs to avoid mutual occlusion; reduce energy consumption, optimize propulsion and steering energy consumption, and control battery loss. Therefore, the optimization objective function is defined as: J=α·J path +β·J collision +γ·J energy The optimal path control strategy is obtained through gradient descent, iterative optimization or Nash equilibrium solution. The minimum value of J and its corresponding parameters are solved to obtain the optimal control path. Among them, α, β, and γ are weight coefficients, which are adjusted according to the actual task. J path is the path error term, where Q d , Q R 、 Weight coefficient, which adjusts the impact of path error, turning radius error, and centripetal acceleration error on the control strategy; e d，k is the distance error, e d，k =d target -d(t+k), d target is the set distance between the UAV and the athlete, which is a preset value; d(t+k) is the actual distance between the UAV and the athlete at the time frame k; e R，k is the turning radius error, e R，k =R target -R(t+k), R target The expected turning radius of the athlete is the predicted value, R(t+k): the actual turning radius of the athlete at time k of the frame; Centripetal acceleration error, a ctarget : The expected centripetal acceleration of the athlete, which is the predicted value, a c (t+k): the actual centripetal acceleration of the athlete at frame k; J collision To avoid obstacles, where d ij，k : The distance between two UAVs i and j at the kth frame; J energy is the energy consumption item, Where α and β are energy consumption weights, α is related to air resistance and propulsion efficiency, and β is related to rotational inertia and friction torque; v d (t) is the linear propulsion speed of the UAV; ω d (t) UAV turning rotation speed.

3. The method according to claim 1, characterized in that Said S3 specifically includes: The deep learning model is used to extract the 2D key points of each camera and perform 3D reconstruction using triangulation. The least squares method is used to optimize the reconstruction of the 3D points. The calculation method is: Where: X = [X, Y, Z] T : coordinates of the three-dimensional point to be found; X i : 2D projection point in the i-th camera; K i : is the camera internal parameter matrix, camera internal parameters; [R i |T i ]: camera extrinsic matrix; R i : rotation matrix; T i is the translation vector, which is calculated according to the camera calibration process; Adjust the three-dimensional trajectory based on the drone's flight altitude data: Assuming that the height of the drone at time t is H(t), the three-dimensional point coordinates become: Among them H ref : The set altitude of the drone; After solving, the three-dimensional motion trajectory data in the camera's local coordinate system is obtained.

4. The method according to claim 1, wherein Said S4 specifically includes: The three-dimensional motion trajectory is subjected to attitude compensation, and its data input is the three-dimensional motion trajectory and IMU heading angle data. The attitude compensation method is: performing IMU data fusion and constructing a rotation matrix: R IMU =R z (ψ)·R y (θ)·R x (f) Where: R z (ψ) is the heading angle rotation matrix of the UAV, which adjusts the heading angle of the UAV to rotate left and right: R y (θ): The pitch angle rotation matrix of the drone, which adjusts the pitch angle of the drone up and down; R x (φ): The roll angle rotation matrix of the drone, which adjusts the roll angle of the drone to roll left and right; The set of three-dimensional points in the local coordinate system obtained from the local coordinate system is: P local =[X local ,Y local ,Z local ] T The formula for converting to a three-dimensional point in the world coordinate system is: P world =R IMU ·P local Then, the human body model scaling and muscle model scaling calculations are performed, and inverse kinematics analysis is performed to obtain inverse kinematics data. The inverse kinematics data includes: human body center of mass motion trajectory, joint motion angles, bone motion trajectory, and the length of some muscles and ligaments.

5. The method according to claim 1, wherein Said S5 specifically includes: Combining the camera parameters, drone attitude changes, and data sampling errors in the drone system, the formula for centripetal acceleration is: Where Δv is the error in the estimated speed of the athlete due to camera distortion correction and frame rate fluctuation, which is calculated based on the optical flow method, frame difference method, and fused IMU data; ΔR is the error in the estimated radius of curvature, which is caused by insufficient camera distortion correction and lens perspective changes, and is corrected based on the radial distortion model; k1, k2, k3 are weight coefficients used to adjust the contribution of attitude angle and perspective error to centripetal acceleration; φ, θ are the roll and pitch angles of the drone. During rapid turns, the change in the drone's attitude will affect the camera's perspective; δ view : Perspective error, the camera field of view offset caused by the change in the drone's yaw angle, which reflects the impact of the camera field of view change on displacement estimation. The calculation formula is: Δx is the viewing angle offset, f is the focal length, is the change in field of view angle.

6. The method according to claim 1, characterized in that Said S6 specifically includes: The ground support force GRF is the result of the acceleration of the human body's center of mass and is expressed as: F GRF =[F GRF,x ,F GRF,y ,F GRF,z ] Among them: F GRF，z Indicates the force in the vertical direction supporting the body weight; F GRF，x Indicates the horizontal force in the direction of the curve / centripetal force; F GRF，y Force in the front-to-back direction indicating acceleration / deceleration direction; F in the vertical direction GRF，z : The vertical GRF is used to support the body weight, F GRF，z =m·(a z +g), where m is the mass of the human body; g is the acceleration due to gravity; a z is the vertical acceleration, which is calculated by taking the second-order derivative of the human body's center of mass trajectory data: Z z (t) is the trajectory data of the center of mass of the human body in the vertical direction; F in the horizontal direction GRF，x : The GRF in the horizontal direction provides the centripetal force during the movement, F GRF，x =m·(a c +a x ), where a c : centripetal acceleration; a x : The translational acceleration in the horizontal direction is calculated by obtaining the human body's center of mass motion trajectory data and performing a second-order derivative: Z x (t) is the trajectory data of the center of mass of the human body in the horizontal direction; F in the front-to-back direction GRF，y : GRF in the front and rear directions is used for propulsion or braking, F GRF，y =m·a y , where a y : The acceleration in the front and rear directions is calculated by taking the second-order derivative of the human body's center of mass trajectory data: z y (t) is the trajectory data of the center of mass of the human body in the front-back direction; Then the ground support force GRF is:

7. A marker-free dynamic analysis system for real-time tracking and shooting of drones, characterized by: The system comprises: An acquisition module is used to obtain dynamic image data of an athlete running on a curved path or a curved path in a playground by controlling the flight of the drone; Synchronous preprocessing module, used to synchronize and preprocess dynamic image data recorded by different cameras; The extraction and reconstruction module is used to extract the 2D key points of the human body from the preprocessed image using the human posture recognition algorithm, and reconstruct the 3D motion trajectory of the human body in the local coordinate system of the camera by combining binocular or multi-camera visual triangulation and drone flight altitude data; A compensation module is used to compensate the posture of the three-dimensional motion trajectory based on the heading angle data recorded by the IMU onboard the drone, ensure the consistency of the human joint trajectory in the ground world reference system, perform human body model scaling calculations and muscle model scaling calculations, perform inverse kinematics analysis, and obtain inverse kinematics data; a calculation module, configured to calculate the centripetal acceleration of the athlete during the curve motion based on the inverse kinematics data; The calculation module is used to perform inverse dynamics calculation based on the centripetal acceleration to calculate the ground support force.

8. An electronic device comprising a processor and a memory, wherein the memory stores at least one instruction, characterized in that: The at least one instruction is loaded and executed by the processor to implement the marker-free dynamic analysis method for real-time tracking and shooting of a drone as described in any one of claims 1 to 6.

9. A computer-readable storage medium, wherein at least one instruction is stored in the storage medium, characterized in that: The at least one instruction is loaded and executed by the processor to implement the marker-free dynamic analysis method for real-time tracking and shooting of a drone as described in any one of claims 1-6.

Images