Human Motion Intelligent Measurement and Digital Training System

Abstract

The present invention relates to a human body motion intelligent measurement and digital training system, the system includes N inertial navigation wearable devices, M cameras, a data comprehensive analysis device and a terminal, the full field of view of the M cameras covers the entire movement scene of the player, the inertial navigation wearable devices are worn and fixed on the player's body to measure and obtain the three-axis acceleration and three-axis angular velocity in the inertial coordinate system of the player's body, the data comprehensive analysis device analyzes the position and speed of each player's movement scene in the world coordinate system and the relative position and posture of each player's limbs in the player's body coordinate system to determine the movement parameters of each player, the terminal establishes a three-dimensional model of the movement scene and the player, and associates the speed and position in the player's movement scene coordinate system, the relative position and posture of the player's limbs in the player's body coordinate system with the three-dimensional model, and displays the movement process and movement parameters of the player in a visualized manner.

Description

Technical Field

[0001] This application claims the priority of a Chinese patent application filed with the China National Patent Office on May 20, 2022, with the application number 202210555949.2 and the invention title "Human Motion Intelligent Measurement and Digital Training System", and all its contents are incorporated herein by reference.

[0002] The present invention relates to a human motion intelligent measurement and digital training system, belonging to the field of intelligent measurement in the electronics industry, providing motion parameters and improving training methods.

Background Art

[0003] With the continuous development of China's sports industry and the progress of science and technology, the innovation of athletes' training modes has increasingly become the main means to improve athletes' competitive levels. How to innovate training modes with the power of science and technology has become the main means to solve various problems during training.

[0004] Monitoring athletes' movement parameters is a necessary means to improve the movement method and enhance the movement performance. The conventional movement parameter monitoring methods use technical means such as videos. Such methods can only roughly observe and judge the movement process, mainly relying on the experience of professional coaches, and cannot achieve precise quantitative analysis. Currently, most movements have a large range, high speed, and high requirements for the coordination of the whole body's limbs. The angle changes between the joints of athletes have a strong correlation with the final movement effect. To measure athletes' movement parameters and provide quantitative technical support for technical improvement and performance enhancement, it is necessary to use wearable products to achieve precise measurement of movement parameters, realize the intelligence of data processing and analysis, and facilitate the use by athletes and coaches. To facilitate the use by athletes and coaches, if an athlete wears an inertial navigation system, the coach can monitor the athlete's movement parameters only by operating a smartphone or a personal computer. The background is supported by a data comprehensive analysis system, which makes the use very convenient and provides important technical support for the assistance of science and technology to sports.

Summary of the Invention

Problems to be Solved by the Invention

[0005] The technical problem to be solved by the present invention is to submit a human movement intelligent measurement and digital training system to overcome the defects of the prior art, realize the measurement of parameters during the movement process, and complete the quantitative analysis during the training process.

Means for Solving the Problems

[0006] The technical solution of the present invention is as follows: a human movement intelligent measurement and digital training system, which includes N inertial navigation wearable devices, M cameras, a data comprehensive analysis device, and a terminal, where N and M are both greater than or equal to 1. The entire field of view of the M cameras covers the entire movement scene of the athlete, captures the images within the field of view, forms an image data frame, and transmits it to the data comprehensive analysis device. The inertial navigation wearable device is worn and fixed on the athlete's limb. Using the athlete's limb as a carrier, it measures and obtains the three-axis linear acceleration of the athlete's limb and the three-axis angular velocity in the inertial coordinate system, and transmits the data to the data comprehensive analysis module. The data comprehensive analysis device stores the basic information of the athlete, establishes and maintains the relationship between the athlete and the inertial navigation wearable device worn by the athlete. Based on the three-axis linear acceleration of the athlete's limb and the three-axis angular velocity in the inertial coordinate system, it performs navigation calculations and coordinate transformations to obtain and store the relative position and posture of the athlete's limb in the athlete's body coordinate system. It collects and stores the images captured by each camera, performs target recognition, tracking, and coordinate transformation on the images captured by each camera to obtain and store the position and velocity of the athlete in the world coordinate system of the sports scene. It analyzes the position and velocity of each athlete in the world coordinate system of the sports scene and the relative position and posture of each athlete's limb in the athlete's body coordinate system to determine and store the motion parameters of each athlete.

[0007] The data comprehensive analysis device includes an inertial navigation calculation module, a motion target detection and tracking module, a motion target velocity recognition module, and a motion parameter analysis module. The inertial navigation calculation module performs navigation calculations based on the three-axis linear acceleration of the athlete's limb and the three-axis angular velocity in the inertial coordinate system to obtain the posture, velocity, and position information of the athlete's limb in the navigation coordinate system. It performs zero-velocity detection on the motion of the athlete's limb. When the athlete's limb is in the zero-velocity interval, it performs zero-velocity error correction on the posture, velocity, and position information of the athlete's limb in the navigation coordinate system. It defines the athlete's body coordinate system and transforms the posture, velocity, and position information of the athlete's limb in the navigation coordinate system to the athlete's body coordinate system. The motion target detection and tracking module collects the images captured by each camera, records the image collection time, performs distortion correction on the images captured by each camera, uses the YOLO model to perform target detection on each corrected image captured at the same timing, obtains the approximate bounding boxes of all players in the pixel coordinate system of the image, and based on the edge detection method, obtains the accurate position and precise bounding box of each player in the pixel coordinate system, matches the precise bounding boxes of the same player at different timings, realizes the tracking of the precise bounding boxes of each player at different timings, converts the coordinates of each player in the pixel coordinate system into the coordinates in the world coordinate system of the corresponding camera field of view coverage area by means of the perspective projection matrix, calculates the coordinates of each player in the global world coordinate system of the motion scene at different timings based on the positional relationship between the coverage areas of each camera field of view, and transmits them to the motion target speed recognition module. The motion target speed recognition module filters the coordinate sequences of each player in the global world coordinate system of the motion scene at different timings to remove noise, and then performs differential processing to obtain the speed of the player in the world coordinate system of the motion scene. The motion parameter analysis module analyzes the relative positions and postures of the player's limbs in the player's body coordinate system to obtain motion parameters, compares the positions and speeds of each player in the world coordinate system of the motion scene, analyzes and sorts these data, ranks the players according to certain rules, and compares the motion parameters of the players with the standard parameters.

[0008] The above-mentioned human motion intelligent measurement and digital training system further includes a terminal that establishes a 3D model of the motion scene and the player, associates the speed and position of the player in the motion scene coordinate system, the relative positions and postures of the player's limbs in the player's body coordinate system, and the corresponding 3D model, and displays the motion process and motion parameters of the player in a visualized manner.

[0009] The terminal supports the use of users with four types of identities: players, coaches, experts, and administrators. The terminal with player permissions includes a "Self-training" module, First a "History Data" check module, and a first "Group Communication" module. The "Self-training" module acquires and records real-time motion parameters from the data comprehensive analysis device. First The "History Data" check module searches for original images, motion parameters, and corresponding training evaluations for a corresponding time period from the data comprehensive analysis device based on the exercise time period and the basic information of the player. The first "Group Communication" module receives player messages and is used for the mutual communication between the player and the coach and experts. The terminal with coach permissions includes a "Player Management" module, Second "history data" check module, a "Match Management" module, and a second "Group Communication" module. The "Player Management" module increases or decreases players and updates the basic information of the players to the data comprehensive analysis device. Second The "History Data" check module searches for original images and motion parameters for a corresponding time period from the data comprehensive analysis device based on the exercise time period input from outside and the basic information of the player, submits a training evaluation, and sends it to the data comprehensive analysis device for storage. The "Match Management" module creates a new in-team competition, sends the grouping and competition rules of the in-team competition to the data comprehensive analysis device for storage, receives coach messages, and is used for the mutual communication between the coach and the players and experts. The terminal with expert permissions includes a "Training Management" module and a third "Group Communication" module. The "Training Management" module checks the training rankings, compares the motion parameters of players in the same session, evaluates and makes suggestions for the players and the training of the corresponding session, forms a data analysis report, and sends it to the data comprehensive analysis device for storage. The third "Group Communication" module receives expert messages and is used for the mutual communication between the expert and the coach and the players. The terminal with an administrator identity set is used to set user information and user identities.

Advantages of the Invention

[0010] The present invention has the following beneficial effects over the prior art. (1) The present invention realizes the measurement of parameters in the human body movement process by using a wearable inertial navigation device. The inertial navigation system has a small volume, light weight, low power consumption, and is easy to wear. Regardless of the athlete's movement scene, the movement parameters can be measured at any time; (2) The present invention uses a string to tie the inertial navigation device to different parts of the human body, collects the angular velocity information of the gyroscope and the measurement information of the accelerometer, and uses the inertial navigation algorithm and the error correction algorithm to obtain the posture information during the movement process; (3) In the inertial navigation system of the present invention, ESP8266 is used as the central processor, and a wireless communication module is integrated, which can realize remote control and data collection; (4) The present invention uses a drone to suspend a PTZ camera to cover the entire movement scene of the athlete, adopts deep learning such as YOLO and DeepSORT to realize the dynamic tracking of the athlete, and further completes the calculation of the relative position and speed; ( 5 ) The present invention uses a 3D model to realize the interaction between the athlete and the model, and may track the athlete's movement process in real time, or display the movement in a visualized form in the form of post-inversion; ( 6 ) In the present invention, in order to realize precise detection of the zero-velocity interval of each part, the inertial navigation wearable device uses a zero-velocity error correction algorithm and a posture error correction algorithm based on a Kalman filter to realize the periodic estimation and correction of the navigation error of different measured parts of the human body, and solves the problem of error divergence in the case of long-term use of the MEMS sensor, and improves the measurement accuracy of the system; ( 7In the present invention, based on the fact that in the process of human walking, in addition to the feet, there are also different zero-velocity intervals in different parts such as the thighs and calves, the inertial navigation wearable device performs zero-velocity detection and correction algorithms on any of the different athlete limbs, and further performs navigation error estimation and correction, solving the problem that the navigation errors of parts other than the feet cannot be corrected regularly; ( 8 )In the present invention, based on the motion data characteristics of different parts of the human body, the inertial navigation wearable device uses different zero-velocity detection algorithms and accurately sets different energy thresholds to realize precise detection of the zero-velocity intervals of all measured parts including the feet, thighs, and calves, providing conditions for regular correction of the navigation errors of each part. ( 9 )In the present invention, the data comprehensive analysis device further recognizes a rough bounding box by edge detection between YOLO model target recognition and DeepSORT tracking to obtain the precise position and precise bounding box of the target, uses DeepSORT to track the precise bounding box, improves the target detection and positioning accuracy, and is suitable for high-precision positioning situations. ( 10 )In the present invention, the data comprehensive analysis device proposes an "extended nine-point calibration method", which does not require the use of a large calibration board and realizes large-range and high-precision calibration. ( 11 )In the present invention, when the data comprehensive analysis device solves the perspective projection matrix, in order to accurately obtain the pixel coordinates of the landmark points, the shape of the landmark points is set to a rhombus. In this way, regardless of the shooting distance, a relatively accurate angular position of the rhombus can be obtained from the captured image, and its center can be accurately positioned.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying out the Invention

[0012] Hereinafter, the present invention will be described in more detail by combining the drawings and specific embodiments.

[0013] As shown in FIG. 1, the human body movement intelligent measurement and digital training system of the present invention includes N inertial navigation wearable devices, M cameras, a data comprehensive analysis device, and a terminal. Both N and M are 1 or more, and the entire field of view of the M cameras covers the entire movement scene of the movement target. 、 habit The inertial navigation wearable device is worn and fixed on the player's limb. Using the player's limb as a carrier, it measures and obtains the three-axis acceleration of the player's limb and the three-axis angular velocity in the inertial coordinate system, and transmits them to the data comprehensive analysis module. It receives the operation mode command sent from the data comprehensive analysis device and operates in different modes including a data collection mode, a data storage mode, a real-time upload mode, and an offline upload mode. The data comprehensive analysis device stores the basic information of the athlete, establishes and maintains the relationship with the inertial navigation wearable device worn by the athlete, and based on the three-axis linear acceleration of the athlete's limbs and the three-axis angular velocity in the inertial coordinate system, performs navigation calculations and coordinate conversions to obtain the relative position and posture of the athlete's limbs in the athlete's body coordinate system, collects the images captured by each camera, performs target recognition, tracking and coordinate conversion on the images captured by each camera to obtain the position and velocity of the athlete's motion scene in the world coordinate system, analyzes the position and velocity of each athlete's motion scene in the world coordinate system and the relative position and posture of each athlete's limb in the athlete's body coordinate system to determine the motion parameters of each athlete, receives the training commands sent from the terminal, obtains the training mode by analyzing the training commands, and guides the athlete to perform reasonable training by sending commands of different training modes to the athlete. The terminal establishes the motion scene and the three-dimensional model of the athlete, associates the velocity and position of the athlete in the motion scene coordinate system, the relative position and posture of the athlete's limbs in the athlete's body coordinate system with the corresponding three-dimensional model, and displays the motion process and motion parameters of the athlete in a visual way, and sends the training commands input by the coach to the data comprehensive analysis device.

[0014] As shown in Figure 2, the inertial navigation wearable device can be worn on the athlete's body parts such as the hip, arm, thigh, calf, and foot in a string or adhesive manner. The origin of the athlete's body coordinate system coincides with the center of gravity of the human body, the X-axis is perpendicular to the sagittal plane, the X-axis is perpendicular to the coronal plane, and the Z-axis is perpendicular to the transverse plane. The coronal plane divides the human body into two complete front and back cross-sections. The sagittal plane is parallel to the direction of the front view of the human and divides the human body into two symmetric left and right parts. The transverse plane is also called the horizontal plane, and the ground horizontally divides the human body into two upper and lower parts.

[0015] The data transmission between the data comprehensive analysis device, the inertial navigation wearable device and the terminal adopts a wireless method.

[0016] The data transmission between the data comprehensive analysis device and the camera adopts a wired method.

[0017] Before the athlete starts exercising, the coach controls, via the terminal, the inertial navigation system worn on the athlete's body to enter the operating mode. In this case, the athlete can exercise according to the requirements, and the coach can view the parameters of the exercise process in real time using a smartphone or a PC data terminal.

[0018] When the training ends, the coach controls, via the terminal, the inertial navigation system to enter the sleep state or stop operating.

[0019] The technical key points of the present invention will be described in detail below. 1. Inertial Navigation Wearable Device The inertial navigation wearable device is worn on the athlete's body, measures the posture information of the entire process of the athlete's training, and includes a MEMS sensor, a signal processing module, a communication module, and a lithium battery. Inside the MEMS sensor, a MEMS gyroscope and a MEMS accelerometer are integrated. The MEMS gyroscope outputs the three-axis angular velocity in the inertial coordinate system, and the MEMS accelerometer outputs the three-axis linear acceleration of the athlete's limbs. The MEMS sensor outputs the measurement results to the signal processing module. The signal processing module frames and packages the measurement results output from the MEMS sensor and then transmits them to the communication module. The communication module transmits the packaged measurement data frame in a wireless communication manner. The lithium battery provides power to the MEMS sensor, the signal processing module, and the communication module.

[0020] In a specific embodiment of the present invention, the inertial navigation wearable device is composed of an MTI-3 attitude sensor, a processing circuit centered on ESP8266, a lithium battery, etc., and measures the athlete's posture. The MTI-3 attitude sensor uses a highly integrated MTI-3 micro inertial sensing unit, integrates information such as a three-axis gyroscope and a three-axis accelerometer, and has characteristics such as a small volume and a light weight.

[0021] The inertial navigation wearable device is worn on different limb parts of one or multiple athletes, and the output data of N inertial navigation wearable devices are synchronized, enabling simultaneous measurement of motion intelligence and digital training for N athletes.

[0022] When an athlete performs a swimming motion, the inertial navigation wearable device should have a waterproof function. By using silicone rubber to seal the upper housing and lower housing structures of the inertial navigation device, an IP68 waterproof rating can be achieved.

[0023] 2. Camera In a specific embodiment of the present invention, a camera is installed above the sports scene. According to the physical characteristics of the athlete, in the complex environment of athlete training, the camera is used to shoot videos of athlete training, and a series of image analysis, processing, and tracking are performed on the videos to finally realize the functions of target recognition, positioning, and speed measurement for the athlete.

[0024] 3. Data comprehensive analysis device The data comprehensive analysis device mainly includes an inertial navigation calculation module, a motion target detection and tracking module, a motion target speed recognition module, and a motion parameter analysis module. Based on the three-axis linear acceleration of the athlete's limb and the three-axis angular velocity in the inertial coordinate system, the inertial navigation calculation module performs navigation calculations to obtain the posture, speed, and position information of the athlete's limb in the navigation coordinate system, and performs zero-velocity detection on the motion of the athlete's limb. When the athlete's limb is in the zero-velocity interval, zero-velocity error correction is performed on the posture, speed, and position information of the athlete's limb in the navigation coordinate system, and the athlete's body coordinate system is defined to convert the posture, speed, and position information of the athlete's limb in the navigation coordinate system to the athlete's body coordinate system. The motion target detection and tracking module collects the images captured by each camera, records the image collection time, performs distortion correction on the images captured by each camera, uses the YOLO model to perform target detection on each corrected image captured at the same timing, obtains the approximate bounding boxes of all players in the pixel coordinate system of the image, and based on the edge detection method, obtains the accurate position and precise bounding box of each player in the pixel coordinate system, matches the precise bounding boxes of the same player at different timings, realizes the tracking of the precise bounding boxes of each player at different timings, converts the coordinates of each player in the pixel coordinate system into the coordinates in the corresponding camera field of view coverage area world coordinate system by means of the perspective projection matrix, calculates the coordinates of each player in the motion scene global world coordinate system at different timings based on the positional relationship between the coverage areas of each camera field of view, and transmits them to the motion target speed recognition module. The motion target speed recognition module filters the coordinate sequences of each player in the motion scene global world coordinate system at different timings to remove noise, then performs differential processing to obtain the speed of the player in the motion scene world coordinate system. The motion parameter analysis module analyzes the relative positions and postures of the player's limbs in the player body coordinate system to obtain motion parameters, compares the positions and speeds of each player in the motion scene world coordinate system, analyzes and sorts these data, ranks the players according to certain rules, and also analyzes the comparison between each player and the standard motion parameters, enabling the coach to analyze the deficiencies of the player during the process and improve the training process.

[0025] 3.1 Inertial navigation calculation module In the present invention, the "east - north - sky" geographical coordinate system is selected as the navigation coordinate system, and navigation calculations are performed using a recursive update algorithm to obtain the posture, speed, and position information of the player's limbs in the navigation coordinate system. The inertial navigation update algorithm is divided into three parts: the update of posture, speed, and position, and the posture update algorithm is the core.

[0026] As shown in FIG. 3, the inertial navigation calculation module is specifically implemented as follows: S1: Select the "east - north - up" geodetic coordinate system as the navigation coordinate system, obtain the three - axis linear acceleration of the athlete's limb and the three - axis angular velocity in the inertial coordinate system, perform navigation calculations, and obtain the posture, velocity, and position information of the athlete's limb in the navigation coordinate system; S1.1: Obtain the three - axis angular velocity of the athlete's limb in the inertial coordinate system

[0027]

Number

[0028] ;

[0029] S1.2: Calculate based on the three - axis angular velocity of the athlete's limb in the inertial coordinate system

[0030]

Number

[0031] to obtain the three - axis angular velocity of the athlete's limb in the navigation coordinate system

[0032]

Number

[0033] ;

[0034] Obtain the following from the angular velocity equation.

[0035]

Number

[0036]

Number

[0037] is the projection in the carrier coordinate system of the angular velocity of the carrier coordinate system with respect to the navigation coordinate system,

[0038]

Number

[0039] is the projection in the carrier coordinate system of the angular velocity of the carrier coordinate system with respect to the inertial coordinate system,

[0040]

Number

[0041] is the projection in the carrier coordinate system of the angular velocity of the Earth coordinate system with respect to the inertial coordinate system,

[0042]

Number

[0043] is the projection in the carrier coordinate system of the angular velocity of the navigation coordinate system with respect to the Earth coordinate system.

[0044] Since the accuracy of the MEMS sensor is low and it cannot detect the angular velocity of the Earth's rotation,

[0045]

Number

[0046] it may be ignored. Since the speed of a human in a general sports scene or walking scene is less than 10 m / s and the radius of the Earth R = 6371393 m,

[0047]

Number

[0048] therefore,

[0049] [Number]

[0050] is 10 ―7 ~10 ―6 order and can be ignored as well. Therefore, for the MEMS sensor, the above formula is equivalent as follows.

[0051] [Number]

[0052] S1.3: Calculate the posture quaternion Q of the athlete's limb at the current sampling timing k and

[0053] [Number]

[0054] That is,

[0055] [Number]

[0056] Δt is the sampling interval of the 3-axis angular velocity in the inertial coordinate system

[0057] [Number]

[0058] i.e., the output interval of the MEMS sensor, and Q k-1 is the posture quaternion of the athlete's limb at the previous sampling timing. Q k The initial value of is obtained by calculation from the initial posture angles θ0, γ0, ψ0 of the athlete's limb in the navigation coordinate system by the initial alignment, and then is obtained by calculation from the quaternion that is continuously updated.

[0059] S1.4: Calculate the coordinate transformation matrix from the athlete's limb body coordinate system to the navigation coordinate system based on the quaternion Q of the athlete's limb posture at the current sampling timing. k

[0060]

Number

[0061]

[0062] S1.5: The coordinate transformation matrix from the athlete's limb body coordinate system to the navigation coordinate system

[0063]

Number

[0064] Based on this, calculate the posture of the athlete's limb in the navigation coordinate system, and the posture of the athlete's limb in the navigation coordinate system includes the pitch angle θ, roll angle γ, and yaw angle ψ of the athlete's limb; The specific calculation method is to obtain the following from

[0065]

Number

[0066]

[0067]

Number

[0068] S1.6: The coordinate transformation matrix from the athlete's limb body coordinate system to the navigation coordinate system

[0069]

Number

[0070] ​​​Substitute it into the specific force equation to obtain the projection of the acceleration of the navigation coordinate system with respect to the Earth coordinate system in the navigation coordinate system

[0071]

Number

[0072] ;

[0073] The specific force equation is as follows.

[0074]

Number

[0075] f b is the three-axis linear acceleration in the inertial coordinate system of the athlete's limb,[[]]

[0076]

Number

[0077] is the projection of the angular velocity of the Earth coordinate system with respect to the inertial coordinate system in the navigation coordinate system,[[]]

[0078]

Number

[0079] is the projection of the angular velocity of the navigation coordinate system with respect to the Earth coordinate system in the navigation coordinate system, and g n is the projection of the gravitational acceleration in the navigation coordinate system; In general scenes, since the human movement speed is less than 10 m / s, the projection of the angular velocity of the Earth coordinate system with respect to the inertial coordinate system in the navigation coordinate system

[0080]

Number

[0081] Projection in the navigation coordinate system of the velocity of the navigation coordinate system with respect to the earth coordinate system

[0082]

Number

[0083] And the projection in the navigation coordinate system of the angular velocity of the navigation coordinate system with respect to the earth coordinate system

[0084]

Number

[0085] All of them may be ignored, and g n Since it is the projection in the navigation coordinate system of the gravitational acceleration, calculate

[0086]

Number

[0087] That is, the projection in the navigation coordinate system of the acceleration of the human body with respect to the earth can be obtained.

[0088] S1.7: Equation

[0089]

Number

[0090] By this, the projection in the navigation coordinate system of the velocity of the navigation coordinate system with respect to the earth coordinate system, that is, the velocity in the navigation coordinate system of the athlete's limbs is updated,

[0091]

Number

[0092] is the projection in the navigation coordinate system of the velocity of the navigation coordinate system with respect to the earth coordinate system at the previous sampling timing,

[0093]

Number

[0094] is the projection in the navigation coordinate system of the velocity of the navigation coordinate system with respect to the earth coordinate system at the current sampling timing.

[0095] S1.7: The position of the athlete's limb in the navigation coordinate system is updated by the following equation.

[0096]

Number

[0097] Δt is the sampling interval of the MEMS sensor, P k-1 is the position at the previous sampling timing, P k is the position at the current sampling timing,

[0098]

Number

[0099] is the projection in the navigation coordinate system of the velocity of the navigation coordinate system with respect to the earth coordinate system at the previous sampling timing.

[0100] S2: Taking the attitude angle error, velocity error, position error, gyro zero offset and accelerometer zero offset in the MEMS sensor of the athlete's limb as state variables, and taking the velocity error and attitude error within the zero velocity interval of the athlete's limb as measurement variables, establish a Kalman filter; The state variable X in the Kalman filtering method is as follows.

[0101]

Number

[0102]

Number

[0103] is the posture angle error of the athlete's limb in the navigation coordinate system,

[0104]

Number

[0105] is the velocity error of the athlete's limb in the navigation coordinate system,

[0106]

Number

[0107] is the position error of the athlete's limb in the navigation coordinate system,

[0108]

Number

[0109] is the gyro zero offset,

[0110]

Number

[0111] is the accelerometer zero offset; The state equation is as follows.

[0112]

Number

[0113] X is a state quantity, Φ is a one-step transition matrix, Γ is a process noise allocation matrix, W is a process noise matrix, k-1 and k respectively indicate the (k-1)th sampling timing and the kth sampling timing, and k / k-1 indicates a one-step prediction from the (k-1)th sampling timing to the kth sampling timing;

[0114]

Number

[0115] W is a process noise matrix,

[0116]

Number

[0117] are respectively the noises of a three-axis gyroscope,

[0118]

Number

[0119] are the noises of a three-axis accelerometer,

[0120]

Number

[0121] is

[0122]

Number

[0123] a skew-symmetric matrix consisting of,

[0124]

Number

[0125] is the three-axis acceleration in the navigation coordinate system of the carrier; The process noise allocation matrix Γ is as follows.

[0126]

Number

[0127] The measured quantities are as follows.

[0128]

Number

[0129]

Number

[0130] are the three-axis components of the velocity in the navigation coordinate system of the athlete's limb respectively;

[0131]

Number

[0132] are the attitude angle data of the athlete's limb at the previous sampling timing and the current sampling timing respectively; The measurement equation is as follows.

[0133]

Number

[0134] ω ie is the angular velocity of the Earth's rotation, L is the Earth's latitude where the carrier is located, and U is the measurement noise matrix;

[0135]

Number

[0136] are respectively the three-axis velocity error noises,

[0137] [Number]

[0138] is the attitude angle error noise, θ, γ and ψ are respectively the pitch angle, roll angle and yaw angle of the athlete's limb, and Δt is the sampling interval of the MEMS sensor.

[0139] S3: At each sampling timing of the MEMS sensor, perform a one-step prediction of the Kalman filter state quantity, calculate the state one-step prediction mean square error matrix, and proceed to step S4.

[0140] S4: Determine whether the athlete's limb is within the zero-velocity interval. If it is within the zero-velocity interval, proceed to step S5; otherwise, proceed to step S6; Due to the low accuracy of the MEMS inertial sensor, it is the main error factor affecting the navigation accuracy of the system. When used for a long time, the navigation error continuously accumulates over time, severely affecting the accuracy of the final measurement result. Different zero-velocity detection algorithms can be used to detect the stationary interval during human movement, and further, the parameters can be corrected within the zero-velocity interval to effectively remove the velocity error and constrain the position and course error.

[0141] During the human walking process, as the foot is lifted, stepped out, landed, and stationary, the IMU sensors worn on different parts of the human body can detect corresponding periodic changes. According to the analysis, in addition to the foot during the human walking process, there are also periodic zero-velocity intervals in parts such as the thigh and calf. By accurately setting different energy thresholds using different detection algorithms, the periodic zero-velocity intervals of different parts of the human body can be detected.

[0142] In the present invention, the method for determining whether the speed of the athlete's limb is within the zero-speed interval is as follows. The original data output from the MEMS gyroscope and the MEMS accelerometer is introduced into the zero-speed detector. The zero-speed detector calculates the statistical quantity of the athlete's limb movement energy, sets the corresponding threshold of the zero-speed detector. If the statistical quantity of the zero-speed detector is lower than the predetermined threshold of the zero-speed detector, it is considered that the athlete's limb is within the zero-speed interval; otherwise, it is considered that the athlete's limb is outside the zero-speed interval.

[0143] For different athlete's limbs, the zero-speed detector uses different algorithms to calculate the statistical value of the limb movement energy. Specifically, if the athlete's limb is the human foot, the zero-speed detector uses the GLRT or ARE algorithm to calculate the energy statistical value; if the athlete's limb is the human thigh or calf, the zero-speed detector uses the MAG or MV algorithm to calculate the energy statistical value.

[0144] In a specific embodiment of the present invention, based on the movement data characteristics of different parts of the human body during the movement process, the foot zero-speed detection algorithm can use GLRT, and the energy detection threshold can be set to 25000. The calf zero-speed detection algorithm can use the MAG algorithm, and the energy detection threshold can be set to 1000. The thigh zero-speed detection algorithm can use the MAG algorithm, and the energy detection threshold can be set to 750. By reasonably setting the energy detection threshold using different zero-speed detection algorithms, the zero-speed interval of the corresponding part, that is, the interval where the statistical value of the athlete's limb movement energy is smaller than the detection threshold, can be effectively detected.

[0145] S5: Update the measurement and measurement matrix of the Kalman filter, calculate the filtering gain based on the measurement, the state one-step prediction mean square error matrix, the state estimation mean square error matrix, and the measurement noise covariance matrix to update the state estimation mean square error matrix, estimate the state using the filtering gain and the measurement matrix, obtain the velocity error, position error, and attitude angle error in the navigation coordinate system of the athlete's limb, and then correct the attitude, velocity, and position information in the navigation coordinate system of the athlete's limb based on the estimated error.

[0146] S6: Output the attitude, velocity, and position information in the navigation coordinate system of the athlete's limb.

[0147] In the present invention, the detailed content of the Kalman filtering and zero velocity error correction algorithm is as follows. The principle of Kalman filtering is to establish a Kalman filter using the velocity error and attitude angle error within the zero velocity interval as the measurement, estimate the velocity error, position error, and attitude angle error of the athlete's limb, compensate the corresponding variables with the estimated errors, and obtain an estimate approaching the true value of the state variable.

[0148] Since the state variables of the Kalman filter include velocity error, position error, and attitude error, it is necessary to establish an appropriate state equation based on the error equation of inertial navigation, the characteristics of MEMS sensors, and the characteristics of human motion.

[0149] 3.1 Error Equation (a) Attitude Error Equation The MEMS attitude error equation is as follows.

[0150]

Equation

[0151] Φ is the attitude angle error, and ε b is the gyro zero offset.

[0152] (b) Velocity Error Equation The MEMS velocity error equation is as follows.

[0153] [Number]

[0154] δV is the velocity error, and f n is the projection of the acceleration in the navigation coordinate system, and ∇ b is the accelerometer zero offset.

[0155] (c) Position error equation The MEMS position error equation is as follows.

[0156] [Number]

[0157] δP is the position error and δV is the velocity error.

[0158] 3.2 Correction algorithm and measurement equation (a) Zero velocity error correction When it is detected that the motion is in a stationary stage, theoretically, its actual velocity is zero. However, since there are large measurement errors in MEMS sensors, the velocity calculated by MEMS inertial navigation is not actually zero. The zero velocity error correction method aims to suppress the navigation parameter error by taking the velocity calculated by MEMS inertial navigation in the stationary stage as the velocity error and performing Kalman filtering estimation with this velocity error as the measurement quantity.

[0159] Therefore, the velocity error by the zero velocity error correction algorithm is ΔV, and moreover,

[0160] [Number]

[0161] [Number]

[0162] They are the three-axis components of the speed value of the athlete's limbs by navigation calculation respectively.

[0163] (b) Attitude error correction On a stationary staircase, theoretically, the attitude angles do not change at two timings before and after. Similarly, since there are large measurement errors in the MEMS sensor, the difference in the attitude angles solved at two timings before and after is not zero. Therefore, the attitude angle error is suppressed using the difference in the attitude angles at two timings before and after within the zero-velocity interval as the measured quantity. Therefore, the measured quantity by the attitude error correction algorithm is

[0164] [Number]

[0165] and,

[0166] [Number]

[0167] ω ie is the angular velocity of the Earth's rotation, and L is the Earth's latitude where the human body to be measured is located.

[0168] 3.3 Kalman filtering (a) State equation By combining the attitude error equation, velocity error equation, and position error equation, the following expression of the state equation can be obtained.

[0169] [Number]

[0170] X is a state quantity, Φ is a one-step transition matrix, Γ is a process noise allocation matrix, W is a process noise matrix, k-1 and k respectively represent the (k-1)-th sampling timing and the k-th sampling timing, and k / k-1 represents a one-step prediction from the (k-1)-th sampling timing to the k-th sampling timing.

[0171]

Number

[0172]

Number

[0173] is the posture angle error in the navigation coordinate system of the athlete's limb,

[0174]

Number

[0175] is the velocity error in the navigation coordinate system of the athlete's limb,

[0176]

Number

[0177] is the position error in the navigation coordinate system of the athlete's limb,

[0178]

Number

[0179] is the gyro zero offset,

[0180]

Number

[0181] is the accelerometer zero offset; The one-step transition matrix is as follows.

[0182]

Number

[0183] The process noise matrix is as follows.

[0184]

Number

[0185] W is the process noise,

[0186]

Number

[0187] are the noises of the three-axis gyroscopes respectively,

[0188]

Number

[0189] are the noises of the three-axis accelerometers,

[0190]

Number

[0191] is

[0192]

Number

[0193] a strain-symmetric matrix consisting of,

[0194]

Number

[0195] is the three-axis acceleration in the navigation coordinate system of the carrier; The process noise allocation matrix is as follows.

[0196]

Number

[0197] (b) Measurement equation By integrating the zero-velocity error correction and the attitude error correction, the following expression of the measurement equation can be obtained.

[0198]

Number

[0199] The measured quantities are as follows.

[0200]

Number

[0201]

Number

[0202] are the three-axis components of the velocity in the navigation coordinate system of the athlete's limbs respectively;

[0203]

Number

[0204] are the attitude angle data of the athlete's limbs at the previous sampling timing and the current sampling timing respectively; The measurement matrix is as follows.

[0205]

Number

[0206] ω ie is the angular velocity of the Earth's rotation, L is the Earth's latitude where the carrier is located, θ, γ, and ψ are the pitch angle, roll angle, and yaw angle of the athlete's limb respectively, and Δt is the sampling interval of the MEMS sensor. The measurement noise matrix U is as follows.

[0207]

Number

[0208]

Number

[0209] are the three-axis velocity error noises respectively,

[0210]

Number

[0211] is the attitude angle error noise.

[0212] (c) Filtering algorithm Based on the Kalman filtering algorithm, after discretizing the continuous equations, substitute them into the following formula. State one-step prediction

[0213]

Number

[0214]

Number

[0215] is the optimal state estimation at the previous sampling timing,

[0216]

Number

[0217] is the state estimation from the previous sampling timing to the current sampling timing,

[0218]

Number

[0219] is the one-step transition matrix from the previous sampling timing to the current sampling timing. State one-step prediction mean square error matrix

[0220]

Number

[0221] P k / k-1 is the mean square error matrix from the previous sampling timing to the current timing, and P k-1 is the mean square error matrix at the previous sampling timing, and Γ k-1 is the process noise allocation matrix at the previous sampling timing, and Q k-1 is the process noise covariance matrix at the previous sampling timing. Filtering gain

[0222]

Number

[0223] K k is the filtering gain at the current sampling timing, and P k / k-1 is the mean square error matrix at the current sampling timing, and H k is the measurement matrix at the current sampling timing, and R kis the covariance matrix of the current sampling timing measurement noise. State estimation

[0224]

Number

[0225]

Number

[0226] is the optimal state estimation at the current sampling timing,

[0227]

Number

[0228] is the state estimation from the previous sampling timing to the current sampling timing, K k is the filtering gain at the current sampling timing, Z k is the measured value at the current sampling timing, H k is the measurement matrix at the current sampling timing. State estimation mean square error matrix

[0229]

Number

[0230] P k is the mean square error matrix at the current sampling timing, P k / k-1 is the mean square error matrix from the previous sampling timing to the current sampling timing, I is the identity matrix, K k is the filtering gain at the current sampling timing, H k is the measurement matrix at the current sampling timing. Since there is only a zero-velocity measurement in the zero-velocity interval, within the zero-velocity interval, the Kalman filter does not perform measurement updates but only time updates. After detecting the zero-velocity interval, the filter performs both time updates and measurement updates.

[0231] 3.2, Moving Target Detection and Tracking Module The camera imaging principle is shown by the following formula.

[0232]

Number

[0233] (u, v) are pixel coordinates, and (X w , Y w , Z w ) are world coordinates. M1 is the internal parameter matrix, and f x = f / dx is the normalized focal length in the x-axis direction of the camera, and f y = f / dy is said to be the normalized focal length in the y-axis direction of the camera, with the unit being pixels. f is the focal length of the camera, and dx and dy are the physical sizes of the pixels in the x and y-axis directions of the camera, respectively. (u0, v0) are the coordinates in the pixel coordinate system at the center of the image, with the unit being pixels. M2 is the external parameter matrix.

[0234] The radial distortion formula is as follows.

[0235]

Number

[0236] k1 is the coefficient of the quadratic term of radial distortion, and k2 is the coefficient of the quartic term of radial distortion. k3 is the coefficient of the sixth-order term of radial distortion; The tangential distortion formula is as follows.

[0237]

Number

[0238] p1 is the first tangential distortion coefficient, p2 is the second tangential distortion coefficient. In the formula, (x, y) are the ideal image coordinates without distortion,

[0239]

Number

[0240] are the image coordinates after distortion, r is the distance from a certain point in the image to the image center point, that is, r 2 = x 2 + y 2 is.

[0241] The moving target detection and tracking module uses the undistort function in the computer vision library opencv to perform distortion correction on the images captured by each camera. The undistort function is as follows. void undistort(InputArray src, OutputArray dst, InputArray cameraMatrix, InputArray distCoeffs, InputArray newCameraMatrix) src is the pixel matrix of the original image, dst is the pixel matrix of the corrected image; cameraMatrix is the internal parameter of the camera.

[0242]

Number

[0243] f x = f / dx is the normalized focal length of the camera in the x-axis direction, f y = f / dy is the normalized focal length of the camera in the y-axis direction, with the unit of pixel. f is the focal length of the camera, dx and dy are the physical sizes of the pixels in the x and y-axis directions of the camera respectively, and (u0, v0) are the coordinates in the pixel coordinate system of the image center, with the unit of pixel; distCoeffs are distortion parameters.

[0244]

Number

[0245] k1 is the coefficient of the quadratic term of radial distortion, k2 is the coefficient of the quartic term of radial distortion, k3 is the coefficient of the sextic term of radial distortion, p1 and p2 are the first tangential distortion parameter and the second tangential distortion parameter respectively, and InputArray newCameraMatrix is a zero matrix.

[0246] The calibration process of the camera internal parameters cameraMatrix and distortion parameters distCoeffs is as follows. As shown in Figure 4, prepare one Zhangyou calibration method checkerboard as the calibration board, use the camera to take pictures of the calibration board at different angles, and obtain a set of W checkerboard images, where 15 ≤ W ≤ 30.

[0247] Use the camera calibration tool Camera Calibration in the matlab toolbox to read the W checkerboard images, automatically detect the corner points on the checkerboard, and obtain the coordinates of the corner points in the pixel coordinate system. Input the actual size of the cells of the checkerboard into the calibration tool Camera Calibration, and the calibration tool Camera Calibration calculates the world coordinates of the corner points. Based on the coordinates of the corner points in the pixel coordinate system and the world coordinate system in the W images, the calibration tool Camera Calibration performs parameter calculation to obtain the camera internal parameters IntrinsicMatrix and distortion parameters distCoeffs.

[0248] The motion target detection and tracking module calls the perspectiveTransform function in the computer vision library opencv to convert the coordinates of the athlete in the pixel coordinate system to the coordinates in the world coordinate system of the camera field of view coverage area.

[0249] The process of obtaining the perspective projection matrix is as follows. S2.1: Arrange and fix the cameras in the athlete's motion scene so that the entire fields of view of M cameras cover the entire athlete's motion scene and adjacent camera screens have overlapping areas; S2.2: Define the field plane of the motion scene as the XOY plane of the global world coordinate system, arrange R rows and C columns of landmark points on the field plane, with the rows of the landmark points parallel to the X-axis of the global world coordinate system and the columns of the landmark points parallel to the Y-axis of the global world coordinate system. Each landmark point is provided with a diamond pattern, and the connecting lines of the opposite vertices of the diamond pattern are parallel to the X-axis and Y-axis of the global world coordinate system. The center point position of the diamond is used as the position of the landmark point. As shown in Figure 5, each camera field of view contains a 2 number of landmark points. The landmark points are uniformly distributed in the form of an a*a matrix, and each landmark point located at the periphery is close to the edge of the camera field of view. The overlapping area of adjacent camera fields of view contains a common landmark points. In an embodiment of the present invention, the value of a is set to 3. S2.3: For each camera, select the upper left corner landmark point in the camera field of view as the origin, that is, select it as having coordinates (0,0), establish the camera field of view region world coordinate system, measure the position of each landmark point relative to the origin, and obtain the coordinates of the 9 landmark points in the camera field of view region world coordinate system; S2.4: Use the camera to take pictures, and each camera obtains an image containing a 2 number of landmark points; S2.5: Perform distortion correction on the image taken by the camera; S2.6: Determine the coordinates of the a 2 number of landmark points in the pixel coordinate system of the distortion-corrected image taken by each camera; Display the image after distortion correction in Matlab, display the position of the point pointed by the mouse in the image using the impixelinfo command, place the mouse at the center of the diamond marker, and a 2 Obtain the positions of the images of a 2 markers in the image. Define the center of the upper left diamond marker in the image as the origin of the pixel coordinate system, record the coordinates as (0, 0), and record the coordinates of the opposing position between the other a -1 non-origin marker points and the origin as the coordinates in the pixel coordinate system of the non-origin marker point. 2 S2.7: For each camera, record the coordinates of each marker point in the pixel coordinate system and the coordinates in the corresponding camera field-of-view region world coordinate system as a set of coordinates, and a

[0250] 3.3 Motion Target Detection and Tracking Module 3.1 YOLO Model The YOLO model is an object recognition and positioning algorithm based on a deep neural network, and the algorithm is as follows. (1) Convert the resolution of the image collected by the camera to 416*416 and divide it into S*S grid cells. In a specific embodiment of the present invention, generally, the value of S is set to 7. (2) Each grid predicts B bounding boxes (Bbox, bounding box) and the confidence score of the bounding box. In a specific embodiment of the present invention, B is 2. (3) The bounding box information is represented by four values (x, y, w, h), where (x, y) is the center coordinate of the bounding box, and w and h are the width and height of the bounding box. (4) The confidence level includes two aspects. The first is the possibility that the bounding box contains the target, and the second includes the accuracy of the bounding box. The former is denoted as Pr(object). When the bounding box contains the target, Pr(object) = 1; otherwise, Pr(object) = 0 (only including the background). The latter is characterized by the IOU (intersection over union) between the predicted box and the actual box (ground truth),

[0251]

Number

[0252] and is denoted as such. Then, the confidence level is

[0253]

Number

[0254] defined as. (5) Besides the bounding box, each grid further predicts C class probability values, representing the probability that the target of the bounding box to be predicted in the cell belongs to each class, denoted as Pr(classi|object).

[0255] As described above, each grid needs to predict (B * 5 + C) values. Set B = 2 and C = 20, then the numerical values contained in each grid will be as shown in Figure 2.

[0256] When the input picture is divided into an S * S grid, the final predicted values are S * S * (B * 5 + C).

[0257] When actually conducting the test, it is also necessary to further calculate the class confidence scores of each bounding box.

[0258]

Number

[0259] For C categories, i = 1, 2, ..., C.

[0260] After obtaining the category confidence of each bounding box, a threshold is set (the threshold is 0.5 in this embodiment), the bounding boxes with low scores are filtered, and the retained bounding boxes are processed by NMS (Non-Maximum Suppression algorithm) to obtain the final detection result. For each detected target, the final output includes seven values: four position values (x, y, w, h) (i.e., the final bounding box), one bounding box confidence, one category confidence, and one category code.

[0261] Since edge detection performs pixel-level processing on an image, accurate pixel-level positioning of the target can be achieved, and the processing flow is shown in FIG. 6. The moving target detection and tracking module performs processing such as edge detection on the bounding box marking area (hereinafter referred to as ROI, Region Of Interest) by YOLO detection to obtain the accurate position and precise bounding box of each player in the pixel coordinate system. S3.1: Perform grayscale conversion and Gaussian filtering on the rough bounding box marking area of the player by YOLO detection; S3.2: Use the Canny-Devernay algorithm to perform edge detection on the rough bounding box marking area of the player to obtain the accurate contour of the player and a set of player contour point coordinates; S3.3: Calculate the characteristic moments of the contour based on the player contour point coordinates; S3.4: Use the characteristic moments of the contour to calculate the centroid of the player

[0262]

Number

[0263] That is, calculate the exact position of the player in the pixel coordinate system; Specifically, use the opencv function cv::moments to obtain the object cv::Moments, and from it, obtain the zeroth moment m 00 and the first moment m 10 , m 01 is obtained.

[0264]

Number

[0265] S3.5: Take the minimum bounding rectangle of the target contour as the precise bounding box of the player.

[0266] The motion target detection and tracking module uses the DeepSORT method to track the precise bounding boxes of each player at different times.

[0267] The DeepSORT algorithm is an extension of the SORT algorithm. The SORT algorithm is an algorithm for realizing multi-target tracking, and its calculation process is as follows. Before tracking, all player detections were completed by the target detection algorithm.

[0268] When the first frame image enters, initialize it with the detected target Bbox, establish a new tracker, and assign an id.

[0269] When the next frame enters, obtain the state prediction and covariance prediction generated from the previous frame Bbox by the Kalman tracker (Kalman Filter). Then, calculate the IOU between all target states of the tracker and the Bbox detected in this frame, and use the Hungarian Algorithm to obtain the unique matching with the maximum IOU (the relevant part of the data), and remove the matching pairs whose matching value is less than iou_threshold (generally 0.3).

[0270] Update the Kalman tracker with the target detection Bbox that matches in the frame, and perform state update and covariance update. Output the state update value as the tracking Bbox of the frame. For the targets that do not match in the frame, re-initialize the tracker. Then, the Kalman tracker makes the next prediction.

[0271] The DeepSORT algorithm does not significantly change the overall framework of SORT, but adds a matching cascade and target confirmation to improve the effectiveness of tracking.

[0272] For the position sequence of the athlete in the global world coordinate system, perform filtering using the method of grouping and averaging, and then perform differential operation on the average to obtain the movement speed of the target.

[0273] Figure 7 is the overall flow of target recognition and tracking in a specific embodiment of the present invention.

[0274] When applied to the field of swimming, recognize the color marks on the swimming cap as targets, and perform speed and position tracking on the swimmers.

[0275] The present invention recognizes multiple athletes simultaneously and completes the calculation of speed and position.

[0276] 3.4, Motion Parameter Analysis Module Analyze the relative position and posture of the athlete's limbs in the athlete's body coordinate system to obtain the joint angles of movement, the stride and walking rate of the athlete, compare the positions and speeds of each athlete in the world coordinate system of the motion scene, obtain the ranking among the athletes, analyze and compare the differences between the motion parameters of the athletes and the standard motion parameters, provide an improved training method, and guide the athletes to improve the training level.

[0277] For swimming, by further conversion, exercise parameters such as the breathing rate, stroke frequency, stroke width, stroke count, and turn time of different swimming strokes are obtained.

[0278] Since the computational load of the data comprehensive analysis device is extremely large, in a specific embodiment of the present invention, the data comprehensive analysis device is realized by setting up a high-performance server. Specifically, it includes a cloud computing server, a cloud storage server, and a service management server.

[0279] The cloud computing server supports the second-generation Intel Xeon scalable processor, supports 8 Tesla GPU accelerator cards in a 2U space, and is currently the server with the highest GPU density in a unit space. It supports two types of interface GPU cards, SXM2 and PCIe, and supports NVIDIA (registered trademark) NVLink2.0 high-speed interconnect technology, realizing an aggregation bandwidth of 300GB / s between GPUs. The Hybrid CubeMesh interconnect improves the latency situation of multi-GPU data sharing, provides better acceleration for computing, reduces system latency, and has strong overall performance. It is suitable for applications in fields such as deep learning model training, offline estimation, scientific computing, engineering computing, and research. The cloud computing server mainly realizes all the functions of the inertial navigation calculation module, the moving target detection and tracking module, the moving target speed recognition module, and the exercise parameter analysis module included in the data comprehensive analysis device.

[0280] The memory server is a network memory product targeting data storage needs. It provides unified IP SAN and NAS characteristics, realizes flexible deployment of the system architecture, provides the Snapshot Copy (data snapshot copy) function in the iSCSI deployment, supports up to 36 3.5-inch high-capacity hard disks in stand-alone mode, the system supports the expansion function of SAS JBOD, the mixed insertion of SAS and SATA disks, and 10TB high-capacity hard disks, supports automatic abnormal power switching and hot swapping of abnormal power supplies, can protect the cache data of the device, the memory system and the data are independent of each other, do not occupy the data storage space, and uses a dedicated memory operating system to ensure the performance and reliability of system access. The Chinese management interface for visualization is more convenient and easy to use, and users can perform deployment operations and status monitoring on magnetic disks, Raid arrays, etc. through the GUI management interface. The memory server stores all the original data sent from the inertial navigation wearable device and the camera collected by the data comprehensive analysis device, the position and speed of each athlete in the world coordinate system of the motion scene calculated from the data comprehensive analysis device itself, and the relative position and posture of each athlete's limbs in the athlete's body coordinate system, and determines the motion parameters of each athlete, etc. These information are stored based on athlete information and time so that they can be checked and analyzed.

[0281] The service management server mainly completes the interaction function with the terminal and the inertial navigation wearable device, realizes the data synchronization between the inertial navigation system and the camera, and is also used to realize the data synchronization between different inertial navigation wearable devices; The service management server supports the new generation Intel Xeon series processors and up to 24 DIMMs, significantly improves the application performance, and the computing performance increases by up to 70%.

[0282] 4. Terminal In a specific embodiment of the present invention, the terminal may be a personal computer, a tablet computer, or a smart phone.

[0283] In addition to completing the above display function, the terminal further supports the use of users with four types of identities: hands, coaches, experts, and administrators based on user needs. , select The terminal with player authority set includes a "self-training" module, a "history data" check module, and a first "group communication" module. After the player identity logs in to the application, basic settings for the "training mode", "history data" check, "group" communication, and "my" application can be performed. The "self-training" module obtains and records real-time motion parameters from the data comprehensive analysis device. First The "history data" check module searches for the original image, motion parameters, and corresponding training evaluations of the corresponding time period from the data comprehensive analysis device based on the exercise time period and the basic information of the player, objectively recognizes the shortcomings during personal training, and borrows the help of experts and coaches to accurately adjust the training to achieve improvement. The first "group communication" module receives player messages and uses them for mutual communication between players, coaches, and experts to share relevant data and further achieve improvement. First The terminal with coach authority set includes a "player management" module, a "match management" module, and a second "group communication" module. After the coach identity logs in to the application, basic settings for "player management", "match management", "group" communication, and "my" application can be performed. The "player management" module increases or decreases players and updates the basic information of the players to the data comprehensive analysis device. Second "history data" check module, The "match management" module searches for the original image, motion parameters, and corresponding training evaluations of the corresponding time period from the data comprehensive analysis device based on the exercise time period and the basic information of the player, objectively recognizes the shortcomings during personal training, and borrows the help of experts and coaches to accurately adjust the training to achieve improvement. The first "group communication" module receives player messages and uses them for mutual communication between players, coaches, and experts to share relevant data and further achieve improvement. SecondThe "Historical Data" check module searches for the original images and motion parameters of the corresponding time period from the data comprehensive analysis device based on the externally input exercise time period and the basic information of the athlete, submits a training evaluation, sends it to the data comprehensive analysis device for storage. The "Match Management" module creates a new intra-team competition, sends the grouping and competition rules of the intra-team competition to the data comprehensive analysis device for storage, and invites personnel such as coaches, athletes, and experts to participate jointly. The second "Group Communication" module receives coach messages and uses them for the mutual communication between coaches and athletes and experts. The terminal with expert authority settings includes the "Training Management" module and the third "Group Communication" module. After logging in with the expert identity, basic installations of "Training Management", "Group" communication, and "My" applications can be performed. The "Training Management" module checks the training rankings, compares the motion parameters of athletes in the same session, evaluates and makes suggestions for the athletes and the training of the corresponding session, forms a data analysis report, and sends it to the data comprehensive analysis device for storage. The third "Group Communication" module receives expert messages and uses them for the mutual communication between experts and coaches and athletes, checks the data shared by athletes, conducts one-on-one precise analysis, and provides personalized guidance. After the administrator identity is set on the terminal and the administrator logs in, simple operation processing can be performed on the mobile terminal, such as user information reset, user identity authentication, training data management, and consultation message reply.

[0284] In a specific embodiment of the present invention, all data installed by the terminal is stored in the data comprehensive analysis device.

[0285] As described above, in the present invention, quantitative analysis and control of each motion parameter of an athlete are realized by means of an inertial navigation wearable device, a data comprehensive analysis device, etc., interaction and communication between the athlete and the coach are realized by real-time data, and support means are provided for the realization of the integration of standardization and personalization of training parameters.

[0286] The above is only the optimal specific embodiment of the present invention, and the protection scope of the present invention is not limited thereto. Any changes or replacements that can be easily conceived by those skilled in the art within the technical scope disclosed by the present invention should fall within the protection scope of the present invention.

[0287] Contents not described in detail in the specification belong to the known technologies of those skilled in the art.

Explanation of Reference Numerals

[0288] Step S1 Step S2 Step S3 Step S4 Step S5 Step S6

Claims

Claim 1 A human motion intelligent measurement and digital training system, comprising N inertial navigation wearable devices, M cameras, a data comprehensive analysis device, and a terminal, where N and M are both greater than or equal to 1, The entire field of view of the M cameras covers the entire athlete's motion scene, captures images within the field of view, forms image data frames, and transmits them to the data comprehensive analysis device, The inertial navigation wearable device is worn and fixed on the athlete's limb. Using the athlete's limb as a carrier, it measures and obtains the three-axis linear acceleration of the athlete's limb and the three-axis angular velocity in the inertial coordinate system, and transmits them to the data comprehensive analysis module, The data comprehensive analysis device stores the basic information of the athlete, establishes and maintains the relationship between the athlete and the inertial navigation wearable device worn by the athlete. Based on the three-axis linear acceleration of the athlete's limb and the three-axis angular velocity in the inertial coordinate system, it performs navigation calculations and coordinate transformations to obtain and store the relative position and posture of the athlete's limb in the athlete's body coordinate system. It collects and stores the images captured by each camera, performs target recognition, tracking, and coordinate transformation on the images captured by each camera to obtain and store the position and velocity of the athlete in the world coordinate system of the motion scene. It analyzes the position and velocity of each athlete in the world coordinate system of the motion scene and the relative position and posture of each athlete's limb in the athlete's body coordinate system to determine and store the motion parameters of each athlete, The data comprehensive analysis device includes an inertial navigation calculation module, a motion target detection and tracking module, a motion target velocity recognition module, and a motion parameter analysis module, The inertial navigation calculation module performs navigation calculations based on the three-axis linear acceleration of the athlete's limb and the three-axis angular velocity in the inertial coordinate system to obtain the posture, velocity, and position information of the athlete's limb in the navigation coordinate system. It performs zero-velocity detection on the motion of the athlete's limb. When the athlete's limb is within the zero-velocity interval, it performs zero-velocity error correction on the posture, velocity, and position information of the athlete's limb in the navigation coordinate system. It defines the athlete's body coordinate system and transforms the posture, velocity, and position information of the athlete's limb in the navigation coordinate system to the athlete's body coordinate system, The motion target detection and tracking module collects the images captured by each camera, records the image collection time, performs distortion correction on the images captured by each camera, uses the YOLO model to perform target detection on each corrected image captured at the same timing, obtains the approximate bounding boxes of all players in the pixel coordinate system of the image, and based on the edge detection method, obtains the accurate position and precise bounding boxes of each player in the pixel coordinate system, matches the precise bounding boxes of the same player at different timings, realizes the tracking of the precise bounding boxes of each player at different timings, converts the coordinates of each player in the pixel coordinate system into the coordinates in the corresponding camera field of view coverage area world coordinate system by the perspective projection matrix, and based on the positional relationship between the coverage areas of each camera field of view, calculates the coordinates of each player in the motion scene global world coordinate system at different timings and transmits them to the motion target speed recognition module. The motion target speed recognition module filters the coordinate sequences of each player in the motion scene global world coordinate system at different timings to remove noise, and then performs difference processing to obtain the speed of the player in the motion scene world coordinate system. The motion parameter analysis module is characterized by analyzing the relative positions and postures of the player's limbs in the player body coordinate system to obtain motion parameters, comparing the positions and speeds of each player in the motion scene world coordinate system, analyzing and sorting these data, ranking the players according to certain rules based on the results of the analysis and sorting, and comparing the motion parameters of the players with the standard parameters, for a human body motion intelligent measurement and digital training system.

2. The human body motion intelligent measurement and digital training system according to claim 1, characterized in that the inertial navigation wearable device is worn on different limb parts of at least one player, and the output data of N inertial navigation wearable devices are synchronized.

3. Establish a motion scene and a 3D model of the athlete, associate the speed and position of the athlete in the motion scene coordinate system, the relative position and posture of the athlete's limbs in the athlete's body coordinate system, and the corresponding 3D model, and further include a terminal that displays the athlete's motion process and motion parameters in a visual manner. The human motion intelligent measurement and digital training system according to claim 1 is characterized in that.

4. The inertial navigation wearable device includes a MEMS sensor, a signal processing module, a communication module, and a lithium battery. Inside the MEMS sensor, a MEMS gyro and a MEMS accelerometer are integrated. The MEMS gyro outputs the three-axis angular velocity in the inertial coordinate system, and the MEMS accelerometer outputs the three-axis linear acceleration of the athlete's limbs. The MEMS sensor outputs the measurement result to the signal processing module. The signal processing module frames and packages the measurement result output from the MEMS sensor, and then transmits it to the communication module. The communication module transmits the measurement data frame packaged in a wireless communication manner. The lithium battery provides power to the MEMS sensor, the signal processing module, and the communication module. The human motion intelligent measurement and digital training system according to claim 1 is characterized in that.

5. Specifically, the inertial navigation calculation module is as follows: S1: Select the "east-north-up" geographic coordinate system as the navigation coordinate system, obtain the three-axis linear acceleration of the athlete's limbs and the three-axis angular velocity in the inertial coordinate system, perform navigation calculations, and obtain the posture, speed, and position information of the athlete's limbs in the navigation coordinate system. S2: Use the posture angle error, speed error, position error of the athlete's limbs in the navigation coordinate system, the gyro zero offset and accelerometer zero offset in the MEMS sensor as state quantities, and the speed error and posture error within the zero velocity interval of the athlete's limbs as measurement quantities to establish a Kalman filter. S3: At each sampling timing of the MEMS sensor, perform a one-step prediction of the Kalman filter state quantity, calculate the state one-step prediction mean square error matrix, and proceed to step S4. S4: Determine whether the athlete's limbs are within the zero velocity interval. If they are within the zero velocity interval, proceed to step S5; otherwise, proceed to step S6. S5: Update the measurement and measurement matrix of the Kalman filter, calculate the filtering gain based on the measurement, the state one-step prediction mean square error matrix, the state estimation mean square error matrix, and the measurement noise covariance matrix to update the state estimation mean square error matrix, estimate the state using the filtering gain and the measurement matrix, obtain the velocity error, position error, and attitude angle error in the navigation coordinate system of the athlete's limb, and then correct the attitude, velocity, and position information of the athlete's limb in the navigation coordinate system based on the estimated error; S6: The human motion intelligent measurement and digital training system according to claim 1, characterized in that it realizes outputting the attitude, velocity, and position information of the athlete's limb in the navigation coordinate system.

6. In step S1, the attitude of the athlete's limb in the navigation coordinate system is calculated in the following steps. S1.1: Obtain the three-axis angular velocity of the athlete's limb in the inertial coordinate system; 【Number 1】 S1.2: Based on the three-axis angular velocity of the athlete's limb in the inertial coordinate system, calculate the three-axis angular velocity of the athlete's limb in the navigation coordinate system; 【Number 2】 【Number 3】 S1.3: Calculate the posture quaternion Q of the athlete's limbs at the current sampling timing ｋ ; [Number 4] Δt is the sampling interval of the MEMS sensor, and Q ｋ－１ is the posture quaternion of the athlete's limb at the previous sampling timing; S1.4: Athlete's body posture quaternion Q at the current sampling timing ｋ Based on this, a coordinate transformation matrix from the athlete's body coordinate system to the navigation coordinate system 【Number 5】 Calculate; S1.5: Calculate the attitude of the athlete's limb in the navigation coordinate system based on the coordinate transformation matrix from the athlete's limb body coordinate system to the navigation coordinate system. The attitude of the athlete's limb in the navigation coordinate system includes the pitch angle θ, roll angle γ, and yaw angle ψ of the athlete's limb; 【Number 6】 The specific calculation method is: 【Number 7】 From 【Number 8】 The human motion intelligent measurement and digital training system according to claim 5, characterized in that it obtains.

7. In step S1, the velocity of the athlete's limb in the navigation coordinate system is calculated in the following steps. S1.6: Substitute the coordinate transformation matrix from the athlete's limb body coordinate system to the navigation coordinate system into the specific force equation to obtain the projection of the acceleration of the navigation coordinate system with respect to the earth coordinate system in the navigation coordinate system; 【Number 9】 【Number 10】 The specific force equation is as follows: 【Number 11】 f ｂ is the three-axis linear acceleration in the inertial coordinate system of the athlete's limbs, 【Number 12】 is the projection of the angular velocity of the earth coordinate system with respect to the inertial coordinate system in the navigation coordinate system. 【Number 13】 is the projection in the navigation coordinate system of the angular velocity of the navigation coordinate system with respect to the earth coordinate system, and g ｎ is the projection in the navigation coordinate system of the gravitational acceleration; S1.7: According to equation 【Number 14】 Update the projection of the velocity of the navigation coordinate system with respect to the earth coordinate system in the navigation coordinate system, that is, the velocity of the athlete's limb in the navigation coordinate system. 【Number 15】 is the projection of the velocity of the navigation coordinate system with respect to the earth coordinate system in the navigation coordinate system at the previous sampling timing. 【Number 16】 The human body movement intelligent measurement and digital training system according to claim 5, characterized in that it is currently the projection in the navigation coordinate system of the velocity of the navigation coordinate system with respect to the earth coordinate system at the sampling timing.

8. In step S1, the position of the athlete's limb in the navigation coordinate system is updated by the following equation: 【Number 17】 Δt is the sampling interval of the MEMS sensor, and P ｋ－１ is the position at the previous sampling timing, and P ｋ is the position at the current sampling timing, 【Number 18】 The human body movement intelligent measurement and digital training system according to claim 5, characterized in that it is currently the projection in the navigation coordinate system of the velocity of the navigation coordinate system with respect to the earth coordinate system at the previous sampling timing.

9. The method for determining whether the velocity of the athlete's limb is within the zero-velocity interval is as follows: The original data output from the MEMS gyroscope and the MEMS accelerometer is introduced into a zero-velocity detector, and the zero-velocity detector calculates the statistical quantity of the athlete's limb movement energy, sets the corresponding threshold of the zero-velocity detector. If the statistical quantity of the zero-velocity detector is lower than the predetermined threshold of the zero-velocity detector, it is considered that the athlete's limb is within the zero-velocity interval; otherwise, it is considered that the athlete's limb is outside the zero-velocity interval. The human body movement intelligent measurement and digital training system according to claim 5 is characterized in that.

10. For different athlete's limbs, the zero-velocity detector uses different algorithms to calculate the statistical value of the movement energy of the athlete's limb. Specifically, if the athlete's limb is the human foot, the zero-velocity detector uses the GLRT or ARE algorithm to calculate the energy statistical value. If the athlete's limb is the human thigh or calf, the zero-velocity detector uses the MAG or MV algorithm to calculate the energy statistical value. The human body movement intelligent measurement and digital training system according to claim 9 is characterized in that.

11. In step S2, the state quantity X of the Kalman filtering method is as follows: 【Number 19】 【Number 20】 is the attitude angle error of the athlete's limb in the navigation coordinate system; 【Number 21】 is the velocity error of the athlete's limb in the navigation coordinate system; 【Number 22】 is the position error of the athlete's limb in the navigation coordinate system; 【Number 23】 is the gyro zero offset; 【24 Points】 is the accelerometer zero offset; The state equation is as follows: 【Number 25】 X is a state quantity, Φ is a one-step transition matrix, Γ is a process noise allocation matrix, W is a process noise matrix, k-1 and k respectively indicate the (k-1)-th sampling timing and the k-th sampling timing, and k / k-1 indicates a one-step prediction from the (k-1)-th sampling timing to the k-th sampling timing; 【Number 26】 W is a process noise matrix, 【Number 27】 are respectively the noises of the three-axis gyroscope, 【Number 28】 is the noise of the three-axis accelerometer, 【No. 29】 is, 【30 numbers】 is a skew-symmetric matrix composed of, 【Number 31】 is the three-axis acceleration in the navigation coordinate system of the carrier; The process noise allocation matrix Γ is as follows, 【Number 32】 The measured quantity is as follows, 【Number 33】 【Number 34】 are respectively the three-axis components of the velocity in the navigation coordinate system of the athlete's limbs; 【Number 35】 are respectively the attitude angle data of the athlete's limbs at the previous sampling timing and the current sampling timing; The measurement equation is as follows 【Number 36】 ω ｉｅ is the angular velocity of the Earth's rotation, L is the Earth's latitude where the carrier is located, U is the measurement noise matrix, 【Number 37】 are respectively the three-axis velocity error noises, 【No. 38】 is the attitude angle error noise, θ, γ and ψ are respectively the pitch angle, roll angle and yaw angle of the athlete's limbs, and Δt is the sampling interval of the MEMS sensor. The human body movement intelligent measurement and digital training system according to claim 5, characterized in that.

12. The motion target detection and tracking module uses the undistort function in the computer vision library openv to perform distortion correction on the images captured by each camera, and the undistort function form is as follows, void undistort(InputArray src, OutputArray dst, InputArray cameraMatrix, InputArray distCoeffs, InputArray newCameraMatrix) src is the pixel matrix of the original image, and dst is the pixel matrix of the corrected image; cameraMatrix is the internal parameter of the camera; 【Number 39】 f ｘ = f / dx is the normalized focal length in the x-axis direction of the camera, and f ｙ = f / dy is the normalized focal length in the y-axis direction of the camera, with the unit of pixel. f is the focal length of the camera, and dx, dy are the physical sizes of the pixel in the x-axis and y-axis directions of the camera respectively. (u ０ , v ０ ) are the coordinates in the pixel coordinate system of the image center, with the unit of pixel. distCoeffs is the distortion parameter: 【Number 40】 k １ is the coefficient of the second-order term of the radial strain, and k ２ is the coefficient of the fourth-order term of the radial strain, k ３ is the coefficient of the sixth-order term of the radial strain, and p １ , p ２ are the first tangential strain parameter and the second tangential strain parameter respectively, and the human body motion intelligent measurement and digital training system according to claim 1, characterized in that InputArray newCameraMatrix is a zero matrix.

13. The calibration process of the camera internal parameter cameraMatrix and the distortion parameter distCoeffs is, Prepare one Zhangyou calibration method checkerboard as the calibration board, use the camera to capture the calibration board at different angles, and obtain a set of W checkerboard images, where 15 ≤ N ≤ 30, Using the Camera Calibration tool in the Matlab toolbox, read W checkerboard images, automatically detect the corner points on the checkerboard, obtain the coordinates of the corner points in the pixel coordinate system, input the actual size of the cells of the checkerboard into the calibration tool Camera Calibration, and the calibration tool Camera Calibration calculates the world coordinates of the corner points, The calibration tool Camera Calibration performs parameter calculation based on the coordinates of the corner points in the pixel coordinate system and the world coordinate system in W images to obtain the internal parameters IntrinsicMatrix and distortion parameters distCoeffs of the camera. The human body motion intelligent measurement and digital training system according to claim 12, characterized in that.

14. The motion target detection and tracking module calls the perspectiveTransform function in the computer vision library openCV to convert the coordinates of the athlete in the pixel coordinate system into the coordinates in the world coordinate system of the camera field of view coverage area. The human body motion intelligent measurement and digital training system according to claim 1.

15. The process of obtaining the perspective projection matrix is as follows: S2.1: Arrange and fix the cameras in the athlete's motion scene so that the entire field of view of M cameras covers the entire athlete's motion scene and adjacent camera screens have overlapping areas; S2.2: Define the field plane of the motion scene as the XOY plane of the global world coordinate system, arrange R rows and C columns of landmark points on the field plane, where the rows of the landmark points are parallel to the X-axis of the global world coordinate system, the columns of the landmark points are parallel to the Y-axis of the global world coordinate system, each landmark point is provided with a rhombus pattern, the connecting line of the opposite vertices of the rhombus pattern is parallel to the X-axis and Y-axis of the global world coordinate system, the center point position of the rhombus is used as the position of the landmark point, and there are a ２ landmark points included in each camera's field of view. The landmark points are uniformly distributed in the form of an a*a matrix, each landmark point located at the periphery is close to the edge of the camera's field of view, and the overlapping area of adjacent camera fields of view contains a common landmark points; S2.3: For each camera, select the top-left corner landmark point in the camera field of view as the origin, that is, select it as having coordinates (0, 0), establish the camera field of view area world coordinate system, measure the positions of each landmark point relative to the origin, and obtain the coordinates of 9 landmark points in the camera field of view area world coordinate system; S2.4: Photograph using a camera, and each camera acquires one image including a ２ number of a fiducial points; S2.5: Perform distortion correction on the images captured by the cameras; S2.6: Determine the coordinates of the a ２ marker points in the pixel coordinate system of the undistorted images captured by each camera; S2.7: For each camera, record the coordinates of each landmark point in the pixel coordinate system and the corresponding coordinates in the world coordinate system of the camera's field of view region as a set of coordinates, a ２ The human body motion intelligent measurement and digital training system according to claim 1, characterized in that a set of coordinates is introduced into the findHomography function in the computer vision library openCV to calculate the perspective projection matrix of the camera.

16. The coordinates of the ２ a number of landmark points in the pixel coordinate system of the image after distortion correction are determined. The specific method is as follows: Display the image after distortion correction in Matlab, display the position of the point pointed by the mouse in the image using the impixelinfo command, place the mouse at the center of the diamond marker, and a ２ Obtain the positions of the images of a markers, define the center of the upper left diamond marker in the image as the origin of the pixel coordinate system, record the coordinates as (0, 0), and for the other a ２ -1 non-origin marker points, record the opposing positions between the non-origin marker points and the origin as the coordinates of the non-origin marker points in the pixel coordinate system. The human body movement intelligent measurement and digital training system according to claim 15, characterized in that.

17. The motion target detection and tracking module obtains the accurate position and precise bounding box of each athlete in the pixel coordinate system by the following method. S3.1: Perform grayscale conversion and Gaussian filtering processing on the rough bounding box marking area of the athlete by YOLO detection; S3.2: Edge detection is performed on the approximate boundary box marking area of the athlete using the Canny-Devernay algorithm to obtain the exact contour of the athlete and a set of athlete contour point coordinates; S3.3: Calculate the characteristic moments of the contour based on the athlete contour point coordinates; S3.4: Use the characteristic moments of the contour to calculate the center of gravity of the athlete 【Number 41】 That is, calculate the exact position of the athlete in the pixel coordinate system of the athlete; S3.5: The human body motion intelligent measurement and digital training system according to claim 1, characterized in that the minimum circumscribed rectangle of the target contour is taken as the precise boundary box of the athlete.

18. The human body motion intelligent measurement and digital training system according to claim 1, characterized in that the motion target detection and tracking module uses the DeepSORT method to track the precise boundary box of each motion target at different times.

Images