A method and system for detecting boxing punches based on imu and calculating attribute values thereof

Abstract

This application discloses a method and system for detecting punch techniques and calculating attribute values ​​in boxing based on IMU (Instrument Detection Unit). The method first generates a set of candidate punch regions from the received raw IMU data; then, a neural network is used to identify the punch techniques within these candidate regions; if a punch technique is identified, an integral algorithm is further used to calculate the attribute value information of the punch region; finally, the punch technique and attribute value information are output. This method solves the problems of high difficulty in punch technique recognition, high difficulty in attribute value quantification, limited technology, and low accuracy in boxing evaluation.

Description

Technical Field

[0001] This application relates to the field of data processing and motion recognition of an IMU device, and in particular to a method and system for recognizing boxing punches and obtaining their attribute values. Background Technology

[0002] In the field of boxing motion recognition, optical capture is currently the main method used in the market. Optical capture can capture subtle movements, but its application is limited by its high price, inconvenience of carrying, complex operation, and susceptibility to color and light. As for obtaining boxing attribute value information, most instruments used at present are force measuring platforms, photoelectric gates, etc. These instruments are mostly laboratory equipment that can obtain accurate values, but they also have problems such as inconvenience of carrying and inability to calculate results in real time.

[0003] IMU devices can detect and measure acceleration and rotational motion in real time. Using these values ​​and the method described in this invention, the punch technique and attribute values ​​can be accurately calculated. Furthermore, IMU devices can be integrated into wearable devices and, through data interaction with software, enable real-time calculation and display of punch technique and attribute values ​​regardless of location or time, making them more convenient for athletes to wear and use. Summary of the Invention

[0004] This application proposes a method for detecting and calculating the attribute values ​​of boxing punches to address the difficulties in recording the attribute values ​​of boxing punches, recognizing complex movements, and identifying continuous movements. This method enables accurate identification of boxing movements and quantitative recording of boxing attribute values.

[0005] The technical solution adopted in this application is as follows:

[0006] A method for recognizing boxing movements includes the following steps:

[0007] Step 1: Use an IMU device fixed to the wrist to collect data on the punching motion, and perform data preprocessing to obtain the preprocessed punching candidate area;

[0008] Step 2: Using the changes in accelerometer, Euler angles, rotation matrix, and neural network to identify punch candidate area data, determine the punch technique. If the punch technique is identified as correct, the punch candidate area is used as the information area for the identification attribute value.

[0009] Step 3: If the punch candidate area identified in Step 2 is the correct punch technique, then use the integral algorithm to calculate the punch attribute information;

[0010] Step 4: Output the punching data and punching attribute information obtained in Step 2 and Step 3.

[0011] Furthermore, this application also relates to a system for calculating fist techniques and their attribute values, including:

[0012] The punching candidate area generation module is used to receive data input from the IMU device and generate a set of punching candidate areas.

[0013] The punch detection module is used to determine the changes in the accelerometer, Euler angles, and rotation matrix, and to use a neural network to identify the punch candidate area generated by the punch candidate area generation module. If the punch is correct, the punch candidate area information is used as the punch information.

[0014] The attribute value calculation module is used to calculate the punch attribute value information of the data in the punch candidate area when the punch recognition module identifies the punch.

[0015] The output module is used to output the punching technique and attribute value information obtained by the punching technique recognition module and the attribute value calculation module.

[0016] This application uses a neural network to identify punching techniques in candidate areas, achieving high accuracy. It also utilizes an integral algorithm to calculate punching attribute values, providing accurate quantification. The identification and calculation method of this application addresses the problems of high difficulty in punching technique identification, significant challenge in attribute value quantification, limited available technology, and low accuracy in existing technologies. Attached Figure Description

[0017] Figure 1 This is a flowchart of a method for detecting boxing techniques and calculating their attribute values, as described in this application;

[0018] Figure 2 This is a flowchart of a method for determining a punching candidate area in this application;

[0019] Figure 3 This is a schematic diagram of a set of punch data visualizations and alternative regions in this application;

[0020] Figure 4 This is a schematic diagram of the neural network structure used in a fist technique detection method in this application;

[0021] Figure 5 This is a flowchart illustrating the calculation order of punch attribute values ​​in this application;

[0022] Figure 6 This is a schematic diagram of the output result of a method for identifying punch techniques and punch attribute values ​​in this application;

[0023] Figure 7 This is an overall framework diagram of a punching technique recognition and punching attribute value recognition system in this application. Detailed Implementation

[0024] To make the above-mentioned objectives and advantages of this application more apparent and understandable, this application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] First, the examples and terms used in this application will be explained.

[0026] IMU (Inertial Measurement Unit) is a sensor primarily used to detect and measure acceleration and rotational motion. It includes accelerometers and gyroscopes (angular velocity meters). Accelerometers obtain acceleration along each axis, while gyroscopes obtain angular velocities along each axis, thus determining angle information. Some inertial measurement units also include magnetometers to obtain information about the surrounding magnetic field.

[0027] Combined with appendix Figure 1-6 This application provides a detailed explanation of a punching technique recognition method and its attribute value calculation. (See attached document.) Figure 1 As shown, the method specifically includes the following steps:

[0028] Step 1: In the received IMU data, a set of punch candidate regions is generated. The punch candidate region actually refers to the relevant IMU data information when the punching action to be identified is performed. In the subsequent processing, it is necessary to determine the punching method to be identified by judging the punch candidate region information.

[0029] Athletes typically perform continuous punching motions; therefore, collecting information on potential punching areas includes:

[0030] Determine the action range of the potential punching area;

[0031] Collect the fist techniques to be identified within the aforementioned action range;

[0032] Specifically, the punching action range refers to the time period from the starting posture to the ending posture, that is, the time period between the punching retraction posture and the punching completion posture.

[0033] Once the punching motion range is determined, a punching motion is considered complete, and punching motion information within the range can be collected by an IMU device worn on the wrist.

[0034] The IMU device in this application can acquire information from a three-axis accelerometer and a three-axis gyroscope during motion.

[0035] like Figure 2In practical applications, we use the raw IMU data to generate the action range of the punch candidate area, calculate whether the region acceleration meets the threshold and the changes in Euler angles and rotation matrix to identify the action posture, and use a neural network to identify whether it is a correct punch, and finally output the correct punch area.

[0036] Step 2: Use accelerometers, Euler angles, rotation matrices, and neural networks to identify whether the punch candidate area generated in Step 1 is a correct punch.

[0037] Step 21: First, the candidate punch region from Step 1 is passed to the preliminary candidate region determination function in Step 22. If the conditions are met, the data is input into the neural network in a specified format for punch determination. This step further includes:

[0038] Step 22: Calculate whether the value of the acceleration in the y-direction of the received region meets the threshold. At the same time, the attitude of the IMU's raw data needs to be solved by Euler angles and rotation matrix. If the condition is not met, return to receive the next candidate region. If the condition is met, the region is passed into the neural network for punch recognition.

[0039] Step 23: Construct a neural network. This neural network can be used to determine whether the punch candidate area in Step 22 represents a correct punch. See the appendix for the structure of this neural network. Figure 4 The network consists of one flattened layer, one dropout layer, and two fully connected layers.

[0040] The flattening layer receives raw inertial navigation data as input. The IMU transmits one data point at a time, containing nine data points in three directions from each of the two accelerometers and one gyroscope. From the candidate region, 30 data points are taken before and after the position corresponding to the maximum acceleration value in the y-direction, forming an array of shape (61, 9). The flattening layer outputs this set of data as a feature vector of length 549.

[0041] The input to a fully connected layer is the feature vector output by the flattened layer or the previous fully connected layer. The input feature vector is multiplied by the connection weights of the fully connected layer, and the output is a fixed-length feature vector or the final prediction result. The output lengths of the two fully connected layers are 64 and 3, respectively.

[0042] The network was trained using data calibrated to indicate the punching area of ​​the fist technique. The neural network constructed in step 23 was then trained using the backpropagation algorithm to obtain the network parameters. These parameters, including the connection weights of the fully connected layers, were obtained through the aforementioned training method.

[0043] The neural network obtained by training the network parameters in step 23 can be used to determine the punching technique in the input punching area.

[0044] The neural network in step 23 identifies whether the candidate punching area in step 22 is a correct punching action. If it is a correct punch, the information of that area is output and designated as the correct punching area.

[0045] like Figure 3 As shown, the IMU data is output in a continuous data format. Based on the filtering of the punching area, the original punching information for each interval can be obtained.

[0046] The specific method for generating the punching area is as follows:

[0047] First, the acceleration is determined based on the received candidate region to see if it meets the threshold. This is based on the rapid movement during a punch, which causes a significant change in acceleration direction. Then, the acceleration changes at the start and end of the punch approach zero to determine the action range of a single punch. (Based on the attached...) Figure 3 The visualization results clearly show the range of the punch.

[0048] In practical applications, the fist technique recognition model needs to be pre-trained, and training the fist technique model also requires corresponding model training data. Therefore, the model method also includes:

[0049] Use an IMU device to collect information on punching motions and punching motion types;

[0050] The collected punching information is processed into unified punching posture information;

[0051] Extract punching motion feature information from punching motion information and punching posture information;

[0052] A punching technique recognition model is trained based on punching action feature information and punching action type.

[0053] In the specific operation, the punching action information and punching action type are both known information, and the data can be labeled manually. For example, after a boxer wears an IMU device, he needs to perform the technical actions of straight punch, hook punch, and uppercut in sequence. At the same time, the IMU device collects the corresponding punching action information, and the punching action types are straight punch, hook punch, and uppercut in sequence.

[0054] After obtaining the punching action information, the punching action feature information and punching action type can be combined into a training sample set, which is used to train the punching action recognition model.

[0055] Specifically, training a punching technique recognition model based on the punching action feature information and punching action type includes:

[0056] The punching motion feature information is input into the punching technique recognition model;

[0057] Obtain the predicted punching action type output by the fist technique recognition model;

[0058] Based on the predicted punching action type and the punching action type, the model loss value is calculated using the backpropagation algorithm;

[0059] The model parameters of the fist technique recognition model are adjusted based on the model loss value, and the fist technique recognition model is trained again until the training conditions are met.

[0060] In the actual training of the punch recognition model, there will be multiple training sample sets. Each training sample set includes punch action feature information and the punch action type corresponding to the punch action information. Specifically, the punch action feature information is input into the punch recognition model to be trained for training, and the punch recognition model responds to the punch action feature information by outputting the punch action type.

[0061] During training, the predicted punch type is generated by an untrained punch recognition model, which differs from the actual punch type. Therefore, it is necessary to compare the predicted punch type with the actual punch type. Specifically, the model loss value is calculated by comparing the predicted punch type with the actual punch type. In this application, we use a multi-class loss function to calculate the loss value.

[0062] After calculating the loss value, the model loss value can be propagated to the punch recognition model through the backpropagation algorithm to adjust the parameters of each layer of the punch recognition model. At this point, the training of the current batch is completed. Next, the next batch of punch sample datasets is used to continue training the model until the model training stops.

[0063] In this application, the model training stopping conditions include five consecutive loss values ​​being less than a preset threshold or the model training rounds reaching a preset number of rounds.

[0064] The punch recognition model in this application is trained using a large number of punch action sample sets, making the punch recognition model more accurate, with smaller errors, reducing the recognition error rate, and improving the user experience.

[0065] Finally, based on the weight file generated by the punch recognition model, the parameters of each layer of the model are extracted, and the model is reproduced in C++. Finally, the reproduction code is implanted into the computing module of the IMU device, so that punch recognition of the raw punch information collected by the IMU can be completed in real time.

[0066] Step 3: If the punching area identified in Step 2 is a correct punch, then the punching attribute values ​​are further calculated using an integral algorithm. These attribute values ​​include punching speed, force, power, acceleration, and motion time. This step further includes:

[0067] Step 31: Receive the punching area data determined in Step 2.

[0068] Step 32 uses an integral algorithm to calculate the punch speed.

[0069]

[0070] The output V represents the speed of the punch, and t represents the time interval between two sampling points (x k y k z k ) represents the acceleration value at a certain moment in different directions output by the IMU device.

[0071] Step 33: Extract the triaxial acceleration corresponding to the moment when the accelerometer returns the maximum acceleration value within the fist area to calculate the total acceleration. Specifically:

[0072]

[0073] Where a X a Y a Z These are the values ​​of triaxial acceleration, respectively.

[0074] Step 34: Calculate the punching force using Newton's second law, specifically:

[0075] F = ma

[0076] Where m is the user's arm weight. If the arm weight is unknown, it is calculated from the user's body weight. The arm weight of a male is generally 0.057 of his body weight, and the arm weight of a female is generally 0.0497 of her body weight. a is the total acceleration.

[0077] Step 35: Calculate the punch action time by retrieving the index from the start of the punch to the maximum acceleration value. Specifically:

[0078] T=(t max -t start )

[0079] Where (t) max t start The values ​​are the time corresponding to the maximum acceleration and the start time of the punch, respectively.

[0080] Step 36: Calculate the power of the punch using dynamic formulas, specifically:

[0081] P = FV

[0082] Where F and V are the punching force and speed calculated earlier, respectively.

[0083] Step 37: After integrating the punching technique and attribute values, output the data. Specifically, this can be achieved through Bluetooth communication with mobile devices, sending the punching information to the backend, and displaying the information via a screen, touchscreen, or other output device. (Appendix) Figure 6 This is a diagram illustrating the punching technique and attribute values ​​corresponding to a single punch.

[0084] In a practical application, this application provides a system for detecting punching techniques and identifying their attribute values; the structure is shown in the appendix. Figure 7 Specifically, the system includes the following modules:

[0085] The punch candidate area generation module is used to generate a set of punch candidate areas from the raw data transmitted by the IMU.

[0086] like Figure 3 As shown, the IMU device outputs data in a continuous data format, and the received data may or may not include punches.

[0087] The punching technique recognition module is used to determine the punching technique in the candidate punching area.

[0088] Specifically, the punch technique recognition module performs two steps of judgment:

[0089] The first step is to determine whether it is a punching motion by measuring the changes in accelerometer, Euler angles, and rotation matrix;

[0090] The second step is to input the candidate region into the neural network in step 23 to determine whether it is a correct punch.

[0091] If both of the above two steps meet the conditions, proceed to the third step: the attribute value calculation module.

[0092] The attribute value calculation module calculates the attributes of a punch using an integral algorithm, Newton's second theorem, and dynamic formulas. These attributes include punch speed, force, power, acceleration, and action time.

[0093] After the attribute values ​​are calculated, the output module can communicate with mobile devices via Bluetooth to send the punch information to the backend and display the information through output devices such as displays and touch screens.

[0094] The above disclosure is only intended to help understand the method and core ideas of this disclosure; at the same time, those skilled in the art will know that there will be changes in the specific implementation methods and application scope based on the ideas of this disclosure. Therefore, the content of this specification should not be construed as a limitation of this disclosure.

Claims

1. A method for detecting boxing punches and calculating their attribute values ​​based on IMU, characterized in that, The method includes: Step 1: Generate a set of punch candidate areas from the received IMU input data; Step 2: Use the candidate region judgment function to determine whether the candidate region for punching meets the requirements for punching. If it does, use a neural network to identify whether the candidate region for punching is a correct punch. If so, use the candidate region information as a set of punching information and return it to the neural network to identify the punching method. Step 3: If the punch candidate area identified in Step 2 is the correct punch, then the punch attribute value information is calculated using an integral algorithm, where the attribute value is calculated using the following formula: The outputs (V, a, F, P, T) represent boxing-related attribute values, where V is the punching speed, a is the total acceleration, F is the punching force, P is the punching power, T is the time to complete the punching motion, and t is the interval between two sampling points. k y k z k ) represents the acceleration value at a specific moment in different directions output by the IMU device, a X a Y a Z These represent the triaxial acceleration values ​​at the time corresponding to the maximum acceleration value, where m is the user's arm weight. If the arm weight is unknown, it is calculated from body weight. For men, arm weight typically accounts for 0.057% of body weight, and for women, it typically accounts for 0.0497%. (t) max t start These represent the time corresponding to the maximum acceleration value and the starting time of the punch, respectively. Step 4: Output the punching technique and punching attribute values ​​obtained in Steps 2 and 3; The punching options in step 1 include: If the acceleration value returned by the IMU exceeds the threshold, the point corresponding to the maximum acceleration value within the interval is selected. The starting time of the punch candidate area is the second zero point before the maximum value point, and the ending time is the first zero point after the maximum value point.

2. The method as described in claim 1, characterized in that, Step 2 further includes: Step 21: Receive the initial punch selection area; Step 22: Calculate whether the value of the acceleration in the y-direction of the received region meets the threshold. At the same time, the attitude of the IMU’s raw data needs to be solved by Euler angles and rotation matrix. If the condition is not met, return to receive the next candidate region. If the condition is met, the region is passed into the neural network for punch recognition. Step 23: Construct a neural network; Step 24: Use some training data with calibrated punch candidate areas to train the network and obtain network parameters; Step 25: Using the network parameters obtained in Step 24, a neural network is obtained to determine the punching method in the candidate punching area in Step 22, and the punching method is identified using the neural network.

3. The method as described in claim 2, characterized in that, The neural network in step 23 includes: It consists of one flattening layer, one dropout layer, and two fully connected layers. The flattening layer flattens the punch candidate region data into one-dimensional data, the dropout layer discards some data during training to improve the robustness of the neural network, and the fully connected layers output a fixed-length feature vector or result.

4. A system for detecting boxing punches and identifying their attribute values ​​based on IMU, characterized in that, The system includes: The punching technique candidate area generation module is used to receive data input from the IMU device and generate a set of punching candidate areas; The punch detection module uses Euler angles and rotation matrices to determine whether the punch candidate area meets the punching action posture. If it does, a neural network is used to identify what punch candidate area generated by the punch candidate area generation module is. If the punch is correct, the punch candidate area information is used as punch information. The index calculation module is used to calculate the punch attribute value information of the data in the punch candidate area when the punch detection module identifies the punch. The output module is used to output the punching technique and punching attribute values ​​obtained by the punching technique detection module and the index calculation module. The method of generating regions in the fist technique candidate region generation module is as follows: If the acceleration value returned by the IMU has been detected to exceed the threshold, the point corresponding to the maximum acceleration value within the interval is taken. The starting time of the punch candidate area is the second zero point before the maximum value point, and the ending time is the first zero point after the maximum value point. The formula for calculating the attribute values ​​of the candidate regions is: The outputs (V, a, F, P, T) represent boxing-related attribute values, where V is the punching speed, a is the total acceleration, F is the punching force, P is the punching power, T is the time to complete the punching motion, and t is the interval between two sampling points. k y k z k ) represents the acceleration value at a specific moment in different directions output by the IMU device, a X a Y a Z These represent the triaxial acceleration values ​​at the time corresponding to the maximum acceleration value, where m is the user's arm weight. If the arm weight is unknown, it is calculated from body weight. For men, arm weight typically accounts for 0.057% of body weight, and for women, it typically accounts for 0.0497%. (t) max t start The values ​​are the time corresponding to the maximum acceleration and the start time of the punch, respectively.

5. The system as described in claim 4, characterized in that, The punching technique detection module further includes: The neural network building block is used to build a neural network. The neural network training module is used to train the network using data that calibrates punching techniques, and to obtain network parameters. The punch technique judgment module is used to obtain a neural network through the network parameters trained by the neural network training module. This network is used to determine what punch technique is used in the punch candidate area generated by the punch candidate area generation module, and the neural network is used to identify punch technique information.

6. The system as described in claim 5, characterized in that, The neural network constructed by the neural network building module consists of one flattening layer, one dropout layer, and two fully connected layers connected in sequence. The flattening layer is used to flatten the punch candidate region data into one-dimensional data. The dropout layer is used to discard some data during training to improve the robustness of the neural network. The fully connected layers are used to output a fixed-length feature vector or result.

Images