Sports movement counting and rule violation judging method based on human posture recognition

Abstract

The application discloses a sports movement counting and rule-breaking action judging method based on human body posture recognition, and comprises the following steps: step A, human body recognition; step B, personnel screening; step C, repeated action detection based on human body key point detection technology; and step D, rule-breaking action judgment. The application uses human body key point detection technology of deep learning, calculates the relative position and relative angle of each limb part according to the position information, thereby judging what action the test personnel is doing, whether the action is in place, and whether there is a rule-breaking situation. Finally, in combination with time domain information, the preceding and subsequent multiple frames of actions are associated, and whether the human body accurately completes each standard action is judged. When each action is sequentially and normally completed, the system automatically counts. The application automatically adapts to different human body features, and compared with infrared radio frequency technology, the counting accuracy is high.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a method for sports activity counting and violation action identification based on human posture recognition. Background Technology

[0002] In sports activities (such as rope skipping, sit-ups, and pull-ups), traditional manual counting is characterized by high workload and serious human error.

[0003] To address the shortcomings of manual counting, existing technologies use infrared counting sensors, such as the invention patent entitled "Combination Device of Automatic Long Rope Counting Rope Swinger and High Jump Stand" (application publication number: CN101757767A). This requires on-site installation and equipment adjustment based on different human heights, weights, and other characteristics, and the counting accuracy is not high. Summary of the Invention

[0004] The purpose of this invention is to solve the aforementioned technical problems in the prior art and to provide a method for sports activity counting and violation action identification based on human posture recognition, thereby improving the counting accuracy.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A method for sports activity counting and violation detection based on human posture recognition is characterized by the following steps:

[0007] Step A, Human Recognition: Acquire real-time video data from the camera and use deep learning-based object detection technology to detect people in the video;

[0008] Step B, Personnel Screening: Calculate the testing area, eliminate irrelevant personnel, and retain athletes;

[0009] Step C: Detect repetitive movements based on human key point detection technology:

[0010] 1) Data collection and annotation: Collect image or video datasets containing repetitive actions and annotate the start and end key points of each action in order to identify a complete action cycle;

[0011] 2) Model training: Use a convolutional neural network to train the prepared dataset;

[0012] 3) Keypoint detection: Using a trained human keypoint detection model, the location of keypoints is predicted in each frame of the dataset;

[0013] 4) Motion cycle recognition: Based on the positional changes of key points, the start and end points of the motion are identified;

[0014] 5) Action counting: Calculate the number of actions based on the identified action cycles;

[0015] Step D, Determination of Violations:

[0016] 1) Feature extraction: Extracting features from the keypoint sequence;

[0017] 2) Establish rules: Develop corresponding rules based on the characteristics of the violations;

[0018] 3) Model training and evaluation: Use deep learning methods to train a classification model to judge illegal actions, then train it using a labeled dataset, and evaluate it using test data;

[0019] 4) Illegal Action Judgment: Based on the extracted features and the trained classification model, the actions in the image or video are judged. If a situation is detected that does not match the violation rules or model prediction, it is judged as an illegal action.

[0020] 5) Feedback and warnings: For detected violations, provide warnings, prompts, and logs.

[0021] Furthermore, in step B, the test area can be obtained in two ways:

[0022] Method 1: Designate a test area on-site; Method 2: Utilize deep learning-based target detection technology to automatically identify the test equipment and automatically generate a test area around the equipment.

[0023] Furthermore, the deep learning model employs a convolutional neural network (CNN), which includes convolutional layers, pooling layers, fully connected layers, activation functions, and batch normalization layers. The convolutional layer uses a set of filters for learning to slide across the input image, performing local feature extraction. Each filter performs a convolution operation on a region of the input image, generating a feature map. The pooling layer reduces the spatial size of the feature maps, lowering computational complexity and making the network translation-invariant. The fully connected layer connects the feature maps extracted by the convolutional and pooling layers to the output layer for classification or regression tasks. The activation function, following the convolutional and fully connected layers, introduces non-linear properties, enabling the network to learn complex feature maps. The batch normalization layer normalizes the input of each feature channel to zero mean and unit variance, reducing the vanishing gradient problem, accelerating the training process, and improving model stability.

[0024] Furthermore, in step D, a lightweight classification model based on a convolutional neural network is selected, and the operation steps are as follows:

[0025] S1. Import software libraries and data: First, import the software libraries, load the image dataset, preprocess the data, and divide the data into training and test sets.

[0026] S2. Load the pre-trained model: Use a classification model that has been pre-trained on a large-scale image dataset as the base model.

[0027] S3. Modify the model architecture: The pre-trained model includes output layers for classifying a large number of different categories. To adapt the structure of the output layers to the specific classification problem, the last fully connected layer needs to be replaced with a new fully connected layer that matches the number of classification categories.

[0028] S4. Freeze some layers: In order to speed up training and prevent the loss of feature extraction capability of the classification model, freeze the convolutional layers so that they remain unchanged during training.

[0029] S5. Training the model: The modified model is trained using the training set. During the training process, the model's weights are updated through backpropagation so that it can learn from the training data.

[0030] S6. Evaluate the model: Use a test set to evaluate the model's performance, typically by calculating classification accuracy, confusion matrix, and other evaluation metrics.

[0031] S7. Fine-tuning: When the model's performance is not good enough in actual operation, it is necessary to fine-tune the model's parameters or try different data augmentation strategies.

[0032] S8. Predicting New Data: After training, use the model to classify new unlabeled images.

[0033] Furthermore, in step C, 1), regular human movements and sports movements are collected, as well as video movements from publicly available internet channels.

[0034] Furthermore, in step C, 2), the training process adjusts the network parameters by minimizing the error between the predicted keypoint location and the labeled location, so that it can accurately predict the keypoint.

[0035] Furthermore, in step C, 4), this is achieved by determining whether the position of a specific key point has passed a threshold.

[0036] Furthermore, in step D, 1), the distance, angle, and velocity characteristics between key points are used to describe changes in human movement.

[0037] The present invention, by adopting the above-described technical solution, has the following beneficial effects:

[0038] This invention utilizes deep learning-based human keypoint detection technology to detect the positional information of key points on the human body, such as eyes, ears, nose, mouth, shoulders, hands, elbows, hips, knees, and feet. Based on this positional information, the relative positions and angles of each limb are calculated to determine what action the test subject is performing, whether the action is performed correctly, and whether there are any violations. Finally, by combining temporal information, multiple frames of action are correlated to determine whether the human body has accurately completed each standardized action. When each action is completed correctly in sequence, the system automatically counts. This allows the system to record the total number of repetitive movements performed by the human body over a period of time.

[0039] The entire process has the following characteristics:

[0040] 1. By adopting a modular approach, developers can quickly build according to design requirements, improving development efficiency, shortening the development cycle, and reducing development costs.

[0041] 2. Asynchronous parallel processing is adopted to make full use of computing resources, reduce deployment costs, improve detection efficiency, and shorten response time.

[0042] This invention automatically adapts to different human body characteristics, and compared with infrared radio frequency technology, this invention has a higher counting accuracy. Attached Figure Description

[0043] The present invention will be further described below with reference to the accompanying drawings:

[0044] Figure 1 This is a schematic diagram of the key point detection method of the present invention;

[0045] Figure 2 This is a comparison chart of the accuracy of the present invention and infrared radio frequency technology. Detailed Implementation

[0046] like Figure 1 and Figure 2 As shown, the present invention provides a method for sports activity counting and violation action identification based on human posture recognition, which includes the following steps:

[0047] Step A, Human Recognition: Acquire real-time video data from the camera and use deep learning-based object detection technology to detect people in the video.

[0048] Step B, Personnel Screening: Calculate the test area, eliminate irrelevant personnel, and retain athletes. There are two ways to obtain the test area: (1) Designate the test area on-site. (2) Use deep learning target detection technology to automatically identify the test equipment and automatically generate the test area around the equipment.

[0049] Step C: Detect repetitive movements based on human key point detection technology:

[0050] 1) Data Collection and Annotation: We collected image or video datasets containing repetitive movements and annotated the start and end points of each movement to identify a complete movement cycle. To improve the detection performance of this technology, enabling it to adapt to different human anatomy features and detect extreme movements in special situations, in addition to regular human and sports movements, we also collected video movements from publicly available internet sources, including yoga, fitness, gymnastics, and dance.

[0051] 2) Model Training: A convolutional neural network is used to train the prepared dataset. The training process adjusts the network parameters by minimizing the error between the predicted keypoint locations and the labeled locations, so that it can accurately predict keypoints.

[0052] 3) Key point detection: Using a trained human key point detection model, the location of key points is predicted in each frame of the dataset.

[0053] 4) Movement cycle recognition: Identify the start and end points of a movement based on changes in the position of key points. This can be achieved by determining whether the position of a specific key point has passed a threshold; for example, raising the arm to a certain height marks the start of the movement, and lowering the arm to the same height marks the end of the movement.

[0054] 5) Action Counting: Calculate the number of actions based on the identified action cycles. A count is performed each time a complete action cycle is detected.

[0055] In summary, human keypoint detection technology can monitor the start and end points of movements, thereby identifying the cycle of repetitive movements and enabling counting. This method can be applied to scenarios requiring the counting of repetitive movements, such as fitness, sports training, and production lines.

[0056] Step D: Determining Violations. In the routine action recognition process, it is necessary to identify and eliminate violations. Determining violations involves analyzing human posture, action characteristics, and predefined rules. The specific implementation steps are as follows:

[0057] 1) Feature extraction: Extracting features from the key point sequence, such as the distance, angle, and speed between key points, to describe changes in human movement.

[0058] 2) Establish rules: Develop corresponding rules based on the characteristics of the violation. For example, some violations may involve improper body posture, abnormal hand or foot positioning, etc.

[0059] 3) Model Training and Evaluation: Relying solely on rule-based judgments is insufficient for accuracy. Therefore, a deep learning method is used to train a classification model to identify violations. The model is trained using a labeled dataset and evaluated using test data.

[0060] 4) Illegal Action Judgment: Based on the extracted features and the trained model, actions in images or videos are judged. If an action is detected that does not match the violation rules or model predictions, it is judged as a violation.

[0061] 5) Feedback and warnings: For detected violations, provide warnings, prompts, and logs.

[0062] Convolutional neural networks (CNNs) are deep learning models widely used in computer vision tasks such as image classification, object detection, and image segmentation. Their design is inspired by biological visual systems, particularly the way the human visual cortex processes visual information.

[0063] The following are the main features and components of convolutional neural networks:

[0064] Convolutional Layers: Convolutional operations are the core of convolutional neural networks. They use a set of learnable filters (also called kernels) to slide across the input image, performing local feature extraction. Each filter performs a convolution operation on a small region of the input image, generating a feature map. This helps capture local features in the image, such as edges and textures.

[0065] Pooling layers: Pooling operations are used to reduce the spatial size of feature maps, decrease computational complexity, and make the network translation invariant. Common pooling operations include max pooling and average pooling, which respectively take the maximum or average value of a local region to reduce the size of the feature map.

[0066] Activation functions: Following convolutional and fully connected layers, activation functions introduce non-linear properties, enabling the network to learn complex feature maps. Commonly used activation functions include ReLU (Rectified Linear Unit), Sigmoid, and Tanh.

[0067] Fully connected layer: The fully connected layer is used to connect the feature maps extracted by the convolutional layer and pooling layer to the output layer for classification or regression tasks.

[0068] Batch normalization layer: Batch normalization is a regularization technique that helps accelerate the training process and improve model stability. It normalizes the input of each feature channel to zero mean and unit variance, reducing the gradient vanishing problem.

[0069] Convolutional Neural Network (CNN) architectures typically employ a stack of multiple convolutional and pooling layers to progressively extract abstract and high-level features. Common CNN architectures include LeNet, AlexNet, VGG, GoogLeNet, and ResNet, which perform exceptionally well on various tasks.

[0070] Convolutional neural networks have achieved great success in image processing and are also being applied in other fields, such as natural language processing and speech recognition. Their success is partly due to the locality of convolution and pooling operations, and the ability of deep networks to automatically learn feature representations.

[0071] The aforementioned classification model is a machine learning or deep learning model used to categorize input data into different classes or labels. This invention selects a lightweight convolutional neural network-based classification model, specifically designed for operation in resource-constrained environments, such as mobile devices and embedded systems. The following are the operating steps:

[0072] S1. Import software libraries and data: First, import the necessary software libraries, load the image dataset, preprocess the data, and divide the data into training and test sets.

[0073] S2. Loading a pre-trained model: Use a classification model that has already been pre-trained on a large-scale image dataset as the base model. This can be achieved through the functions provided in deep learning frameworks.

[0074] S3. Modify the model architecture: Pre-trained models typically include output layers for classifying a large number of different categories. The structure of these output layers needs to be modified to suit the specific classification problem. This requires replacing the last fully connected layer with a new fully connected layer that matches the number of classification categories.

[0075] S4. Freeze some layers: In order to speed up training and prevent the loss of feature extraction capabilities of the classification model, some convolutional layers are frozen so that they remain unchanged during training.

[0076] S5. Training the Model: Train the modified model using the training set. During training, the model's weights are updated through backpropagation, enabling it to learn from the training data.

[0077] S6. Evaluate the model: Use a test set to evaluate the model's performance, typically by calculating classification accuracy, confusion matrix, and other evaluation metrics.

[0078] S7. Fine-tuning: When the model's performance is not good enough in actual operation, it is necessary to fine-tune the model's parameters or try different data augmentation strategies.

[0079] S8. Predicting New Data: After training, use the model to classify new unlabeled images.

[0080] This invention utilizes deep learning-based human keypoint detection technology to detect the positional information of key points on the human body, such as eyes, ears, nose, mouth, shoulders, hands, elbows, hips, knees, and feet. Based on this positional information, the relative positions and angles of each limb are calculated to determine what action the test subject is performing, whether the action is performed correctly, and whether there are any violations. Finally, by combining temporal information, multiple frames of action are correlated to determine whether the human body has accurately completed each standardized action. When each action is completed correctly in sequence, the system automatically counts. This allows the system to record the total number of repetitive movements performed by the human body over a period of time.

[0081] The entire process has the following characteristics:

[0082] 1. By adopting a modular approach, developers can quickly build according to design requirements, improving development efficiency, shortening the development cycle, and reducing development costs.

[0083] 2. Asynchronous parallel processing is adopted to make full use of computing resources, reduce deployment costs, improve detection efficiency, and shorten response time.

[0084] like Figure 2 As shown, the horizontal axis represents the accuracy rate, and the vertical axis represents the number of tests that meet the accuracy rate. The counting accuracy rate of infrared radio frequency is above 98%, while the counting accuracy rate of this invention is above 99%. The counting accuracy rate of this invention is superior to that of infrared radio frequency counting. This invention has undergone high-intensity testing in real-world scenarios with 50,000 participants, achieving an accuracy rate of 99.7%.

[0085] This invention automatically adapts to different human body characteristics, and compared with infrared radio frequency technology, this invention has a higher counting accuracy.

[0086] The above are merely specific embodiments of the present invention, but the technical features of the present invention are not limited thereto. Any simple changes, equivalent substitutions, or modifications made based on the present invention to solve essentially the same technical problems and achieve essentially the same technical effects are all covered within the protection scope of the present invention.

Claims

1. A method for sports activity counting and violation detection based on human posture recognition, characterized in that, Includes the following steps: Step A, Human Recognition: Acquire real-time video data from the camera and use deep learning-based object detection technology to detect people in the video; Step B, Personnel Screening: Calculate the testing area, eliminate irrelevant personnel, and retain athletes; Step C: Detect repetitive movements based on human key point detection technology: 1) Data collection and annotation: Collect image or video datasets containing repetitive movements and annotate the start and end key points of each movement in order to identify a complete movement cycle. The dataset includes regular human movements, sports movements, and video movements collected from publicly available Internet channels. 2) Model training: The prepared dataset is trained using a deep learning model. The deep learning model adopts a convolutional neural network, which includes convolutional layers, pooling layers, fully connected layers, activation functions, and batch normalization layers. The convolutional layer uses a set of filters for learning to slide across the input image to perform local feature extraction. Each filter performs a convolution operation on a region of the input image to generate a feature map. The pooling layer is used to reduce the spatial size of the feature mapping, reduce computational complexity, and make the network translation invariant. The fully connected layer is used to connect the feature maps extracted by the convolutional and pooling layers to the output layer for classification or regression tasks. The activation function is placed after the convolutional and fully connected layers. The activation function introduces non-linear properties, enabling the network to learn complex feature maps. The batch normalization layer normalizes the input of each feature channel to zero mean and unit variance, reducing the gradient vanishing problem, which helps to accelerate the training process and improve the stability of the model. The training process adjusts the network parameters by minimizing the error between the predicted keypoint location and the labeled location, so that it can accurately predict keypoints. 3) Keypoint detection: Using a trained human keypoint detection model, the location of keypoints is predicted in each frame of the dataset; 4) Action cycle recognition: Based on the positional changes of key points, the start and end points of the action are identified. This recognition is achieved by determining whether the position of a specific key point has passed a threshold. 5) Action counting: Calculate the number of actions based on the identified action cycles; Step D, Determination of Violations: 1) Feature extraction: Extract features from the keypoint sequence, using the distance, angle and velocity features between keypoints to describe changes in human movement; 2) Establish rules: Develop corresponding rules based on the characteristics of the violations; 3) Model Training and Evaluation: A classification model is trained using deep learning methods to identify violations. This classification model is a machine learning model used to categorize input data into different classes or labels. It is trained using a labeled dataset and evaluated using test data. A lightweight convolutional neural network-based classification model is selected. The operation steps are as follows: S1. Import software libraries and data: First, import the software libraries, load the image dataset, preprocess the data, and divide the data into training and testing sets. S2. Load the pre-trained model: Use a classification model that has been pre-trained on a large-scale image dataset as the base model; S3. Modify the model architecture: The pre-trained model includes output layers for classifying a large number of different categories. The structure of the output layers needs to be modified according to the specific classification problem. The last fully connected layer needs to be replaced with a new fully connected layer that matches the number of classification categories. S4. Freeze some layers: In order to speed up training and prevent the loss of feature extraction ability of the classification model, freeze the convolutional layers so that they remain unchanged during training. S5. Training the model: The modified model is trained using the training set. During the training process, the model's weights are updated through backpropagation so that it can learn from the training data. S6. Evaluate the model: Use the test set to evaluate the model's performance; S7. Fine-tuning: When the model's performance is not good enough in actual operation, it is necessary to fine-tune the model's parameters or try different data augmentation strategies. S8. Predicting New Data: After training, use the model to classify new unlabeled images; 4) Illegal Action Judgment: Based on the extracted features and the trained classification model, the actions in the image or video are judged. If a situation is detected that does not match the violation rules or model prediction, it is judged as an illegal action. 5) Feedback and warnings: For detected violations, provide warnings, prompts, and logs.

2. The method for sports activity counting and violation action identification based on human posture recognition according to claim 1, characterized in that: In step B, the test area can be obtained in two ways: Method 1: Designate a testing area on-site; Method 2: Use deep learning-based target detection technology to automatically identify the test equipment and automatically generate a test area around the equipment.

Images