A sports training system and method based on big data and deep learning

Abstract

The invention relates to the technical field of deep learning, and discloses a sports training system based on big data and deep learning, comprising: an image feature extraction module, which is used to generate a limb-dependent CNN model; a feature conversion module, which is used to vectorize the output of the limb-dependent CNN model; a feature input module, which sorts the vectorized results of the output of the convolution chain of the first convolution space corresponding to each limb according to time to generate sequence data; a state generation module, which inputs the sequence data corresponding to each limb into a GMM-HMM model, and outputs a hidden state; the invention comprehensively considers human body movements represented by general image features and human body movement features represented by infrared images to perform image decomposition on human limbs, then comprehensively introduces movement features to perform comprehensive convolution, and uses the comprehensive movement features obtained by convolution to express the force state based on the HMM model, and assists in visualization to achieve guidance on students' horizontal bar pulling.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and more specifically, to a sports training system based on big data and deep learning. Background Art

[0002] Pull-ups are an exercise that trains the upper limb strength of the human body. In the existing smart sports teaching technology, the number of pull-ups is only counted based on image processing, and the guidance of students' skills and force generation methods still depends on the guidance of teachers. Summary of the invention

[0003] The present invention provides a sports training system based on big data and deep learning, which solves the technical problem in the related art that the guidance of horizontal bar pull-up exercise can only rely on manual guidance of teachers.

[0004] The present invention provides a sports training system based on big data and deep learning, comprising:

[0005] A first image acquisition module, which is used to acquire a first image of a student doing a horizontal bar pull;

[0006] A second image acquisition module is used to acquire a second image of the student doing horizontal bar pulling;

[0007] The first image is a general image, and the second image is an infrared image;

[0008] A limb recognition module, which is used to recognize limbs from the first image and segment the limb area to obtain a first limb image of each limb;

[0009] A first limb segmentation module, which is used to map the limb area of ​​the first image to the corresponding second image, and then segment the second image to obtain a second limb image of each limb;

[0010] An image feature extraction module is used to generate a limb-dependent CNN model, the limb-dependent CNN model includes a first convolution space and a second convolution space, each convolution space includes a plurality of parallel convolution chains, each convolution chain includes a plurality of convolution layers; the convolution chain of the first convolution space inputs a first limb image of a limb; the convolution chain of the second convolution space inputs a second limb image of a limb

[0011] A feature conversion module, which is used to vectorize the output of the limb-dependent CNN model;

[0012] The feature input module sorts the vectorized output of the convolution chain of the first convolution space corresponding to each limb in time to generate sequence data;

[0013] The state generation module inputs the sequence data corresponding to each limb into the GMM-HMM model. The input sequence data is used to generate the observation state sequence. The GMM-HMM model outputs the hidden state sequence as L = {l1, l2…l n}, the state set of the hidden state is H = {h1, h2, h3}, and the state set of the hidden state contains three states, corresponding to excessive force, normal force and insufficient force respectively.

[0014] Furthermore, the formula for the kth convolutional layer of the i-th convolutional chain in the first convolutional space is:

[0015]

[0016] in, represents the convolution weight of the kth layer of the ith convolution chain, represents the bias parameter, * represents convolution, represents the input of the kth layer of the i-th convolution chain, and f represents the activation function;

[0017]

[0018] Where ⊙ represents the Hadamard product operation, σ represents the sigmoid activation function, represents the output of the kth layer of the ith convolution chain of the first convolution space, M represents the set of convolution chains of the second convolution space corresponding to the limb associated with the limb corresponding to the first limb image input by the ith convolution chain; Represents the output of the k-1th layer of the cth convolutional chain in the second convolutional space.

[0019] Furthermore, the parameters of each convolutional chain of the limb-dependent CNN model are trained separately.

[0020] Furthermore, for a convolution chain, its connected classifier is mapped to a classification set Q, which corresponds to the classification of limbs, and is trained using a training set to obtain the parameters of the convolution chain.

[0021] Furthermore, the vectorization method is to expand the output of the convolution chain of the first convolution space by flattening, and then reduce the dimension using the dimensionality reduction weight matrix.

[0022] Furthermore, the first limb image and the second limb image input into the limb-dependent CNN model at the same time are derived from the first image and the second motion image at the same time.

[0023] Furthermore, the parameters of the GMM-HMM model are γ = (π, A, C, μ, U), where π is the initial state matrix, A is the state transition probability matrix, C is the mixing weight matrix, u is the mean vector, and U is the covariance matrix.

[0024] Furthermore, it also includes a result visualization module, which is used to visualize the results output by the state generation module.

[0025] Furthermore, one method of visualization processing is: marking the hidden state of the limb corresponding to the limb in the first image at time t; the marking is through character marking or color marking.

[0026] A sports training method based on big data and deep learning, using the above-mentioned sports training system based on big data and deep learning to perform the following steps:

[0027] Step 101, collecting a first image and a second image of a student doing a horizontal bar pull;

[0028] Step 102, identifying limbs from the first image, and segmenting limb regions to obtain a first limb image of each limb;

[0029] Step 103, segmenting the second image to obtain a second limb image of each limb;

[0030] Step 104, inputting the first limb image and the second limb image into a limb-dependent CNN model;

[0031] Step 105, sorting the vectorized results of the output of the convolution chain of the first convolution space corresponding to each limb according to time to generate sequence data;

[0032] Step 106, input the sequence data into the GMM-HMM model, and then output the force state of each limb at the corresponding time.

[0033] The beneficial effects of the present invention are:

[0034] The present invention comprehensively considers human body movements represented by general image features and human body motion features represented by infrared images to decompose the image of human limbs, and then comprehensively introduces motion features for comprehensive convolution, and considers the motion involvement of adjacent limbs. The comprehensive motion features obtained by convolution are based on the HMM model to express the force state, and auxiliary visualization can realize the guidance of students' horizontal bar pulling exercises. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a module schematic diagram of a sports training system based on big data and deep learning of the present invention;

[0036] Figure 2 It is a flow chart of a sports training method based on big data and deep learning of the present invention.

[0037] In the figure: a first image acquisition module 101, a second image acquisition module 102, a limb recognition module 103, a first limb segmentation module 104, an image feature extraction module 105, a feature conversion module 106, a feature input module 107, a state generation module 108, and a result visualization module 109. DETAILED DESCRIPTION

[0038] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the contents of this specification. Each example may omit, replace or add various processes or components as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0039] Embodiment 1

[0040] like Figure 1 As shown, a sports training system based on big data and deep learning includes:

[0041] A first image acquisition module 101, which is used to acquire a first image of a student doing a horizontal bar pull;

[0042] The second image acquisition module 102 is used to acquire a second image of the student doing horizontal bar pulling;

[0043] The first image is a general image, and the second image is an infrared image;

[0044] A limb recognition module 103, which is used to recognize limbs from the first image and segment the limb area to obtain a first limb image of each limb;

[0045] A first limb segmentation module 104, which is used to map the limb area of ​​the first image to the corresponding second image, and then segment the second image to obtain a second limb image of each limb;

[0046] The image feature extraction module 105 is used to generate a limb-dependent CNN model, wherein the limb-dependent CNN model includes a first convolution space and a second convolution space, each convolution space includes a plurality of parallel convolution chains, each convolution chain includes a plurality of convolution layers; the convolution chain of the first convolution space inputs a first limb image of a limb; the convolution chain of the second convolution space inputs a second limb image of a limb

[0047] The formula for the kth convolutional layer of the ith convolutional chain in the first convolutional space is:

[0048]

[0049] in, represents the convolution weight of the kth layer of the ith convolution chain, represents the bias parameter, * represents convolution, represents the input of the kth layer of the i-th convolution chain, and f represents the activation function;

[0050]

[0051] Where ⊙ represents the Hadamard product operation, σ represents the sigmoid activation function, represents the output of the kth layer of the ith convolution chain of the first convolution space, M represents the set of convolution chains of the second convolution space corresponding to the limb associated with the limb corresponding to the first limb image input by the ith convolution chain; Represents the output of the k-1th layer of the cth convolution chain in the second convolution space;

[0052] For example, for the lower left torso, its associated limbs include the left thigh, and the corresponding convolution chain of the second convolution space is the 9th (9 is only an example, and other numbers can be used) convolution chain, so the convolution chain set of the corresponding second convolution space includes only one element, that is, the 9th convolution chain of the second convolution space;

[0053] In one embodiment of the present invention, the parameters of each convolution chain of the limb-dependent CNN model are trained separately;

[0054] For example, for a convolution chain, its connected classifier is mapped to a classification set Q, which corresponds to the classification of limbs. The parameters of the convolution chain are obtained by training it with a training set.

[0055] A feature conversion module 106 for vectorizing the output of the limb-dependent CNN model;

[0056] In one embodiment of the present invention, the vectorization method is: flattening the output of the convolution chain of the first convolution space, and then reducing the dimension using a dimensionality reduction weight matrix.

[0057] The feature input module 107 sorts the vectorized output of the convolution chain of the first convolution space corresponding to each limb according to time to generate sequence data;

[0058] The first limb image and the second limb image input to the limb-dependent CNN model at the same time are derived from the first image and the second motion image at the same time;

[0059] The state generation module 108 inputs the sequence data corresponding to each limb into the GMM-HMM model. The input sequence data is used to generate the observation state sequence. The GMM-HMM model outputs the hidden state sequence L = {l1, l2…ln}, the state set of the hidden state is H = {h1, h2, h3}, and the state set of the hidden state contains three states, corresponding to excessive force, normal force and insufficient force respectively;

[0060] The parameters of the GMM-HMM model are γ = (π, A, C, μ, U), where π is the initial state matrix, A is the state transition probability matrix, C is the mixing weight matrix, u is the mean vector, and U is the covariance matrix;

[0061] The parameter training of the GMM-HMM model is a conventional technical method and will not be described in detail here.

[0062] For example, a possible hidden state sequence is {h1, h1, h3…h2}, l1=h1 is abbreviated as h1;

[0063] The result visualization module 109 is used to visualize the result output by the state generation module 108 .

[0064] In one embodiment of the present invention, a method of visualization processing is:

[0065] Mark the hidden state of the limb corresponding to the first image at time t;

[0066] The marking can be through character marking or color marking.

[0067] like Figure 2 As shown, based on the above-mentioned sports training system based on big data and deep learning, the present invention provides a sports training method based on big data and deep learning, comprising the following steps:

[0068] Step 101, collecting a first image and a second image of a student doing a horizontal bar pull;

[0069] Step 102, identifying limbs from the first image, and segmenting limb regions to obtain a first limb image of each limb;

[0070] Step 103, segmenting the second image to obtain a second limb image of each limb;

[0071] Step 104, inputting the first limb image and the second limb image into a limb-dependent CNN model;

[0072] Step 105, sorting the vectorized results of the output of the convolution chain of the first convolution space corresponding to each limb according to time to generate sequence data;

[0073] Step 106, input the sequence data into the GMM-HMM model, and then output the force state of each limb at the corresponding time.

[0074] The above describes an embodiment of the present embodiment, but the present embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present embodiment, ordinary technicians in this field can also make many forms, all of which are within the protection of the present embodiment.

Claims

1. A sports training system based on big data and deep learning, characterized in that: include: A first image acquisition module, which is used to acquire a first image of a student doing a horizontal bar pull; A second image acquisition module, which is used to acquire a second image of the student doing horizontal bar pulling; The first image is a general image, and the second image is an infrared image; A limb recognition module, which is used to recognize limbs from the first image and segment the limb area to obtain a first limb image of each limb; A first limb segmentation module, which is used to map the limb area of ​​the first image to the corresponding second image, and then segment the second image to obtain a second limb image of each limb; An image feature extraction module is used to generate a limb-dependent CNN model, the limb-dependent CNN model includes a first convolution space and a second convolution space, each convolution space includes a plurality of parallel convolution chains, each convolution chain includes a plurality of convolution layers; the convolution chain of the first convolution space inputs a first limb image of a limb; the convolution chain of the second convolution space inputs a second limb image of a limb; A feature conversion module, which is used to vectorize the output of the limb-dependent CNN model; The feature input module sorts the vectorized output of the convolution chain of the first convolution space corresponding to each limb in time to generate sequence data; The state generation module inputs the sequence data corresponding to each limb into the GMM-HMM model. The input sequence data is used to generate the observation state sequence. The GMM-HMM model outputs the hidden state sequence as L = {l1, l2…l n }, the state set of the hidden state is H = {h1, h2, h3}, and the state set of the hidden state contains three states, corresponding to excessive force, normal force and insufficient force respectively.

2. A sports training system based on big data and deep learning according to claim 1, characterized in that: The formula for the kth convolutional layer of the ith convolutional chain in the first convolutional space is: in, represents the convolution weight of the kth layer of the ith convolution chain, represents the bias parameter, * represents convolution, represents the input of the kth layer of the i-th convolution chain, and f represents the activation function; Where ⊙ represents the Hadamard product operation, σ represents the sigmoid activation function, represents the output of the kth layer of the ith convolution chain of the first convolution space, M represents the set of convolution chains of the second convolution space corresponding to the limb associated with the limb corresponding to the first limb image input by the ith convolution chain; Represents the output of the k-1th layer of the cth convolutional chain in the second convolutional space.

3. A sports training system based on big data and deep learning according to claim 1, characterized in that: The parameters of each convolution chain of the limb-dependent CNN model are trained separately.

4. A sports training system based on big data and deep learning according to claim 1, characterized in that: For a convolution chain, its connected classifier is mapped to a classification set Q, which corresponds to the classification of limbs. The parameters of the convolution chain are obtained by training it with the training set.

5. A sports training system based on big data and deep learning according to claim 1, characterized in that: The vectorization method is to expand the output of the convolution chain in the first convolution space by flattening, and then use the dimensionality reduction weight matrix to reduce the dimension.

6. A sports training system based on big data and deep learning according to claim 1, characterized in that: The first limb image and the second limb image input to the limb-dependent CNN model at the same time are derived from the first image and the second motion image at the same time.

7. A sports training system based on big data and deep learning according to claim 1, characterized in that: The parameters of the GMM-HMM model are γ = (π, A, C, μ, U), where π is the initial state matrix, A is the state transition probability matrix, C is the mixing weight matrix, u is the mean vector, and U is the covariance matrix.

8. A sports training system based on big data and deep learning according to claim 1, characterized in that: It also includes a result visualization module, which is used to visualize the results output by the state generation module.

9. A sports training system based on big data and deep learning according to claim 8, characterized in that: One method of visualization processing is: marking the hidden state of the limb corresponding to the first image at time t; marking is through character marking or color marking.

10. A sports training method based on big data and deep learning, characterized in that: The sports training system based on big data and deep learning as described in any one of claims 1 to 9 is applied to perform the following steps: Step 101, collecting a first image and a second image of a student doing a horizontal bar pull; Step 102, identifying limbs from the first image, and segmenting limb regions to obtain a first limb image of each limb; Step 103, segmenting the second image to obtain a second limb image of each limb; Step 104, inputting the first limb image and the second limb image into a limb-dependent CNN model; Step 105, sorting the vectorized results of the output of the convolution chain of the first convolution space corresponding to each limb according to time to generate sequence data; Step 106, input the sequence data into the GMM-HMM model, and then output the force state of each limb at the corresponding time.

Images