A physical training data feature extraction method based on BERT
Through the BERT-based physical training data feature extraction method, using feature engineering and Transformer encoding, the problem of subjectivity of physical training data feature extraction in the prior art is solved, and efficient data analysis and training target prediction are achieved.
Patent Information
- Application Number
- CN202210628842.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-06
- Publication Date
- 2024-12-10
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the prior art, the fuzzy classification method and AHP method are difficult to achieve scientific and effective extraction of physical fitness training data features due to subjective factors, which affects the data analysis effect.
The BERT-based physical training data feature extraction method is used to cluster through feature engineering, word2vec pre-trained physical training project embedding, improve MLM for mask training, integrate age, gender, physiological state and psychological state distribution, and use 12-layer Transformer Coding is carried out to achieve effective feature extraction of physical training data.
The characteristic dimensions of physical training data are reduced, the data analysis efficiency and effect are improved, and scientific prediction of training goals is achieved.
Smart Images
Figure CN115098623B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a method for extracting physical training data features based on BERT. Background Art
[0002] With the continuous development of economic level and national quality, the number of people doing physical exercise is increasing, and the data of various physical training is also increasing. When a large amount of physical training data is classified and further analyzed, only by extracting features from the physical training data and mapping the high-dimensional original data to a low-dimensional manifold can the physical training data become separable.
[0003] The journal article "Application of Data Mining Technology in Physical Training of Fire Forces [J]. Fire Technology and Product Information, 2017 (08): 74-77." uses fuzzy classification to achieve system classification and provide a basis for classification training. The AHP method is used to combine the training results and standards of trainees in different projects and decompose and calculate them one by one.
[0004] However, the fuzzy classification method and AHP method adopted in the technical solution of the above paper use the subjective factors of experts as the basis for judging key features. The subjective factors have too much influence, making it difficult to realize implicit knowledge mining and unable to achieve scientific and effective feature extraction, which will affect the effect of data analysis. Summary of the invention
[0005] The embodiment of the present invention provides a method for extracting features from physical training data based on BERT. Through feature engineering, corresponding key features are extracted according to the differences in time-series physical training data, the feature dimension of training data is reduced, and the difficulty of data analysis for physical training is reduced, the efficiency and effectiveness of data analysis are improved, and finally the prediction of the training target effect is achieved.
[0006] The embodiment of the present invention provides a method for extracting features from physical training data based on BERT. The method uses 80% of the daily training data of people doing physical training as a training set of the model, 10% as a validation set, and 10% as a test set, and performs K-fold cross validation. The method includes:
[0007] In the BERT part, the embedding of the physical training project pre-trained by word2vec is used to cluster the training; MLM is improved to change the original form of directly masking the entire training information including attributes to only masking part of the training information;
[0008] The integration part specifically integrates the age, gender, physiological state and psychological state distribution of the physical training population that meets the preset conditions into the network;
[0009] In the output part, the age, gender, physiological state, and psychological state distribution of the trainees are classified and predicted, and the probability of user categories that can achieve effective training by using different training methods is output;
[0010] The model is trained until convergence and then the training is terminated, wherein the BERT part of the model is the training feature extractor.
[0011] In some embodiments of the present invention, the step of integrating the age, gender, physiological state and psychological state distribution of the physical training group meeting the preset conditions into the network includes:
[0012] Divide the group of people who meet the preset conditions and are engaged in physical training into multiple groups;
[0013] If the trainees undergo multiple trainings and come from the population in the first part, the remaining ones are used as the seed population, and the distribution of age, gender, physiological state, and psychological state for each training is calculated and fused using a 12-layer Transformer.
[0014] In some embodiments of the present invention, the group of people who meet the preset conditions and carry out physical training are divided into multiple groups, specifically:
[0015] The people who meet the preset conditions and carry out physical training are divided into 64 groups.
[0016] In some embodiments of the present invention, the fusion is performed using a 12-layer Transformer, specifically:
[0017] The BERT model containing 12 layers of Transformer is used as the encoder for encoding. Each layer of Transformer takes the latent variables of the previous layer as input and outputs latent variables with higher information interaction.
[0018] In some embodiments of the present invention, satisfying the preset condition is satisfying a preset training duration and performing training at a preset frequency.
[0019] In some embodiments of the present invention, the BERT-based physical training data feature extraction method further includes:
[0020] When using the improved MLM prediction training, the most recent specific project name ID and its attributes can be used for prediction so that the embeddings of different attributes are aligned in the semantic space.
[0021] The BERT-based physical training data feature extraction method provided in the embodiment of the present invention has the following advantages: after receiving the physical training data input, it performs physical training feature extraction through the improved BERT method. Assuming that each training is a word, the training sequence of the trained user is regarded as a sentence, that is, the original problem is converted into a text classification problem in the field of NLP (natural language processing). With the help of the mature embedding method in this field, the training target effect prediction problem is converted into a classification problem based on the type of stable trainees, thereby realizing physical training data feature extraction and training effect prediction. A physical training feature extraction method based on BERT is established, and with the help of BERT's mature and effective language feature extraction capabilities, effective extraction of physical training data features is achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 Flow chart of a method for extracting features from physical training data based on BERT according to an embodiment of the present invention;
[0023] Figure 2 It is a framework diagram of a physical training data feature extraction method based on BERT according to an embodiment of the present invention. DETAILED DESCRIPTION
[0024] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods.
[0025] The phrases "in one embodiment", "in another embodiment", "in yet another embodiment", "in an embodiment", "in some embodiments" or "in other embodiments" may be used in this specification to refer to one or more of the same or different embodiments according to the present invention.
[0026] Specific embodiments of the present invention are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments invented are merely embodiments of the present invention, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to determine the true intent based on the user's historical operations and to avoid unnecessary or redundant details that obscure the present invention. Therefore, the specific structural and functional details of the present invention are not intended to be limiting, but only serve as the basis and representative basis for the claims to teach those skilled in the art to use the present invention in a variety of substantially any appropriate detailed structures.
[0027] The embodiment of the present invention provides a method for extracting features from physical training data based on BERT. Figure 1 and Figure 2As shown, the method uses 80% of the daily training data of people doing physical training as the training set of the model, 10% as the validation set, and 10% as the test set, and performs K-fold cross validation. The method includes:
[0028] The BERT (Bidirectional Encoder Representation From Transformer) part uses the embedding of the word2vec pre-trained physical training project to cluster the training; improves the MLM, changing the original direct mask of the entire training information including attributes to only masking part of the training information;
[0029] The integration part specifically integrates the age, gender, physiological state and psychological state distribution of the physical training population that meets the preset conditions into the network;
[0030] In the output part, the age, gender, physiological state, and psychological state distribution of the trainees are classified and predicted, and the probability of user categories that can achieve effective training by using different training methods is output;
[0031] The model is trained until convergence and then the training is terminated, wherein the BERT part of the model is the training feature extractor.
[0032] In some embodiments of the present invention, the step of integrating the age, gender, physiological state and psychological state distribution of the physical training group meeting the preset conditions into the network includes:
[0033] Divide the group of people who meet the preset conditions and are engaged in physical training into multiple groups;
[0034] If the trainees undergo multiple trainings and come from the population in the first part, the remaining ones are used as the seed population, and the distribution of age, gender, physiological state, and psychological state for each training is calculated and fused using a 12-layer Transformer.
[0035] In some embodiments of the present invention, the group of people who meet the preset conditions and carry out physical training are divided into multiple groups, specifically:
[0036] The people who meet the preset conditions and carry out physical training are divided into 64 groups.
[0037] In some embodiments of the present invention, the fusion is performed using a 12-layer Transformer, specifically:
[0038] The BERT model containing 12 layers of Transformer is used as the encoder for encoding. Each layer of Transformer takes the latent variables of the previous layer as input and outputs latent variables with higher information interaction.
[0039] In some embodiments of the present invention, satisfying the preset condition is satisfying a preset training duration and performing training at a preset frequency.
[0040] In some embodiments of the present invention, the BERT-based physical training data feature extraction method further includes:
[0041] When using the improved MLM prediction training, the most recent specific project name ID and its attributes can be used for prediction so that the embeddings of different attributes are aligned in the semantic space.
[0042] Through the above technical solutions, it can be known that the physical training data feature extraction method based on BERT provided in the above embodiment of the present invention has the following advantages: after receiving the physical training data input, it performs physical training feature extraction through the improved BERT method. Assuming that each training is a word, the training sequence of the trained user is regarded as a sentence, that is, the original problem is converted into a text classification problem in the field of NLP (natural language processing). With the help of mature embedding methods in this field, the training target effect prediction problem is converted into a classification problem based on the type of stable trainees, thereby realizing physical training data feature extraction and training effect prediction. A physical training feature extraction method based on BERT is established, and with the help of BERT's mature and effective language feature extraction capabilities, effective extraction of physical training data features is achieved.
[0043] To facilitate the detailed description of the above technical solution, a specific example is given below to provide a detailed description. 80% of the daily training data of people who regularly perform physical training are selected as the training set of the model, 10% as the validation set, and 10% as the test set, and K-fold cross-validation is performed. Figure 1 This is a model introduction diagram of the technical solution of the present invention. Figure 1 The physical training feature extraction method of the present invention includes the following parts:
[0044] (1) BERT part: Assuming that each training item is a word and the training sequence of the trained user is regarded as a sentence, the original problem can be converted into a text classification problem in the field of NLP (Natural Language Processing). The BERT model containing 12 layers of Transformers is used as the encoder for encoding. Each layer of Transformer takes the latent variables of the previous layer as input and finally outputs latent variables with higher information interaction. The following two-stage pre-training method is used to train BERT: First, use word2vec to pre-train the embedding of physical training items and cluster similar training; second, improve MLM, change the original direct masking of the entire training information including attributes to only masking part of the training information (training items or attributes). When using the improved MLM (masked language model) for predictive training, not only can the recent specific project name ID be used, but also its attributes can be used for prediction, so that the embeddings of different attributes can be aligned in the semantic space.
[0045] (2) Fusion part: In this part, the age, gender, physiological state, and psychological state distribution of people who have been engaged in long-term and stable physical training are integrated into the network. The specific method is to divide the population into 64 parts. Assuming that the trainees undergo n training and come from the first population, the remaining population can be used as the seed population, and the age, gender, physiological state, and psychological state distribution of each training can be calculated, and then fused using a 12-layer Transformer.
[0046] (3) Output part: Classify and predict the age, gender, physiological state, and psychological state distribution of the trainees, and output the probability of user categories that can be effectively trained using different training methods.
[0047] The training ends after the model is trained until convergence. The BERT part of the model is the training feature extractor.
[0048] Although the content of the present invention has been described in detail through the above preferred embodiments, it should be appreciated that the above description should not be considered as a limitation of the present invention. After reading the above content, it will be apparent to those skilled in the art that various modifications and substitutions of the present invention will occur. Therefore, the protection scope of the present invention should be limited by the appended claims.
Claims
1. A method for extracting features from physical training data based on BERT, characterized in that: The method uses 80% of the daily training data of people who perform physical training as a training set of the model, 10% as a validation set, and 10% as a test set, and performs K-fold cross validation. The method includes: The BERT part specifically uses word2vec to pre-train the embedding of physical training items and cluster similar training. The MLM is improved by changing the original form of directly masking the entire training information including attributes to only masking part of the training information, where the part of the information for mask training is the training items and attributes. The integration part specifically integrates the age, gender, physiological state and psychological state distribution of the physical training population that meets the preset conditions into the network; In the output part, the age, gender, physiological state, and psychological state distribution of the trainees are classified and predicted, and the probability of user categories that can achieve effective training by using different training methods is output; The model is trained until convergence and then the training is terminated, wherein the BERT part of the model is the training feature extractor; The age, gender, physiological state and psychological state distribution of the physical training population meeting the preset conditions are integrated into the network, including: Divide the group of people who meet the preset conditions and are engaged in physical training into multiple groups; If the trainee undergoes multiple trainings and comes from the first part of the population, the remaining ones are used as seed populations, and the distribution of age, gender, physiological state, and psychological state for each training is calculated and fused using a 12-layer Transformer; The physical training population meeting the preset conditions is divided into multiple groups, specifically: The physical training population that meets the preset conditions is divided into 64 groups; The 12-layer Transformer is used for fusion, specifically: The BERT model with 12 layers of Transformers is used as the encoder for encoding. Each layer of Transformer takes the latent variables of the previous layer as input and outputs latent variables with higher information interaction. The preset condition is to meet the preset training duration and perform training at a preset frequency; When using the improved MLM prediction training, the most recent specific project name ID and its attributes can be used for prediction so that the embeddings of different attributes are aligned in the semantic space.