Motion type identification and model training method and system, wearable device and medium

Abstract

The invention relates to a motion type identification and model training method and system, wearable equipment and a medium. The training method comprises the following steps: acquiring a training sample set and a corresponding label set comprising motion types; wherein each training sample comprises multi-axis acceleration data in a preset sampling duration; inputting the multi-axis acceleration data into a to-be-trained motion type recognition model, extracting corresponding motion features, and obtaining a corresponding predicted motion type according to the motion features; calculating label loss of the predicted motion type and the corresponding real motion type based on the predicted motion type and the corresponding real motion type; based on model parameters of a full connection layer of the motion type identification model, sparse loss is calculated; calculating center loss based on the motion features; and obtaining total loss based on the label loss, the sparse loss and the center loss, and updating parameters of the motion type recognition model according to the total loss to obtain a trained motion type recognition model. According to the invention, the operation cost of the algorithm is reduced, and the corresponding motion category can be identified more accurately.

Description

Technical Field

[0001] The present invention relates to the field of signal processing technology, and in particular to a method, system, wearable device and medium for motion type recognition and model training. Background Art

[0002] With the popularity of wearable devices, their applications in sports health monitoring are becoming more and more extensive. Sports type recognition is a core function of wearable devices. By accurately identifying the user's sports type, it can provide users with more accurate sports analysis and health advice.

[0003] Existing motion type recognition is generally solved by sensor data feature extraction and machine learning. The solution mainly includes: first, collect three-axis / six-axis / nine-axis sensor data, and perform data preprocessing, such as normalization, bandpass filtering, etc., and then extract features based on experience, such as mean, variance, etc., and then select some highly distinguishing features from the extracted features for classification to identify the type of motion. The type of extracted features needs to be determined based on experience. At the same time, if new types of motion need to be identified, new features may need to be added, or old features may need to be removed, which will require significant changes to the algorithm and require higher costs. In addition, the machine learning classifier may not be consistent in distinguishing different types of motion, and the type of classifier used also needs to be determined based on experience. Therefore, it is necessary to provide a method, system, wearable device and medium for motion type recognition and model training. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a method, system, wearable device and medium for motion type recognition and model training, which improves the problems of high cost and poor accuracy of related algorithms for motion type recognition in the prior art.

[0005] To achieve the above-mentioned purpose and other related purposes, the present invention provides a method for training a motion type recognition model, comprising: obtaining a training sample set and a corresponding label set including motion types; wherein each training sample includes multi-axis acceleration data within a preset sampling duration; inputting the multi-axis acceleration data into the motion type recognition model to be trained, extracting the corresponding motion features, and obtaining the corresponding predicted motion type based on the motion features; based on the predicted motion type and the corresponding true motion type, calculating the label loss of both; based on the model parameters of the fully connected layer of the motion type recognition model, calculating the sparse loss; based on the motion features, calculating the center loss; obtaining the total loss based on the label loss, the sparse loss and the center loss, and updating the parameters of the motion type recognition model according to the total loss to obtain a trained motion type recognition model.

[0006] In one embodiment of the present invention, the obtaining of the training sample set includes: obtaining an initial training sample set; for each initial training sample of the initial training sample set: if the length of the initial training sample is equal to a preset length threshold, then it remains unchanged; if the length of the initial training sample is not equal to the length threshold, then the initial training sample is resampled at least once according to the preset sampling length, and the resampled multi-axis acceleration data is used as the initial training sample, so that the sample length of the initial training sample reaches the length threshold; all initial training samples with unified sample lengths are merged to obtain a final training sample set.

[0007] In one embodiment of the present invention, the center loss Among them, N batch is the number of training samples in the current training batch, f k is the motion feature of the kth training sample, For sports type y k The center vector of .

[0008] In one embodiment of the present invention, the motion type recognition model is a lightweight model.

[0009] In one embodiment of the present invention, the multi-axis acceleration data is input into a motion type recognition model to be trained, corresponding motion features are extracted, and corresponding predicted motion types are obtained according to the motion features, including: inputting the multi-axis acceleration data into a feature extraction module group of the motion type recognition model, performing feature processing through each feature extraction module to generate motion features, and using the motion features generated by the last feature extraction module as the final motion features; the feature extraction module group includes a plurality of cascaded feature extraction modules, each feature extraction module includes a cascaded depthwise separable convolutional layer, a normalization layer, an activation layer, and a pooling layer; the motion features are input into a fully connected layer of the motion type recognition model to obtain the corresponding predicted motion type.

[0010] In one embodiment of the present invention, the multi-axis acceleration data is input into a feature extraction module group of a motion type recognition model, and feature processing is performed through each feature extraction module to generate motion features, including: inputting the multi-axis acceleration data into the first feature extraction module of the feature extraction module group, performing time series convolution on the acceleration of each axis respectively to obtain the motion features of each axis, and fusing the motion features of all axes to obtain the initial motion features; for each remaining feature extraction module: inputting the initial motion features / the motion features of the previous feature extraction module into the current feature extraction module, performing time series convolution on each channel of the input features independently to obtain the motion features of each channel, and fusing the motion features of all channels to generate the corresponding motion features.

[0011] In one embodiment of the present invention, a method for identifying a type of motion is also provided, the method comprising: periodically sampling and caching multi-axis acceleration data within a preset first time period, and calculating the average mean square error of the multi-axis acceleration data based on the acceleration of each axis; determining the activity state according to the average mean square error, and continuing to sample and cache the multi-axis acceleration data when the activity state is motion, until the cumulative sampling time of the cached multi-axis acceleration data is equal to a preset second time period; inputting the cached multi-axis acceleration data into a motion type identification model to obtain a predicted motion type.

[0012] In one embodiment of the present invention, the cached multi-axis acceleration data is input into a motion type recognition model to obtain a predicted motion type, including: downsampling the cached multi-axis acceleration data to reduce the cumulative sampling time of the multi-axis acceleration data to a preset third time; inputting the downsampled multi-axis acceleration data into the motion type recognition model to obtain a predicted motion type.

[0013] In one embodiment of the present invention, after inputting the cached multi-axis acceleration data into the motion type recognition model to obtain the predicted motion type, it also includes: continuing to obtain multi-axis acceleration data within a preset fourth time period, and inputting the acceleration within the fourth time period into the motion type recognition model to obtain the corresponding predicted motion type; determining whether the two motion types are consistent, and generating and sending a reminder message if they are consistent.

[0014] In one embodiment of the present invention, a motion type recognition model training system is also provided, the system comprising: a sample acquisition module, used to acquire a training sample set and a corresponding label set including motion types; wherein each training sample comprises multi-axis acceleration data within a preset sampling duration; a motion type prediction module, used to input the multi-axis acceleration data into the motion type recognition model to be trained, extract the corresponding motion features, and obtain the corresponding predicted motion type according to the motion features; a label loss calculation module, used to calculate the label loss of the predicted motion type and the corresponding true motion type based on the two; a sparse loss calculation module, used to calculate the sparse loss based on the model parameters of the fully connected layer of the motion type recognition model; a center loss calculation module, used to calculate the center loss based on the motion features; a parameter update module, used to obtain the total loss based on the label loss, the sparse loss and the center loss, and update the parameters of the motion type recognition model according to the total loss to obtain a trained motion type recognition model.

[0015] In one embodiment of the present invention, a wearable device is also provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the wearable device implements any of the above-described methods for identifying the type of exercise.

[0016] In one embodiment of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes any of the above-mentioned motion type identification methods or motion type identification model training methods.

[0017] As described above, a method, system, wearable device and medium for motion type recognition and model training of the present invention have the following beneficial effects: a training sample set is constructed using multi-axis acceleration data and corresponding real motion types. By calculating the label loss between the predicted motion type and the real motion type, so that the model has a higher accuracy in the classification task, the sparse loss is calculated based on the model parameters of the fully connected layer, and the weight of the fully connected layer is constrained, thereby improving the generalization ability of the model. The center loss calculated according to the motion features further enhances the model's ability to discriminate multiple categories by reducing the distance between sample features of the same category. The present invention adopts an end-to-end approach. When the trained motion type recognition model needs to expand new motion types, it only needs to increase the number of output neurons in the last layer and update the relevant parameters to achieve expansion, which greatly simplifies the upgrade and maintenance process of the model. Not only can the computational cost of the algorithm be reduced, but the corresponding motion category can also be identified more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic flow chart of a method for training a sports category recognition model provided by an embodiment of the present invention;

[0019] Figure 2 A schematic diagram of a flow chart of a method for identifying a type of motion provided by an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a process for sending a three-axis acceleration data reminder message provided by an embodiment of the present invention;

[0021] Figure 4 Shown is a structural block diagram of a sports type recognition model training system provided by an embodiment of the present invention;

[0022] Figure 5 Shown is a structural schematic diagram of a wearable device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0024] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0025] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0026] The inventors found that in order to ensure long battery life, wearable devices can only detect some physiological characteristics (heart rate) and record some exercise information (number of steps, distance) by default. More detailed information, such as the current heart rate range, exercise duration, stride, pace, etc., will only be detected when the special sports APP is turned on. However, there is a situation where users forget to open the special sports APP during exercise, resulting in the loss of exercise information, making the device experience poor. In addition, existing wearable devices have certain limitations in the recognition of sports categories. For example, the type of extracted features needs to be determined based on experience. At the same time, if new sports categories need to be identified, new features may need to be added, or old features may need to be removed, which requires a large change to the algorithm and requires a higher cost. In addition, the machine learning classifier may not be consistent in distinguishing different types of sports, and the type of classifier used also needs to be determined based on experience. Therefore, an improved method is urgently needed to improve user experience and algorithm efficiency.

[0027] The present invention provides a method for training a motion type recognition model. Through an end-to-end lightweight network and by using the average mean square deviation of multi-axis acceleration, accurate recognition of various motion types (such as walking, running, elliptical machine, rowing machine, etc.) can be achieved, while also having strong scalability and low power consumption characteristics. The present invention uses the average mean square deviation to preliminarily screen the multi-axis acceleration data, reducing the number of algorithm runs in the stationary state and effectively reducing power consumption. By downsampling the data with a second duration T2 to a third duration T3 (T3 < T2), the input dimension of the network is reduced, and the consumption of memory and computing power is decreased. In addition, during the training phase, the diversity of the input data is enriched through data augmentation methods, enabling the model to better adapt to different motion patterns. The generalization ability of the model is improved by effectively constraining the weights of the fully connected layer through sparse loss. The center loss further enhances the discriminative ability of the model for multiple categories by reducing the distance between the features of samples of the same category. When new motion types need to be extended, only the number of output neurons in the last layer needs to be increased and the relevant parameters need to be updated to achieve the extension, greatly simplifying the model upgrade and maintenance process.

[0028] Please refer to Figure 1 , the method for training a motion type recognition model includes the following steps:

[0029] S11. Obtain a training sample set and a corresponding label set including motion types; wherein, each training sample includes multi-axis acceleration data within a preset sampling duration.

[0030] When a user is exercising, within a preset sampling duration, continuously record the multi-axis acceleration data generated by the sensors built in the wearable device to form a training sample, and use the corresponding real motion type (such as running, swimming, etc.) as a label. The wearable device includes but is not limited to smart watches, sports bracelets, smart glasses, etc. It should be noted that the multi-axis acceleration described in the present invention includes but is not limited to three-axis acceleration, six-axis acceleration, and nine-axis acceleration. For the convenience of description, the following content takes three-axis acceleration as an example for elaboration, but it should be understood that other types such as six-axis acceleration and nine-axis acceleration are also within the protection scope of the present invention.

[0031] In an embodiment of the present invention, the obtaining of the training sample set includes:

[0032] Obtain an initial training sample set;

[0033] For each initial training sample in the initial training sample set:

[0034] If the length of the initial training sample is equal to the preset length threshold, keep it unchanged;

[0035] If the length of the initial training sample is not equal to the length threshold, resampling the initial training sample at least once according to a preset sampling length, and using the resampled multi-axis acceleration data as the initial training sample, so that the sample length of the initial training sample reaches the length threshold;

[0036] All initial training samples with unified sample lengths are merged to obtain the final training sample set.

[0037] When constructing a training sample, multi-axis acceleration data is continuously collected from the sensor built into the wearable device at a preset sampling frequency until the preset duration is reached, and the collection sequence within the preset duration is used as an initial training sample. Therefore, for each initial training sample, it is a multi-axis acceleration data sequence arranged in time sequence. For each initial training sample, check whether its length is the same as the preset length threshold T3. If the length is consistent, the initial training sample can be directly retained. If the length is different, the sample can be adjusted by resampling. Specifically, according to the length threshold, a corresponding segment extraction range (such as 0.5*T3 to 1.8*T3) is preset. Based on the preset segment extraction range, a segment of acceleration data is randomly extracted from the initial training sample, and the extracted segment is compared with the length threshold to perform resampling in different ways: if the segment length is greater than the length threshold, the segment length is shortened to the length threshold by downsampling; if the segment length is less than the length threshold, the segment length is expanded to the length threshold based on the interpolation method. After the above operation, the resampled segment is used as a training sample. This random interception and resampling greatly enriches the diversity of data, allowing originally shorter samples to be appropriately lengthened, and longer samples to extract different time segments, thereby presenting different movement rates, rhythm changes, and timing characteristics. Through this resampling strategy, the model can fully contact and learn the movement characteristics of different rhythms such as too fast and too slow during the training phase, thereby enhancing the model's robustness and adaptability to different movement rhythms.

[0038] S12, inputting the multi-axis acceleration data into a motion type recognition model to be trained, extracting corresponding motion features, and obtaining a corresponding predicted motion type according to the motion features.

[0039] The collected multi-axis acceleration data is input into the motion type recognition model to be trained, and the model will extract features that can characterize different types of motion. According to the extracted motion features, the predicted probabilities of various types of motion are obtained through the fully connected layer and the Softmax output layer of the model, thereby generating the corresponding predicted motion types. The motion type recognition model can be any model that can accurately identify the type of motion, and the network structure of the model includes but is not limited to convolutional networks, long short-term memory units, gated recurrent units, Transformer structures, lightweight models, etc. Preferably, considering that wearable devices usually rely on battery power, in order to reduce power consumption and extend the continuous use time of wearable devices, in one embodiment of the present invention, the motion type recognition model is a lightweight model. Of course, it can be understood that those skilled in the art can also adaptably select models of other network structures based on other purposes, which are not limited here.

[0040] In one embodiment of the present invention, the step of inputting the multi-axis acceleration data into a motion type recognition model to be trained, extracting corresponding motion features, and obtaining the corresponding predicted motion type according to the motion features includes:

[0041] Inputting the multi-axis acceleration data into a feature extraction module group of a motion type recognition model, performing feature processing through each feature extraction module to generate motion features, and taking the motion features generated by the last feature extraction module as the final motion features; the feature extraction module group includes a plurality of cascaded feature extraction modules, each feature extraction module includes a cascaded depth-separable convolution layer, a normalization layer, an activation layer and a pooling layer;

[0042] The motion features are input into the fully connected layer of the motion type recognition model to obtain the corresponding predicted motion type.

[0043] When the model is a lightweight model, the motion type recognition model includes multiple cascaded feature extraction modules and fully connected layers. Each feature extraction module includes a depth-separable convolutional layer, a normalization layer, an activation layer, and a pooling layer connected in series. For each training sample: After the multi-axis acceleration data passes through the feature extraction module group, the motion features are gradually extracted, and the motion features of the last feature extraction module are input into the fully connected layer. The motion features are linearly mapped, and the predicted probabilities of the motion features corresponding to various motion categories are generated through the Softmax activation function. The largest one is selected from these predicted probabilities as the predicted motion type corresponding to the training sample.

[0044] In one embodiment of the present invention, the step of inputting the multi-axis acceleration data into a feature extraction module group of a motion type recognition model, performing feature processing through each feature extraction module, and generating motion features includes:

[0045] Input the multi-axis acceleration data into the first feature extraction module of the feature extraction module group, perform time series convolution on the acceleration of each axis respectively, obtain the motion feature of each axis, and fuse the motion features of all axes to obtain the initial motion feature;

[0046] For each remaining feature extraction module: input the initial motion feature / the motion feature of the previous feature extraction module into the current feature extraction module, perform temporal convolution on each channel of the input feature independently to obtain the motion feature of each channel, and fuse the motion features of all channels to generate the corresponding motion feature.

[0047] For the first feature extraction module, the following operations are performed: the X-axis acceleration, Y-axis acceleration, and Z-axis acceleration are respectively input into the corresponding deep convolutional layers, and multiple convolution kernels are used to slide and extract features on the time dimension of the X-axis acceleration to obtain the first number channel motion features of the X-axis. The Y and Z axes are operated in the same way, which will not be described in detail here, and the second number channel motion features and the third number channel motion features are obtained accordingly. After the motion features of each axis are extracted, the extracted motion features of each axis are merged into several channels and feature fusion is performed to obtain the fused initial motion features. The fused initial motion features are input into the normalization layer to ensure that the feature distribution is stable, and the normalized initial motion features are obtained. The normalized initial motion features are input into the activation layer, and nonlinear activation is performed through the activation function (such as ReLU) to obtain the activated initial motion features. The activated initial motion features are input into the pooling layer, and the key features are highlighted by downsampling to obtain the final initial motion features and output.

[0048] For each of the remaining feature extraction modules, the following operations are performed: its input is the output feature of the previous module, and the number of channels of the deep convolutional network layer in the feature extraction module is consistent with the number of channels of the output feature of the previous module. By independently convolving each channel, the temporal motion features of the channel are extracted, and the motion features of each channel are fused so that the model can learn the association between different feature channels. The fused motion features are normalized, nonlinearly activated, and pooled in turn to obtain the motion features output by the module.

[0049] S13. Based on the predicted motion category and the corresponding true motion category, calculate the label loss of both.

[0050] In each iterative training, the multi-axis acceleration data of the current training batch is input into the model to obtain the corresponding predicted motion type. According to the predicted motion type and the actual motion type corresponding to the acceleration, the label loss L of the two is calculated as shown in formula (1): CE :

[0051]

[0052] Among them, N batch is the number of training samples in the current training batch, and They represent the true motion type and the corresponding predicted motion type of the i-th sample respectively. It can be understood that the calculation method of label loss includes but is not limited to cross entropy loss, mean square error loss or KL divergence, etc., but in order to more accurately measure the difference in probability distribution so that the model can obtain a more robust effect, this implementation uses cross entropy loss as label loss.

[0053] S14. Calculate the sparse loss based on the model parameters of the fully connected layer of the motion type recognition model.

[0054] In order to make the model pay more attention to more important parameters, filter out parameters that contribute less to the model prediction results, reduce the risk of overfitting, and improve the prediction speed and accuracy of the model, the present invention also calculates the sparse loss by averaging the L1 norm of the weights of the fully connected layer, as shown in formula (2):

[0055]

[0056] Among them, L Sparse is the sparse loss, N T is the number of weight parameters in the fully connected layer, W j is the jth weight parameter in the fully connected layer, and ||·||1 represents L1 regularization. By calculating the sparse loss, the values ​​of some weight parameters with lower importance in the fully connected layer can be made smaller or even zero, so that the entire model has a sparse structure, which can not only reduce the model's dependence on specific weights, but also reduce the model's size and computational overhead, and improve generalization ability and overall adaptability.

[0057] S15. Calculate the center loss based on the motion features.

[0058] In order to make the motion features of the same category more densely clustered in the feature space and improve the prediction accuracy of the model, the present invention also implements the constraint on the feature distribution by calculating the center loss. Specifically, for each training batch: the motion feature extracted from each training sample in the training batch is compared with the center vector corresponding to its true category, and the center loss is calculated according to formula (3):

[0059]

[0060] Among them, L Center is the center loss, N batch is the number of training samples in the current training batch, f k is the motion feature of the kth training sample, is the true motion type y of the kth training sample k Corresponding center vector. By minimizing the Euclidean distance between the motion feature and the center vector, the model can achieve accurate classification of inter-class features during training while also improving the ability to judge intra-class features. The center vector in the present invention changes dynamically with the iterative process of the model. At the beginning of training, in order to obtain a more accurate feature set center, each center vector is first initialized. The center vector of each class of samples can be randomly initialized by noise that conforms to the Gaussian distribution to ensure that the initial center vector is evenly distributed in the corresponding feature space, avoiding the problem of unreasonable distribution of the center vector due to arbitrary random initialization, thereby causing inaccurate model recognition. With the continuous iteration of training, the center vector of each category needs to be updated after each round of training to match the changes in the feature distribution of the current batch of samples. Specifically, it can be updated by formula (4) (5):

[0061]

[0062] in, is the average offset between the center vector of motion category j and the motion features corresponding to the training samples belonging to this category in the current training batch during the e-th round of training, N batch is the number of training samples in the current training batch, y i is the true motion category label of the i-th training sample, θ(y i =j) is an indicator function used to determine whether the i-th training sample belongs to category j. i =j is 1, otherwise it is 0. is the center vector of motion category j in the current e-th round of training, f i is the motion feature of the i-th training sample (i.e., the feature input to the fully connected layer), ρ is the learning rate of the preset central loss function, and its value is between 0 and 1. is the center vector of motion category j in the e+1th round of training. The core idea of ​​the center loss function is to use the distance between the feature and the center of the feature set of the same category as the loss, so that the spacing between the features extracted by the network from the same category samples after training is smaller.

[0063] Specifically, for the e-th round of training, after the e-th round of backpropagation is completed, the center vector of the j-th motion category The updating process is as follows: count the number of samples belonging to the jth motion category from the training batch, and accumulate all motion features f of this category in the current training batch i With the current center vector The average distance between And calculate the center vector of the next round, i.e. the e+1th round, by formula (4): Through continuous iterative updates, the center vector can gradually approach the true mean position of the current category in the feature space.

[0064] S16. Obtain a total loss based on the label loss, the sparse loss and the center loss, and update the parameters of the motion type recognition model according to the total loss to obtain a trained motion type recognition model.

[0065] In the present invention, the total loss is obtained by weighted summing of label loss, sparse loss and center loss, as shown in formula (6):

[0066] L=L CE +α1*L Sparse +α2*L Center (6)

[0067] Wherein, L is the total loss, α1 and α2 are preset weighting coefficients, which are used to balance the orders of magnitude of different loss functions, wherein the order of magnitude of the center loss is smaller than the order of magnitude of the label loss. It should be noted that the present invention prioritizes ensuring that the model correctly classifies the types of motion, and on this basis, clustering of intra-class features is performed, so that the label loss plays a dominant role and the center loss plays an auxiliary role, so that the model can more accurately predict the motion category. During each iterative training, after calculating the total loss, back propagation is performed through the gradient descent method and the model parameters are updated according to the gradient. Repeat the iteration many times until the total loss is reduced below the preset threshold to obtain a trained motion type recognition model.

[0068] See also Figure 2 In another embodiment of the present invention, a method for identifying a type of motion is further provided, the method comprising:

[0069] S21, periodically sampling and caching the multi-axis acceleration data within a preset first time period, and calculating the mean square error of the multi-axis acceleration data based on the acceleration of each axis;

[0070] S22, determining the activity state according to the average mean square error, and continuing to sample and cache the multi-axis acceleration data when the activity state is motion, until the cumulative sampling time length of the cached multi-axis acceleration data is equal to a preset second time length;

[0071] S23, inputting the cached multi-axis acceleration data into a motion type recognition model to obtain a predicted motion type.

[0072] After the user wears the wearable device, in order to monitor the user's movement in real time, a first time length (such as unit time T0) can be preset. During the first time length, multi-axis acceleration data is continuously collected and cached according to the preset sampling frequency, and the mean value of the acceleration of each axis is used as the acceleration of the corresponding axis during the first time length. The average mean square error of the multi-axis acceleration during the first time length is calculated according to formula (7):

[0073]

[0074] Among them, std(.) is the standard deviation, acc x ,acc y and acc z They are the acceleration data of X, Y and Z axes respectively, f std It can be understood that for six-axis or nine-axis acceleration, the corresponding average mean square error. The average mean square error can reflect the change of acceleration in the first time period, and the average mean square error f std Compared with the preset error threshold T1, if the mean square error f std If the error is less than the error threshold T1, it means that the user is in a relatively static state. In the next time period T0, we can continue to collect new multi-axis acceleration data and periodically determine whether the user starts to move. std If the error is greater than or equal to the error threshold T1, it means that the user has entered the motion state, and the data needs to be sampled until the total sampling time reaches the second time T2, so as to obtain enough data for subsequent model analysis. These data are input into the previously trained motion type recognition model to predict the user's current motion type.

[0075] In one embodiment of the present invention, the step of inputting the cached multi-axis acceleration data into a motion type recognition model to obtain a predicted motion type includes:

[0076] Downsampling the cached multi-axis acceleration data to reduce the cumulative sampling time of the multi-axis acceleration data to a preset third time;

[0077] The downsampled multi-axis acceleration data is input into the motion type recognition model to obtain the predicted motion type.

[0078] In order to unify the amount of data input to the model, when the cumulative sampling time of the cached multi-axis acceleration data reaches the second time length T2, the cached data can be downsampled, thereby reducing the duration of the data from T2 to the third time length T3, so as to reduce the amount of data and compress the length of the multi-axis acceleration sequence. The downsampled multi-axis acceleration data is input into the aforementioned trained motion type recognition model to predict the user's current motion type. By downsampling the data before inputting into the model, not only can the amount of data be reduced to meet the model's requirements for input length, but also the real-time responsiveness of the model can be improved.

[0079] In one embodiment of the present invention, after inputting the cached multi-axis acceleration data into the motion type recognition model to obtain the predicted motion type, the method further includes:

[0080] Continue to obtain multi-axis acceleration data within a preset fourth time period, and input the acceleration within the fourth time period into the motion type recognition model to obtain a corresponding predicted motion type;

[0081] Determine whether the two motion types are consistent, and generate and send a reminder message if they are consistent.

[0082] After recognizing that the user has started to exercise, the multi-axis acceleration data in the fourth duration can be continuously obtained, and input into the sports category recognition model to predict the sports category. If the sports category predicted here is the same as the sports category predicted last time, it means that the user has been doing the same kind of exercise during this period. At this time, a reminder message can be generated and sent to the user, and the user can be reminded by sound, vision, vibration, etc. to prompt the user to open the APP and record more detailed sports data. On the contrary, if the sports categories predicted twice are different, it means that the user has changed a new sport. At this time, the latest sports category can be continued as the object to be compared, and the data is repeatedly obtained again to determine whether the user still maintains the same kind of exercise. It can be understood that the present invention can first sample multi-axis acceleration data in the fourth duration, and after reaching the fourth duration, the sampled data is sent to the model for identification. It can also be sampled according to the first duration T0, and it is determined whether the user is in motion after each sampling. If in, the sampling is continued to the second duration T2, and the collected data is downsampled to the third duration T3 and sent to the model to identify the sports category. The identified sports category is compared with the previous one. If they are the same, sampling continues until the cumulative sampling time reaches the fourth time, and then a reminder message is sent. Furthermore, when it is identified that the user is doing a certain sport, the background defaults to calling the APP to calculate and save the relevant physiological information and sports information. After reminding the user to open the APP, the previous sports information can be retained without loss.

[0083] See also Figure 3, which shows a schematic diagram of the process of sending a three-axis acceleration data reminder message. First, the three-axis acceleration data of the wearable device is continuously collected and cached within the first time length T0, and the average mean square error f of the three-axis acceleration in the first time length is calculated std . Determine the mean square error f std Is it less than the preset error threshold T1: If so, it means that the user is not exercising, and the previously cached three-axis acceleration data and all exercise durations can be directly cleared to end the monitoring process here. In order to ensure the sustainability of monitoring, new three-axis acceleration data can be sampled in the next first duration T0 and the above process can be repeated. On the contrary, if the calculated mean square error f std Greater than or equal to the error threshold T1, indicating that the user is in motion, at this time, sampling needs to be continued until the cumulative sampling duration reaches the first duration T1. The collected data are downsampled so that the sampling duration of the data is reduced to the third duration T3 to achieve compression of the length of the three-axis acceleration data. The downsampled data is input into the motion category recognition model to identify the motion category. If the motion category is not identified, it means that the user is not in any motion state. At this time, all the previously cached data and motion duration can be directly cleared, and a new round of data sampling and recognition can be started. If it is identified that the user is doing a certain sport (such as A sport), when this sport is the same as the sport identified last time, the second duration T2 is accumulated, and it is determined whether the accumulated duration is greater than the fourth duration T4. If greater than, a reminder message is directly sent to the user and all cached data and motion duration are cleared, and a new round of data sampling and recognition is started. If not greater than, it is necessary to continue to sample new three-axis acceleration data in the next first duration T0 and repeat the above process until the cumulative motion duration reaches the fourth duration or the user changes the sport.

[0084] See also Figure 4, the motion type recognition model training system 100 includes: a sample acquisition module 110, a motion type prediction module 120, a label loss calculation module 130, a sparse loss calculation module 140, a center loss calculation module 150 and a parameter update module 160. The above-mentioned sample acquisition module 110 is used to obtain a training sample set; wherein each training sample includes multi-axis acceleration data within a preset sampling time and a corresponding real motion type. The motion type prediction module 120 is used to input the multi-axis acceleration data into the motion type recognition model to be trained, extract the corresponding motion features, and obtain the corresponding predicted motion type according to the motion features. The label loss calculation module 130 is used to calculate the label loss of the predicted motion type and the corresponding real motion type. The sparse loss calculation module 140 is used to calculate the sparse loss based on the model parameters of the fully connected layer of the motion type recognition model. The center loss calculation module 150 is used to calculate the center loss based on the motion features. The parameter updating module 160 is used to obtain the total loss based on the label loss, the sparse loss and the center loss, and update the parameters of the motion type recognition model according to the total loss to obtain a trained motion type recognition model.

[0085] For the specific limitations of the sports type recognition model training system, please refer to the limitations of the sports type recognition model training method above, which will not be repeated here. Each module in the above-mentioned sports type recognition model training system can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware format, or can be stored in the memory of the computer device in software format, so that the processor can call the operations corresponding to the above modules.

[0086] It should be noted that, in order to highlight the innovative part of the present invention, the present embodiment does not introduce modules that are not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other modules in the present embodiment.

[0087] See also Figure 5 The wearable device 1 may include a memory 12, a processor 13 and a bus, and may also include a computer program stored in the memory 12 and executable on the processor 13, such as a sports type recognition program.

[0088] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the wearable device 1, such as a mobile hard disk of the wearable device 1. In other embodiments, the memory 12 can also be an external storage device of the wearable device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the wearable device 1. Further, the memory 12 can also include both the internal storage unit of the wearable device 1 and the external storage device. The memory 12 can not only be used to store application software and various types of data installed on the wearable device 1, such as codes for identifying types of sports, etc., but can also be used to temporarily store data that has been output or is to be output.

[0089] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 13 is the control core (Control Unit) of the wearable device 1, and uses various interfaces and lines to connect the various components of the entire wearable device 1, and executes or executes programs or modules (such as sports type recognition programs, etc.) stored in the memory 12, and calls the data stored in the memory 12 to execute various functions of the wearable device 1 and process data.

[0090] The processor 13 executes the operating system and various installed applications of the wearable device 1. The processor 13 executes the applications to implement the steps in the above-mentioned method for identifying the type of exercise.

[0091] Exemplarily, the computer program may be divided into one or more modules, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the wearable device 1. For example, the computer program may be divided into a sample acquisition module 110, a motion type prediction module 120, a label loss calculation module 130, a sparse loss calculation module 140, a center loss calculation module 150, and a parameter update module 160.

[0092] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium, and the computer-readable storage medium can be non-volatile or volatile. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to perform part of the functions of the sports category recognition model training method or the sports category recognition method described in each embodiment of the present application.

[0093] In summary, the present invention discloses a method, system, wearable device and medium for identifying and training a type of motion, and uses multi-axis acceleration data and corresponding real motion types to construct a training sample set. By calculating the label loss between the predicted motion type and the real motion type, the model has a higher accuracy in the classification task, and the sparse loss is calculated based on the model parameters of the fully connected layer, and the weight of the fully connected layer is constrained, thereby improving the generalization ability of the model. The center loss calculated according to the motion feature further enhances the model's ability to distinguish multiple categories by reducing the distance between the features of samples of the same category. The present invention adopts an end-to-end approach. When the trained motion type recognition model needs to expand new motion types, it only needs to increase the number of output neurons in the last layer and update the relevant parameters to achieve expansion, which greatly simplifies the upgrade and maintenance process of the model. Not only can the algorithm's computing cost be reduced, but the corresponding motion category can also be identified more accurately. Therefore, the present invention effectively overcomes the various shortcomings in the prior art and has a high industrial utilization value.

[0094] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for training a sports type recognition model, characterized in that: The method comprises: Obtaining a training sample set and a corresponding label set including motion types; wherein each training sample includes multi-axis acceleration data within a preset sampling time; Inputting the multi-axis acceleration data into a motion type recognition model to be trained, extracting corresponding motion features, and obtaining a corresponding predicted motion type according to the motion features; Based on the predicted motion category and the corresponding true motion category, calculating the label loss of both; Calculating a sparse loss based on model parameters of a fully connected layer of the motion category recognition model; Based on the motion features, calculating a center loss; A total loss is obtained based on the label loss, the sparse loss and the center loss, and the parameters of the motion type recognition model are updated according to the total loss to obtain a trained motion type recognition model.

2. The method for training a sports type recognition model according to claim 1, characterized in that: The step of obtaining a training sample set includes: Obtain an initial training sample set; For each initial training sample in the initial training sample set: If the length of the initial training sample is equal to the preset length threshold, it remains unchanged; If the length of the initial training sample is not equal to the length threshold, resampling the initial training sample at least once according to a preset sampling length, and using the resampled multi-axis acceleration data as the initial training sample, so that the sample length of the initial training sample reaches the length threshold; All initial training samples with unified sample lengths are merged to obtain the final training sample set.

3. The method for training a sports type recognition model according to claim 1, characterized in that: The center loss Among them, N batch is the number of training samples in the current training batch, f k is the motion feature of the kth training sample, For sports type y k The center vector of .

4. The method for training a sports type recognition model according to claim 1, characterized in that: The sports type recognition model is a lightweight model.

5. The method for training a sports type recognition model according to claim 4, characterized in that: The step of inputting the multi-axis acceleration data into a motion type recognition model to be trained, extracting corresponding motion features, and obtaining corresponding predicted motion types according to the motion features includes: Inputting the multi-axis acceleration data into a feature extraction module group of a motion type recognition model, performing feature processing through each feature extraction module to generate motion features, and taking the motion features generated by the last feature extraction module as the final motion features; the feature extraction module group includes a plurality of cascaded feature extraction modules, each feature extraction module includes a cascaded depth-separable convolution layer, a normalization layer, an activation layer and a pooling layer; The motion features are input into the fully connected layer of the motion type recognition model to obtain the corresponding predicted motion type.

6. The method for training a sports type recognition model according to claim 5, characterized in that: The step of inputting the multi-axis acceleration data into a feature extraction module group of a motion type recognition model, performing feature processing through each feature extraction module, and generating motion features comprises: Input the multi-axis acceleration data into the first feature extraction module of the feature extraction module group, perform time series convolution on the acceleration of each axis respectively, obtain the motion feature of each axis, and fuse the motion features of all axes to obtain the initial motion feature; For each remaining feature extraction module: input the initial motion feature / the motion feature of the previous feature extraction module into the current feature extraction module, perform temporal convolution on each channel of the input feature independently to obtain the motion feature of each channel, and fuse the motion features of all channels to generate the corresponding motion feature.

7. A method for identifying a type of motion, characterized in that: The identification method comprises: Periodically sampling and caching the multi-axis acceleration data within a preset first time period, and calculating the average mean square error of the multi-axis acceleration data based on the acceleration of each axis; Determine the activity state according to the average mean square error, and continue to sample and cache the multi-axis acceleration data when the activity state is motion, until the cumulative sampling time length of the cached multi-axis acceleration data is equal to a preset second time length; The cached multi-axis acceleration data is input into the motion type recognition model to obtain the predicted motion type.

8. The method for identifying the type of movement according to claim 7, characterized in that: The step of inputting the cached multi-axis acceleration data into a motion type recognition model to obtain a predicted motion type includes: Downsampling the cached multi-axis acceleration data to reduce the cumulative sampling time of the multi-axis acceleration data to a preset third time; The downsampled multi-axis acceleration data is input into the motion type recognition model to obtain the predicted motion type.

9. The method for identifying the type of movement according to claim 7, characterized in that: After inputting the cached multi-axis acceleration data into the motion type recognition model to obtain the predicted motion type, the method further includes: Continue to obtain multi-axis acceleration data within a preset fourth time period, and input the acceleration within the fourth time period into the motion type recognition model to obtain a corresponding predicted motion type; Determine whether the two motion types are consistent, and generate and send a reminder message if they are consistent.

10. A sports type recognition model training system, characterized in that: The system comprises: A sample acquisition module, used to acquire a training sample set and a corresponding label set including a motion type; wherein each training sample includes multi-axis acceleration data within a preset sampling time; A motion type prediction module, used for inputting the multi-axis acceleration data into a motion type recognition model to be trained, extracting corresponding motion features, and obtaining a corresponding predicted motion type according to the motion features; A label loss calculation module, used to calculate the label losses of the two based on the predicted motion type and the corresponding real motion type; A sparse loss calculation module, used to calculate the sparse loss based on the model parameters of the fully connected layer of the motion type recognition model; A center loss calculation module, used for calculating the center loss based on the motion feature; A parameter updating module is used to obtain a total loss based on the label loss, the sparse loss and the center loss, and to update the parameters of the motion type recognition model according to the total loss to obtain a trained motion type recognition model.

11. A wearable device, characterized in that: The wearable device comprises: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the wearable device to implement the method for identifying the type of exercise as described in any one of claims 7 to 9.

12. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute a motion type recognition model training method as described in any one of claims 1 to 6 or a motion type recognition method as described in any one of claims 7 to 9.

Images