Method and apparatus for assisting in running training

Abstract

The present application relates to the technical field of reinforcement learning, and particularly relates to a method and device for assisting running training, comprising: recording physiological data and running actions of a person in running training, and assigning an initialized quality score; taking physiological data, running actions and a reward score determined by the quality score in adjacent time periods as experience samples; training a reinforcement learning model of the person by using a data set of the person; obtaining a plurality of experience samples with the maximum predicted score in the data set, and updating the quality score by using the change degree corresponding to the plurality of experience samples; continuing to train the reinforcement learning model by using a data set of a target person with the same updated quality score; and generating running actions by using the reinforcement learning model when the person runs again, and determining a reward score by using the updated quality score. The present application realizes adaptive and personalized running training assistance based on individual differences and real-time physiological state, and improves training effect and safety.

Description

Technical Field

[0001] This invention relates to the field of reinforcement learning technology, and more specifically to methods and devices for assisting running training. Background Technology

[0002] In current running training practices, whether aimed at improving athletic performance or enhancing physical fitness, training programs typically rely on pre-set, standardized running motion parameters (such as a fixed combination of stride frequency and stride length). These pre-set parameters are often based on the average physiological characteristics or rules of thumb for specific events (such as sprinting and long-distance running) and are applied to all trainees. During running training, real-time monitoring of the individual trainee's physiological state (such as heart rate and respiration) is primarily used to assess whether they have reached the pre-set training intensity range.

[0003] Because trainees have significantly different initial physical qualities (such as cardiorespiratory capacity, muscular endurance, and athletic experience), their optimal training load and suitable running form also vary. A training intensity that is restorative for a professional athlete may be the limit for a beginner. Therefore, using a standardized, fixed running form for instruction cannot provide each individual with the optimal training stimulus that their current physical condition can handle and that effectively promotes performance improvement. This can lead to overtraining and injury for those with weaker physical conditions, while undertraining and slow progress for those with stronger physical conditions. Ultimately, the training process is not optimal for a significant number of individuals, affecting training effectiveness and safety.

[0004] In summary, existing running training methods cannot overcome the limitations of existing standardized running training guidelines, and cannot provide adaptive, efficient, and safe running training assistance for different individuals. Summary of the Invention

[0005] To address the above problems, the present invention provides a method and device for assisting running training.

[0006] The method and device for assisting running training of the present invention adopt the following technical solution: One embodiment of the present invention provides a method for assisting running training, the method comprising the following steps: Record the physiological data and running movements of any individual during each running training session, and assign an initial fitness score to that individual; use the physiological data, running movements, and reward scores of any adjacent time periods as an experience sample; the reward score is determined by the fitness score: calculate the degree of change of the physiological data of any adjacent time periods in the experience sample; when the degree of change is less than or equal to the fitness score, the reward score is positively correlated with the range of motion of the running movements; when the degree of change is greater than the fitness score, the reward score is negatively correlated with the range of motion of the running movements. After each running training session, the dataset for that person is first constructed from all the experience samples from the most recent running training sessions, and then the person's reinforcement learning model is trained using that dataset. Obtain several empirical samples with the highest predicted scores in the dataset, and update the competency score using the degree of change corresponding to the empirical samples; the predicted score is output by the reinforcement learning model; personnel other than the person with the same updated competency score are recorded as target personnel, and the reinforcement learning model for the target personnel is further trained using the target personnel's dataset; When the person runs again, a reinforcement learning model is used to generate running motions, and the updated competency score is used to determine the reward score.

[0007] Preferably, the specific steps for calculating the degree of change in physiological data between any adjacent time periods in the experience sample are as follows: For any two adjacent time periods, the earlier time period is designated as the first time period, and the later time period is designated as the second time period. Calculate the first intensity index of physiological data within the first time period; calculate the second intensity index of physiological data within the second time period; the mean of the first intensity index and the second intensity index is recorded as the average intensity index of physiological activity intensity; the difference between the second intensity index and the first intensity index is recorded as the change index of physiological activity intensity; the degree of change is positively correlated with the average intensity index and the change index, respectively.

[0008] Preferably, the specific steps for obtaining several empirical samples with the highest predicted scores in the dataset and updating the quality score using the degree of change corresponding to the several empirical samples are as follows: For the currently trained reinforcement learning model and all experience samples recorded during the most recent running training, the physiological data in each experience sample is input into the Actor network of the reinforcement learning model. The predicted running action output by the Actor network is input into the Critic network of the reinforcement learning model using the physiological data in the experience samples. The Critic network outputs a predicted score. For the predicted scores output for all experience samples, the first mean of the degree of change calculated for the several experience samples with the largest predicted scores is obtained. The updated quality score is positively correlated with this first mean.

[0009] Preferably, individuals other than the stated individual who have the same updated competency score are designated as target individuals. The reinforcement learning model for these target individuals is then trained using their dataset. The specific steps include the following: After the person updates their competency score, the competency scores of all people are clustered to obtain all categories. If the competency score of any other person has never been updated, the competency score of the other person is not included in the clustering, and the competency scores in the same category are used as the same updated competency scores. Obtain the category to which the person's competency score belongs. People in that category but not the person are designated as target people. Merge the datasets of all target people and the person's dataset into a single dataset. Use this single dataset to train the reinforcement learning model for that person.

[0010] Preferably, the specific steps of generating running motions using a reinforcement learning model are as follows: After each preset running training period, the person inputs the physiological data of the most recent preset period into the Actor network in the reinforcement learning model, and the Actor network outputs a predicted running action; in the next preset period, the person trains running using the predicted running action; the preset period is equal to the length of any one of the adjacent time periods.

[0011] Preferably, the initialization method for the competency score is as follows: For any given individual, if the number of running training sessions completed by that individual is less than a certain number, that individual will perform running training according to a preset running motion, and the reward score in the experience sample will not be calculated. If the number of running training sessions completed by that individual is greater than or equal to a certain number, the degree of change corresponding to all experience samples collected during the most recent few running training sessions will be obtained. The average of the corresponding degree of change will be used as the initial value of that individual's quality score. The reward score in the experience sample obtained when the number of running training sessions is less than a certain number will be calculated using this initial value.

[0012] Preferably, the specific steps for obtaining the range of motion of the running motion are as follows: The running motion includes cadence and stride length. The cadence and stride length in the running motion are normalized, and the square of the normalized cadence and the square of the normalized stride length are summed and then the square root is taken to obtain the range of motion of the running motion.

[0013] Preferably, the specific steps for obtaining the first strength index and the second strength index are as follows: All physiological data are normalized; the mean of all normalized physiological data within the first time period is calculated and denoted as the first intensity index; the mean of all normalized physiological data within the second time period is calculated and denoted as the second intensity index.

[0014] Preferably, the specific formula for calculating the degree of change is as follows: The degree of change is E = w1 × E0 + (1 - w1) × E1; where E0 represents the change index, E1 represents the average intensity index, and w1 represents the weighting coefficient. The weighting coefficient w1 is positively correlated with the ratio E0 / E1.

[0015] Another embodiment of the present invention provides a device for assisting running training, the device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements all the steps of the method for assisting running training described above.

[0016] The beneficial effects of the technical solution of the present invention are: This invention constructs a reinforcement learning model based on individual physiological data and fitness scores to generate personalized running movements for each trainee, achieving adaptive assistance in running training. The overall technical effect lies in solving the technical problem that existing standardized training programs cannot match individual differences, achieving personalized and adaptive training guidance. Specifically, by recording physiological data and running movements and constructing experience samples, and combining them with fitness scores to determine reward scores, a training feedback signal based on individual endurance is provided to the reinforcement learning model. Using individual datasets to train a dedicated reinforcement learning model ensures that the running movement recommendations output by the model are based on personalized summaries of the individual's own historical data. By updating the fitness score using the experience sample with the highest predicted score, online adaptive optimization of the fitness score is achieved, allowing the score to dynamically converge to the optimal training intensity level that the individual can actually bear. Continuing to train the reinforcement learning model using a target dataset of individuals with the same updated fitness scores significantly expands the number of effective training samples, improving the efficiency and robustness of model training. Finally, using the reinforcement learning model to generate running movements and combining them with the updated fitness scores to determine reward scores, real-time, personalized, dynamic, and safe training guidance based on the individual's current actual state and the optimal strategy learned by the model is achieved. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the steps of a method for assisting running training according to an embodiment of the present invention. Detailed Implementation

[0019] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the method and device for assisting running training proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0021] The specific solutions for the method and equipment for assisting running training provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0022] Please see Figure 1 The diagram illustrates a flowchart of a method for assisting running training according to an embodiment of the present invention, the method comprising the following steps: Step S101: Record the physiological data and running movements of any individual during each running training session.

[0023] The physiological data (such as heart rate or respiratory data) directly reflects an individual's exercise load, cardiopulmonary function, and fatigue level. This embodiment collects time-series physiological data, enabling the capture of continuous trends in an individual's physiological state during training. The physical meaning of these trends lies in their direct correlation with the immediate and cumulative effects of training intensity on the individual's body.

[0024] In this embodiment, running actions refer to parameters such as stride length and stride frequency, which are actively controllable running training variables imposed on the individual. Subsequently, this embodiment uses a reinforcement learning model to generate running actions, which the individual then uses for efficient and safe running training.

[0025] Step S102: Assign an initial fitness score to any individual; calculate the degree of change in physiological data in any adjacent time period in the experience sample. When the degree of change is less than or equal to the fitness score, the reward score is positively correlated with the range of motion of the running action; otherwise, it is positively correlated.

[0026] The fitness score is not intended for absolutely precise medical assessment, but rather as a dynamic similarity measure. It is a relative, dynamic, and personalized scalar estimate of an individual's current physical fitness baseline (such as cardiopulmonary function and exercise tolerance).

[0027] The degree of variability describes the fluctuations or changes in physiological indicators contained in physiological data over adjacent time periods, quantifying the magnitude of the immediate disturbance or stress caused by exercise load to the physiological system. A small degree of variability may indicate that the body is adapting to the current load or is in a low-load state; a large degree of variability may indicate that the body is under greater stress or is approaching its limit. This degree of variability transforms continuous physiological signals into a key criterion that can be used for comparison with fitness scores.

[0028] The aforementioned experience samples encapsulate physiological data, running movements, and reward scores observed between any two adjacent time periods, where the reward score represents a feedback result of performing the running movement. This structured data encapsulation method is a standard input format for training reinforcement learning models. It enables raw data collected during continuous running training to be effectively organized into discrete experience units that can be learned and optimized by reinforcement learning models.

[0029] Furthermore, this embodiment provides an adaptive reward scoring evaluation mechanism. The reward score, serving as a signal guiding the optimization direction of the reinforcement learning model, relies on a comparison of personalized competence scores with the degree of change in real-time physiological data. Specifically: When the degree of change is less than or equal to the fitness score: the reward score is positively correlated with the range of motion of the running motion. The purpose is to encourage reinforcement learning models to explore or recommend running motions with larger ranges and higher intensity when the individual's physiological response is within their personal tolerance range (i.e., not exceeding the threshold of their current fitness estimate), so as to promote improved training results.

[0030] When the degree of change exceeds the fitness score: reward is negatively correlated with movement amplitude. The purpose is to punish or suppress running movements that cause physiological responses exceeding an individual's estimated tolerance, guiding the model to recommend gentler movements, thereby preventing overtraining and ensuring training safety.

[0031] This reward scoring and evaluation mechanism achieves personalized and adaptive reward thresholds: individuals with strong physical fitness (high fitness score) receive a correspondingly higher threshold for encouraging them to increase training; while individuals with weak physical fitness (low fitness score) receive a lower threshold for suggesting they reduce their workload. This directly solves the technical problem of using a uniform, fixed standard that cannot match individual differences in endurance.

[0032] Step S103: After each running training session, firstly, construct the person's dataset from all the experience samples from the most recent running training sessions, and then use the person's dataset to train the person's reinforcement learning model.

[0033] This embodiment uses experience samples from several recent running training sessions to ensure the temporal relevance of the dataset. Since an individual's physical condition changes dynamically with training, using recent data ensures that the training model learns the behavior-response mapping relationship of the individual in their current state, avoiding the misleading influence of outdated historical data and thus guaranteeing the timeliness and current applicability of the model's recommendations.

[0034] Furthermore, this embodiment maintains and trains a dedicated reinforcement learning model for each individual. Through training, the model learns how to obtain a higher expected reward (output) for that specific individual. This ensures that the running motion recommendations output by the model are based on personalized summaries of the individual's own historical data, rather than general rules, thus directly solving the problem that standardized solutions cannot be individually adapted.

[0035] Step S104: Obtain several experience samples with the largest predicted scores in the dataset, and update the competency scores using the degree of change corresponding to these experience samples; the predicted scores are output by the reinforcement learning model; personnel other than this person who have the same updated competency scores as this person are recorded as target personnel, and the reinforcement learning model for this person is further trained using the dataset of the target personnel.

[0036] The above steps, while aiming to make reinforcement learning models timely and applicable in the present moment, and to address the issue of individualized adaptation, have two potential limitations: The reward mechanism relies on competency scores, but these initial scores may be inaccurate, and individuals' actual competency changes during training. Furthermore, to ensure timeliness, the individual dataset only contains recent data, resulting in a limited number of experience samples for training. These two factors may collectively limit the accuracy of the initial reinforcement learning model training.

[0037] This step is used to further resolve the problem, specifically: The prediction score is an evaluation by the reinforcement learning model of the expected benefit obtained from performing an action in a given experience sample. The experience samples with the highest prediction scores represent instances that achieved the best expected training effect in historical training under the current individual model's understanding. Selecting this experience sample as a benchmark has the technical advantage of using the trained individual model as a filter to automatically identify training segments from historical data that are of high intensity or ideal state for the current individual.

[0038] By updating the competency score using the degree of change corresponding to the best-expected empirical sample, online adaptive optimization of the score is achieved. This allows the competency score to dynamically converge from a potentially inaccurate initial estimate to a level of physiological change that reflects the optimal training intensity that the individual can actually achieve or tolerate. The updated score is more reliable, thus providing a foundation for more accurate reward calculation and personnel matching in the next round of running training, and solving the problem of dependence on the initial score.

[0039] Among these steps, other individuals with the same score are selected as target individuals, and these target individuals are identified as a group with the same or similar physical fitness as the current individuals.

[0040] The use of datasets from target individuals (those with similar physical abilities) to supplement the training of individual models significantly expands the number of effective training samples. Since similar individuals may have experienced different training movements and physiological responses, but their basic physical abilities are similar, this data is a highly relevant and safe supplementary sample for training the current model. This addresses the problem of insufficient training and model instability that can result from purely individual models due to limited experience samples (only recent data), improving the efficiency and robustness of model training.

[0041] Step S105: When the personnel run again, the running action is generated using a reinforcement learning model, and the reward score is determined using the updated competency score.

[0042] At this point, each individual has undergone reinforcement learning model training after each running training session. When that individual runs again, during their training, the reinforcement learning model outputs recommended personalized running movements (such as stride length and cadence) based on real-time or near-real-time physiological data (state input). This step transforms the personalized knowledge accumulated through data collection, sample construction, model training, and scoring updates into real-time, dynamic training guidance. This replaces fixed, standardized movement schemes and provides optimal or suboptimal training stimuli tailored to the individual's current capabilities based on their actual state (reflected by physiological data) and the best strategy learned by the model.

[0043] For the more reliable competency score updated in step S104, the updated score is used to calculate the reward score after the second run training, ensuring that the individual ability benchmark on which the reward mechanism relies is up-to-date and more realistic. This allows the reward signal to more accurately reflect whether the newly performed action is beneficial or detrimental relative to the individual's latest ability level, thereby providing higher-quality training labels for the next round of model training (step S103) and promoting continuous optimization of the entire learning loop.

[0044] In summary, while initial issues such as model and scoring inaccuracies may exist, this embodiment constructs a long-term, iterative online learning loop, enabling each stage to mutually reinforce the others: Each running training session updates the individual's dataset. This dataset trains the individual's reinforcement learning model, making it more accurate. A more accurate reinforcement learning model can select more reliable best experiences to update the competency score. More reliable competency scores enable more reliable reward calculations and find more matching, similar individuals to expand the data. Richer data on similar individuals further trains an even more accurate individual model. This cycle continues, and as running training progresses, personalized recommendations become increasingly accurate and efficient. Essentially, this involves continuous, collaborative online learning and optimization of each individual's competency score and personalized model.

[0045] This embodiment overcomes the limitations of existing standardized running training guidance programs and provides a method that can dynamically generate and optimize personalized running training movements (stride length and cadence) based on individual differences and real-time physiological states, thereby achieving adaptive, efficient and safe running training assistance for different individuals.

[0046] As an example, the methods for recording the physiological data and running movements of any individual during each running training session include: The physiological data described in this example refers to heart rate data (in beats per minute). Specifically, during running training, a person wears a smartwatch / wristband to collect heart rate data at a frequency of 1Hz. The smartwatch / wristband uses a green LED light to illuminate the capillaries under the skin of the wrist, and the heart rate is calculated by detecting changes in light absorption caused by blood flow through sensors. This process is existing technology and will not be described in detail in this embodiment.

[0047] It should be noted that each experience sample in the subsequent process corresponds to two physiological data points in any two adjacent time periods and a running action. In this embodiment, for the time period that is earlier in the time sequence of two adjacent time periods, the running action in that time period is always the same. The running action recorded in this example (that is, the running action corresponding to each experience sample) refers to the running action that was synchronously read when the last physiological data was collected in the time period that was earlier in the time sequence (which can be read directly from the assisted running training device).

[0048] Thus, during the running training process, time-series data of physiological data and running movements at different time periods were collected.

[0049] In other examples, the physiological data refers to respiratory data (in breaths per minute). Specifically, during running training, a person wears a respiratory monitoring belt to collect respiratory data at a frequency of 1 Hz. The belt uses embedded stretch sensors or accelerometers to measure the periodic expansion and contraction of the chest or abdomen during running, thereby calculating the respiratory rate. This process is existing technology and will not be described in detail in this embodiment.

[0050] In some examples, respiratory and heart rate data are collected simultaneously and used as physiological data.

[0051] As a preferred example, the method for calculating the degree of change in physiological data over any adjacent time periods in an empirical sample includes: For any two adjacent time periods, the time period that occurs earlier is designated as the first time period, and the time period that occurs later is designated as the second time period.

[0052] Calculate the first intensity index of physiological data within the first time period. Calculate the second intensity index of physiological data within the second time period. The first and second intensity indices describe the intensity of physiological activity, such as heart rate or respiratory rate, in adjacent time periods.

[0053] The average of the first and second intensity indices is recorded as the average intensity index of physiological activity intensity. The larger this value, the higher the frequency of breathing or heartbeat in adjacent time periods, indicating a greater intensity of exercise endured in adjacent time periods. The difference between the second and first intensity indices is recorded as the change index of physiological activity intensity. The larger this difference, the more significant the increase in breathing or heartbeat between adjacent time periods.

[0054] The degree of change is positively correlated with both the average intensity index and the change index. This indicates that the greater the intensity of physiological activity within adjacent time periods, or the greater the significant increase in physiological activity between adjacent time periods, the greater the degree of change. This embodiment reflects whether an individual's physiological response is within their personal tolerance range by comparing the degree of change with their competence score.

[0055] As an example, the time sequence of physiological data collected during each running training session is divided into several subsequences. In this example, the duration of each subsequence is one minute, and the time intervals corresponding to any two adjacent subsequences are considered as adjacent time intervals.

[0056] Specifically, if the last subsequence has a duration of less than one minute, then the last subsequence is deleted. If the total number of subsequences obtained is less than 10, it means that the running training session was too short to achieve the desired training effect, and the running training session is not considered.

[0057] As an example, the methods for obtaining the first and second intensity indices are as follows: All physiological data are normalized to remove dimensions. The mean of all normalized physiological data within the first time period is calculated and denoted as the first intensity index. The mean of all normalized physiological data within the second time period is calculated and denoted as the second intensity index.

[0058] As an example, the method for calculating the degree of change is as follows: The degree of change is E = w1 × E0 + (1 - w1) × E1. Here, E0 represents the change index, E1 represents the average intensity index, and w1 represents the weighting coefficient. This weighting coefficient w1 is positively correlated with the ratio E0 / E1. Its function is that when the change in physiological activity intensity between adjacent time periods is larger than the average intensity index (i.e., when w1 is larger), it indicates that the physiological activity generated during running training is primarily reflected in the steady increase in physiological activity intensity. In this case, the increase in physiological activity intensity better reflects the physiological impact of running training, and therefore, the magnitude of E0 is more important. When the change in physiological activity intensity between adjacent time periods is smaller than the average intensity index (i.e., when w1 is smaller), it indicates that the physiological activity during running training is strong and has reached its limit (i.e., the physiological activity intensity cannot increase further). In this case, the average intensity index of physiological activity intensity better reflects the physiological impact of running training, and therefore, the magnitude of E1 is more important.

[0059] As an example, w1 = 2 × sigmoid(E0 / E1) - 1, where sigmoid() represents the sigmoid function, used to normalize the weighting coefficient w1. Specifically, when E0 is less than 0, it indicates a decrease in physiological activity intensity. In this case, setting E0 to 0 ensures that the degree of change E only considers the average intensity index E1. Since sigmoid(E0 / E1) is greater than or equal to 0.5 and less than 1, this example uses 2 × sigmoid(E0 / E1) - 1 to ensure that w1 is greater than or equal to 0 and less than 1.

[0060] As an example, the normalization of all physiological data includes the following methods: When the physiological data refers to heart rate (or heartbeat data), the physiological data is divided by a preset maximum heart rate (e.g., 200 beats / minute) to normalize each physiological data point. When the physiological data refers to respiratory data, the physiological data is divided by a preset maximum respiratory rate (e.g., 65 breaths / minute) to normalize each physiological data point. It should be noted that the purpose of normalization in this embodiment is to remove dimensions, allowing for the special case where the normalized physiological data is greater than 1.

[0061] As an example, if the physiological data includes both heart rate and respiratory data, the degree of change is calculated separately for heart rate and respiratory data according to the example above, and then the average of the degree of change is taken to obtain the degree of change obtained in this example.

[0062] As an optional example, methods for calculating the degree of change in physiological data over any adjacent time periods in an empirical sample include: The method for obtaining the degree of change in the above preferred example has a very low computational cost. However, since the changes in physiological data during running training are complex, the reliability of the obtained degree of change may be low. In this example, physiological data from any adjacent time period are input into a neural network (e.g., LSTM), and the neural network outputs the degree of change.

[0063] The training method for this neural network is as follows: A large amount of physiological data from individuals during various running training sessions at adjacent time points is collected. Each pair of adjacent time points is considered a sample, and all samples constitute the first dataset. Each sample is manually assigned a label with a value ranging from {0.1, 0.2, ..., 1}, where a larger label value indicates a greater degree of variation. The neural network is trained using the samples and labels from the first dataset. The loss function is the mean squared error loss function, the appropriate optimizer is Adam, the learning rate is 0.01, the batch size is 8, and the maximum number of training iterations is set to 10. 5 .

[0064] It should be noted that the degree of change obtained in this example is based on manually labeled tags, which is highly reliable, but requires a neural network and involves a large amount of computation.

[0065] As a preferred example, the reward score is calculated as follows: For each running motion, the cadence and stride length are normalized. The square of the normalized cadence and the square of the normalized stride length are summed and then the square root is taken to obtain the range of motion of the running motion.

[0066] When the degree of change is less than or equal to the fitness score, the reward score is equal to the range of motion. When the degree of change is greater than the fitness score, the reward score is equal to 1 minus the range of motion. It should be noted that this example allows the reward score to be less than 0. In this example, as long as the degree of change is slightly greater than the fitness score (e.g., the exercise intensity is slightly higher), the exercise intensity is immediately reduced, making the trend of reducing exercise intensity quite sensitive. Although this may result in the exercise intensity being reduced immediately when there is a small disturbance in the exercise intensity, since the fitness score describes the limit level of exercise that an individual can withstand, this situation further ensures the safety of the exercise process.

[0067] As an example, the normalization of cadence and stride length for each running motion includes the following methods: Divide the cadence and stride length by 220 steps / minute and 1.75m respectively to obtain the normalized cadence and stride length.

[0068] As a preferred example, the reinforcement learning model utilizes the well-known DDPG (Deep Deterministic Policy Gradient) model. The training method for this model is as follows: For any individual's dataset (in this example, the dataset consists of all experience samples from the last 5 running training sessions), extract experience samples sequentially from the dataset (in the order in which the experience samples were obtained). On one hand, the physiological data from the first time period in the experience sample, i.e., the time series of physiological data within the first time period, is input into the Actor network in DDPG. The Actor network outputs a predicted running action (including stride length and cadence). This predicted running action and the time series of physiological data within the first time period are then input into the Critic network in DDPG. The Critic network outputs a predicted score, which represents the evaluation of the expected benefit obtained from the running action. The parameters of the Actor network in DDPG are updated using the Adam optimizer and the Actor loss function (policy gradient loss function) to maximize the predicted score of the Critic network. The specific method for updating the Actor network parameters (including the Actor loss function) is described in well-known DDPG techniques and will not be elaborated on in this embodiment. On the other hand, physiological data in the time sequence and running movements in the experience samples within the first time period are input into the Critic network. The Critic network outputs a predicted score. The parameters of the Critic network (the optimizer is adam) are updated by constructing the Critic network loss function (temporal difference error loss function) by the predicted score and the reward score in the experience samples. The specific method for updating the Critic network parameters (including the Critic network loss function) is described in the well-known DDPG technology, and will not be described in detail again in this embodiment.

[0069] The parameters of the Actor and Critic networks are continuously updated by taking empirical samples from the dataset (which is equivalent to the empirical replay pool in DDPG).

[0070] As an example, both the Actor and Critic networks employ an LSTM network architecture. The learning rate for both networks is set to 0.01, the batch size (i.e., the number of experience samples retrieved from the experience replay pool each time) is 6, and the maximum number of training iterations is set to 10. 5 .

[0071] It should be noted that, since the predicted running motion and physiological data are time series from different modalities within the first time period, this example processes the data as follows when inputting it into the Critic network: First, a convolutional layer is used to process the time sequence to obtain convolutional features. The predicted stride frequency and stride length of running actions are then input into a fully connected neural network with one hidden layer (two neurons). The output of this fully connected network is fully connected features (two-dimensional fully connected features). The convolutional features and fully connected features are then concatenated and input into the Critic network. At this point, the convolutional layer, the hidden layer, and the Critic network are considered as a whole.

[0072] As an example, the convolutional kernel of the convolutional layer is one-dimensional with a length of 3 and a stride of 1, and the convolutional features are single-channel sequences.

[0073] In other examples, the time series can be flattened, and the predicted running cadence and stride length can be directly concatenated after the flattened time series before being input into the Critic network. While this example suffers from the relatively low accuracy of the Critic network, it has a relatively small number of parameters.

[0074] As an example, training is also considered complete when the standard deviation of the most recent (e.g., 50) predicted scores in DDPG is less than 0.2.

[0075] In other examples, methods to reduce training time include: Considering that running training focuses on long-term changes in physiological data rather than short-term local physiological data, the time-series physiological data within the first time period is downsampled to reduce the amount of input data for the Actor and Critic networks. This example uses Max Pooling for downsampling, with a sampling window size of 2, a stride of 2, and 3 downsampling iterations. Both the Actor and Critic networks employ a fully connected neural network structure. As an example, the fully connected neural network structure has 5 layers: the first and last layers are the input and output layers, and the middle three layers are hidden layers. The number of neurons in the input and output layers equals the number of input and output data points, respectively, while the number of neurons in the hidden layers is 6, 4, and 2, respectively.

[0076] This example significantly reduces the number of parameters and training difficulty of the DDPG model, making it suitable for running training scenarios with a large number of participants and high frequency of running training.

[0077] As a preferred example, the quality score is updated by obtaining several empirical samples with the highest predicted scores from the dataset and using the degree of change corresponding to these empirical samples. This includes the following methods: For the currently trained reinforcement learning model, and all experience samples from the most recent (e.g., the most recent) running training sessions (specifically, the completed running training sessions), the physiological data (specifically, its temporal sequence within the first time period) of each experience sample is input into the Actor network. The predicted running action output by the Actor network is then input into the Critic network using the physiological data (specifically, its temporal sequence within the first time period) of the experience sample. The Critic network outputs a predicted score. For all experience samples, the multiple experience samples with the highest predicted scores are obtained. For each of these experience samples, a degree of change is calculated (the specific calculation process has been described above). The first mean of the degree of change of the multiple experience samples with the highest predicted scores (e.g., 5) is obtained. The updated quality score is positively correlated with this first mean.

[0078] As an example, the updated competency score is equal to the first mean. In other examples, the updated competency score can also be equal to a multiple of the first mean (e.g., 0.8 to 1 times). In other examples, 0.9 times the first mean is used as an example.

[0079] As a preferred example, individuals other than the target person who have the same updated competency score are designated as the target person. The reinforcement learning model for this target person is then trained using their dataset, including the following methods: Once the individual has updated their competency score, all competency scores are clustered to obtain all categories. In particular, if the competency scores of any other individual have never been updated, then those other individuals' competency scores are not included in the clustering.

[0080] If the quality scores in each category are the same or similar, and all the people in each category have the same or similar physical fitness conditions, then obtain the category to which the quality score of the person belongs. People in that category other than the person are recorded as people with the same updated quality score as the person, and are recorded as target people. In this example, "same" means the same or similar under the same category.

[0081] In other alternative examples, the competency scores can be rounded to one decimal place, and competency scores with identical values ​​can be grouped into one category. This example is more computationally efficient.

[0082] The datasets of all target personnel and the dataset of the selected individual are merged into a single dataset (equivalent to adding the experience samples of other personnel to the experience replay pool of this individual). The reinforcement learning model for this individual is then trained using this single dataset. This involves sequentially extracting experience samples from the single dataset and updating the reinforcement learning model parameters according to the method described above. This continuous extraction of experience samples achieves ongoing training of the reinforcement learning model. Sequentially extracting experience samples means: from all target personnel and a randomly selected individual, experience samples are extracted sequentially from all of this individual's experience samples according to their chronological order.

[0083] In one example, the probability of a target person being randomly selected is 60% of the probability of that person being selected (the sum of the probabilities of all target people and that person being selected equals 1). The purpose is to avoid contamination from the experience samples of other people.

[0084] As an example, using the K-Means algorithm for clustering, the number of clusters is set to one-tenth of the total number of people (rounded up), meaning that each cluster contains an average of ten people (note that this is not equivalent to making the number of people in each cluster always equal to ten). Other examples may set the number of clusters to other values; this example does not impose restrictions.

[0085] As a preferred example, when the person trains running again, a reinforcement learning model is used to generate running motions, and an updated competency score is used to determine the reward score. The methods include: After each minute of running training (the duration of the first time period), the individual inputs their most recent minute of physiological data into the Actor network in the DDPG (Directed Running Projection System). The network outputs a predicted running motion, including stride length and cadence. Then, for the next minute (the duration of the second time period), the individual trains with this stride length and cadence. After another minute of running training (the duration of the first time period), the most recent minute of physiological data is input into the Actor network in the DDPG, which outputs a predicted running motion, including stride length and cadence. This process continues until the running training is complete.

[0086] After the running training session ends, the reward score is determined based on the physiological data collected during the training session using the updated fitness score (the specific process has been described above).

[0087] As an example, if the number of running training sessions completed by the individual is less than a certain number (e.g., less than 5 times), the individual will run according to a preset stride length and cadence. The period before the start of each running training session (e.g., one minute before the start) is considered the warm-up phase. During the warm-up phase, the individual will run with the preset stride length and cadence, and no physiological data will be recorded during the period before the start.

[0088] Specifically, during running training, if the reinforcement learning model has not yet completed its training, the running motion is generated using the most recently trained reinforcement learning model to guide the current running training process. If no most recently trained reinforcement learning model is available, then running training is conducted according to the preset stride length and stride frequency.

[0089] It should be noted that the cold start process in this embodiment is as follows: when the number of running training sessions completed by the person is greater than or equal to a certain number (e.g., less than 5 times), the reinforcement learning model is trained using the experience samples obtained from the most recent running training sessions. After the reinforcement learning model has completed its first training, the running action is generated using the reinforcement learning model to guide subsequent running training.

[0090] As an example, a total running training duration (e.g., 100 hours) is set for each person. When the total duration of running training guidance using a reinforcement learning model exceeds this total running training duration, the reinforcement learning model is no longer used for subsequent running training guidance for each person. The reinforcement learning model is deleted to reduce the size of the reinforcement learning model for all people.

[0091] As a preferred example, the initialization method for the competency score is as follows: For any given individual, if the number of running training sessions completed by that individual is less than a certain number (e.g., less than 5 times), then the individual is training according to a preset stride length and stride frequency. Once the number of running training sessions completed is greater than or equal to a certain number, the degree of change corresponding to all experience samples collected during the most recent running training sessions is obtained. The average of these degree of change is used as the initial value of the individual's quality score. Then, the reward score in the experience samples obtained when the number of running training sessions was less than a certain number is calculated using this initial value, and the reinforcement learning model is trained using the experience samples.

[0092] As an example, to ensure that the running motion generated by the reinforcement learning model does not deviate significantly from the preset stride length and stride frequency, this example performs the following processing: The preset stride length and stride frequency are denoted as A and B, respectively. During the training of the reinforcement learning model, the stride length and stride frequency output by the Actor network are denoted as A1 and B1, respectively. When A1 is greater than 130% of A or less than 70% of A, or B1 is greater than 120% of B or less than 80% of B, the reward score is directly set to -10. The purpose is to discourage the generated running movements from deviating too much from the preset stride length and stride frequency.

[0093] As an example, taking long-distance running training, the preset stride length is 0.9 meters and the preset cadence is 175 steps per minute. In other examples, the preset stride length can be set using an empirical formula: the preset stride length is equal to 0.45 to 0.6 times the height (this example uses 0.5 times as an example).

[0094] Another embodiment of the present invention provides a device for assisting running training, the device being used to assist a person in running training.

[0095] In one alternative example, the device includes a treadmill and a pattern projection device for projecting patterns (e.g., footprint patterns) onto the treadmill belt. The projected patterns are used to guide the stride length and cadence of the running training. The distance between two adjacent projected patterns is equal to the stride length generated by the reinforcement learning model, and the projection frequency of the patterns is equal to the cadence generated by the reinforcement learning model.

[0096] In a preferred example, the device includes a running guidance robot that moves along a specific track in a running training area and projects a pattern backward (similar to the above). The projected pattern is used to provide running training guidance for people following behind the robot. This running guidance robot is prior art; for example, CN113350770B discloses a running robot for auxiliary training, which describes its specific implementation process. This embodiment will not elaborate on it in detail.

[0097] In all embodiments of the present invention, the device includes a running motion generation module. The running motion generation module includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor runs the computer program, it implements all the steps of the above-mentioned method for assisting running training and stores the motion generated by the reinforcement learning model into the treadmill or running guidance robot for guiding running training by projecting patterns.

[0098] As an example, the device also includes an identity recognition module. Specifically, this module uses fingerprint or facial recognition technology to enable registration and login for running trainees and login for administrators. After logging in, administrators (such as coaches) can write data such as preset stride length and cadence from the above embodiments into the auxiliary running training device, or set the total running training duration mentioned above.

[0099] As an example, the module also includes an anomaly detection module. Specifically, this module uses a camera to capture video of each person's running motion (the camera capture frequency is 12Hz), and uses YOLOv5 to detect the bounding boxes of the person's feet and the projected pattern in each frame of the image. The bounding boxes of the feet detected by YOLOv5 include two categories: one is the foot landing category, and the other is the foot not landing category.

[0100] Determine whether there is an overlapping area between the bounding box of each foot and the bounding box of the projected pattern under the foot landing category, and whether the area of ​​the overlapping area is greater than 50% of the bounding box of the foot. If not, mark the corresponding frame image as a suspected abnormal frame.

[0101] When recording experience samples, if the number of suspected abnormal frames in the time period of the recorded experience sample is greater than 20% of the total number of frames in that time period, the recorded experience sample is determined to be abnormal, indicating that the running training was not carried out in accordance with the guidance of the assisted running training device. At this time, the recorded experience sample is deleted to avoid polluting the dataset.

[0102] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for assisting running training, characterized in that, The method includes the following steps: Record the physiological data and running movements of any individual during each running training session, and assign an initial fitness score to that individual; use the physiological data, running movements, and reward scores of any adjacent time periods as an experience sample; the reward score is determined by the fitness score: calculate the degree of change of the physiological data of any adjacent time periods in the experience sample; when the degree of change is less than or equal to the fitness score, the reward score is positively correlated with the range of motion of the running movements; when the degree of change is greater than the fitness score, the reward score is negatively correlated with the range of motion of the running movements. After each running training session, the dataset for that person is first constructed from all the experience samples from the most recent running training sessions, and then the person's reinforcement learning model is trained using that dataset. Obtain several empirical samples with the highest predicted scores in the dataset, and update the competency score using the degree of change corresponding to the empirical samples; the predicted score is output by the reinforcement learning model; personnel other than the person with the same updated competency score are recorded as target personnel, and the reinforcement learning model for the target personnel is further trained using the target personnel's dataset; The specific steps for obtaining the highest predicted scores from several experience samples in the dataset and updating the competency score using the degree of change corresponding to these experience samples are as follows: For the currently trained reinforcement learning model and all experience samples recorded during the most recent running training, the physiological data in each experience sample is input into the Actor network of the reinforcement learning model. The predicted running action output by the Actor network and the physiological data in the experience samples are input into the Critic network of the reinforcement learning model. The Critic network outputs a predicted score. For the predicted scores output for all experience samples, the first mean of the calculated degree of change corresponding to the highest predicted scores from several experience samples is obtained. The updated competency score is positively correlated with this first mean. The personnel other than the current personnel who have the same updated competency score are designated as target personnel. The reinforcement learning model for the target personnel is then trained using their dataset. The specific steps are as follows: After the current personnel updates their competency score, the competency scores of all personnel are clustered to obtain all categories. If the competency score of any other personnel has never been updated, the competency scores of those other personnel are not included in the clustering, and the competency scores in the same category are considered as the same updated competency score. The category in which the current personnel's competency score belongs is obtained, and the personnel other than the current personnel included in that category are designated as target personnel. The datasets of all target personnel and the current personnel are merged together as a single dataset, and the reinforcement learning model for the current personnel is then trained using this single dataset. When the individual resumes running training, a reinforcement learning model is used to generate running motions, and the updated competency score is used to determine the reward score.

2. The method for assisting running training according to claim 1, characterized in that, The specific steps involved in calculating the degree of change in physiological data across any adjacent time periods within the empirical sample are as follows: For any two adjacent time periods, the earlier time period is designated as the first time period, and the later time period is designated as the second time period. Calculate the first intensity index of physiological data within the first time period; calculate the second intensity index of physiological data within the second time period; the mean of the first intensity index and the second intensity index is recorded as the average intensity index of physiological activity intensity; the difference between the second intensity index and the first intensity index is recorded as the change index of physiological activity intensity; the degree of change is positively correlated with the average intensity index and the change index, respectively.

3. The method for assisting running training according to claim 1, characterized in that, The specific steps involved in generating running motions using a reinforcement learning model are as follows: After each preset running training period, the person inputs the physiological data of the most recent preset period into the Actor network in the reinforcement learning model, and the Actor network outputs a predicted running action; in the next preset period, the person trains running using the predicted running action; the preset period is equal to the length of any one of the adjacent time periods.

4. The method for assisting running training according to claim 1, characterized in that, The initialization method for the competency score is as follows: For any given individual, if the number of running training sessions completed by that individual is less than a certain number, that individual will perform running training according to a preset running motion, and the reward score in the experience sample will not be calculated. If the number of running training sessions completed by that individual is greater than or equal to a certain number, the degree of change corresponding to all experience samples collected during the most recent few running training sessions will be obtained. The average of the corresponding degree of change will be used as the initial value of that individual's quality score. The reward score in the experience sample obtained when the number of running training sessions is less than a certain number will be calculated using this initial value.

5. The method for assisting running training according to claim 1, characterized in that, The specific steps for obtaining the range of motion of the running action are as follows: The running motion includes cadence and stride length. The cadence and stride length in the running motion are normalized, and the square of the normalized cadence and the square of the normalized stride length are summed and then the square root is taken to obtain the range of motion of the running motion.

6. The method for assisting running training according to claim 2, characterized in that, The specific steps for obtaining the first strength index and the second strength index are as follows: All physiological data are normalized; the mean of all normalized physiological data within the first time period is calculated and denoted as the first intensity index; the mean of all normalized physiological data within the second time period is calculated and denoted as the second intensity index.

7. The method for assisting running training according to claim 2, characterized in that, The specific formula for calculating the degree of change is as follows: The degree of change is E = w1 × E0 + (1 - w1) × E1; where E0 represents the change index, E1 represents the average intensity index, and w1 represents the weighting coefficient. The weighting coefficient w1 is positively correlated with the ratio E0 / E1.

8. A device for assisting running training, the device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor runs the computer program, it implements all the steps of the method for assisting running training as described in any one of claims 1 to 7.

Images