Method, device and equipment for motion monitoring based on multi-stage network learning and medium

Abstract

This invention provides a motion monitoring method, device, equipment, and medium based on multi-stage network learning, relating to the field of data processing technology. The method includes: acquiring several user motion video segments, preprocessing them to generate training samples, establishing an initial model, and training to obtain a target model that determines motion height based on sensor data; real-time acquisition of target sensor data from a specific user, and using a classification network in the target model to classify the data based on its volatility; when the target sensor data volatility is high, a first-stage processing network after training is used for motion height monitoring; when the target sensor data volatility is low, a second-stage processing network after training is used for motion height monitoring; and outputting a target result including motion height for motion monitoring. This method overcomes the problem that existing rope skipping exercises can only calculate the number of repetitions using sensors and cannot achieve autonomous motion monitoring.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, equipment and medium for motion monitoring based on multi-stage network learning. Background Technology

[0002] Studies have shown that exercise is a powerful physiological stimulus to the body's own growth hormone secretion, but this stimulation requires the simultaneous fulfillment of two factors: exercise intensity and exercise duration. Under the same exercise duration, lower-intensity exercise has little effect on promoting growth hormone secretion. High-intensity exercise has a ceiling effect on growth hormone secretion, and it may also lead to sports injuries. Therefore, exercise intensity needs to be within an effective and safe range.

[0003] Rope skipping, as an aerobic exercise, is now very popular in primary and secondary schools across the country. It not only improves reaction time and coordination but also enhances immunity to some extent. Furthermore, as a full-body workout, rope skipping requires the coordinated participation of multiple muscle groups, significantly promoting strength and physical fitness in teenagers.

[0004] Most existing jump rope devices are divided into ordinary jump ropes without sensors and jump rope devices that can count jump ropes. Neither of them can monitor the intensity of the jump rope. Most of them can only monitor the movement with manual assistance and cannot monitor the movement independently based on the intensity and number of jump ropes. Summary of the Invention

[0005] In order to overcome the above-mentioned technical defects, the purpose of this invention is to provide a motion monitoring method, device, equipment and medium based on multi-stage network learning, which can overcome the problem that existing rope skipping exercises can only calculate the number of times through sensors and cannot achieve autonomous motion monitoring.

[0006] This invention discloses a motion detection method based on multi-stage network learning, comprising:

[0007] Several user motion video segments are collected, preprocessed, and then used to generate training samples. The training samples contain several training data, each of which includes sensor data and facial centroid height of a user at different times.

[0008] An initial model is established and trained using the training samples to obtain a target model for determining motion height based on sensor data; wherein, the target model includes a classification network, a first processing network based on a transformer network, and a second processing network.

[0009] Real-time acquisition of target sensor data for a specific user; encoding of the target sensor data in the target model; classification based on the volatility of the target sensor data using a classification network; when the volatility of the target sensor data is high, a first training network is used for motion height monitoring; when the volatility of the target sensor data is low, a second training network is used for motion height monitoring; outputting a target result including motion height for motion monitoring.

[0010] Preferably, the step of collecting several user motion video segments, preprocessing them, and generating training samples includes:

[0011] For any user, obtain the sensor data for that user at each time point;

[0012] The sensor data is formatted and then standardized.

[0013] Acquire each frame of data from the user's motion video segment, and use a pre-trained object detection network to perform face recognition on each frame of data to determine the face centroid height corresponding to each frame of data;

[0014] By associating the sensor data with the face centroid height at different times, several training data are obtained, and training samples are generated.

[0015] Preferably, the step of training with the training samples to obtain a target model for determining motion height based on sensor data includes:

[0016] During training, sensor data from the training samples is used as input. The mean square error of the output of the initial model is compared with the corresponding face centroid. The network is trained and optimized by minimizing the mean square error to obtain a target model that determines the motion height based on sensor data.

[0017] Preferably, the classification based on the volatility of the target sensor data using a classification network includes:

[0018] Based on the target sensor data, obtain sensor data from a predetermined number of historical data acquisitions;

[0019] Feature extraction is performed on sensor data collected a preset number of times in history, and the mean is calculated to generate a reference vector;

[0020] Feature extraction is performed on the target sensor data to obtain a target vector, and fluctuation calculation is performed on the target vector based on the reference vector to classify the target sensor data.

[0021] Preferably, the step of performing fluctuation calculations on the target vector based on the reference vector to classify the target sensor data includes:

[0022] Calculate the distance between the target vector and the reference vector;

[0023] If the ratio of the distance to the target vector exceeds 50%, it is determined that the target sensor data has large fluctuations.

[0024] If the ratio of the distance to the target vector does not exceed 50%, then the target sensor data is determined to have low fluctuation.

[0025] Preferably, after classifying the target sensor data using a classification network based on the volatility, the method further includes:

[0026] The target sensor data is labeled according to the category using a single valid code.

[0027] Preferably, it further includes:

[0028] A mapping table of the relationship between the heat of movement and the height of movement is provided in advance;

[0029] The heat change curve of the target user within a preset time period is obtained by accumulating the target results.

[0030] The present invention also provides a motion monitoring device based on multi-stage network learning, comprising:

[0031] The preprocessing module is used to collect several user motion video segments, and generate training samples after preprocessing. The training samples contain several training data, and each training data includes sensor data and face centroid height of a user at different times.

[0032] The model processing module is used to build an initial model and train it using the training samples to obtain a target model that determines the motion height based on sensor data; wherein, the target model includes a classification network, a first processing network based on a transformer network, and a second processing network.

[0033] The execution module is used to collect target sensor data of a target user in real time. In the target model, the target sensor data is encoded and classified based on the volatility of the target sensor data using a classification network. When the volatility of the target sensor data is large, a first processing network after training is used to monitor the motion height; when the volatility of the target sensor data is small, a second processing network after training is used to monitor the motion height. The result includes the motion height and is used to perform motion monitoring.

[0034] The present invention also provides a computer device, the computer device comprising: a memory for storing executable program code; and a processor for calling the executable program code in the memory and executing steps including the motion monitoring method based on multi-stage network learning.

[0035] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the motion monitoring method based on multi-stage network learning.

[0036] Compared with existing technologies, the above technical solution has the following advantages:

[0037] The motion monitoring method provided by this invention establishes and trains a multi-stage learning transformer network model to obtain a target model for determining motion height based on sensor data. The target model is used to classify the collected sensor data, and a first processing network and a second processing network are used to predict motion height for sensor data with large and small fluctuations, respectively. This method can accurately predict the user's real-time height off the ground during motion, overcoming the problem that existing rope skipping exercises can only calculate the number of times through sensors and cannot achieve autonomous motion monitoring, thereby improving the accuracy of the calculated motion amount. Attached Figure Description

[0038] Figure 1 This is a flowchart of an embodiment of the motion monitoring method based on multi-stage network learning described in this invention;

[0039] Figure 2 This is a schematic diagram of the network structure for multi-stage processing of the target model in Embodiment 1 of the motion monitoring method based on multi-stage network learning described in this invention;

[0040] Figure 3 This is a schematic diagram of a module in Embodiment 2 of the motion monitoring device based on multi-stage network learning described in this invention;

[0041] Figure 4 This is a schematic diagram of the modules of the computer device described in this invention.

[0042] Figure label:

[0043] 4-Motion monitoring device based on multi-stage network learning; 41-Preprocessing module; 42-Model processing module; 43-Execution module; 5-Computer equipment; 51-Memory; 52-Processor. Detailed Implementation

[0044] The advantages of the present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments.

[0045] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0046] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0047] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0048] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0049] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0050] In the following description, suffixes such as "module," "part," or "unit" used to denote elements are used only for the convenience of the description of the invention and have no specific meaning in themselves. Therefore, "module" and "part" can be used interchangeably.

[0051] Example 1: This example discloses a motion monitoring method based on multi-stage network learning. This method collects data from the jump rope's sensors and combines this data with neural network learning to determine the user's motion height, thereby monitoring the motion state. This data can then be used to monitor calorie consumption, overcoming the limitations of existing technologies that only record the number of jumps and require real-time manual monitoring. Specifically, this monitoring method can be applied to devices embedded in jump rope handles. (See reference...) Figures 1-2 ,include:

[0052] S100: Collect several user motion video segments, preprocess them to generate training samples, wherein the training samples contain several training data, and each training data includes sensor data and face centroid height of a user at different times.

[0053] In the above steps, it should be noted that sensor data and user motion status (height) can be obtained from the user's motion video segments. Specifically, as an example, a jump rope handle template can be designed, integrating a gyroscope and an accelerometer into the jump rope handle. 200 volunteers can be selected to conduct jump rope experiments, and sensor signals can be collected. A camera can be used to record the jump rope process directly in front of the volunteers to obtain the aforementioned user motion video.

[0054] Specifically, several user motion video segments are collected, preprocessed, and then used to generate training samples, including those for any user:

[0055] S110: Obtain sensor data for the user at various times;

[0056] For illustrative purposes, the aforementioned sensor data includes data collected by gyroscopes and accelerometers.

[0057] S120: Convert the format of the sensor data and perform consistency processing;

[0058] Specifically, in the above steps, the acceleration data in the sensor coordinate system is converted to the vertical acceleration data in the geodetic coordinate system, and the signal is processed by error calibration, noise reduction, normalization, etc.; as an example and not a limitation, let the sensor data obtained after signal processing at time t be denoted as

[0059] D t =(x 1t ,x 2t ,y 1t ,y 2t ,z 1t ,z 2t )

[0060] Where x1, y1, z1 represent angular velocities in three directions, and x2, y2, z2 represent angular accelerations in three directions.

[0061] S130: Obtain each frame of data in the user's motion video segment, and use a pre-trained target detection network to perform face recognition on each frame of data to determine the face centroid height corresponding to each frame of data;

[0062] In the above steps, an initial detection model is established based on the YOLO network. It is trained using labeled training images. By reducing the loss function value and updating the network weight parameters, the object detection network is generated. For example, several volunteer video files are randomly selected, frames are extracted and saved as images, and the saved images are labeled. Rectangular bounding boxes are used to delineate face locations, and a neural network for face detection is designed. This object detection neural network uses the YOLO model, with the class set to 1. The predicted bounding boxes include the confidence level of the target class and the probability of each bounding box region. Redundant windows are removed using non-maximum suppression. The prepared training set images and labels are input into the YOLO network for training. By reducing the loss function value and updating the network weight parameters, the learned network weight parameters are obtained after several training iterations.

[0063] S140: Based on the sensor data and the face centroid height associated with the time, obtain several training data and generate training samples.

[0064] After the above face recognition, the face centroid can be determined according to a preset range, and then the face centroid height can be obtained to obtain the change curve of the face centroid in the corresponding video segment. As time changes, the sensor data changes and the face centroid changes accordingly. Therefore, the corresponding sensor data and face centroid height can be correlated with time and used as sample data for training.

[0065] S200: Establish an initial model and train it using the training samples to obtain a target model for determining motion height based on sensor data; wherein, the target model includes a classification network, a first processing network based on a transformer network, and a second processing network;

[0066] In the above steps, it should be noted that the target model provided in this embodiment is divided into two extreme processing stages. The first stage is the processing of the classification model, and the second stage is the processing of the first processing network and the second processing network. Both the first processing network and the second processing network use transformer networks, mainly to divide the data into data with large fluctuations and data with small fluctuations and process them differently, so as to make the monitoring results more accurate.

[0067] Specifically, the step of training using the training samples to obtain a target model for determining motion height based on sensor data includes:

[0068] During training, sensor data from the training samples is used as input. The mean square error of the output of the initial model is compared with the corresponding face centroid. The network is trained and optimized by minimizing the mean square error to obtain a target model that determines the motion height based on sensor data.

[0069] Based on the above, a multi-stage transformer network is constructed (i.e., classification in the first stage and processing in the second stage as described above). Sensor data is used as input, and the network output is compared with the face centroid at the corresponding time point using the mean squared error (MSE). The network is trained and optimized by minimizing the MSE. After several training cycles, the learned network weight parameters are obtained.

[0070] S300: Real-time acquisition of target sensor data for a target user; encoding the target sensor data in the target model; classifying the target sensor data based on its volatility using a classification network; when the target sensor data has high volatility, using a trained first processing network for motion height monitoring; when the target sensor data has low volatility, using a trained second processing network for motion height monitoring; outputting a target result containing motion height for motion monitoring.

[0071] As mentioned above, refer to Figure 2 In this embodiment, the target model establishes a multi-stage transformer network based on a transformer network (i.e., the first stage is classification, and the second stage is processing, as described above). Classifying sensor data with large and small fluctuations allows for more accurate prediction of the real-time jump rope height, resulting in more precise calculations of the exercise intensity. Both the first and second processing networks are transformer networks, but their corresponding training data differ. Each training data point in the training samples is also classified by the classification network and then used to train both the first and second processing networks. Specifically, data with high fluctuations is used to train the first processing network, and data with low fluctuations is used to train the second processing network. Before the encoding, data preprocessing (including but not limited to format conversion or standardization processing) as described in step S120 can also be performed.

[0072] It should be noted that the aforementioned volatility refers to the length of change in current data relative to historical data. Since large and small volatility can affect the prediction results of subsequent motion height, different processing networks are used based on the volatility of the sensor data. First, the volatility classification of the sensor data is determined. Specifically, the classification based on the volatility of the target sensor data using a classification network includes:

[0073] S310: Obtain sensor data from a preset number of historical acquisitions based on the target sensor data; extract features from the sensor data from the preset number of historical acquisitions and calculate the mean to generate a reference vector;

[0074] In the above steps, volatility is determined based on the change of the current data relative to the 10 historical data points.

[0075] Specifically, in this embodiment, the mean of the vector obtained by extracting features from sensor data at 10 recording times before time t is used as a reference vector to be compared with the target sensor data to determine the volatility.

[0076] S320: Extract features from the target sensor data to obtain a target vector, and perform fluctuation calculation on the target vector based on the reference vector to classify the target sensor data.

[0077] Specifically, the step of calculating the volatility of the target vector based on the reference vector to classify the target sensor data includes: calculating the distance between the target vector and the reference vector; if the ratio of the distance to the target vector exceeds 50%, the target sensor data is determined to have high volatility; if the ratio of the distance to the target vector does not exceed 50%, the target sensor data is determined to have low volatility.

[0078] In the above steps, the volatility of the target sensor is determined by the ratio of the distance between the target sensor data and the mean of multiple historical records to the target sensor data. The specific distance can be, but is not limited to, Euclidean distance. For example, let S... t Let D be the sensor data at time t. t The vector obtained after extracting feature d through the network Let be the mean of the vectors obtained after feature extraction from sensor data at the 10 recorded times prior to time t using a classification network. Comparing the mean vectors at time t with the mean vectors at the previous 10 times, the magnitude of the change can be categorized into two types: large fluctuation and small fluctuation. If the Euclidean distance between the two vectors is greater than 1 / 2... If the volatility is 50%, it is classified as Category I (i.e., high volatility); otherwise, it is classified as Category II (i.e., low volatility).

[0079]

[0080] Specifically, after classifying the target sensor data based on the volatility using a classification network, the method further includes: labeling the target sensor data according to the category using a one-bit effective encoding. That is, one-hot encoding is used to convert the first and second categories into (1,0) and (0,1) vectors, respectively, as labels for the classification network.

[0081] The second-stage high-precision prediction model uses a basic transformer model. The prediction model corresponding to data with large fluctuations is designated as Model 1 (first processing network), and the prediction model corresponding to data with small fluctuations is designated as Model 2 (second processing network). The prediction model selected for the second stage is determined based on the output of the first stage. During the training phase, Model 1 and Model 2 are pre-trained using the corresponding datasets, and the weights of the optimal model are fixed.

[0082] In this embodiment, based on the above, the motion height can be predicted according to sensor data, thereby performing motion monitoring. Furthermore, based on the user's motion height, exercise consumption can also be calculated. Specifically, it also includes:

[0083] A mapping table of exercise heat relative to exercise height is provided in advance; the heat change curve of the target user within a preset time period is obtained by accumulating the target results.

[0084] In the above steps, the target user's exercise calories can be automatically calculated based on the obtained target results including the height of the jump rope, achieving fully automatic detection of the jump rope's height and exercise intensity. This enables real-time monitoring of the target user's exercise and can also provide warnings when the exercise threshold is exceeded or insufficient. Alternatively, it can read sensor data offline for user exercise data analysis. In this embodiment, the sensor signals are processed and modeled together with the video captured by the camera. The relationship between the two is analyzed through a multi-stage deep learning model. The trained model processes the sensor signals to obtain the jump rope height and exercise intensity, achieving intelligent monitoring of the jump rope process.

[0085] In this embodiment, multi-stage transformer learning accurately detects the user's height off the ground during rope skipping and can further calculate the amount of exercise. The device implementing this method can be integrated with the rope skipping device, and the user's exercise data can be recorded for subsequent data analysis. This solves the problem of requiring manual supervision and counting in existing methods, making it convenient and beneficial for developing personalized and reasonable exercise plans for users, while also providing guidance and reference for sports programs in schools, communities, and families.

[0086] Example 2: This embodiment provides a motion monitoring device 4 based on multi-stage network learning, which can be integrated with a jump rope device to autonomously monitor the user's motion status. For details, please refer to... Figure 3 ,include:

[0087] The preprocessing module 41 is used to collect several user motion video segments, and generate training samples after preprocessing. The training samples contain several training data, and each training data includes sensor data and face centroid height of a user at different times.

[0088] Specifically, the operational steps for collecting experimental sensor signals and recording jump rope videos in the training phase can be as follows: Recruit 200 volunteers, aged 7 to 18. Have the volunteers jump rope for 1 to 10 minutes, repeated 10 times. Collect data from various sensors generated during the volunteers' jump rope process and record the experimental video files, i.e., the aforementioned user movement video segments. The preprocessing includes, but is not limited to, converting the acceleration in the sensor coordinate system to vertical acceleration data in the geodetic coordinate system, and performing error calibration, noise reduction, normalization, and other processing on the signals.

[0089] The model processing module 42 is used to establish an initial model and train it using the training samples to obtain a target model that determines the motion height based on sensor data; wherein, the target model includes a classification network, a first processing network based on a transformer network, and a second processing network.

[0090] In the above module, sensor data from the training samples is used as input. The mean square error of the output of the initial model is compared with the corresponding face centroid. The network is trained and optimized by minimizing the mean square error to obtain a target model that determines the motion height based on sensor data.

[0091] The execution module 43 is used to collect target sensor data of a target user in real time. In the target model, after encoding the target sensor data, a classification network is used to classify the target sensor data based on its volatility. When the target sensor data has high volatility, a first processing network after training is used to monitor motion height. When the target sensor data has low volatility, a second processing network after training is used to monitor motion height. The goal is to output a target result that includes motion height for motion monitoring.

[0092] In the above module, the target model establishes a multi-stage transformer network based on the transformer network (first stage classification, second stage processing). It classifies and processes sensor data with large and small fluctuations, which can more accurately predict the real-time height of the rope off the ground, thus making the calculated amount of exercise more accurate. It can also automatically calculate the exercise calories of the target user based on the target results including the height of the exercise, realizing fully automatic detection of the height of the rope off the ground and the amount of exercise during the rope jumping process, thereby realizing real-time monitoring of the target user's exercise.

[0093] Example 3:

[0094] To achieve the above objectives, the present invention also provides a computer device 5. This computer device can be a smartphone, tablet computer, laptop computer, desktop computer, etc., that executes programs, used to receive sensor data for offline analysis, and may even be hardware integrated into the jump rope device. The computer device in this embodiment includes, but is not limited to, at least: a memory 51 and a processor 52 that can communicate with each other, such as… Figure 4 As shown. It should be noted that only a computer device with components is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0095] In this embodiment, the memory 51 includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), disk, etc. In some embodiments, the memory 51 may be an internal storage unit of a computer device, such as the hard disk or RAM of the computer device. In other embodiments, the memory 51 may also be an external storage device of the computer device. Of course, the memory 51 may include both internal storage units and external storage devices of the computer device. In this embodiment, the memory 51 is typically used to store the operating system and various application software installed on the computer device, such as the program code and training samples of the motion monitoring method based on multi-stage network learning in Embodiment 1. In addition, the memory 51 may also be used to temporarily store various types of data that have been output or will be output.

[0096] In some embodiments, processor 52 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. This processor 52 is typically used to control the overall operation of a computer device. In this embodiment, processor 52 is used to run program code stored in memory 51 or process data, such as performing data queries, to implement the motion monitoring method based on multi-stage network learning in Embodiment 1.

[0097] Example 4:

[0098] To achieve the above objectives, the present invention also provides a computer-readable storage system, which includes multiple storage media, such as flash memory, hard disk, multimedia card, server, App application store, etc., on which computer programs are stored. When the programs are executed by the processor 52, they perform corresponding functions. In this embodiment, the computer-readable storage medium is used to store a data storage and retrieval device, and when executed by the processor 52, it implements the motion monitoring method based on multi-stage network learning in Embodiment 1.

[0099] It should be noted that the embodiments of the present invention have better implementability and are not intended to limit the present invention in any way. Any person skilled in the art may use the above-disclosed technical content to change or modify it into equivalent effective embodiments. However, any modifications or equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A method of motion monitoring based on multi-stage network learning, characterized by, include: Several user motion video segments are collected, preprocessed, and then used to generate training samples. The training samples contain several training data, each of which includes sensor data and facial centroid height of a user at different times. An initial model is established and trained using the training samples to obtain a target model for determining motion height based on sensor data; wherein, the target model includes a classification network, a first processing network and a second processing network based on a transformer network, and the training using the training samples includes: pre-training the first processing network using a training sample set corresponding to high volatility, pre-training the second processing network using a training sample set corresponding to low volatility, and fixing the weights of the optimal model; Real-time acquisition of target sensor data for a specific user; encoding the target sensor data in the target model; and classifying the data based on its volatility using a classification network. If the target sensor data exhibits high volatility, a first, trained processing network is used for motion height monitoring; if the target sensor data exhibits low volatility, a second, trained processing network is used for motion height monitoring. The goal is to output a target result including motion height for motion monitoring. The classification based on the volatility of the target sensor data using a classification network includes: Based on the target sensor data, obtain sensor data from a predetermined number of historical data acquisitions; Feature extraction is performed on sensor data collected a preset number of times in history, and the mean is calculated to generate a reference vector; The classification includes: using one-hot encoding to transform the categories with high volatility and low volatility into (1,0) and (0,1) vectors, respectively, as labels for the classification network; Feature extraction is performed on the target sensor data to obtain a target vector, and fluctuation calculation is performed on the target vector based on the reference vector to classify the target sensor data; the fluctuation calculation on the target vector based on the reference vector to classify the target sensor data includes: Calculate the distance between the target vector and the reference vector; If the ratio of the distance to the target vector exceeds 50%, it is determined that the target sensor data has large fluctuations. If the ratio of the distance to the target vector does not exceed 50%, then the target sensor data is determined to have low fluctuation.

2. The motion monitoring method of claim 1, wherein, The process of collecting several user motion video segments, preprocessing them, and generating training samples includes: For any user, obtain the sensor data for that user at each time point; The sensor data is formatted and then standardized. Acquire each frame of data from the user's motion video segment, and use a pre-trained object detection network to perform face recognition on each frame of data to determine the face centroid height corresponding to each frame of data; By associating the sensor data with the face centroid height at a given time, several training data points are obtained, and training samples are generated.

3. The motion monitoring method according to claim 1, characterized in that, The step of training using the training samples to obtain a target model for determining motion height based on sensor data includes: During training, sensor data from the training samples is used as input. The mean square error of the output of the initial model is compared with the corresponding face centroid. The network is trained and optimized by minimizing the mean square error to obtain a target model that determines the motion height based on sensor data.

4. The motion monitoring method according to claim 1, characterized in that, After classifying the target sensor data using a classification network based on its volatility, the method further includes: The target sensor data is labeled according to the category using a single valid code.

5. The motion monitoring method according to claim 1, characterized in that, Also includes: A mapping table of the relationship between the heat of motion and the height of motion is provided in advance; The heat change curve of the target user within a preset time period is obtained by accumulating the target results.

6. A motion monitoring device based on multi-stage network learning, characterized in that, include: The preprocessing module is used to collect several user motion video segments, and generate training samples after preprocessing. The training samples contain several training data, and each training data includes sensor data and face centroid height of a user at different times. A model processing module is used to establish an initial model and train it using the training samples to obtain a target model for determining motion height based on sensor data. The target model includes a classification network, a first processing network and a second processing network based on a transformer network. The training using the training samples includes: pre-training the first processing network using a training sample set corresponding to high volatility, pre-training the second processing network using a training sample set corresponding to low volatility, and fixing the weights of the optimal model. The execution module is used to collect target sensor data of a target user in real time. In the target model, the target sensor data is encoded and then classified based on the volatility of the target sensor data using a classification network. When the volatility of the target sensor data is high, a first trained processing network is used for motion height monitoring; when the volatility of the target sensor data is low, a second trained processing network is used for motion height monitoring. The output includes a target result containing motion height information, thus performing motion monitoring. The classification based on the volatility of the target sensor data using a classification network includes: Based on the target sensor data, obtain sensor data from a predetermined number of historical data acquisitions; Feature extraction is performed on sensor data collected a preset number of times in history, and the mean is calculated to generate a reference vector; The classification includes: using one-hot encoding to transform the categories with high volatility and low volatility into (1,0) and (0,1) vectors, respectively, as labels for the classification network; Feature extraction is performed on the target sensor data to obtain a target vector, and fluctuation calculation is performed on the target vector based on the reference vector to classify the target sensor data; the fluctuation calculation on the target vector based on the reference vector to classify the target sensor data includes: Calculate the distance between the target vector and the reference vector; If the ratio of the distance to the target vector exceeds 50%, it is determined that the target sensor data has large fluctuations. If the ratio of the distance to the target vector does not exceed 50%, then the target sensor data is determined to have low fluctuation.

7. A computer device, characterized in that: The computer device includes: A memory for storing executable program code; and a processor for calling the executable program code in the memory to perform steps including the motion monitoring method as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the motion monitoring method according to any one of claims 1 to 5.

Images