Motion event classification method and device, equipment and medium

Abstract

The embodiment of the invention discloses a motion event classification method and device, equipment and a medium, the method and device are applied to wearable equipment, the wearable equipment comprises target sensors, the target sensors at least comprise a first sensor and a second sensor, and the method comprises the steps that sensor data collected by the target sensors are detected in real time, obtaining a motion impact core point of the current motion event; determining sensor data corresponding to the current motion event from the collected sensor data according to the motion impact core point; wherein the sensor data corresponding to the current motion event comprises sensor data in a set first time window before the motion impact core point, sensor data of the motion impact core point and sensor data in a set second time window after the motion impact core point; and according to the sensor data corresponding to the current motion event and a set motion event classification model, obtaining a category label of the current motion event.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of wearable devices, and more particularly, to a motion event classification method, device, apparatus and medium. BACKGROUND

[0002] At present, a wearable device has become one of the devices frequently used to capture human actions in the field of sports training. For example, in the field of sports training, the wearable device can cut data streams collected by different sensor channels into fixed-length data windows respectively and input them into a trained motion event classification model, so as to obtain a classification result of a motion event through the trained motion event classification model.

[0003] However, the fixed data window not only causes the data streams collected by different sensor channels to be incorrectly aligned and segmented in time, so that any one data window fails to capture a complete motion event, thereby affecting the classification accuracy of the motion event classification model, and even if a user is in a resting or meaningless action state, the motion event classification model will still continuously perform invalid calculations, causing great waste of resources and computing power burden. SUMMARY

[0004] Embodiments of the present disclosure aim to provide a motion event classification method, device, apparatus and medium.

[0005] According to a first aspect of embodiments of the present disclosure, a motion event classification method is provided, applied to a wearable device, the wearable device comprising a target sensor, the target sensor comprising at least a first sensor and a second sensor, the method comprising: detecting sensor data collected by the target sensor in real time to obtain a motion impact core point of a current motion event; determining sensor data corresponding to the current motion event from the collected sensor data according to the motion impact core point; wherein the sensor data corresponding to the current motion event comprises sensor data within a set first time window before the motion impact core point, sensor data of the motion impact core point and sensor data within a set second time window after the motion impact core point; obtaining a class label of the current motion event according to the sensor data corresponding to the current motion event and a set motion event classification model.

[0006] Optionally, the first sensor comprises an inertial measurement unit, the sensor data comprises an IMU signal, and the IMU signal comprises an acceleration signal, the real-time detection of the sensor data collected by the target sensor to obtain the motion impact core point of the current motion event comprises: detecting the acceleration signal in real time to obtain a jerk value of the acceleration signal; cumulatively summing the jerk value of the acceleration signal in a third time window to obtain a cumulative sum signal in the third time window; determining a motion impact core point of a current motion event according to the cumulative sum signal. Optionally, the determining the motion impact core point of the current motion event according to the cumulative sum signal comprises: continuously detecting whether a local extreme point of the cumulative sum signal appears; in a case where the local extreme point of the cumulative sum signal is detected, determining the motion impact core point of the current motion event according to the local extreme point. Optionally, the determining the motion impact core point of the current motion event according to the local extreme point in a case where the local extreme point of the cumulative sum signal is detected comprises: in a case where the local extreme point of the cumulative sum signal is detected, continuously detecting whether a new local extreme point of the cumulative sum signal appears in a fourth time window; in a case where the new local extreme point of the cumulative sum signal is not detected, marking the local extreme point as the motion impact core point of the current motion event. Optionally, the method further comprises: in a case where the new local extreme point of the cumulative sum signal is detected, updating the local extreme point according to the new local extreme point, and continuing to perform the step of continuously detecting whether a new local extreme point of the cumulative sum signal appears in a fourth time window in a case where a local extreme point of the cumulative sum signal appears. Optionally, before the detecting the sensor data collected by the target sensor in real time, the method further comprises: continuously collecting the sensor data collected by the target sensor; and, storing the collected sensor data to a set buffer according to a first-in-first-out order. Optionally, after the determining the sensor data corresponding to the current motion event from the collected sensor data according to the motion impact core point, the method further comprises: updating the sensor data stored in the set buffer to retain the sensor data in a latest fifth time window. According to a second aspect of the embodiments of the present disclosure, a motion event classification device is provided, which is applied to a wearable device, the wearable device comprises a target sensor, the target sensor comprises at least a first sensor and a second sensor, and the device comprises: The detection module is used to detect the sensor data collected by the target sensor in real time and obtain the core point of motion impact of the current motion event; The determination module is used to determine the sensor data corresponding to the current motion event from the collected sensor data based on the core point of motion impact; wherein, the sensor data corresponding to the current motion event includes sensor data within a set first time window before the core point of motion impact, sensor data at the core point of motion impact, and sensor data within a set second time window after the core point of motion impact. The acquisition module is used to obtain the category label of the current motion event based on the sensor data corresponding to the current motion event and the set motion event classification model.

[0007] According to a third aspect of the present disclosure, a wearable device is provided, the wearable device comprising: Memory is used to store executable computer instructions; A processor, configured to execute the motion event classification method according to the first aspect above, under the control of the executable computer instructions.

[0008] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having computer instructions stored thereon, which, when executed by a processor, perform the motion event classification method described in the first aspect above.

[0009] One beneficial effect of this disclosure is that by determining the sensor data corresponding to the current motion event through real-time detection of the core point of motion impact, it is possible to ensure that the sensor data collected by multiple sensors are accurately aligned in time. Furthermore, the sensor data corresponding to the current motion event includes sensor data within a set first time window before the core point of motion impact, sensor data at the core point of motion impact, and sensor data within a set second time window after the core point of motion impact. This can completely capture the core point of motion impact and its preceding and following context, providing high-quality and complete input for setting the motion event classification model, improving the classification accuracy of the set motion event classification model. In addition, it only extracts and classifies sensor data when the core point of motion impact is detected, avoiding continuous calculation of a large amount of invalid sensor data and reducing the power consumption and computing power consumption of the wearable device.

[0010] Other features and advantages of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present specification and, together with the description, serve to explain the principles of the present specification.

[0012] Figure 1 is a schematic diagram of a hardware configuration of a wearable device to which the method provided by the embodiments of the present disclosure can be applied; Figure 2 is a schematic diagram of a flow of a motion event classification method provided by the embodiments of the present disclosure; Figure 3 is a schematic diagram of a flow of another motion event classification method provided by the embodiments of the present disclosure; Figure 4 is a block diagram of a motion event classification apparatus provided by the embodiments of the present disclosure; Figure 5 is a block diagram of a wearable device provided by the embodiments of the present disclosure. DETAILED DESCRIPTION

[0013] Various exemplary embodiments of the present disclosure will now be described in detail below with reference to the accompanying drawings. Note that the relative arrangement, numerical expressions, and numerical values of components and steps set forth in these embodiments are not limiting to the scope of the embodiments of the present disclosure unless specifically stated otherwise.

[0014] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way limiting to the scope of the disclosure and its applications or uses.

[0015] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail herein, but should be considered as part of the specification, where appropriate.

[0016] In all of the examples shown and discussed herein, any specific values should be interpreted as merely illustrative and not as a limitation on the scope of the exemplary embodiments. Thus, other examples of the exemplary embodiments can have different values.

[0017] Note that like reference numerals and letters indicate like items in the accompanying drawings and, as such, once an item is defined in one drawing, that item is not discussed further in subsequent drawings.

[0018] <Hardware Configuration> Figure 1 is a block diagram of a hardware configuration of a wearable device 1000 according to the embodiments of the present disclosure.

[0019] As shown in Figure 1 , the wearable device 1000 may, for example, be a smart watch, a smart bracelet, or the like, and the embodiments of the present disclosure are not limited thereto.

[0020] In one embodiment, as shown inFigure 1 As shown, the wearable device 1000 can include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, an output device 1500, an input device 1600, an audio device 1700, a first sensor 1800, and a second sensor 1900, etc.

[0021] The processor 1100 can include, but is not limited to, a central processing unit (CPU), a microprocessor (MCU), etc. The memory 1200 includes, for example, a ROM (Read-Only Memory), a RAM (Random Access Memory), a nonvolatile memory such as a hard disk, etc. The interface device 1300 includes, for example, various bus interfaces, such as a serial bus interface (including a USB interface), a parallel bus interface, etc. The communication device 1400 can perform wired or wireless communication, for example. The output device 1500 can include at least one of a display screen, a vibrator, a buzzer, etc. The input device 1600 includes, for example, a touch screen, a body-sensing input, etc. The audio device 1700 can be used to input / output voice information. The first sensor 1800 can include an inertial measurement unit (IMU) which can include an accelerometer (ACC) and a gyroscope (GYRO), wherein the accelerometer can be a three-axis accelerometer and the gyroscope can be a three-axis angular velocity meter. The second sensor 1900 can include an electromyogram (EMG) sensor.

[0022] Those skilled in the art should understand that although a plurality of devices of the wearable device 1000 are shown in Figure 1 the above description, the wearable device 1000 of the embodiments of the present disclosure can only involve part of the devices, and can also include other devices, which are not limited herein.

[0023] In the present embodiment, the memory 1200 of the wearable device 1000 is configured to store instructions for controlling the processor 1100 to operate to implement or support implementation of the motion event classification method according to any embodiment. The skilled person can design the instructions according to the schemes disclosed in the present specification. How the instructions control the processor to operate is well known in the art, and thus will not be described in detail herein.

[0024] In the above description, the skilled person can design the instructions according to the schemes provided in the present disclosure. How the instructions control the processor to operate is well known in the art, and thus will not be described in detail herein.

[0025] Figure 1 The wearable device shown is merely illustrative and is by no means intended to limit the present disclosure, its application or use.

[0026] <Method Embodiment> Figure 2 A motion event classification method of one embodiment of the present disclosure is shown, which can be implemented by a wearable device, which can include a target sensor including at least a first sensor and a second sensor, and the wearable device can be Figure 1 the wearable device 1000 shown. As Figure 2 shown, the motion event classification method can include the following steps S2100-S2300: Step S2100, real-time detection of sensor data collected by the target sensor to obtain a motion impact core point of a current motion event.

[0027] Among them, the target sensor includes at least a first sensor and a second sensor, and the first sensor and the second sensor are different types of sensors. The first sensor can include an inertial measurement unit, which can include an accelerometer and a gyroscope, wherein the accelerometer can be a three-axis accelerometer and the gyroscope can be a three-axis angular velocity meter. The second sensor can include an electromyography sensor, such as a first electromyography sensor and a second electromyography sensor.

[0028] Among them, the sensor data collected by the target sensor includes at least the sensor data collected by the first sensor and the sensor data collected by the second sensor. The sensor data collected by the first sensor can include the IMU signal collected by the inertial measurement unit, such as the acceleration signal collected by the accelerometer and the angular velocity signal collected by the gyroscope, such as the three-axis acceleration signal and the three-axis angular velocity signal. The sensor data collected by the second sensor can include the electromyography signal collected by the electromyography sensor, such as the first electromyography signal collected by the first electromyography sensor and the second electromyography signal collected by the second electromyography sensor.

[0029] In this embodiment, before performing this step S2100 to real-time detect the sensor data collected by the target sensor, the motion event classification method of the present embodiment can further include: continuously collecting the sensor data collected by the target sensor; and storing the collected sensor data to a set buffer.

[0030] Among them, the set buffer includes a first buffer and a second buffer, and the first buffer can be used to store the IMU signal collected by the inertial measurement unit, and the second buffer can be used to store the electromyography signal collected by the electromyography sensor.

[0031] Generally, the size of the first buffer and the second buffer can be the same or different, for example, the first buffer and the second buffer can each accommodate at least 1100 milliseconds of data to ensure that a complete motion event window can be constructed.

[0032] For example, the wearable device can be worn on the user's body, such as the user's wrist, and can continuously receive the IMU signals collected by the inertial measurement unit and the electromyographic signals collected by the electromyographic sensor, and store the received IMU signals in a first buffer in a first-in-first-out order, and store the received electromyographic signals in a second buffer in a first-in-first-out order.

[0033] It should be noted that the first buffer can include a first sub-buffer and a second sub-buffer, the first sub-buffer can be used to store, for example, acceleration signals collected by an accelerometer in a first-in-first-out order, and the second sub-buffer can be used to store, for example, angular velocity signals collected by a gyroscope in a first-in-first-out order. The second buffer can include a third sub-buffer and a fourth sub-buffer, the third sub-buffer can be used to store, for example, first electromyographic signals collected by a first electromyographic sensor in a first-in-first-out order, and the fourth sub-buffer can be used to store, for example, second electromyographic signals collected by a second electromyographic sensor in a first-in-first-out order. The first sub-buffer, the second sub-buffer, the third sub-buffer, and the fourth sub-buffer can each accommodate at least 1100 milliseconds of data.

[0034] The motion impact core point refers to a key moment in a motion event that plays a decisive role in the motion event. In the field of sports training, such as table tennis, the motion impact core point can refer to the moment of hitting the ball. Locking the sensor data corresponding to the motion event through the motion impact core point of the motion event can ensure that different sensor data are aligned in time.

[0035] In one embodiment of the present disclosure, the step S2100 of detecting sensor data collected by a target sensor in real time to obtain a motion impact core point of a current motion event can further include the following steps S2110-S2130: Step S2110, detecting the acceleration signal in real time to obtain a jerk value of the acceleration signal.

[0036] The jerk of the acceleration signal can refer to the first derivative of the acceleration signal along the time axis, which can effectively represent the instantaneous change rate of the impact force. The moment of hitting the ball is usually the moment when the jerk is the largest.

[0037] Step S2120, accumulating and summing the jerk value of the acceleration signal within a set third time window to obtain a cumulative sum signal within the set third time window.

[0038] The third time window can be a value set in advance according to actual scenarios and experience, for example, 30 ms. The third time window can be set relatively short, so as to obtain the cumulative sum signal by accumulating and summing the jerk value in a very short time window. The cumulative sum signal can present a very sharp and significant local extreme value when the impact event occurs. The high jerk signal generated by the impact event can be effectively amplified through the cumulative sum signal, so that the instantaneous event such as the ping-pong ball hitting can be accurately captured.

[0039] In step S2130, the motion impact core point of the current motion event is determined according to the cumulative sum signal.

[0040] Optionally, the step S2130 of determining the motion impact core point of the current motion event according to the cumulative sum signal can further include: continuously detecting whether the local extreme value point of the cumulative sum signal appears; and in a case where the local extreme value point of the cumulative sum signal is detected to appear, determining the motion impact core point of the current motion event according to the local extreme value point.

[0041] In one example, in a case where the local extreme value point of the cumulative sum signal is detected to appear, the motion impact core point of the current motion event is determined according to the local extreme value point can further include: in a case where the local extreme value point of the cumulative sum signal is detected to appear, continuously detecting whether a new local extreme value point of the cumulative sum signal appears within a fourth time window; and in a case where the new local extreme value point of the cumulative sum signal is detected to not appear, marking the local extreme value point as the motion impact core point of the current motion event.

[0042] The fourth time window can be a value set in advance according to actual scenarios and experience, for example, 500 ms. The fourth time window can be set to ensure that the set fourth time window can include, for example, the swing and reset actions after the hitting action.

[0043] In the present example, the wearable device continuously detects the cumulative sum signal. In a case where the local extreme value point of the cumulative sum signal is detected to appear, the local extreme value point can be taken as a candidate motion impact core point. After the candidate motion impact core point is detected, the wearable device enters a dynamic confirmation stage to ensure the effectiveness of the motion event and capture the complete action.

[0044] In the dynamic confirmation phase, the wearable device generally continues to detect whether a new local extreme point of the cumulative sum signal appears within a set fourth time window. If no new local extreme point appears, the wearable device takes the candidate motion impact core point as the motion impact core point of the current motion event, so as to determine the sensor data corresponding to the current motion event from the collected sensor data based on the motion impact core point.

[0045] In one example, in the case where a new local extreme point of the cumulative sum signal is detected, the local extreme point is updated according to the new local extreme point, and the step of continuing to detect whether a new local extreme point of the cumulative sum signal appears within a set fourth time window in the case where a local extreme point of the cumulative sum signal is detected is continued to be performed.

[0046] In the present example, as described above, in the dynamic confirmation phase, the wearable device generally continues to detect whether a new local extreme point of the cumulative sum signal appears within a set fourth time window. If a new local extreme point appears, it indicates that the wearable device detects a more significant local extreme point than the current candidate motion impact core point, and the new local extreme point is the real ball hit. The above candidate motion impact core point may be generated due to an unintentional swing (generating a small extreme point) of the athlete before hitting the ball. The wearable device updates the new local extreme point as the candidate motion impact core point, and continues to detect whether a new local extreme point of the cumulative sum signal appears within a set fourth time window until the motion impact core point of the current motion event is determined.

[0047] In step S2200, the sensor data corresponding to the current motion event is determined from the collected sensor data based on the motion impact core point.

[0048] The sensor data corresponding to the current motion event includes sensor data within a set first time window before the motion impact core point, sensor data of the motion impact core point, and sensor data within a set second time window after the motion impact core point. In this way, the motion impact core point and the context before and after the motion impact core point can be completely captured, and a high-quality and complete input is provided for the set motion event classification model.

[0049] In the present embodiment, after the motion impact core point of the current motion event is determined, the wearable device extracts the latest sensor data within a set sixth time window from each buffer as the sensor data corresponding to the current motion event.

[0050] The sixth time window can be set according to actual experience and a scene, and the sensor data in the sixth time window can be composed of the sensor data in the first time window, the sensor data of the motion impact core point, and the sensor data in the second time window. The second time window can be the fourth time window.

[0051] For example, the fourth time window is set to be 500 ms, and the sixth time window is set to be 900 ms. The length of 900 ms is sufficient to ensure that the window can cover the backswing, power preparation action before hitting the ball, and the forward swing, braking and reset action after hitting the ball. In this way, each segmented data window is a context-rich data containing a complete action cycle centered on the hitting moment, which is beneficial to subsequent setting of the motion event classification model for action classification.

[0052] For example, after determining the motion impact core point of the current motion event, the wearable device can intercept the latest 900 ms of IMU signals from the IMU signals stored in the first buffer area up to the current time, and intercept the latest 900 ms of electromyographic signals from the electromyographic signals stored in the second buffer area up to the current time, so as to obtain the sensor data corresponding to the current motion event.

[0053] In step S2300, a category label of the current motion event is obtained according to the sensor data corresponding to the current motion event and the set motion event classification model.

[0054] The category label of the motion event is used to represent the category to which the motion event belongs. In the field of sports training, for example, in table tennis, the category label of the motion event includes, but is not limited to, backhand pull, backhand push, and forehand attack.

[0055] The set motion event classification model can be a convolutional long short-term memory network, which can simultaneously process the time dependence in time series data and the spatial correlation between multiple sensor channels. Generally, the convolutional long short-term memory network includes two or more convolutional long short-term memory layers for automatically extracting deep temporal and spatial features from input data. The convolutional long short-term memory network also includes a batch normalization layer and a random inactivation layer, which can accelerate model convergence and prevent overfitting. The convolutional long short-term memory network also includes a flattening layer and multiple fully connected layers for nonlinear integration and mapping of the extracted temporal and spatial features. The convolutional long short-term memory network also includes an output layer, which adopts a Softmax activation function. The Softmax activation function can output a probability distribution, and the category corresponding to the highest probability value in the probability distribution is the recognition result of the current motion event, for example, the probability of backhand pull is 95%.

[0056] The input of the set motion event classification model can be sensor data corresponding to the current motion event, and the output of the set motion event classification model can be a category label of the current motion event.

[0057] For example, the wearable device inputs IMU data and electromyography data corresponding to the current motion event into the set motion event classification model, and the set motion event classification model can obtain the category label of the current motion event based on the IMU data and the electromyography data corresponding to the current motion event.

[0058] According to the embodiments of the present disclosure, the sensor data corresponding to the current motion event is determined by detecting the motion impact core point in real time, which can ensure that the sensor data collected by the plurality of sensors is accurately aligned in time, and the sensor data corresponding to the current motion event includes sensor data within a set first time window before the motion impact core point, sensor data of the motion impact core point, and sensor data within a set second time window after the motion impact core point, which can completely capture the motion impact core point and the context before and after the motion impact core point, provide high-quality and complete input for the set motion event classification model, improve the classification accuracy of the set motion event classification model, and further, only extract and classify the sensor data when the motion impact core point is detected, avoid continuous calculation of a large amount of invalid sensor data, and reduce the power consumption and computing power consumption of the wearable device.

[0059] In one embodiment, after the sensor data corresponding to the current motion event is determined from the collected sensor data according to the motion impact core point in the above step S2200, the method further includes updating the sensor data stored in the set buffer to retain the sensor data within a set fifth time window.

[0060] The set fifth time window can be a value set according to actual scenarios and experience, and for example, the set fifth time window can be 400 ms.

[0061] In the present embodiment, after the wearable device successfully generates and outputs the sensor data of the current motion event, the wearable device updates the sensor data in the buffer in order to prepare for the next detection and ensure smooth transition between windows. Instead of completely emptying, only the latest 400 ms of sensor data is retained. This overlapping retention mechanism can effectively prevent the real follow-up shot event from being completely missed due to accidental false triggering of the previous event, greatly improving the robustness of the device. The buffer overlapping update strategy enhances the stability and anti-interference ability of the system in continuous and rapid motion.

[0062] In an embodiment, after the above step S2300, the sensor data corresponding to the current motion event and the set motion event classification model are executed to obtain the category label of the current motion event, the motion event classification method of the embodiments of the present disclosure further comprises: obtaining the evaluation value of the set motion event classification model according to the category label of the current motion event and the actual category label of the current motion event, and updating the set motion event classification model according to the evaluation value, so that the classification result of the set motion event classification model is more accurate.

[0063] <Example> Next, an example of a motion event classification method is shown, referring to Figure 3 The motion event classification method can include: Step 301, collecting sensor data and storing it in a set buffer.

[0064] The data collection and buffer module of the wearable device continuously collects the IMU signals collected by the inertial measurement unit and the electromyographic signals collected by the electromyographic sensor, stores the collected IMU signals in the first buffer according to the first-in first-out principle, and stores the collected electromyographic signals in the second buffer according to the first-in first-out principle.

[0065] The IMU signal can include three-axis acceleration signals and three-axis angular velocity signals.

[0066] Step 302, real-time detection of acceleration signals to obtain the jerk value of the acceleration signals.

[0067] Step 303, cumulative summation of the jerk value of the acceleration signals within a set third time window to obtain the cumulative sum signal within the set third time window.

[0068] Step 304, continuously detecting whether the cumulative sum signal appears a local extreme point, if a local extreme point appears, executing the following step 305, otherwise, continuing to execute the present step 304.

[0069] For the above steps 302~304, the motion event real-time segmentation module of the wearable device detects the tri-axial acceleration signal in real time, and calculates the first derivative of the tri-axial acceleration signal along the time axis, which can effectively represent the instantaneous change rate of the impact force. In order to amplify the instantaneous signal response generated by physical impact (for example, the collision of a table tennis ball with a racket), the motion event real-time segmentation module can perform cumulative summation on the calculated first derivative within a very short time window (for example, 30 milliseconds) to obtain a cumulative sum signal. This cumulative sum signal will present a very sharp and significant local extreme point when such an impact event occurs. The motion event real-time segmentation module continuously monitors the cumulative sum signal, and once a local extreme point is detected, it is considered that a candidate motion impact core point is captured, and the following step 305 is continued, otherwise, the detection of whether a local extreme point appears in the cumulative sum signal is continued.

[0070] Step 305, the local extreme point is taken as the captured candidate motion impact core point, and a confirmation period is started, that is, the cumulative sum signal is continuously detected within the confirmation period, for example, 500 ms, to see whether a new local extreme point appears, if no new local extreme point appears, the following step 306 is executed, otherwise, the step 305 is continued.

[0071] The step 305 can be executed by the motion event real-time segmentation module of the wearable device.

[0072] Step 306, after the confirmation period ends, the candidate motion impact core point is taken as the motion impact core point of the current motion event, and the latest sensor data within a set sixth time window, for example, 900 ms, ending at the current time is intercepted from the set buffer.

[0073] The step 306 can be executed by the motion event real-time segmentation module of the wearable device.

[0074] For example, after the confirmation period ends, the candidate motion impact core point is taken as the motion impact core point of the current motion event, and the latest IMU data within a set sixth time window, for example, 900 ms, ending at the current time can be intercepted from the first buffer, and the latest electromyography data within a set sixth time window, for example, 900 ms, ending at the current time can be intercepted from the second buffer.

[0075] Step 307, the sensor data corresponding to the current motion event is input into a set motion event classification model to obtain a category label of the current motion event.

[0076] According to the example, in one aspect, the segmentation accuracy and data integrity are improved by anchoring the data window by detecting the motion impact core point in real time, ensuring accurate alignment of different sensor data in time. Moreover, the dynamic confirmation mechanism can completely capture the action core and the context before and after, providing high-quality and complete information input for subsequent motion event classification model setting, significantly improving the recognition accuracy.

[0077] On the other hand, the system running efficiency and robustness are improved by using an event-driven intelligent segmentation mechanism, which only extracts and classifies the window when valid motion is detected, avoiding continuous calculation of a large amount of invalid data, and greatly reducing system power consumption and computing power consumption.

[0078] <Device Embodiment> Figure 4 is a schematic diagram of the principle of a motion event classification device according to an embodiment, applied to a wearable device, which includes a target sensor, the target sensor including at least a first sensor and a second sensor. Referring to Figure 4 , the motion event classification device 400 includes a detection module 410, a determination module 420, and an acquisition module 430.

[0079] The detection module 410 is configured to detect sensor data collected by the target sensor in real time to obtain a motion impact core point of a current motion event. The determination module 420 is configured to determine sensor data corresponding to the current motion event from the collected sensor data according to the motion impact core point; wherein the sensor data corresponding to the current motion event includes sensor data within a set first time window before the motion impact core point, sensor data of the motion impact core point, and sensor data within a set second time window after the motion impact core point. The acquisition module 430 is configured to obtain a class label of the current motion event according to the sensor data corresponding to the current motion event and a set motion event classification model.

[0080] In one embodiment, the first sensor includes an inertial measurement unit, and the sensor data includes an IMU signal, the IMU signal including an acceleration signal. The detection module 410 is specifically configured to detect the acceleration signal in real time to obtain a jerk value of the acceleration signal; accumulate and sum the jerk value of the acceleration signal within a set third time window to obtain a cumulative sum signal within the set first third time window; and determine the motion impact core point of the current motion event according to the cumulative sum signal.

[0081] In one embodiment, the detection module 410 is specifically configured to continuously detect whether the cumulative sum signal has a local extreme point; and in a case where it is detected that the cumulative sum signal has a local extreme point, determine a motion impact core point of the current motion event according to the local extreme point. In one embodiment, the detection module 410 is specifically configured to, in a case where it is detected that the cumulative sum signal has a local extreme point, continuously detect whether the cumulative sum signal has a new local extreme point within a set fourth time window; and in a case where it is detected that the cumulative sum signal does not have a new local extreme point, mark the local extreme point as the motion impact core point of the current motion event. In one embodiment, the detection module 410 is specifically configured to, in a case where it is detected that the cumulative sum signal has a new local extreme point, update the local extreme point according to the new local extreme point, and continue to perform the step of continuously detecting whether the cumulative sum signal has a new local extreme point within a set fourth time window in a case where it is detected that the cumulative sum signal has a local extreme point. In one embodiment, the motion event classification apparatus 400 further comprises a collection module (not shown in the figure) configured to continuously collect sensor data collected by the target sensor; and store the collected sensor data in a set buffer according to a first-in-first-out order. In one embodiment, the motion event classification apparatus 400 further comprises an updating module (not shown in the figure) configured to, after the determination module 420 determines the sensor data corresponding to the current motion event from the collected sensor data according to the motion impact core point, update the sensor data stored in the set buffer to retain the latest sensor data within a set fifth time window. According to embodiments of the present disclosure, the motion impact core point is determined by real-time detection, and the sensor data corresponding to the current motion event is determined according to the motion impact core point, which can ensure that the sensor data collected by the plurality of sensors is accurately aligned in time, and the sensor data corresponding to the current motion event includes sensor data within a set first time window before the motion impact core point, sensor data of the motion impact core point, and sensor data within a set second time window after the motion impact core point, which can completely capture the motion impact core point and the context before and after the motion impact core point, provide a high-quality and complete input for the set motion event classification model, improve the classification accuracy of the set motion event classification model, and further, only extract and classify the sensor data when the motion impact core point is detected, which avoids continuous calculation of a large amount of invalid sensor data, and reduces the power consumption and computing power consumption of the wearable device.

[0082] <Device Embodiment> Figure 5is a hardware structure schematic diagram of a wearable device according to an embodiment. As shown in Figure 5 The wearable device 1000 includes a processor 1100 and a memory 1200.

[0083] The memory 1200 can be used to store executable computer instructions.

[0084] The processor 1100 can be used to execute the motion event classification method according to the embodiments of the method of the present disclosure according to the control of the executable computer instructions.

[0085] The wearable device 1000 can be the wearable device 1000 as shown in Figure 1

[0086] In another embodiment, the wearable device 1000 can include the above motion event classification apparatus 400.

[0087] In an embodiment, each module of the above motion event classification apparatus 400 can be implemented by the processor 1100 running the computer instructions stored in the memory 1200.

[0088] <Computer readable storage medium> The embodiments of the present disclosure further provide a computer readable storage medium, having stored thereon computer instructions, which when executed by a processor perform the motion event classification method provided by the embodiments of the present disclosure.

[0089] The present disclosure can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions embodied therewith, which when executed by a processor, implement various aspects of the present disclosure.

[0090] ​Computer readable storage media can be tangible storage media which can retain and store instructions for use by an instruction execution device. Computer readable storage media can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer readable storage media include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0091] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0092] Computer readable program instructions for carrying out operations of the present disclosure can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0093] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0094] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other data storage device. When the computer readable program instructions are loaded into the computer and other programmable data processing apparatus, a series of operational steps are implemented that provide processes such that the instructions which execute on the computer or other programmable data processing apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0095] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0096] The flow diagrams and the block diagrams in the drawings are presented to illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logic functions. In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0097] Embodiments of the present disclosure have been described above, and the description is intended to be illustrative of the embodiments and not restrictive of the disclosure. Many modifications and variations of the described embodiments are possible and will be apparent to those of ordinary skill in the art. The use of the terms "including", "containing", "comprising", "having" and the like are meant to be inclusive in a manner similar to the term "comprising", and are intended to mean that anything disclosed can be included in the embodiments without limitation to a specific number of instances and further is not meant to exclude or eliminate other moieties, constituents, elements, items and / or steps. The selection of the terms "first", "second" and like terms is not intended to imply that one came before or after the other. It is therefore intended that the disclosure be interpreted by the scope of the appended claims.

Claims

1. A method of classifying a motion event, characterized by, The method is applied to a wearable device including a target sensor, the target sensor including at least a first sensor and a second sensor, and the method includes: detecting sensor data collected by the target sensor in real time to obtain a motion impact core point of a current motion event; determining, according to the motion impact core point, sensor data corresponding to the current motion event from the collected sensor data; wherein the sensor data corresponding to the current motion event includes sensor data within a set first time window before the motion impact core point, sensor data of the motion impact core point, and sensor data within a set second time window after the motion impact core point; obtaining a category label of the current motion event according to the sensor data corresponding to the current motion event and a set motion event classification model.

2. The method of claim 1, wherein, The first sensor includes an inertial measurement unit, and the sensor data includes an IMU signal, and the IMU signal includes an acceleration signal, detecting sensor data collected by the target sensor in real time to obtain a motion impact core point of a current motion event includes: detecting the acceleration signal in real time to obtain a jerk value of the acceleration signal; cumulatively summing the jerk value of the acceleration signal within a set third time window to obtain a cumulative sum signal within the set third time window; determining the motion impact core point of the current motion event according to the cumulative sum signal.

3. The method of claim 2, wherein, Determining the motion impact core point of the current motion event according to the cumulative sum signal includes: continuously detecting whether a local extreme point of the cumulative sum signal appears; in a case where it is detected that the local extreme point of the cumulative sum signal appears, determining the motion impact core point of the current motion event according to the local extreme point.

4. The method of claim 3, wherein, In a case where it is detected that the local extreme point of the cumulative sum signal appears, determining the motion impact core point of the current motion event according to the local extreme point includes: in a case where it is detected that the local extreme point of the cumulative sum signal appears, continuously detecting whether a new local extreme point of the cumulative sum signal appears within a set fourth time window; in a case where it is detected that the new local extreme point of the cumulative sum signal does not appear, marking the local extreme point as the motion impact core point of the current motion event.

5. The method of claim 4, wherein, The method further includes: in a case where it is detected that the new local extreme point of the cumulative sum signal appears, updating the local extreme point according to the new local extreme point, and continuing to perform the step of continuously detecting whether a new local extreme point of the cumulative sum signal appears within a set fourth time window in a case where it is detected that the local extreme point of the cumulative sum signal appears.

6. The method of claim 1, wherein, Before the step of detecting sensor data collected by the target sensor in real time, the method further includes: continuously collecting sensor data collected by the target sensor; and storing the collected sensor data to a set buffer area in a first-in-first-out order.

7. The method of claim 6, wherein, After the step of determining, according to the motion impact core point, sensor data corresponding to the current motion event from the collected sensor data, the method further includes: The sensor data stored in the setting buffer is updated to retain the latest sensor data within a set fifth time window.

8. A sports event classification apparatus characterized by comprising: The application is applied to a wearable device, the wearable device comprises a target sensor, the target sensor at least comprises a first sensor and a second sensor, and the device comprises: A detection module is configured to detect sensor data collected by the target sensor in real time to obtain a motion impact core point of a current motion event. A determination module is configured to determine sensor data corresponding to the current motion event from the collected sensor data according to the motion impact core point, wherein the sensor data corresponding to the current motion event comprises sensor data within a set first time window before the motion impact core point, sensor data of the motion impact core point, and sensor data within a set second time window after the motion impact core point. An acquisition module is configured to obtain a category label of the current motion event according to the sensor data corresponding to the current motion event and a set motion event classification model.

9. A wearable device, comprising: The wearable device comprises: A memory is configured to store executable computer instructions. A processor is configured to execute the method according to any one of claims 1-7 under the control of the executable computer instructions.

10. A computer-readable storage medium, characterized in that, A computer program product is provided, and a computer program product comprises a computer readable storage medium storing computer instructions, wherein the computer instructions are executed by a processor to perform the method according to any one of claims 1-7.

Images