Method, apparatus, device and medium for sports event classification

Abstract

This disclosure provides a method, apparatus, device, and medium for classifying motion events, applied to wearable devices. The wearable device includes a target sensor, which includes at least a first sensor and a second sensor. The method includes: real-time detection of sensor data collected by the target sensor to obtain the core impact point of the current motion event; determining sensor data corresponding to the current motion event from the collected sensor data based on the core impact point; wherein the sensor data corresponding to the current motion event includes sensor data within a predetermined first time window before the core impact point, sensor data at the core impact point, and sensor data within a predetermined second time window after the core impact point; and obtaining a category label for the current motion event based on the sensor data corresponding to the current motion event and a predetermined motion event classification model.

Description

Technical Field

[0001] This disclosure relates to the field of wearable device technology, and more specifically, to a method, apparatus, device, and medium for classifying motion events. Background Technology

[0002] Currently, wearable devices have become one of the most frequently used devices in the field of sports training for capturing human movements. For example, in the field of sports training, wearable devices can cut the data streams collected by different sensor channels into fixed-length data windows in real time and input them into a trained motion event classification model so that the classification results of motion events can be predicted through the trained motion event classification model.

[0003] However, the fixed data window not only causes the data streams collected by different sensor channels to be incorrectly aligned and segmented in time, making it impossible for any data window to capture the complete motion event, thus affecting the classification accuracy of the motion event classification model, but also causes the motion event classification model to continue to perform invalid calculations even when the user is in a resting or meaningless state, resulting in a huge waste of resources and a heavy burden on computing power. Summary of the Invention

[0004] The purpose of this disclosure is to provide a method, apparatus, device, and medium for classifying motion events.

[0005] According to a first aspect of the present disclosure, a motion event classification method is provided, applied to a wearable device, the wearable device including a target sensor, the target sensor including at least a first sensor and a second sensor, the method comprising:

[0006] Real-time detection of sensor data collected by the target sensor to obtain the core point of motion impact of the current motion event;

[0007] Based on the core point of motion impact, sensor data corresponding to the current motion event is determined from the collected sensor data; wherein, the sensor data corresponding to the current motion event includes sensor data within a set first time window before the core point of motion impact, sensor data at the core point of motion impact, and sensor data within a set second time window after the core point of motion impact.

[0008] Based on the sensor data corresponding to the current motion event and the established motion event classification model, the category label of the current motion event is obtained.

[0009] Optionally, the first sensor includes an inertial measurement unit (IMU), and the sensor data includes IMU signals, which include acceleration signals.

[0010] The real-time detection of sensor data collected by the target sensor to obtain the core point of motion impact of the current motion event includes:

[0011] The acceleration signal is detected in real time to obtain the jerk value of the acceleration signal;

[0012] The jerk value of the acceleration signal is accumulated and summed within a set third time window to obtain the accumulated sum signal within the set third time window;

[0013] Based on the accumulated signal, the core point of motion impact of the current motion event is determined.

[0014] Optionally, determining the core point of the motion impact of the current motion event based on the accumulated signal includes:

[0015] Continuously monitor whether the accumulated sum signal exhibits local extrema;

[0016] If a local extreme point is detected in the accumulated signal, the motion impact core point of the current motion event is determined based on the local extreme point.

[0017] Optionally, determining the motion impact core point of the current motion event based on the local extreme point when a local extreme point is detected in the accumulated sum signal includes:

[0018] If a local extreme point is detected in the accumulated sum signal, the accumulated sum signal is continuously monitored for new local extreme points within a set fourth time window.

[0019] If no new local extreme point is detected in the accumulated signal, the local extreme point is marked as the motion impact core point of the current motion event.

[0020] Optionally, the method further includes:

[0021] If a new local extremum is detected in the accumulated sum signal, the local extremum is updated according to the new local extremum, and the step of continuously detecting whether a new local extremum is detected in the accumulated sum signal within a set fourth time window continues.

[0022] Optionally, before the real-time detection of the sensor data collected by the target sensor, the method further includes:

[0023] Continuously collect sensor data acquired by the target sensor; and,

[0024] The collected sensor data is stored in a set buffer in a first-in, first-out (FIFO) order.

[0025] Optionally, after determining the sensor data corresponding to the current motion event from the collected sensor data based on the core point of the motion impact, the method further includes:

[0026] The sensor data stored in the set buffer is updated to retain the latest sensor data within the set fifth time window.

[0027] According to a second aspect of the present disclosure, a motion event classification device is provided, applied to a wearable device, the wearable device including a target sensor, the target sensor including at least a first sensor and a second sensor, the device comprising:

[0028] The detection module is used to detect the sensor data collected by the target sensor in real time and obtain the core point of motion impact of the current motion event;

[0029] The determination module is used to determine the sensor data corresponding to the current motion event from the collected sensor data based on the core point of motion impact; wherein, the sensor data corresponding to the current motion event includes sensor data within a set first time window before the core point of motion impact, sensor data at the core point of motion impact, and sensor data within a set second time window after the core point of motion impact.

[0030] The acquisition module is used to obtain the category label of the current motion event based on the sensor data corresponding to the current motion event and the set motion event classification model.

[0031] According to a third aspect of the present disclosure, a wearable device is provided, the wearable device comprising:

[0032] Memory is used to store executable computer instructions;

[0033] A processor, configured to execute the motion event classification method according to the first aspect above, under the control of the executable computer instructions.

[0034] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having computer instructions stored thereon, which, when executed by a processor, perform the motion event classification method described in the first aspect above.

[0035] One beneficial effect of this disclosure is that by determining the sensor data corresponding to the current motion event through real-time detection of the core point of motion impact, it is possible to ensure that the sensor data collected by multiple sensors are accurately aligned in time. Furthermore, the sensor data corresponding to the current motion event includes sensor data within a set first time window before the core point of motion impact, sensor data at the core point of motion impact, and sensor data within a set second time window after the core point of motion impact. This can completely capture the core point of motion impact and its preceding and following context, providing high-quality and complete input for setting the motion event classification model, improving the classification accuracy of the set motion event classification model. In addition, it only extracts and classifies sensor data when the core point of motion impact is detected, avoiding continuous calculation of a large amount of invalid sensor data and reducing the power consumption and computing power consumption of the wearable device.

[0036] Other features and advantages of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of this specification.

[0038] Figure 1 This is a schematic diagram of the hardware configuration of a wearable device to which the methods provided in the embodiments of this disclosure can be applied;

[0039] Figure 2 This is a flowchart illustrating the motion event classification method provided in this embodiment of the disclosure;

[0040] Figure 3 This is a flowchart illustrating another motion event classification method provided in this embodiment of the disclosure;

[0041] Figure 4 This is a block diagram of the motion event classification device provided in the embodiments of this disclosure;

[0042] Figure 5 This is a block diagram of a wearable device provided in an embodiment of this disclosure. Detailed Implementation

[0043] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the embodiments of the present disclosure.

[0044] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0045] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0046] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0047] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0048] <Hardware Configuration>

[0049] Figure 1 This is a block diagram of the hardware configuration of a wearable device 1000 according to an embodiment of the present disclosure.

[0050] like Figure 1 As shown, the wearable device 1000 may be, for example, a smartwatch, a smart bracelet, etc., and this embodiment of the present disclosure does not limit it.

[0051] In one embodiment, such as Figure 1 As shown, the wearable device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, an output device 1500, an input device 1600, an audio device 1700, a first sensor 1800, and a second sensor 1900, etc.

[0052] The processor 1100 may include, but is not limited to, a central processing unit (CPU) or a microprocessor (MCU). The memory 1200 may include, for example, ROM (Read-Only Memory), RAM (Random Access Memory), or non-volatile memory such as a hard disk. The interface device 1300 may include, for example, various bus interfaces, such as serial bus interfaces (including USB interfaces), parallel bus interfaces, etc. The communication device 1400 may be capable of wired or wireless communication. The output device 1500 may include at least one of a display screen, a vibrator, and a buzzer. The input device 1600 may include, for example, a touchscreen or motion-sensing input. The audio device 1700 may be used for inputting / outputting voice information. The first sensor 1800 may include an inertial measurement unit (IMU), which may include an accelerometer (ACC) and a gyroscope (GYRO), wherein the accelerometer may be a three-axis accelerometer and the gyroscope may be a three-axis angular velocity meter. The second sensor 1900 may include an electromyography (EMG) sensor.

[0053] Those skilled in the art should understand that, although in Figure 1 The wearable device 1000 is shown in the specification, but the wearable device 1000 of the embodiments herein may only involve some of the devices, or may also include other devices, which is not limited here.

[0054] In this embodiment, the memory 1200 of the wearable device 1000 is used to store instructions that control the processor 1100 to operate in order to implement or support the implementation of the motion event classification method according to any embodiment. Those skilled in the art can design instructions based on the schemes disclosed in this specification. How the instructions control the processor to operate is well known in the art and will not be described in detail here.

[0055] In the above description, those skilled in the art can design instructions based on the solutions provided in this disclosure. How the instructions control the processor to operate is well known in the art, and therefore will not be described in detail here.

[0056] Figure 1 The wearable devices shown are for illustrative purposes only and are not intended to limit this disclosure, its application, or its use.

[0057] <Method Implementation>

[0058] Figure 2 An embodiment of the motion event classification method of this disclosure is illustrated. This motion event classification method can be implemented by a wearable device, which may include a target sensor, which includes at least a first sensor and a second sensor. The wearable device may be... Figure 1 The wearable device 1000 shown is an example. Figure 2 As shown, the motion event classification method may include the following steps S2100 to S2300:

[0059] Step S2100: Real-time detection of sensor data collected by the target sensor to obtain the core point of motion impact of the current motion event.

[0060] The target sensor includes at least a first sensor and a second sensor, which are sensors of different types. The first sensor may include an inertial measurement unit (IMU), which may include an accelerometer and a gyroscope, wherein the accelerometer may be a three-axis accelerometer and the gyroscope may be a three-axis angular velocity meter. The second sensor may include an electromyography (EMG) sensor, for example, including both a first EMG sensor and a second EMG sensor.

[0061] The sensor data acquired by the target sensor includes at least sensor data acquired by a first sensor and sensor data acquired by a second sensor. The sensor data acquired by the first sensor may include IMU signals acquired by an inertial measurement unit, such as acceleration signals acquired by an accelerometer and angular velocity signals acquired by a gyroscope. The acceleration signals may be, for example, triaxial acceleration signals, and the angular velocity signals may be, for example, triaxial angular velocity signals. The sensor data acquired by the second sensor may include electromyographic (EMG) signals acquired by an EMG sensor, such as a first EMG signal acquired by a first EMG sensor and a second EMG signal acquired by a second EMG sensor.

[0062] In this embodiment, before performing step S2100 to detect the sensor data collected by the target sensor in real time, the motion event classification method of this embodiment may further include: continuously collecting the sensor data collected by the target sensor; and storing the collected sensor data in a set buffer.

[0063] The buffer includes a first buffer and a second buffer. The first buffer can be used to store IMU signals acquired by the inertial measurement unit, and the second buffer can be used to store electromyographic signals acquired by the electromyography sensor.

[0064] Typically, the first and second buffers can be the same size or different. For example, both the first and second buffers can hold at least 1100 milliseconds of data to ensure that a complete motion event window can be constructed.

[0065] For example, the wearable device can be worn on the user's body, such as the user's wrist, and can continuously receive IMU signals collected by the inertial measurement unit and electromyographic signals collected by the electromyographic sensor. The received IMU signals are stored in a first buffer in a first-in-first-out order, and the received electromyographic signals are stored in a second buffer in a first-in-first-out order.

[0066] It should be noted that the first buffer may include a first sub-buffer and a second sub-buffer. The first sub-buffer can be used to store, for example, acceleration signals acquired by an accelerometer in a first-in-first-out (FIFO) order, and the second sub-buffer can be used to store, for example, angular velocity signals acquired by a gyroscope in a FIFO order. The second buffer may include a third sub-buffer and a fourth sub-buffer. The third sub-buffer can be used to store, for example, a first electromyography (EMG) signal acquired by a first EMG sensor in a FIFO order, and the fourth sub-buffer can be used to store, for example, a second EMG signal acquired by a second EMG sensor in a FIFO order. Each of the first, second, third, and fourth sub-buffers can hold at least 1100 milliseconds of data.

[0067] The impact core point refers to the critical moment in a motion event that plays a decisive role in that event. In sports training, such as table tennis, the impact core point can be the instant the ball is struck. Locating the sensor data corresponding to a motion event by identifying its impact core point ensures temporal alignment of data from different sensors.

[0068] In one embodiment of this disclosure, step S2100, which involves real-time detection of sensor data collected by the target sensor to obtain the core point of motion impact in the current motion event, may further include the following steps S2110 to S2130:

[0069] Step S2110: Detect the acceleration signal in real time and obtain the jerk value of the acceleration signal.

[0070] The jerkiness of the acceleration signal can refer to the first derivative of the acceleration signal along the time axis, which can effectively characterize the instantaneous rate of change of the impact force. The moment of impact is usually the moment with the greatest jerkiness.

[0071] Step S2120: Accumulate and sum the jerk values ​​of the acceleration signal within a set third time window to obtain the accumulated sum signal within the set third time window.

[0072] The third time window can be a pre-set value based on the actual scenario and experience. For example, the third time window could be 30ms. The third time window is usually set relatively short to accumulate and sum the jerk values ​​within a very short time window to obtain an accumulated sum signal. This accumulated sum signal can exhibit a very sharp and significant local extremum when an impact event occurs. Therefore, this accumulated sum signal can effectively amplify the high jerk signal truly generated by the impact event, thus enabling precise capture of instantaneous events such as the impact of a ping-pong ball.

[0073] Step S2130: Determine the core point of the motion impact of the current motion event based on the accumulated signal.

[0074] Optionally, step S2130, which determines the core point of motion impact of the current motion event based on the accumulated sum signal, may further include: continuously detecting whether the accumulated sum signal has a local extreme point; and if the accumulated sum signal has a local extreme point, determining the core point of motion impact of the current motion event based on the local extreme point.

[0075] In one example, when a local extreme point is detected in the accumulated sum signal, determining the motion impact core point of the current motion event based on the local extreme point may further include: when a local extreme point is detected in the accumulated sum signal, continuously detecting whether a new local extreme point appears in the accumulated sum signal within a set fourth time window; when no new local extreme point is detected in the accumulated sum signal, marking the local extreme point as the motion impact core point of the current motion event.

[0076] The fourth time window can be a value preset based on the actual scenario and experience. For example, the fourth time window can be 500ms to ensure that the set fourth time window can include, for example, the swing action and reset action that follow the hitting action.

[0077] In this example, the wearable device continuously monitors the aforementioned cumulative signal. If a local extreme point is detected in the cumulative signal, this local extreme point can be considered as a candidate motion impact core point. After detecting the candidate motion impact core point, the wearable device enters a dynamic verification phase to ensure the validity of the motion event and capture the complete action.

[0078] During the dynamic confirmation phase, the wearable device typically continuously detects whether a new local extreme point appears in the accumulated signal within a set fourth time window. If no new local extreme point appears, the wearable device will take the candidate motion impact core point as the motion impact core point of this motion event, so as to determine the sensor data corresponding to the current motion event from the collected sensor data based on the motion impact core point.

[0079] In one example, if a new local extremum is detected in the accumulated sum signal, the local extremum is updated according to the new local extremum, and the step of continuously detecting whether a new local extremum is detected in the accumulated sum signal within a set fourth time window continues.

[0080] In this example, as described above, during the dynamic confirmation phase, the wearable device typically continuously detects whether a new local extreme point appears in the accumulated sum signal within a set fourth time window. If a new local extreme point appears, it indicates that the wearable device has detected a more significant local extreme point than the current candidate motion impact core point. This new local extreme point is the actual hit. The aforementioned candidate motion impact core point may be caused by an unintentional tremor (generating a small extreme point) before the athlete hits the ball. The wearable device will then update the new local extreme point as the candidate motion impact core point and continue to continuously detect whether a new local extreme point appears in the accumulated sum signal within the set fourth time window until the motion impact core point of this motion event is determined.

[0081] Step S2200: Based on the core point of the motion impact, determine the sensor data corresponding to the current motion event from the collected sensor data.

[0082] The sensor data corresponding to the current motion event includes sensor data within a set first time window before the core point of motion impact, sensor data at the core point of motion impact, and sensor data within a set second time window after the core point of motion impact. This can completely capture the core point of motion impact and its preceding and following context, providing high-quality and complete input for setting the motion event classification model.

[0083] In this embodiment, after the wearable device determines the core point of the motion impact of the current motion event, it will extract the latest sensor data within the set sixth time window up to the current moment from each buffer and use it as the sensor data corresponding to the current motion event.

[0084] The sixth time window can be set based on practical experience and the scenario, and the sensor data within the sixth time window can consist of the sensor data from the first time window, the sensor data from the core point of the motion impact, and the sensor data from the second time window. The second time window can be the fourth time window mentioned above.

[0085] For example, the fourth time window is set to 500ms, and the sixth time window to 900ms. The length of 900ms is sufficient to ensure that the window can cover the backswing and power preparation before the shot, and the follow-through, braking, and recovery after the shot. In this way, each segmented data window is a context-rich data point centered on the moment of the shot, containing the complete motion cycle, which is beneficial for setting up a motion event classification model for motion classification.

[0086] For example, after determining the core point of the motion impact of this motion event, the wearable device can extract the latest 900ms IMU signal up to the current moment from the IMU signal stored in the first buffer, and extract the latest 900ms EMG signal up to the current moment from the EMG signal stored in the second buffer, in order to obtain the sensor data corresponding to this motion event.

[0087] Step S2300: Based on the sensor data corresponding to the current motion event and the set motion event classification model, obtain the category label of the current motion event.

[0088] The category label of a sports event is used to identify the category to which the sports event belongs. In the field of sports training, such as table tennis, the category label of a sports event includes, but is not limited to, backhand topspin, backhand flick, and forehand attack.

[0089] The motion event classification model can be a convolutional long short-term memory (SSM) network, capable of simultaneously handling temporal dependencies and spatial correlations between multiple sensor channels in time-series data. Typically, an SSM network includes two or more SSM layers to automatically extract deep spatiotemporal features from the input data. It also includes batch normalization layers and random deactivation layers to accelerate model convergence and prevent overfitting. Furthermore, it incorporates flattening layers and multiple fully connected layers for nonlinear integration and mapping of the extracted spatiotemporal features. Finally, it includes an output layer that uses the Softmax activation function. The Softmax activation function outputs a probability distribution, where the category corresponding to the highest probability value is the recognition result of the motion event; for example, the probability of a backhand topspin is 95%.

[0090] The input to the motion event classification model can be the sensor data corresponding to the current motion event, and the output of the motion event classification model can be the category label of the current motion event.

[0091] For example, the wearable device inputs the IMU data and electromyography data corresponding to the current motion event into the setting motion event classification model. The setting motion event classification model can obtain the category label of the current motion event based on the IMU data and electromyography data corresponding to the current motion event.

[0092] By using the embodiments of this disclosure, the sensor data corresponding to the current motion event is determined by real-time detection of the core point of motion impact. This ensures that the sensor data collected by multiple sensors are precisely aligned in time. Furthermore, the sensor data corresponding to the current motion event includes sensor data within a set first time window before the core point of motion impact, sensor data at the core point of motion impact, and sensor data within a set second time window after the core point of motion impact. This allows for the complete capture of the core point of motion impact and its preceding and following context, providing high-quality and complete input for setting a motion event classification model and improving the classification accuracy of the model. Moreover, sensor data extraction and classification are performed only when the core point of motion impact is detected, avoiding continuous calculation of a large amount of invalid sensor data and reducing the power consumption and computing power consumption of the wearable device.

[0093] In one embodiment, after performing step S2200 above, which involves determining the sensor data corresponding to the current motion event from the collected sensor data based on the core point of the motion impact, the method further includes: updating the sensor data stored in the set buffer to retain the latest sensor data within the set fifth time window.

[0094] The fifth time window can be set as a value based on the actual scenario and experience. For example, the fifth time window can be set to 400ms.

[0095] In this embodiment, after successfully generating and outputting sensor data for the current motion event, the wearable device updates the sensor data in the buffer to prepare for the next detection and ensure a smooth transition between windows. It does not completely clear the buffer, but only retains, for example, the latest 400ms sensor data. This overlapping retention mechanism effectively prevents subsequent real ball-hitting events from being completely missed due to accidental false triggers from the previous one, greatly improving the robustness of the device. The buffer overlapping update strategy enhances the system's stability and anti-interference capability during continuous rapid movement.

[0096] In one embodiment, after performing the above step S2300 to obtain the category label of the current motion event based on the sensor data corresponding to the current motion event and the set motion event classification model, the motion event classification method of this embodiment further includes: obtaining the evaluation value of the set motion event classification model based on the category label of the current motion event and the actual category label of the current motion event, and updating the set motion event classification model based on the evaluation value, so that the classification result of the set motion event classification model is more accurate.

[0097] <Example>

[0098] The following is an example of a motion event classification method, see reference. Figure 3 The motion event classification method may include:

[0099] Step 301: Collect sensor data and store it in the set buffer.

[0100] The wearable device's data acquisition and buffering module continuously collects IMU signals acquired by the inertial measurement unit and electromyographic signals acquired by the electromyographic sensor. The acquired IMU signals are stored in the first buffer in a first-in-first-out (FIFO) manner, and the acquired electromyographic signals are stored in the second buffer in a first-in-first-out (FIFO) manner.

[0101] The IMU signal can include triaxial acceleration signals and triaxial angular velocity signals.

[0102] Step 302: Real-time detection of acceleration signal to obtain jerk value of acceleration signal.

[0103] Step 303: Accumulate and sum the jerk values ​​of the acceleration signal within a set third time window to obtain the accumulated sum signal within the set third time window.

[0104] Step 304: Continuously detect whether the accumulated sum signal has a local extreme point. If a local extreme point is found, proceed to step 305. Otherwise, continue to execute step 304.

[0105] For steps 302-304 above, the real-time motion event segmentation module of the wearable device detects the triaxial acceleration signal in real time and calculates the first derivative of the triaxial acceleration signal along the time axis. This jerk value can effectively characterize the instantaneous rate of change of the impact force. In order to amplify the instantaneous signal response generated by physical impact (such as the collision of a ping-pong ball and a racket), the real-time motion event segmentation module can accumulate and sum the calculated first derivative within a very short time window (e.g., 30 milliseconds) to obtain an accumulated sum signal. This accumulated sum signal will present a very sharp and significant local extremum point when such an impact event occurs. The real-time motion event segmentation module continuously monitors this accumulated sum signal. Once a local extremum point is detected, it is considered that a candidate motion impact core point has been captured, and the next step 305 is continued. Otherwise, the detection of whether a local extremum point appears in the accumulated sum signal continues.

[0106] Step 305: Use the local extreme point as the candidate motion impact core point for capture, and start the confirmation period, that is, continuously detect whether a new local extreme point appears in the accumulated signal within the confirmation period, for example, 500ms. If no new local extreme point appears, proceed to step 306; otherwise, continue to execute step 305.

[0107] Step 305 can be executed through the motion event real-time segmentation module of the wearable device.

[0108] Step 306: After the confirmation period ends, the candidate motion impact core point is taken as the motion impact core point of the current motion event, and the latest sensor data within 900ms up to the current moment is extracted from the set buffer.

[0109] This step 306 can be executed through the motion event real-time segmentation module of the wearable device.

[0110] For example, after the confirmation period ends, the candidate motion impact core point is taken as the motion impact core point of the current motion event, and the latest set sixth time window, such as 900ms, can be extracted from the first buffer, and the latest set sixth time window, such as 900ms, can be extracted from the second buffer.

[0111] Step 307: Input the sensor data corresponding to the current motion event into the set motion event classification model to obtain the category label of the current motion event.

[0112] Based on this example, on the one hand, it improves segmentation accuracy and data integrity by anchoring the data window through real-time detection of the core point of motion impact, ensuring precise temporal alignment of data from different sensors. Furthermore, its dynamic confirmation mechanism can fully capture the core of the action and its preceding and following context, providing high-quality, complete input for subsequent motion event classification model design, significantly improving recognition accuracy.

[0113] On the other hand, it improves system operating efficiency and robustness by adopting an event-driven intelligent segmentation mechanism, which only extracts and classifies windows when valid motion is detected, avoiding continuous calculation of a large amount of invalid data and significantly reducing system power consumption and computing power consumption.

[0114] <Device Embodiment>

[0115] Figure 4 This is a schematic diagram of a motion event classification device according to one embodiment, applied to a wearable device. The wearable device includes a target sensor, which includes at least a first sensor and a second sensor. (See reference...) Figure 4 As shown, the motion event classification device 400 includes a detection module 410, a determination module 420, and an acquisition module 430.

[0116] The detection module 410 is used to detect the sensor data collected by the target sensor in real time and obtain the core point of motion impact of the current motion event.

[0117] The determining module 420 is used to determine the sensor data corresponding to the current motion event from the collected sensor data based on the core point of motion impact; wherein, the sensor data corresponding to the current motion event includes sensor data within a set first time window before the core point of motion impact, sensor data at the core point of motion impact, and sensor data within a set second time window after the core point of motion impact.

[0118] The acquisition module 430 is used to obtain the category label of the current motion event based on the sensor data corresponding to the current motion event and the set motion event classification model.

[0119] In one embodiment, the first sensor includes an inertial measurement unit, the sensor data includes an IMU signal, the IMU signal includes an acceleration signal, and the detection module 410 is specifically used to detect the acceleration signal in real time and obtain the jerk value of the acceleration signal; accumulate and sum the jerk value of the acceleration signal within a set third time window to obtain the accumulated sum signal within the set first and third time windows; and determine the motion impact core point of the current motion event based on the accumulated sum signal.

[0120] In one embodiment, the detection module 410 is specifically used to continuously detect whether the accumulated sum signal has a local extreme point; if a local extreme point is detected in the accumulated sum signal, the motion impact core point of the current motion event is determined based on the local extreme point.

[0121] In one embodiment, the detection module 410 is specifically configured to, when a local extreme point is detected in the accumulated sum signal, continuously detect whether a new local extreme point appears in the accumulated sum signal within a set fourth time window; and when no new local extreme point is detected in the accumulated sum signal, mark the local extreme point as the motion impact core point of the current motion event.

[0122] In one embodiment, the detection module 410 is specifically configured to, upon detecting a new local extreme point in the accumulated sum signal, update the local extreme point based on the new local extreme point, and continue to execute the step of continuously detecting whether a new local extreme point appears in the accumulated sum signal within a set fourth time window when a local extreme point is detected in the accumulated sum signal.

[0123] In one embodiment, the motion event classification device 400 further includes a data acquisition module (not shown) for continuously collecting sensor data acquired by the target sensor and storing the acquired sensor data in a first-in-first-out (FIFO) order into a set buffer.

[0124] In one embodiment, the motion event classification device 400 further includes an update module (not shown in the figure), which is used to update the sensor data stored in the set buffer after the determination module 420 determines the sensor data corresponding to the current motion event from the collected sensor data based on the motion impact core point, so as to retain the latest set fifth time window of sensor data.

[0125] According to the embodiments of this disclosure, by detecting the core point of motion impact in real time to determine the sensor data corresponding to the current motion event, it is possible to ensure that the sensor data collected by multiple sensors are accurately aligned in time. Furthermore, the sensor data corresponding to the current motion event includes sensor data within a set first time window before the core point of motion impact, sensor data at the core point of motion impact, and sensor data within a set second time window after the core point of motion impact. This can completely capture the core point of motion impact and its preceding and following context, providing high-quality and complete input for setting the motion event classification model, improving the classification accuracy of the set motion event classification model. In addition, it only extracts and classifies sensor data when the core point of motion impact is detected, avoiding continuous calculation of a large amount of invalid sensor data and reducing the power consumption and computing power consumption of the wearable device.

[0126] <Equipment Example>

[0127] Figure 5 This is a schematic diagram of the hardware structure of a wearable device according to one embodiment. Figure 5 As shown, the wearable device 1000 includes a processor 1100 and a memory 1200.

[0128] The memory 1200 can be used to store executable computer instructions.

[0129] The processor 1100 can be used to execute the motion event classification method according to the method embodiments of this disclosure, under the control of the executable computer instructions.

[0130] The wearable device 1000 can be as follows: Figure 1 The wearable device 1000 shown.

[0131] In another embodiment, the wearable device 1000 may include the above-mentioned motion event classification device 400.

[0132] In one embodiment, each module of the motion event classification device 400 can be implemented by the processor 1100 running computer instructions stored in the memory 1200.

[0133] Computer-readable storage media

[0134] This disclosure also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, perform the motion event classification method provided in this disclosure.

[0135] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0136] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0137] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0138] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0139] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0140] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0141] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0142] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are equivalent.

[0143] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A method for classifying motion events, characterized in that, The method is applied to a wearable device, the wearable device including a target sensor, the target sensor including at least a first sensor and a second sensor, and includes: Real-time detection of sensor data collected by the target sensor to obtain the core point of motion impact of the current motion event; Based on the core point of motion impact, sensor data corresponding to the current motion event is determined from the collected sensor data; wherein, the sensor data corresponding to the current motion event includes sensor data within a set first time window before the core point of motion impact, sensor data at the core point of motion impact, and sensor data within a set second time window after the core point of motion impact. Based on the sensor data corresponding to the current motion event and the established motion event classification model, obtain the category label of the current motion event; The first sensor includes an inertial measurement unit (IMU), and the sensor data includes an IMU signal, which includes an acceleration signal. Real-time detection of the sensor data collected by the target sensor to obtain the motion impact core point of the current motion event includes: real-time detection of the acceleration signal to obtain the jerk value of the acceleration signal; accumulating and summing the jerk values ​​of the acceleration signal within a set third time window to obtain an accumulated sum signal within the set third time window, wherein the accumulated sum signal is used to exhibit a local extremum when the impact event occurs; continuously detecting whether the accumulated sum signal exhibits a local extremum point; if a local extremum point is detected, the local extremum point is used as a candidate motion impact core point; after detecting a candidate motion impact core point, a dynamic confirmation phase is entered. In the dynamic confirmation phase, the cumulative sum signal is continuously monitored for new local extreme points within a set fourth time window. If no new local extreme point is detected, the local extreme point is marked as the motion impact core point of the current motion event. If a new local extreme point is detected, the local extreme point is updated according to the new local extreme point, and the step of continuously monitoring the cumulative sum signal for new local extreme points within a set fourth time window continues.

2. The method according to claim 1, characterized in that, Before the real-time detection of the sensor data collected by the target sensor, the method further includes: Continuously collect sensor data acquired by the target sensor; and, The collected sensor data is stored in a set buffer in a first-in, first-out (FIFO) order.

3. The method according to claim 2, characterized in that, After determining the sensor data corresponding to the current motion event from the collected sensor data based on the core point of the motion impact, the method further includes: The sensor data stored in the set buffer is updated to retain the latest sensor data within the set fifth time window.

4. A motion event classification device, characterized in that, Applied to wearable devices, the wearable device including a target sensor, the target sensor including at least a first sensor and a second sensor, the device comprising: The detection module is used to detect the sensor data collected by the target sensor in real time and obtain the core point of motion impact of the current motion event; The determination module is used to determine the sensor data corresponding to the current motion event from the collected sensor data based on the core point of motion impact; wherein, the sensor data corresponding to the current motion event includes sensor data within a set first time window before the core point of motion impact, sensor data at the core point of motion impact, and sensor data within a set second time window after the core point of motion impact. The acquisition module is used to obtain the category label of the current motion event based on the sensor data corresponding to the current motion event and the set motion event classification model; The first sensor includes an inertial measurement unit (IMU), and the sensor data includes an IMU signal, which includes an acceleration signal. The detection module is specifically used to detect the acceleration signal in real time and obtain the jerk value of the acceleration signal; to accumulate and sum the jerk values ​​of the acceleration signal within a set third time window to obtain an accumulated sum signal within the set third time window, wherein the accumulated sum signal is used to exhibit a local extremum when an impact event occurs; and to continuously detect whether a local extremum point appears in the accumulated sum signal. If a local extremum point is detected, it is used as a candidate motion impact core point. After detecting a candidate motion impact core point, a dynamic confirmation phase is entered. In the dynamic confirmation phase, the cumulative sum signal is continuously monitored for new local extreme points within a set fourth time window. If no new local extreme point is detected, the local extreme point is marked as the motion impact core point of the current motion event. If a new local extreme point is detected, the local extreme point is updated according to the new local extreme point, and the step of continuously monitoring the cumulative sum signal for new local extreme points within a set fourth time window continues.

5. A wearable device, characterized in that, The wearable device includes: Memory is used to store executable computer instructions; A processor configured to execute the method according to any one of claims 1-3, under the control of the executable computer instructions.

6. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, perform the method described in any one of claims 1-3.

Images