Gait data-based behavior recognition method, device, storage medium and computer program product

Abstract

The application discloses a behavior recognition method and device based on gait data, related equipment, a storage medium and a computer program product. The method comprises the following steps: a wearable device divides gait data collected by the wearable device into a plurality of first gait data based on peak values in the gait data, and each first gait data represents gait data in a gait cycle; and in the case that the plurality of first gait data meets a set condition, the plurality of first gait data is reported to a terminal, and the plurality of first gait data is used for the terminal to perform behavior recognition on a user wearing the wearable device.

Description

Technical Field

[0001] This application relates to the field of wearable device technology, and in particular to a behavior recognition method, device, related equipment, storage medium and computer program product based on gait data. Background Technology

[0002] In related technologies, wearable devices monitor patients' movement status through continuous monitoring or manual wake-up monitoring. However, continuous monitoring is limited by battery capacity, making it difficult to support long-term standby use and thus limiting the duration of continuous monitoring. Summary of the Invention

[0003] To address the related technical problems, embodiments of this application provide a behavior recognition method, apparatus, related devices, storage medium, and computer program product based on gait data.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a behavior recognition method based on gait data, applied to wearable devices, including:

[0006] Based on the peak values ​​in the gait data collected by the wearable device, the gait data collected by the wearable device is divided into multiple first gait data, and each first gait data represents the gait data within a gait cycle.

[0007] If the defined first-step data meets the set conditions, the first-step data is reported to the terminal. The first-step data is used by the terminal to identify the behavior of the user wearing the wearable device.

[0008] In the above scheme, the set conditions include:

[0009] The multiple first-step state data represent the user continuously walking to reach a set number of steps. The set number of steps is represented as the product of 3 and N. N is determined based on the user's walking speed and the maximum number of steps required to complete the set turning path, and N is an integer greater than 1.

[0010] The method in the above scheme further includes:

[0011] Detect whether the user is walking and obtain the detection result;

[0012] If the detection result indicates that the user is in a walking state, the collected gait data is saved to segment the first gait data.

[0013] The method in the above scheme further includes:

[0014] The first sensing data collected by the multiple sensors in the wearable device are normalized to the quaternion fusion coordinate system under the Earth coordinate system to obtain the second sensing data corresponding to the multiple sensors respectively.

[0015] The second sensing data corresponding to the multiple sensors are fused and corrected to obtain the gait data collected by the wearable device.

[0016] The method in the above scheme further includes:

[0017] Gait data acquisition is triggered at set time intervals; where...

[0018] The set time interval is determined based on the user's medication time, gait data collection time, and corresponding gait data within a historical time period.

[0019] This application also provides a behavior recognition method based on gait data, applied to a terminal, including:

[0020] Receive multiple first step gait data reported by a wearable device, wherein each first step gait data represents gait data within a gait cycle;

[0021] Determine whether the plurality of first step gait data contains a first data segment to obtain a first determination result. The first data segment represents the gait data of the user wearing the wearable device during the process of traversing a complete set turning path.

[0022] If the first judgment result indicates that the first data segment is included in the plurality of first step gait data, then identify the normal gait data and / or abnormal gait data in the first data segment.

[0023] In the above scheme, the multiple first-step state data satisfy the set conditions, which include: the multiple first-step state data represent that the user has walked continuously to reach a set number of steps, the set number of steps is represented as the product of 3 and N, N is determined based on the user's walking speed and the maximum number of steps required to complete the set turning path, and N is an integer greater than 1.

[0024] In the above scheme, determining whether the plurality of first-step state data contains a first data segment includes:

[0025] Based on the first step state data of the first stage and the first step state data of the second stage, it is determined whether the user walks in a straight line in the first stage and the second stage, and a second determination result is obtained. The first step state data of the first stage is the k-3Nth to k-2N-1th gait data among the plurality of first step state data, and the first step state data of the second stage is the k-2Nth to kN-1th gait data among the plurality of first step state data. k is the number of the plurality of first step state data, and k is an integer greater than 3N.

[0026] When the second judgment result indicates that the user walks in a straight line in both the first stage and the second stage, based on the angle between the straight line the user walks in the first stage and the straight line the user walks in the second stage, it is determined that the first data segment is included in the first step state data of the third stage, wherein the first step state data of the third stage is the kNth to kth gait data among the plurality of first step state data.

[0027] The method in the above scheme further includes:

[0028] Determine the set time interval; where,

[0029] The set time interval is used by the wearable device to trigger the collection of gait data; the set time interval is determined based on the user's medication time, gait data collection time and corresponding gait data in a historical period.

[0030] The method in the above scheme further includes:

[0031] M first-step data groups are sent to the server, wherein each of the M first-step data groups includes P second-step data groups reported by the wearable device for each of the M consecutive days, and each second-step data group includes the plurality of first-step data groups, wherein M and P are both integers greater than or equal to 1.

[0032] This application also provides a behavior recognition method based on gait data, applied to a server, including:

[0033] Get M first-step data groups sent by the terminal, wherein the M first-step data groups each include P second-step data groups reported by the wearable device for each of the M consecutive days, each second-step data group includes the plurality of first-step data, and M and P are both integers greater than or equal to 1;

[0034] Based on the M first step gait data sets, the gait data corresponding to the M days and / or the gait data corresponding to each day in the M days are checked to see if they are normal, and the detection results are obtained.

[0035] In the above scheme, based on the M first step gait data groups, detecting whether the gait data corresponding to each day of the M days is normal includes:

[0036] Calculate the first-order forward difference between gait data from adjacent collection times in the gait data for each day to obtain the first calculation result;

[0037] The second calculation result is obtained by calculating the second-order forward difference between multiple first calculation results;

[0038] Based on the obtained second calculation results, it is determined whether the gait data corresponding to each day of the M days is normal.

[0039] In the above scheme, based on the M first step gait data groups, detecting whether the gait data corresponding to the M days is normal includes:

[0040] Calculate the mean of gait data at each identical collection time during the M days;

[0041] Based on the deviation between the gait data collected at the corresponding time each day and the mean, it is determined whether the gait data for the M days is normal.

[0042] This application also provides a behavior recognition device based on gait data, including:

[0043] A segmentation unit is used to divide the gait data collected by the wearable device into multiple first-step gait data based on the peak value in the gait data collected by the wearable device, wherein each first-step gait data represents the gait data within a gait cycle.

[0044] The first reporting unit is used to report the multiple first-step data to the terminal when the multiple first-step data meet the set conditions. The multiple first-step data are used by the terminal to perform behavior recognition on the user wearing the wearable device.

[0045] This application also provides a behavior recognition device based on gait data, including:

[0046] The first receiving unit is used to receive multiple first step gait data reported by the wearable device, wherein each first step gait data represents gait data within a gait cycle;

[0047] The first judgment unit is used to determine whether the plurality of first step gait data contains a first data segment and obtain a first judgment result. The first data segment represents the gait data of the user wearing the wearable device during the process of walking a complete set turning path.

[0048] The identification unit is configured to identify normal gait data and / or abnormal gait data in the first data segment when the first judgment result indicates that the first data segment is included in the plurality of first step gait data.

[0049] This application also provides a behavior recognition device based on gait data, including:

[0050] The acquisition unit is used to acquire M first-step data groups sent by the terminal, wherein the M first-step data groups respectively include P second-step data groups reported by the wearable device for each day in M ​​consecutive days, each second-step data group includes the plurality of first-step data, and M and P are both integers greater than or equal to 1;

[0051] The detection unit is used to detect whether the gait data corresponding to the M days and / or the gait data corresponding to each day in the M days are normal based on the M first step gait data groups, and obtain the detection result.

[0052] This application also provides a wearable device, including: a first processor and a first communication interface; wherein,

[0053] The first processor is configured to divide the gait data collected by the wearable device into multiple first-step gait data based on the peak values ​​in the gait data collected by the wearable device, wherein each first-step gait data represents gait data within a gait cycle.

[0054] The first communication interface is used to report the multiple first-step state data to the terminal when the multiple first-step state data meet the set conditions. The multiple first-step state data are used by the terminal to perform behavior recognition on the user wearing the wearable device.

[0055] This application also provides a terminal, including: a second processor and a second communication interface; wherein,

[0056] The second communication interface is used to receive multiple first step gait data reported by the wearable device, wherein each first step gait data represents gait data within a gait cycle;

[0057] The second processor is used to determine whether the plurality of first step gait data contains a first data segment and obtain a first determination result. The first data segment represents the gait data of the user wearing the wearable device during the process of walking a complete set turning path.

[0058] The second processor is configured to identify normal gait data and / or abnormal gait data in the first data segment when the first determination result indicates that the plurality of first step gait data includes data in the first data segment.

[0059] This application also provides a server, including: a third processor and a third communication interface; wherein,

[0060] The third communication interface is used to acquire M first-step data groups sent by the terminal, wherein the M first-step data groups each include P second-step data groups reported by the wearable device for each of the M consecutive days, and each second-step data group includes the plurality of first-step data groups, wherein M and P are both integers greater than or equal to 1.

[0061] The third processor is used to detect whether the gait data corresponding to the M days and / or the gait data corresponding to each day in the M days are normal based on the M first step gait data groups, and obtain the detection result.

[0062] This application also provides a wearable device, including: a first processor and a first memory for storing a computer program capable of running on the processor.

[0063] Wherein, when the first processor is used to run the computer program, it executes the steps of any of the methods described above for the wearable device.

[0064] This application also provides a terminal, including: a second processor and a second memory for storing computer programs capable of running on the processor.

[0065] Wherein, when the second processor is running the computer program, it executes the steps of any of the methods described above on the terminal side.

[0066] This application also provides a server, including: a third processor and a third memory for storing computer programs capable of running on the processor.

[0067] The third processor is used to execute any of the steps of the above-described server-side method when running the computer program.

[0068] This application embodiment also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the above-described wearable device-side methods, or the steps of any of the above-described terminal-side methods, or the steps of any of the above-described server-side methods.

[0069] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the methods described above for wearable devices, or the steps of any of the methods described above for terminals, or the steps of any of the methods described above for servers.

[0070] In the various embodiments of this application, when the wearable device monitors motion status, it only reports gait data after determining that the user is walking, which can extend the standby capability of the wearable device to a certain extent. Based on the gait data reported by the wearable device, the terminal identifies the gait data corresponding to the user's gait during the process of walking a complete set transition path, and performs behavior recognition on the user based on the identified gait data. In this way, the amount of data used for behavior recognition is reduced, thereby effectively saving terminal power consumption. The server detects the user's gait data within a certain data reporting period, realizing the rational use of communication and computing resources, and also effectively reducing the system power consumption caused by behavior recognition. Attached Figure Description

[0071] Figure 1 This is a schematic diagram of the overall interaction process of the behavior recognition method based on gait data in an embodiment of this application;

[0072] Figure 2 This is a schematic diagram of a subject walking task in an embodiment of this application;

[0073] Figure 3 This is a schematic diagram illustrating the implementation process of a behavior recognition method based on gait data according to an embodiment of this application;

[0074] Figure 4 This is a schematic diagram showing the changes in gait data of subjects in an embodiment of this application after drug administration;

[0075] Figure 5 This is a schematic diagram illustrating the process of calculating the number of steps taken by the subjects in an embodiment of this application.

[0076] Figure 6 This is a schematic diagram illustrating the implementation process of another behavior recognition method based on gait data in an embodiment of this application;

[0077] Figure 7 This is a schematic diagram of the trajectory of foot changes during a stage of foot movement in an embodiment of this application.

[0078] Figure 8 This is a schematic diagram of the trajectory changes of the three feet during the foot movement phase of the subject in the embodiments of this application;

[0079] Figure 9 This is a schematic diagram of the step estimation model design in an embodiment of this application;

[0080] Figure 10 This is a schematic diagram illustrating the implementation process of the third behavior recognition method based on gait data in this application embodiment;

[0081] Figure 11 This is a schematic diagram of the algorithm flow for spatial pose calculation in an embodiment of this application;

[0082] Figure 12 This is a schematic diagram of the structure of a behavior recognition device based on gait data according to an embodiment of this application;

[0083] Figure 13 This is a schematic diagram of another behavior recognition device based on gait data according to an embodiment of this application;

[0084] Figure 14 This is a schematic diagram of the third type of behavior recognition device based on gait data according to an embodiment of this application;

[0085] Figure 15 This application contains a schematic diagram of the wearable device structure.

[0086] Figure 16 This is a schematic diagram of the terminal structure according to an embodiment of this application;

[0087] Figure 17 This is a schematic diagram of the server structure in an embodiment of this application. Detailed Implementation

[0088] Motion status is an important reference for assessing the early onset and recovery progress of chronic diseases such as cardiovascular and neurological disorders. In recent years, professional motion measurement equipment has been increasingly used in the field of assistive medicine. In gait-related motion monitoring scenarios, professional motion measurement equipment in hospitals includes motion capture systems composed of inertial measurement units (IMUs) worn on different parts of the body, optical equipment, and pressure walkways. However, since this motion measurement equipment is located inside the hospital, patients may not be able to exhibit gait symptoms without monitoring due to factors such as tension. Furthermore, due to its limited mobility, this motion measurement equipment cannot be used for daily home use. Therefore, currently, most gait motion monitoring is conducted using wearable devices with a standby time of more than three months that can provide uninterrupted monitoring.

[0089] In related technologies, wearable devices monitor patients' movement status through continuous monitoring or manual wake-up monitoring. However, continuous monitoring is limited by battery capacity and cannot support long-term standby use, while manual wake-up monitoring is prone to missing the collection of high-risk data and may cause patients to fail to exhibit gait symptoms in a natural state.

[0090] Based on this, in the various embodiments of this application, when the wearable device monitors the movement state, it only reports gait data after determining that the user is in a walking state, which can extend the standby capability of the wearable device to a certain extent. Based on the gait data reported by the wearable device, the terminal identifies the gait data corresponding to the user's gait during the process of walking a complete set transition path, and performs behavior recognition on the user based on the identified gait data. In this way, the amount of data used for behavior recognition is reduced, thereby effectively saving terminal energy consumption. The server detects the user's gait data within a certain data reporting period, realizing the rational use of communication and computing resources, and also effectively reducing the system energy consumption caused by behavior recognition.

[0091] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0092] First, the overall interaction flow of the behavior recognition method based on gait data in the embodiments of this application will be described. (Refer to...) Figure 1 The communication entities involved in this interactive process include: a human domain end-user acquisition terminal, a human domain mobile aggregation center, and a cloud-based population modeling application server. The human domain end-user acquisition terminal can be a wearable device, and the human domain mobile aggregation center can be a terminal. Both the human domain end-user acquisition terminal and the human domain mobile aggregation center are deployed on a single user side, i.e., the subject side. The cloud-based population modeling application server is deployed in the cloud. This overall interactive process includes, for example: Figure 1 The seven sub-processes shown will be explained in detail below.

[0093] The first level, executed by the human body domain end-effector acquisition terminal (i.e., wearable device), primarily determines the initial step count condition that triggers gait data reporting. It employs a method that fuses the raw acceleration and angular velocity data into a quaternion representation in Earth coordinates. The method corresponding to this first level requires less storage space and, while fusing the two types of features for mutual verification, reduces storage overhead and improves accuracy.

[0094] In the first level, the subjects, i.e. the users wearing the wearable device, performed a set of "straight walking + turning" gait tasks, and the wearable device collected data.

[0095] In practical applications, subjects can wear a miniature 6-axis IMU device and walk freely. For example, the walking path can be as follows: Figure 2 As shown, the path includes a straight path and a turning path, with a turning angle greater than or equal to 90 degrees. For example, the wearable device can be a shoe insole on which several IMUs are deployed.

[0096] In the first level, the wearable device can remind the subject to begin gait detection and simultaneously activate the data acquisition module to collect triaxial acceleration and triaxial angular velocity data output by the IMU in the wearable device in real time. Furthermore, the wearable device uses motion detection algorithms to determine in real time whether the subject's movement behavior is walking. It only saves the collected gait data and calculates steps when it detects that the subject is walking, thus saving storage resources and improving the efficiency of the wearable device's computing resources.

[0097] Here, gait cycles are calculated based solely on the extreme values ​​of the raw data collected by the wearable device, and step counts are then determined accordingly. Next, based on the zero-gravity acceleration data collected by the wearable device within the initial 10 steps, and combined with human kinematics principles, the average walking speed of the subject is initialized. The step count condition N is adaptively determined; if continuous walking is greater than or equal to 3N steps, the initial step count condition is considered met, triggering the reporting of gait data. Specifically:

[0098] Step 1: Use peak detection on the attitude angle signal to identify toe-off (TO) gait events and heel-strike (HS) gait events;

[0099] Step 2: Filter the pitch angle angular velocity acquired by the IMU in Step 1 using a third-order zero-hysteresis high-pass filter to obtain the filtered pitch angle angular velocity.

[0100] Step 3: Use threshold detection and peak detection methods to identify heel-off (HO) and foot-flat (FF) gait events based on the IMU angular velocity.

[0101] Step 4: Based on the gait events of HO, TO, HS, and FF, output gait data, then divide the gait cycle, and calculate the support phase time and swing phase time of the subject's feet.

[0102] Then, the time point of the HS gait event is used as the dividing mark of each step. The acceleration data within each step within the initial 10 steps is integrated to solve for the velocity within each step. Then, the average velocity of each step is calculated, and the average speed of the subject walking within 10 steps is obtained. This average speed is used as the normal speed of the current walking, thereby predicting the value of the step count condition N.

[0103] Here, the formula for calculating the velocity within each step is:

[0104]

[0105] Among them, v e(t) represents the velocity obtained by integration within each step, a e (τ) represents the zero-gravity acceleration in the Earth coordinate system obtained by the solution, and t represents the time point within one step, T gait This indicates the duration of each step.

[0106] Therefore, the average velocity value for each step is calculated as follows:

[0107]

[0108] in, v represents the average speed at each step. i This represents the i-th velocity value within a step, and n represents the number of velocity values ​​contained within a step.

[0109] The average walking speed of a person is calculated as follows:

[0110]

[0111] in, This represents the average walking speed of a person within 10 steps. Let N represent the average speed of the i-th step within 10 steps, where N equals 10.

[0112] According to the principles of human kinematics, the normal walking speed of a person is known to be v. normal =1.3m / s, the maximum number of steps N required to complete the turning path set in the test of this application. turn =10, from which the ratio of the current walking speed of the subject to the normal speed can be calculated, thereby adaptively deriving the step count condition N value that matches the current walking speed:

[0113]

[0114] Where λ represents the ratio of velocities.

[0115]

[0116] Where N represents the step count condition setting value, in practical applications, in If the integer is not an integer, it can be used for... The result is rounded up to obtain the value of N.

[0117] In this embodiment of the application, the subject's movement state is determined based on the collected gait data. If the determination result meets the set conditions, the process proceeds to the corresponding third-level processing flow.

[0118] The second level is executed by the human body domain end-of-field data acquisition terminal, that is, by wearable devices. It is mainly used to fuse and correct data collected by multiple sensors on wearable devices to improve the accuracy of the data.

[0119] Furthermore, by exploiting the characteristic that the feet do not leave the ground in the two time phases before and after the flatfoot stage, a zero-position fusion filtering scheme is adopted. This scheme uses the small change in the horizontal distance between the two feet in the two time phases before and after the flatfoot stage as the basis for detecting outliers in the acceleration data, thus completing the detection and correction of outliers in the acceleration data. Specifically:

[0120] Step 1: Fusion modification, that is, normalizing the sensor data collected by multiple sensors on the wearable device to the quaternion fusion coordinate system under the Earth coordinate system.

[0121] Here, based on IMU data from the sole of the foot, quaternions are calculated for the spatial attitude changes of the IMU coordinates and the Earth coordinate system, respectively. Euler angles and global acceleration in the zero-gravity Earth coordinate system are also calculated, that is, acceleration and angular velocity information are fused into quaternion representation. By choosing quaternion representation to represent real-time changes in velocity and direction, only 4 bits are used in terms of data overhead, which can reduce storage and computational load. Furthermore, by unifying the coordinate system to the Earth coordinate system, the time overhead and conversion error of multiple coordinate systems can be reduced.

[0122] Step 2: First correction of acceleration error and gyroscope error, that is, initializing and calibrating key event points and determining the fusion gain coefficient K between multiple sensors.

[0123] The calculations for the gyroscope measurement ω and the gained acceleration error term K are as follows:

[0124]

[0125] In the above formula, the initial It is a fixed value, which is inaccurate during initial calculation. This is achieved by adjusting the initialization time t. init The internal gain coefficient K is dynamically converged from a large initial value to a normal value, thereby making Slowly converges to the accurate value:

[0126]

[0127] Among them, t init K is the initialization time in seconds. init K is the initial value of the gain coefficient. normal It is the gain coefficient when calculating quaternions, and K init >K normal t is the actual sampling time point.

[0128] Step 3: Second correction of acceleration error, that is, correction of acceleration error term using fused coordinate system.

[0129] The acceleration error component e is determined by the acceleration measurement value a, and is calculated as follows:

[0130] in

[0131] Where, q x ,q y ,q z ,q ω It is a component of standardized quaternions Element.

[0132] Step 4: Second correction of gyroscope error, that is, using the fused coordinate system to correct and determine the gyroscope error term.

[0133] In practical applications, gyroscopes generate linear errors during measurement. These errors can be subtracted from the estimated error value ω before the complementary filter estimation calculation. bias In the calculation of the quaternion rate of change, ω uses the corrected gyroscope reading ω′:

[0134] ω′=ω-ω bias

[0135] The estimation error value is calculated using the low-pass filtering results:

[0136] ω bias =2πf c ∫pω·dt

[0137]

[0138] Among them, f c ω is the cutoff frequency of the low-pass filter, ω is the gyroscope data, and p is a dynamically determined value, either 1 or 0, indicating that the low-pass filter is enabled when the IMU is stationary. p is determined by the function f. b (ω,ω min This is determined by the fact that the amplitude of each element in ω is less than ω. min The time. Where, ω min This is the minimum angular velocity threshold that enables the deviation estimator. t b It is the minimum static time after the bias estimator is enabled.

[0139] Step 5: The third correction of acceleration error, that is, eliminating the influence of gravitational acceleration.

[0140] Gravity is removed from the acceleration data, and the acceleration in the Earth coordinate system is calculated from the attitude quaternion at the current moment.

[0141] Zero-gravity acceleration is the acceleration measurement after removing gravity. It is obtained by subtracting the component of gravity in the IMU coordinate system from the acceleration measurement.

[0142]

[0143] By standardizing the changes in quaternions, the global acceleration in the Earth coordinate system can be calculated:

[0144]

[0145] in, yes The conjugate vector, The calculation formula is as follows:

[0146]

[0147] Step Six: The fourth correction of acceleration error, namely, tilt angle linear acceleration suppression.

[0148] Here, linear acceleration is suppressed, effectively reducing errors caused by acceleration from linear and rotational motion. The working principle of acceleration suppression is to compare the instantaneous tilt angle measurement provided by the accelerometer with the current tilt angle measurement output by the algorithm. If the angle difference between these two tilt angles is greater than a threshold, the accelerometer measurement can be ignored.

[0149]

[0150] -g d <(||a|-1)<g d

[0151] Among them, the function f(a,g) d The calculation condition for ) is -g d <(||a|-1)<g d Invalid time period, g d To enable acceleration suppression, the acceleration threshold t a Minimum rest time to enable linear acceleration suppression.

[0152] Based on the above description of the first and second level implementation schemes executed by the human body domain end-effector, i.e., by the wearable device, this application provides a behavior recognition method based on gait data, applied to wearable devices. (See also...) Figure 3 The method includes:

[0153] Step 301: Based on the peak values ​​in the gait data collected by the wearable device, divide the gait data collected by the wearable device into multiple first gait data, each first gait data representing the gait data within a gait cycle.

[0154] Step 302: If the multiple first-step data sets meet the set conditions, report multiple first-step data sets to the terminal. These multiple first-step data sets are used by the terminal to perform behavior recognition on users wearing wearable devices.

[0155] In one embodiment, the set conditions include:

[0156] Multiple first-step state data represent the user continuously walking to reach a set number of steps. This set number of steps is represented as the product of 3 and N, where N is determined based on the user's walking speed and the maximum number of steps required to complete the set turning path, and N is an integer greater than 1.

[0157] In one embodiment, the method further includes:

[0158] Detect whether the user is walking and obtain the detection result;

[0159] If the detection results indicate that the user is in a walking state, the collected gait data is saved to segment the first gait data.

[0160] In one embodiment, the method further includes:

[0161] The first sensing data collected by multiple sensors in the wearable device are normalized to the quaternion fusion coordinate system under the Earth coordinate system to obtain the second sensing data corresponding to each of the multiple sensors.

[0162] The second sensing data corresponding to multiple sensors are fused and corrected to obtain the gait data collected by the wearable device.

[0163] In one embodiment, the method further includes:

[0164] At set time intervals, the gait data acquisition operation is triggered.

[0165] The time interval is determined based on the user's medication time, gait data collection time, and corresponding gait data within a historical period.

[0166] Combination Figure 1 The third level is executed by the human body domain mobile convergence center, that is, by the terminal. Based on the gait data reported by the wearable device, combined with business characteristics such as speed, the terminal predicts and verifies turning points step by step on a larger scale, which reduces the computational overhead of traditional traversal retrieval of turning points and verifies them on a spatial scale, thereby improving accuracy. This allows the identification of the gait data corresponding to the user during the process of walking through the complete set turning path, and the user's behavior is identified based on the identified gait data. In this way, the amount of data used for behavior recognition is reduced, thereby effectively saving terminal energy consumption.

[0167] In practical applications, on the terminal side, based on the subject's medical management needs, the doctor sets fixed medication schedules for each medication period during the subject's home rehabilitation. At each scheduled medication schedule, the terminal will trigger a medication reminder. If no medication feedback is received within 2 minutes, the terminal will trigger the reminder again. Each day, the terminal inputs the subject's gait data from the previous day into the detection time planning algorithm. Changes in gait data, such as stride length, after medication administration include... Figure 4 The six scenarios are shown.

[0168] like Figure 4 As shown, t0 to t4 represent sampling times, and P0 to P4 represent gait data values ​​at the corresponding sampling times. Time t0 is the data collection time immediately after medication administration, before the drug takes effect. Therefore, gait indicators before medication administration can be determined from P0. Figure 4 In the image, type 1 on the left represents changes in gait data inhibited by drugs, while type 2 represents changes in gait data promoted by drugs.

[0169] like Figure 4 The standard change curve shown has five detection times evenly distributed throughout the entire drug's action period, with t2 representing the peak drug effect and t4 representing the time when gait data recovers to pre-drug levels. However, in actual measurements, two scenarios may occur: Scenario 1 occurs because the sampling interval is too short, causing the data collection to end before the drug's effect has worn off; Scenario 2 occurs because the sampling interval is too long, causing the drug's effect to have largely disappeared by t3. Both scenarios result in inaccurate detection of the drug's impact on the subject's gait. Therefore, in this embodiment, a detection time planning algorithm is used to adjust the detection time interval so that the gait data changes achieve the effect of the standard change curve. The specific method is as follows:

[0170]

[0171] Where σ is the difference ratio, used to determine whether the gait data at time t4 has recovered to the state before medication, that is, recovered to the state at time t0;

[0172]

[0173] Where ∈ is the rate of change, used to determine whether the drug effect ends at time t3, that is, the change in gait data is very small during the process from t3 to t4;

[0174]

[0175] σ threshold The difference ratio threshold is set to 0.1;

[0176] ∈ threshold The threshold for the rate of change is set to 0.1;

[0177] ΔT last This represents the time interval between the last gait detection.

[0178] Δt adapt The time step for each adjustment of the gait detection time interval is set to 0.1h.

[0179] In ΔT current In the calculation formula, condition ① corresponds to Figure 4 In scenario one, the detection time interval needs to be increased; condition ② corresponds to... Figure 3 In scenario two, the detection time interval needs to be reduced. Both of these scenarios still require this part of the calculation and judgment in the next detection. Finally, condition ③ corresponds to... Figure 3 The standard variation curve in the data does not require adjustment of the detection time interval.

[0180] On the terminal side, multiple gait data points reported by the wearable device are received. Let the number of steps corresponding to the currently received gait data be step k. Steps k-3N to k-2N-1 are divided into the first stage of gait data, steps k-2N to kN-1 into the second stage, and steps kN to k into the third stage. Then:

[0181] Based on the foot position information of the subjects in phases one and three, two directed straight lines for walking paths were fitted respectively. Therefore, the position information of phase one (P) was respectively... k-3N ,P k-3N-1 ,P k-3N-2 ,...,P k-2N ) and the location of the third stage (P k-N ,P k-N-1 ,P k-N-2 ,...,P k Two straight lines, l1 and l2, are fitted, where P i =(x i ,y i This contains the position data on the x-axis and y-axis for each step, and the corresponding fitting function is: in, and These are the slope and intercept of the fitted line, respectively:

[0182]

[0183] Where, x i and y i The location point P used when fitting the straight line i The data is given by n, where n is the number of data points used to fit the straight line.

[0184] By calculating the mean-square error (MSE) between the fitted points and the fitted line, the system comprehensively assesses whether the subjects' walking in phases one and three was along a straight line. The residuals are the vertical distances between the fitted line and the actual data points, and the sum of squared residuals is the sum of the squares of these residuals. A better fit results in a smaller sum of squared residuals. The formula for calculating the sum of squared residuals is as follows:

[0185]

[0186] Among them, y i These are actual data values. To fit the straight line at the corresponding x i The predicted value at point n is the number of data points used to fit the straight line.

[0187] Mean squared error is the sum of squared residuals divided by the average number of data points, and the formula is as follows:

[0188]

[0189] Where n is the number of data points containing the fitted location points in both Phase 1 and Phase 3.

[0190] Mean squared error (MSE) is the average of the squared distances between the fitted line and the actual data points. It reflects the accuracy of the fit; a lower MSE means a better fit and a closer approximation of the line. Here, a threshold th is selected. d =0.5 is used as the threshold for judging the mean square error, and the following binary number is set:

[0191]

[0192] When S k When S = 0, the subject is judged not to be walking in a straight line; when S = 0, the subject is judged not to be walking in a straight line. k When the value is 1, the subject is judged to be walking in a straight line.

[0193] Based on the above scheme for determining whether a subject walks in a straight line, if the straight lines fitted in both Phase 1 and Phase 3 satisfy the criteria for straight walking, then the angle θ between the two fitted straight lines is calculated. If the straight line fitted in either Phase 1 or Phase 3 does not satisfy the criteria for straight walking, then gait data needs to be re-collected by the wearable device.

[0194] When the fitted lines from both Phase 1 and Phase 3 satisfy the criteria for straight-line walking, the formula for calculating the difference in the angle between the two lines is as follows:

[0195]

[0196] Where a1 and a2 represent the slope values ​​of the fitted lines l1 and l2, respectively.

[0197] When the angle θ between two directed straight lines (i.e., lines with a defined positive direction) meets a set threshold condition, a complete turning process is considered to have occurred in stage two. Otherwise, the wearable device needs to re-collect gait data. In practical applications, θ can be set to ≥ 85°.

[0198] When it is determined that a complete turning process exists in stage two, the slope of the angle bisector of the angle between the fitted lines of stage one and stage three is found. Then, based on the slope of the angle bisector, the middle step of the turning process in stage two is located. Starting from the middle step of the subject in stage two, the step that satisfies the angle bisector is found. That is, starting from the middle step of the turn, the starting step and the ending step of the turn are found on both sides.

[0199] The mathematical derivation of the slope formula for the angle bisector of the two lines l1 = a1x + b and l2 = a2x + b², which have already been calculated, is as follows:

[0200] Let the slopes of the two lines be a1 and a2, respectively. Let the angle between them be θ. Then we have:

[0201]

[0202] Using half-width identities, we get:

[0203]

[0204] Considering that sin(θ) and cos(θ) can be expressed using slopes, we have:

[0205]

[0206] Substitute the above expressions for sin(θ) and cos(θ) into In the middle, we get:

[0207]

[0208] Now, considering the slope a of the angle bisector m According to the formula for included angle:

[0209]

[0210] By two If the expressions are equal, then the following equation holds:

[0211]

[0212] Given that a1 and a2 are known values, the slope a of the angle bisector can be calculated. m .

[0213] Starting from the middle step of stage two, retrieve the middle step of the turn. If the number of steps N in stage two is odd, then start from the middle step of stage two. The search begins at the intermediate step of the turn. If the number of steps N in stage two is even, then the search starts from the second stage. The search begins at the middle of the turn.

[0214] The straight line l is fitted based on all points within one step. step The slope 'a' of the fitted line in this step is calculated. step Therefore, calculate the angle θ between the line in the current step and the angle bisector. step :

[0215]

[0216] According to θ step The value of θ is determined at the position of the middle step of the turn in the second stage. step If θ < 90°, then search the steps following the intermediate step in stage two. step If θ > 90°, then search for the steps before the intermediate step in stage two. step If the angle is 90°, then the current step is considered to be the intermediate step in the turning process of the second stage.

[0217] When the middle step of the turn is located, the search for the start step of the turn begins before the middle step and the search for the end step of the turn begins after the middle step. When searching for the start and end steps of the turn, a straight line for each step needs to be fitted to obtain the slope of the fitted line.

[0218] The turning start step is determined by the angle between the line fitted to Phase 1 and the starting line. The formula for calculating the angle is:

[0219]

[0220] If θ start If θ < 10°, then the current step is considered the starting step for a turn; if θ start If the angle is ≥10°, continue searching until a turning start step is found.

[0221] The final step of the turn is determined by the angle between the angle and the straight line fitted in stage three. The formula for calculating the angle is:

[0222]

[0223] When θ end If θ < 10°, then the current step is considered the final step of the turn; if θ end If the angle is ≥10°, continue searching until the turn ends and the search stops.

[0224] The fourth level is executed by the human body domain mobile convergence center, that is, by the terminal. The terminal combines key events in the business time domain to perform a second level of correction on the gait data.

[0225] In the fourth level, abnormal data is filtered and corrected by combining the temporal continuity and spatial correlation of the business.

[0226] First, convert the quaternions to Euler angles, which are then converted into business-related parameters.

[0227] Here, converting quaternions to Euler angles is a common pose representation transformation, used to convert the pose information of quaternions into a more easily understood Euler angle representation. The following equation converts the standardized quaternion to Euler angles, where q... w q x q y and q z These are elements of a standard quaternion:

[0228]

[0229] The results for tan and arcsin are... This does not cover all orientations, and for θ pitch horn The range of values ​​for is already satisfied, so atan2 needs to be used instead of arctan.

[0230] Here, critical gait events are used for anomaly correction:

[0231] 1. Based on the attitude angle signal, peak detection is used to identify TO and HS gait events;

[0232] 2. Based on the attitude angle signal, perform first-order differentiation to obtain the pitch angle and angular velocity;

[0233] 3. The pitch angle angular velocity is filtered by a third-order zero-hysteresis high-pass filter to obtain the filtered pitch angle angular velocity;

[0234] 4. Threshold detection and peak detection methods are used to identify HO and FF gait events based on IMU angular velocity;

[0235] 5. Based on the HO, TO, HS and FF gait events, output gait time parameters, then divide the gait cycle and calculate the support phase time and swing phase time.

[0236] Subsequently, within a more detailed scope, the variation law of yaw angle and angular velocity in each step is analyzed. By judging whether the yaw angle and angular velocity are zero or negative, that is, the turning angular velocity value is positive, the normal step and abnormal step in the turn are determined.

[0237] The yaw rate within one step is Where t i ∈[t start ,t end ], t start It is the starting time of a step, t end This is the end time of one step, and the data length of the yaw rate within one step is M. Calculate the number N of yaw rates that are zero or negative within one step. i :

[0238]

[0239] Among them, w n This represents the nth yaw rate within a step. sign(x) is the sign function, which returns 1 when x>0, -1 when x<0, and 0 when x=0.

[0240] The number N of zero or negative data detected in a certain step i When N i If the length of the data in that step is equal to or greater than 20% of the total length, then that step is considered an abnormal turning step; when N i If the length of the data in this step is less than 20% of the total length, then this step is considered a normal turning step.

[0241]

[0242] Among them, S i This indicates the turning state at that step, and M is the length of the yaw rate data within that step. Once an abnormal turning step is detected, other data within that step will no longer be detected.

[0243] By testing whether the turning gait is normal, the subject's turning ability can be further assessed. The normality of the turning gait is an important indicator for assessing the subject's balance, coordination, and stability. Abnormal turning gait may indicate problems with the subject's nervous system, musculoskeletal system, or other bodily systems.

[0244] The fifth level is executed by the human body domain mobile convergence center, that is, by the terminal, and is used to perform the third level correction in conjunction with key events in the business space domain.

[0245] In practical applications, after collecting gait data for a day, the wearable device begins to calculate the gait data:

[0246] 1. The gait data calculation module determines whether the wearable device has completed a day's gait data collection. For example, a day should include 5 gait data collections.

[0247] 2. If the required number of gait data collections is not completed, wait for the next gait detection period, and the wearable device will continue gait detection. If the required number of gait data collections is completed, the gait data will be calculated by the gait data calculation module.

[0248] 3. After completing the gait data calculation, the gait data calculation module transmits the gait data to the terminal.

[0249] In addition, the terminal's gait anomaly detection module determines whether a gait detection cycle, such as a 7-day gait detection period, has been completed. If not, it continues to remind the subject to take medication and perform gait detection daily. If completed, the gait anomaly detection module uses the following two gait data anomaly detection algorithms to detect anomalies in the daily gait data variation patterns and the gait data deviation values ​​within a gait cycle:

[0250] 1. Lateral detection: Anomaly detection of gait data changes in five separate instances after a single dose of medication on the same day. Clinical observations of Parkinson's patients after medication administration show that the drug's gait-regulating effect gradually increases within two hours of administration, reaching its maximum around two hours, and then gradually weakens. Therefore, there are two possible trends in the corresponding gait data changes, such as... Figure 4 The standard variation curves show that the horizontal axis represents the time after medication administration, and the vertical axis represents the numerical value of gait data. P0 to P4 represent the values ​​at different times t. i The following gait data values.

[0251] Δf(t i )=f(t i+1 )-f(t i )

[0252] Δ 2 f(t j )=Δf(t j+1 )-Δf(t j )

[0253]

[0254] Where, f(t) i ) represents the numerical value of the gait parameters at time i, Δf(t) i ) represents the first-order forward difference, where i takes values ​​of 0, 1, 2, and 3; Δ 2 f(t j) represents the second-order forward difference, with j taking values ​​of 0, 1, and 2; Q represents the output of the lateral detection; condition ① is that the values ​​of the second-order forward difference are all greater than 0, and the corresponding gait data curve is a concave function, which corresponds to type one; condition ② is that the values ​​of the second-order forward difference are all less than 0, and the corresponding gait data curve is a convex function, which corresponds to type two. Both of these cases conform to the normal variation pattern; condition ③ is that none of the above conditions are met, which indicates that the lateral detection is abnormal.

[0255] 2. Longitudinal detection: Anomaly detection of five gait data points after one medication administration within a gait monitoring cycle. The gait data calculated from the j-th gait data point collected on day i is represented as P. i,j Calculate the mean of the gait data of the j-th time over one detection period. The calculation formula is: N=7, then calculate the gait data and mean for the j-th time each day. deviation If it is greater than the threshold P j,max and express The maximum and minimum values ​​in Δ i,j If the value is greater than γ, then an anomaly is determined to exist in the vertical direction.

[0256] Then, based on the judgment results of the gait anomaly detection module, different operations are selected and executed:

[0257] If no gait abnormalities are detected in either the lateral or longitudinal detection, only the processed gait data is transmitted to the server, without the need to transmit the original data, which greatly reduces the amount of data transmitted and the power consumption of communication. If gait abnormalities are detected in either the lateral or longitudinal detection, in addition to transmitting the processed gait data to the server, the system also needs to request the transmission of the cached original data of the abnormal part from the data acquisition module. The data acquisition module transmits the cached original data of the abnormal part to the terminal. After receiving the abnormal gait data, the terminal's data request and transmission module transmits the data along with the gait data to the server.

[0258] In practical applications, the subject's personal attributes and basic walking patterns, such as average walking speed, are crucial for step count estimation because subjects of different body types and ages exhibit different walking patterns. The core of neural network models lies in their ability to learn and recognize the complex relationships between these different attributes, thereby providing more accurate step count estimates. A major advantage of neural networks is their ability to handle large numbers of input variables and uncover hidden patterns and correlations, which is often difficult to achieve in traditional step count methods. The specific implementation process is as follows... Figure 5 As shown.

[0259] Combination Figure 5 Based on individual attribute information, such as gender, age, height, and weight, and the individual's average walking speed, a step count estimation model is used to further refine the step count conditions, reducing storage overhead and improving accuracy. The training dataset contains features X and labels Y, where feature X refers to the individual's attributes and the average walking speed solved in level one, and label Y refers to the number of turning steps n solved in level four based on the initial and final turning steps.

[0260] Next, the error estimation model is trained. During training, the input signal is first propagated forward through the weight connections between layers until an output signal is generated. The difference between the prediction and the actual result of the output layer forms the error signal, which is then used in the backpropagation process, starting from the output layer and propagating back through each layer until the input layer. During backpropagation, the network's weights and bias parameters are adjusted according to the error gradient, a process typically implemented using gradient descent or its variants.

[0261] Based on the above description of the implementation schemes for the third to fifth levels executed by the human body domain mobile convergence center, i.e., executed by the terminal, this application provides a behavior recognition method based on gait data, applied to a terminal. See [link to relevant documentation]. Figure 6 The method includes:

[0262] Step 601: Receive multiple first-state data reported by the wearable device.

[0263] Each step gait data point represents the gait data within a gait cycle.

[0264] Step 602: Determine whether the first data segment is included in the multiple first-step state data to obtain the first determination result.

[0265] The first data segment represents the gait data of a user wearing a wearable device as they traverse a complete set turning path.

[0266] Step 603: If the first judgment result indicates that the first data segment is included in multiple first step gait data, identify the normal gait data and / or abnormal gait data in the first data segment.

[0267] In one embodiment, multiple first-step state data satisfy a set condition, which includes: the multiple first-step state data represent that the user has walked continuously to reach a set number of steps, the set number of steps is represented as the product of 3 and N, N is determined based on the user's walking speed and the maximum number of steps required to complete the set turning path, and N is an integer greater than 1.

[0268] In one embodiment, determining whether a first data segment is included in a plurality of first-step data includes:

[0269] Based on the first step state data of the first stage and the first step state data of the second stage, it is determined whether the user walks in a straight line in the first stage and the second stage, and a second judgment result is obtained.

[0270] In this context, the first step gait data in the first stage consists of the k-3Nth to k-2N-1th gait data from multiple first step gait data, and the first step gait data in the second stage consists of the k-2Nth to kN-1th gait data from multiple first step gait data, where k is the number of multiple first step gait data.

[0271] When the second judgment result indicates that the user walks in a straight line in both the first and second stages, based on the angle between the straight line the user walks in the first stage and the straight line the user walks in the second stage, it is determined that the first gait data of the third stage contains a first data segment. The first gait data of the third stage is the kNth to kth gait data among multiple first gait data, and k is an integer greater than 3N.

[0272] In one embodiment, the method further includes:

[0273] Determine the set time interval.

[0274] The set time interval is used by the wearable device to trigger the collection of gait data; the set time interval is determined based on the user's medication time, gait data collection time and corresponding gait data in the historical period.

[0275] In one embodiment, the method further includes:

[0276] Send M first-step data groups to the server. Each of the M first-step data groups includes P second-step data groups reported by the wearable device for each of the M consecutive days. Each second-step data group includes the multiple first-step data groups. M and P are both integers greater than or equal to 1.

[0277] Level 6: This is executed by the cloud-based crowd modeling application server, which combines crowd clustering to perform a fourth level of correction on the gait data reported by the terminal.

[0278] First, a third-level correction is made by combining key events in the crowd spatial domain, that is, by dividing the temporal phase of the gait cycle of the business event to calculate the position and distance of the subjects' feet.

[0279] Here, based on gait events, the foot stationary period is defined as TS→HO, and the foot movement period is defined as HO→TS. The foot movement period can be further subdivided into three stages: stage 1: push-off period HO→TO, stage 2: swing period TO→HS, and stage 3: landing period HS→TS.

[0280] Based on the analysis of human walking patterns, when both feet are in the support phase (i.e., one foot is in phase one of the movement process and the other foot is in phase three), the horizontal distance changes minimally. At this point, the preliminary positions of both feet in the Earth coordinate system are calculated as follows:

[0281]

[0282] Among them, t start Indicates the start time of the walk, t end p0 represents the end time of the walk and p0 represents the initial position during the walk.

[0283] Formula for calculating the horizontal distance between the two feet in a two-footed support phase:

[0284]

[0285] Among them, T DLS P is the duration of the two-footed stance phase. L P represents the position data of the left foot. R This indicates the position data of the right foot, and D represents the horizontal distance between the two feet.

[0286] Subsequently, based on the small change in the horizontal distance between the two feet as the basis for detecting outliers in the acceleration data, when the subject was in a walking state, the abnormal data during foot movement were corrected by combining the detection of abnormal values ​​in the horizontal distance between the two feet.

[0287] For the calculated horizontal distance within the bipedal support phase, the Mad-Median Rule method is used to detect outliers.

[0288] First, calculate the median of the horizontal distance data within the bipedal support phase:

[0289]

[0290] If the total number of distance data points n is odd, the median is the middle number; if the total number of distance data points n is even, the median is the average of the two middle numbers, where X represents all distance data points after sorting, and median(X) represents the method for finding the median.

[0291] Calculate the absolute deviation of each distance data point from the median:

[0292] Deviation i = |X i -median(X)∣

[0293] Among them, X i This represents the i-th distance data point.

[0294] The calculated median absolute deviation (MAD) is the median absolute deviation among all absolute deviations.

[0295] MAD=median(Deviation1,Deviation2,...,Deviation n )

[0296] Using a constant factor of 0.6745, which is the inverse of the 25th percentile of the standard normal distribution, the corrected Z-score for each distance data point is calculated, transforming MAD into a scale similar to the standard deviation:

[0297]

[0298] If the corrected Z-Score for the i-th distance data point exceeds theta DLS =2.5, then the distance from data point X can be considered as 2.5. (i) If it is an outlier, then the acceleration data of both feet corresponding to that distance data point will be corrected to the previous normal value corresponding to that distance data point.

[0299] Then, the cumulative acceleration error of the two time phases before and after the support phase was calculated using the statistical characteristics of the population.

[0300] 1. Obtain user gender and height. In this embodiment, based on large-scale data, an implicit mapping relationship is constructed between the subject's relevant personal attributes and the subject's shoe length, thereby determining the user's gender and height.

[0301] 2. Stage One: HO→TO, the foot's movement trajectory resembles a circle, through... Figure 7 The relationships shown can be used to calculate the changes in distance along the x, y, and z axes in stage one:

[0302]

[0303]

[0304] Where Δz1, Δy1, and Δx1 represent the distances along the z, y, and x axes respectively in stage one, and L tiptoe This indicates the distance between the IMU placement location and the tip of the insole. This indicates the pitch angle corresponding to the TO gait event. This represents the pitch angle corresponding to the TO gait event.

[0305] According to the equation of motion based on distance:

[0306]

[0307] Given Δz1, Δy1, and Δx1, the true average acceleration value in stage one can be calculated:

[0308]

[0309] The average value of the measurements of the three axes of the inertial unit measured by the IMU in Phase 1 is obtained as follows:

[0310]

[0311] in, It is the average value of the IMU measurements in Phase 1. It is the average value of the actual acceleration in stage one. It is the average error value in stage one.

[0312] Therefore, the formula for the cumulative error in stage one can be calculated:

[0313]

[0314] Where e(T1) represents the cumulative error in stage one, and T1 is the duration of stage one.

[0315] 3. Stage Three: HO→TO, the foot's trajectory is similar to Stage One, resembling a circle, through... Figure 8 The geometric relationships shown can be used to calculate the distance changes along the x, y, and z axes in stage three:

[0316]

[0317] Where Δz3, Δy3, and Δx3 represent the distances along the z, y, and x axes in stage three, respectively, and L heel This indicates the distance between the IMU placement location and the heel of the insole. Indicates the pitch angle corresponding to the HS gait event. This indicates the pitch angle corresponding to the HS gait event.

[0318] The motion process in stage three is the reverse of the motion process in stage one. In stage one, the velocity gradually increases from zero, while in stage three, the velocity gradually decreases to zero. Therefore, the equation of motion for the distance in stage three is the same as that in stage one, as follows:

[0319]

[0320] Given Δz3, Δy3, and Δx3, the true average acceleration value in stage three can be calculated:

[0321]

[0322] Based on the above formula, the average value of the IMU measurements in Phase 3 can be calculated as follows:

[0323]

[0324] in, It is the average value of the IMU measurements in Phase 3. It is the average value of the actual acceleration in stage three. It is the average error value in stage three.

[0325] At this point, the cumulative error formula for stage three can be calculated:

[0326]

[0327] Where e(T3) represents the cumulative error in stage one, and T3 is the duration of stage three.

[0328] Subsequently, a zero-velocity combined with a zero-position scheme was employed to perform geometric modeling and analysis of the foot movement trajectory in the two time phases before and after the flatfoot phase, calculating the corresponding cumulative errors for each phase and thus solving for the cumulative errors during the support phase before and after the flatfoot phase. Through zero-velocity updates and +zero-position updates, the three components of the cumulative error for each step were subdivided, and useful swing phase errors were extracted in a refined manner to correct the gait data.

[0329] 1. During the stationary period, the foot is stationary and its linear velocity is zero. Set the velocity to zero during the stationary period, and integrate the acceleration during the motion period to calculate the velocity.

[0330] Therefore, by integrating the acceleration in a weightless Earth coordinate system, the velocity during bipedal motion in the Earth coordinate system can be calculated:

[0331]

[0332] Where aglobal(t) is the global zero-gravity acceleration in the Earth coordinate system, t HO It is the time point when the heel leaves the ground, t TS It is the point in time when the toe-striking incident occurred.

[0333] 2. Due to deviation errors in acceleration measurement, the estimated foot velocity may not be zero when the foot is stationary. In this embodiment, the difference between the actual velocity (known as zero) and the velocity obtained by integration is used to correct the acceleration deviation error. By calculating the cumulative error during the motion period (HO→TS), the drift error caused by the integration of acceleration is further eliminated.

[0334] For acceleration measurements during motion:

[0335]

[0336] T = t TS -t HO

[0337] in, These are actual IMU measurements; 'e' represents the Earth coordinate system. It is the actual acceleration value generated by the motion, ε is the acceleration deviation error, T is the duration of the motion, and t is the acceleration deviation error. HO It is the time point when the heel leaves the ground, t TS It is the time point in time when the toe struck the ground.

[0338] During the short period of motion, the acceleration bias error is a variable ε(t) that fluctuates slightly with time. Furthermore, at the beginning of the motion, the velocity is zero. Therefore, the formula for calculating the velocity during the motion is rewritten as follows:

[0339]

[0340] in, This represents the velocity calculated by integration, which is derived from the actual velocity. It consists of the cumulative error function e(t), where t∈[0,T].

[0341] 3. At the end of the motion period, i.e., at t=T, when the sole of the foot is fully in contact with the ground, the actual speed is... Since the value is zero, the bias error in acceleration measurement can be calculated as follows:

[0342]

[0343] Therefore, The value is a cumulative error of the integral of acceleration during the foot movement period. From this, the cumulative error of the IMU accelerometer between two full foot landing periods can be obtained:

[0344]

[0345] The cumulative error consists of three stages:

[0346] Phase 1: Extension phase HO→TO;

[0347] Phase Two: Swing Period TO→HS;

[0348] Phase 3: Landing Phase HS→TS

[0349] The bias error is a variable ε(t) that fluctuates slightly over time. The accumulated value of the error is different in the three stages of the motion period. In the embodiments of this application, the mean value of the bias error is calculated for each stage, and then error compensation is performed separately, so as to more accurately correct the bias error of the IMU measurement unit.

[0350] Based on the cumulative error of the two time phases before and after the flat-foot period, the cumulative error within the swing phase can be calculated. Then, the average error values ​​corresponding to the three stages are used to correct the acceleration data of the three stages, thereby more accurately correcting the bias error of the bipedal IMU measurement.

[0351] Given the total cumulative error e(T) during the motion period, the cumulative error e(T1) in stage one of the motion period, and the cumulative error e(T3) in stage three of the motion period, the cumulative error in stage two of the motion period can be calculated:

[0352] e(T2) = e(T) - e(T1) - e(T3)

[0353] Where e(T2) represents the cumulative error of stage two, and T2 is the duration of stage two.

[0354] Level 7: Executed by the cloud-based population modeling application server, this level further refines the adapted individuals and establishes a twin profile error compensation model.

[0355] First, a powerful error compensation model is trained using deep learning to further correct the accumulated error of the oscillating phase, making the accumulated error of the oscillating phase close to the true value.

[0356] Step 1: The abnormal data analysis module receives and stores the data uploaded by the body domain gateway, i.e., the terminal. For abnormal data, it sends it to the doctor's terminal.

[0357] Step 2: The data uploaded to the doctor's end is used to evaluate the therapeutic effect of the drug on the subject's symptoms through multimodal information fusion and a drug efficacy assessment model.

[0358] Here, the multimodal information includes: feature data, numerical data, text data, and label data. Feature data can be derived from relevant characteristic attributes of the subjects, such as gender and age. Numerical data comes from the lateral index Δ of each gait data point detected as abnormal in the third level. 2 f(t j) and longitudinal index Δ i,j The text data comes from the follow-up reports completed by patients after each gait test. For example, the follow-up content can refer to the Comprehensive Parkinson's Disease Rating Scale (UPDRS). The label data comes from each doctor's diagnosis and may include "improvement" and "no improvement" labels.

[0359] Numerical Feature Extractor: For numerical data, convolutional neural networks are used to extract data features and mine semantic feature information between variables and time in gait data.

[0360] Text Feature Extractor: First, the word embedding method (Word2Vec) converts the text data into word vectors. Then, the sequence model (GRU) is used to extract the temporal features from the word vectors. Next, the data dimensionality information is transformed, and a CNN model is used to extract high semantic features. The feature data is then transformed to the same dimensionality as the numerical data features to facilitate subsequent feature concatenation operations.

[0361] Feature fusion module: The concatenated numerical features and text features are fused using an attention mechanism. The attention mechanism adopts CBMA, which can not only achieve feature fusion of different modalities with different weights, but also weight fusion between different variables within the same modality.

[0362] The drug efficacy evaluation model uses a fully convolutional neural network (FCN), which consists of a fully connected network and a final sigmoid activation function, ultimately outputting the degree of therapeutic effect. The loss function employs binary cross-entropy loss.

[0363]

[0364] Where N is the total number of samples. i This is the actual label of sample i, with a value of 0 or 1. It is the predicted probability of sample i, usually expressed as the probability of belonging to the positive class.

[0365] Step 3: The doctor reviews the abnormal gait data and the results of the drug efficacy evaluation model, conducts a comprehensive analysis, and then provides feedback to the subject to make adjustments, such as adjusting the dosage and timing of medication.

[0366] In this embodiment, error compensation is performed using a neural network to predict the difference between the accumulated error of the oscillating phase and the actual oscillating phase error in the third-level correction, thereby correcting the error value and making the final error closer to the true value. Neural networks possess excellent generalization performance, and their unique learning ability can effectively extract data features. Approximating nonlinear functions is an advantage of neural networks. Therefore, a feedforward neural network (BP) is used as the error estimation model here.

[0367] Step 4: Design of the step count estimation model. (Combined with...) Figure 9 The step count estimation model is a multi-layer feedforward neural network, which consists of an input layer, three hidden layers, and an output layer. The input layer takes into account the subject's personal attributes, such as gender, age, height, weight, and average walking speed. The hidden layers transform these inputs and minimize the difference between the output and the target value using weights and biases that are continuously updated during training. The output layer generates a personally adaptive step count condition N.

[0368] Step 5:

[0369] 1. Creating the training dataset for the error estimation model.

[0370] Using data measured by Vicon and IMU, the true cumulative error value of the oscillating phase is calculated. The difference between the calculated true cumulative error value and the cumulative error value of the oscillating phase calculated by the third-level correction is used as the label, and the cumulative error value of the oscillating phase calculated by the third-level correction is used as the feature.

[0371] Based on the changing trends of VICON position data during visualization of walking, the characteristics corresponding to gait events are extracted. HS (i.e., the peak value of TOE_Z axis data) and TO (i.e., the peak and trough of TOE_Z axis data) are determined based on rules.

[0372] Based on the HS and TO events, the gait cycle can be divided into a support phase and a swing phase. Therefore, based on the positional data of the VICON within the swing phase and the motion characteristics of each step taken by the subject during walking, the average acceleration value of the swing phase can be calculated.

[0373] Calculate the distance the foot travels during the swing phase:

[0374] D swing =p HS -p TO

[0375] Calculate the time interval of the oscillation phase:

[0376] tswing =t HS -t TO

[0377] Calculating distance traveled based on acceleration, given that an object accelerates from rest, the formula for distance is:

[0378]

[0379] Therefore, based on the VICON position data of the oscillating phase, the average acceleration of the oscillating phase is calculated:

[0380]

[0381] Based on the acceleration values ​​measured by the IMU, calculate the average value of the acceleration measured by the IMU within the swing phase:

[0382]

[0383] according to and The true cumulative error value Δe within the oscillating phase can be calculated. true .

[0384]

[0385] The cumulative error e(T2) in the second stage of the motion period during the third-level correction is the cumulative error Δe of the oscillation phase obtained by optimizing the zero-velocity update technique. zvu Therefore, Δe can be calculated. zvu and Δe true The difference between the two values ​​is Δe = Δe true -Δe zvu .

[0386] 2. Error estimation model design

[0387] During human walking, the cumulative error compensation of the swing phase may be due to dynamic changes in the foot and external disturbances. To correct this error, the neural network needs to be able to identify and predict these error patterns and generate corresponding compensation differences.

[0388] The error estimation model is a multi-layer feedforward neural network consisting of an input layer, two hidden layers, and an output layer. The input layer takes into account the accumulated error of the oscillating phase calculated by optimizing the zero-velocity update scheme; the hidden layers transform these inputs and minimize the difference between the output and the target value using weights and biases that are continuously updated during training; the output layer generates an accumulated error difference compensation signal to adjust the accumulated error of the oscillating phase in the optimized zero-velocity scheme, making the final oscillating phase error value closer to the true value.

[0389] 3. Error estimation model training

[0390] During training, the input signal is first propagated forward through the weights of layers until an output signal is generated. The difference between the predicted result and the actual result at the output layer forms an error signal, which is then used in the backpropagation process, starting from the output layer and propagating back through each layer until the input layer. During backpropagation, the network's weights and bias parameters are adjusted according to the error gradient. This process is typically implemented using gradient descent, with the goal of minimizing the network's loss function, a quantification of the prediction error. Through multiple iterations of this process, the neural network learns the features of the data and gradually reduces the error between the predicted result and the target value. The backpropagation of error and the updating of weights rely on the chain rule to calculate the gradient, usually combined with the derivatives of the loss function and the activation function. This method minimizes the loss function by gradually adjusting the network weights, allowing the network to learn complex function mappings.

[0391] 4. Finally, based on the cumulative error of the compensated oscillating phase... The average bias error of stage two can be calculated. This allows for bias error correction of the acceleration in the three stages, and then the corrected acceleration can be used to calculate stride and position information, etc.

[0392] First, based on the average bias error obtained in stage one from the above solution... Phase 2 bias average error and the bias average error in stage three It can compensate for the cumulative error caused by the acceleration integral to solve for velocity in the three stages of motion.

[0393] Compared to the speed after traditional zero-speed update compensation, the speed after compensation based on geometric modeling analysis during foot movement is closer to the linear speed generated by the foot when walking. Therefore, the stride length and position calculated by integral method based on the speed after compensation based on geometric modeling analysis during foot movement are more accurate.

[0394] Stride length is calculated by integrating the velocity during phase two of the foot movement:

[0395]

[0396] in The velocity t represents the corrected velocity. HS It is the time point of HS, t TO It is the time point of TO.

[0397] In related technologies, zero-speed update uses the average error of the entire motion period to compensate for the cumulative speed error in stage two, but this compensation is not precise. However, in this embodiment, the average error of stage two of the motion period is used to compensate for the cumulative speed error in stage two, resulting in a speed that more closely matches the true value after stage two compensation, and a more accurate calculated stride.

[0398] Based on geometric modeling analysis of foot movement, the average error of each stage during the movement is used to compensate for the cumulative velocity error of each stage. The compensated velocity is closer to the true velocity value of each stage during the movement, resulting in higher accuracy of the calculated position data at each moment. Error compensation is performed for each step, improving the accuracy of velocity compensation at each step compared to zero-velocity updates in related technologies. The final position data is calculated by integrating the velocity throughout the entire walking process, thus achieving higher accuracy than zero-velocity updates in related technologies.

[0399]

[0400] Among them, t start It is the start time of walking, t end p0 represents the end time of the walk and p0 represents the initial position during the walk.

[0401] Based on the above description of the implementation schemes for the sixth and seventh levels executed by the cloud-based crowd modeling application server, this application provides a behavior recognition method based on gait data, applied to a server. (See also...) Figure 10 The method includes:

[0402] Step 1001: Obtain the M first-step state data sent by the terminal.

[0403] Among them, the M first gait data groups each include the P second gait data groups reported by the wearable device for each of the M consecutive days, and each second gait data group includes the multiple first gait data groups, where M and P are both integers greater than or equal to 1;

[0404] Step 1002: Based on M sets of first step gait data, detect whether the gait data corresponding to M days and / or the gait data corresponding to each day in M ​​days are normal, and obtain the detection results.

[0405] In one embodiment, based on M sets of first step gait data, the method of detecting whether the gait data for each day within the M days is normal includes:

[0406] Calculate the first-order forward difference between gait data from adjacent collection times in the gait data for each day to obtain the first calculation result;

[0407] The second calculation result is obtained by the second-order forward difference between the multiple first calculation results;

[0408] Based on the obtained secondary calculation results, it is determined whether the gait data corresponding to each day of the M-day detection period is normal.

[0409] In one embodiment, based on M sets of first step gait data, detecting whether the gait data corresponding to M days is normal includes:

[0410] Calculate the mean of gait data at the same collection time over M days;

[0411] Based on the deviation between the gait data collected at the corresponding time each day and the mean, it is determined whether the gait data for the M days is normal.

[0412] Figure 11 A schematic diagram illustrating the overall implementation flow of the behavior recognition method based on gait data provided in this application embodiment is shown.

[0413] Based on the above solution, this application provides a more flexible method for identifying abnormal motor signs and behaviors based on wearable devices. This method adapts to the needs and changes in the condition of different subjects, adapts to data collection at different times, intervals, and scenarios, and combines treatment effects for prediction and correction. Furthermore, it saves resources and power caused by inappropriate single-time collection of unqualified data, multiple unqualified data collection sessions, and cyclical unqualified data collection patterns, thereby improving the long-term standby capability of minimalist wearable devices.

[0414] To implement the method on the wearable device side of this application embodiment, this application embodiment also provides a behavior recognition device based on gait data, which is installed on the wearable device, such as... Figure 12 As shown, the device includes:

[0415] The segmentation unit 1201 is used to divide the gait data collected by the wearable device into multiple first-step gait data based on the peak value in the gait data collected by the wearable device, wherein each first-step gait data represents the gait data within a gait cycle.

[0416] The first reporting unit 1202 is used to report the multiple first-step data to the terminal when the multiple first-step data meet the set conditions. The multiple first-step data are used by the terminal to perform behavior recognition on the user wearing the wearable device.

[0417] In one embodiment, the set conditions include:

[0418] The multiple first-step state data represent the user's continuous walking to reach a set number of steps. The set number of steps is represented as the product of 3 and N. N is determined based on the user's walking speed and the maximum number of steps required to complete the set turning path, and N is an integer greater than 1.

[0419] In one embodiment, the device further includes:

[0420] The detection unit is used to detect whether the user is walking and obtain the detection result;

[0421] If the detection result indicates that the user is in a walking state, the collected gait data is saved to segment the first gait data.

[0422] In one embodiment, the device further includes:

[0423] The normalization unit is used to normalize the first sensing data collected by multiple sensors in the wearable device to the quaternion fusion coordinate system under the Earth coordinate system to obtain the second sensing data corresponding to the multiple sensors respectively.

[0424] The fusion correction unit is used to fuse and correct the second sensing data corresponding to the multiple sensors respectively to obtain the gait data collected by the wearable device.

[0425] In one embodiment, the device further includes:

[0426] The acquisition unit is used to trigger the acquisition of gait data at set time intervals; wherein...

[0427] The set time interval is determined based on the user's medication time, gait data collection time, and corresponding gait data within a historical time period.

[0428] To implement the terminal-side method of this application embodiment, this application embodiment also provides a behavior recognition device based on gait data, which is installed on the terminal, such as... Figure 13 As shown, the device includes:

[0429] The first receiving unit 1301 is used to receive multiple first step gait data reported by the wearable device, wherein each first step gait data represents gait data within a gait cycle;

[0430] The first judgment unit 1302 is used to determine whether the plurality of first step gait data contains a first data segment and obtain a first judgment result. The first data segment represents the gait data of the user wearing the wearable device during the process of walking a complete set turning path.

[0431] The identification unit 1303 is used to identify normal gait data and / or abnormal gait data in the first data segment when the first judgment result indicates that the first data segment is included in the plurality of first step gait data.

[0432] In one embodiment, the plurality of first step state data satisfy a set condition, the set condition including: the plurality of first step state data represent that the user has walked continuously to reach a set number of steps, the set number of steps is represented as the product of 3 and N, N is determined based on the user's walking speed and the maximum number of steps required to complete the set turning path, and N is an integer greater than 1.

[0433] In one embodiment, the first determining unit 1302 is configured to:

[0434] Based on the first step state data of the first stage and the first step state data of the second stage, it is determined whether the user walks in a straight line in the first stage and the second stage, and a second determination result is obtained. The first step state data of the first stage is the k-3Nth to k-2N-1th gait data among the plurality of first step state data, and the first step state data of the second stage is the k-2Nth to kN-1th gait data among the plurality of first step state data, where k is the number of the plurality of first step state data and k is an integer greater than 3N.

[0435] When the second judgment result indicates that the user walks in a straight line in both the first stage and the second stage, based on the angle between the straight line the user walks in the first stage and the straight line the user walks in the second stage, it is determined that the first data segment is included in the first step state data of the third stage, wherein the first step state data of the third stage is the kNth to kth gait data among the plurality of first step state data.

[0436] In one embodiment, the device further includes:

[0437] The determining unit is used to determine a set time interval; wherein,

[0438] The set time interval is used by the wearable device to trigger the collection of gait data; the set time interval is determined based on the user's medication time, gait data collection time and corresponding gait data in a historical period.

[0439] In one embodiment, the device further includes:

[0440] The sending unit is used to send M first-step data groups to the server, wherein the M first-step data groups each include P second-step data groups reported by the wearable device for each of the M consecutive days, and each second-step data group includes the plurality of first-step data groups, wherein M and P are both integers greater than or equal to 1.

[0441] To implement the server-side method of this application embodiment, this application embodiment also provides a behavior recognition device based on gait data, which is set on a server, such as... Figure 14 As shown, the device includes:

[0442] The acquisition unit 1401 is used to acquire M first-step data groups sent by the terminal, wherein the M first-step data groups respectively include P second-step data groups reported by the wearable device for each day in M ​​consecutive days, and each second-step data group includes the plurality of first-step data.

[0443] The detection unit 1402 is used to detect whether the gait data corresponding to the M days and / or the gait data corresponding to each day in the M days are normal based on the M first step gait data groups, and obtain the detection result.

[0444] In one embodiment, the detection unit 1402 is used for:

[0445] Calculate the first-order forward difference between gait data from adjacent collection times in the gait data for each day to obtain the first calculation result;

[0446] The second calculation result is obtained by calculating the second-order forward difference between multiple first calculation results;

[0447] Based on the obtained second calculation results, it is determined whether the gait data corresponding to each day of the M days is normal.

[0448] In one embodiment, the detection unit 1402 is used for:

[0449] Calculate the mean of gait data at each identical collection time during the M days;

[0450] Based on the deviation between the gait data collected at the corresponding time each day and the mean, it is determined whether the gait data for the M days is normal.

[0451] In practical applications, the above units can be implemented by processors and / or communication interfaces in gait data-based behavior recognition.

[0452] It should be noted that the gait data-based behavior recognition device provided in the above embodiments is only illustrated by the division of the above-described program modules when performing gait data-based behavior recognition. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the gait data-based behavior recognition device and the gait data-based behavior recognition method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0453] Based on the hardware implementation of the above program modules, and in order to implement the method on the wearable device side of this application embodiment, this application embodiment also provides a wearable device, such as... Figure 15 As shown, the wearable device 1500 includes:

[0454] The first communication interface 1501 is capable of exchanging information with other network nodes;

[0455] The first processor 1502 is connected to the first communication interface 1501 to enable information interaction with other network nodes. When running a computer program, it executes the methods provided by one or more technical solutions on the wearable device side. The computer program is stored in the first memory 1503.

[0456] Specifically, the first processor 1502 is used for:

[0457] Based on the peak values ​​in the gait data collected by the wearable device, the gait data collected by the wearable device is divided into multiple first gait data, and each first gait data represents the gait data within a gait cycle.

[0458] The first communication interface 1501 reports the multiple first-step state data to the terminal when the multiple first-step state data meet the set conditions. The multiple first-step state data are used by the terminal to perform behavior recognition on the user wearing the wearable device.

[0459] It should be noted that the specific processing procedures of the first processor 1502 and the first communication interface 1501 can be understood by referring to the above method.

[0460] Of course, in practical applications, the various components in the wearable device 1500 are coupled together via a bus system 1504. It can be understood that the bus system 1504 is used to implement communication between these components. In addition to a data bus, the bus system 1504 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 15 The general labeled all buses as Bus System 1504.

[0461] The first memory 1503 in this embodiment is used to store various types of data to support the operation of the wearable device 1500. Examples of such data include any computer program used to operate on the wearable device 1500.

[0462] The methods disclosed in the above embodiments of this application can be applied to the first processor 1502, or implemented by the first processor 1502. The first processor 1502 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware or by instructions in the form of software in the first processor 1502. The first processor 1502 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The first processor 1502 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the first memory 1503. The first processor 1502 reads the information in the first memory 1503 and completes the steps of the aforementioned method in combination with its hardware.

[0463] In an exemplary embodiment, the wearable device 1500 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned methods.

[0464] Based on the hardware implementation of the above program modules, and in order to implement the terminal-side method of the embodiments of this application, the embodiments of this application also provide a terminal, such as... Figure 16 As shown, the terminal 1600 includes:

[0465] The second communication interface 1601 is capable of exchanging information with other network nodes;

[0466] The second processor 1602 is connected to the second communication interface 1601 to enable information interaction with other network nodes. When running a computer program, it executes the methods provided by one or more of the aforementioned terminal-side technical solutions. The computer program is stored in the second memory 1603.

[0467] Specifically, the second communication interface 1601 is used for:

[0468] Receive multiple first step gait data reported by a wearable device, wherein each first step gait data represents gait data within a gait cycle.

[0469] The second processor 1602 is used for:

[0470] Determine whether the plurality of first step gait data contains a first data segment to obtain a first determination result. The first data segment represents the gait data of the user wearing the wearable device during the process of traversing a complete set turning path.

[0471] If the first judgment result indicates that the first data segment is included in the plurality of first step gait data, then identify the normal gait data and / or abnormal gait data in the first data segment.

[0472] It should be noted that the specific processing procedures of the second processor 1602 and the second communication interface 1601 can be understood by referring to the above method.

[0473] Of course, in practical applications, the various components in terminal 1600 are coupled together through bus system 1604. It can be understood that bus system 1604 is used to realize the connection and communication between these components. In addition to the data bus, bus system 1604 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 16 The general designated all buses as Bus System 1604.

[0474] The second memory 1603 in this embodiment is used to store various types of data to support the operation of the terminal 1600. Examples of such data include any computer program used to operate on the terminal 1600.

[0475] The methods disclosed in the embodiments of this application can be applied to the second processor 1602, or implemented by the second processor 1602. The second processor 1602 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware or by instructions in the form of software in the second processor 1602. The second processor 1602 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The second processor 1602 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the second memory 1603. The second processor 1602 reads the information in the second memory 1603 and completes the steps of the aforementioned method in conjunction with its hardware.

[0476] In an exemplary embodiment, terminal 1600 may be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic components to perform the aforementioned method.

[0477] Based on the hardware implementation of the above program modules, and in order to implement the server-side method of this application embodiment, this application embodiment also provides a server, such as... Figure 17 As shown, the server 1700 includes:

[0478] The third communication interface 1701 is capable of exchanging information with other network nodes;

[0479] The third processor 1702 is connected to the third communication interface 1701 to enable information interaction with other network nodes and to execute the methods provided by one or more of the aforementioned server-side technical solutions when running a computer program. The computer program is stored in the third memory 1703.

[0480] Specifically, the third processor 1702 is used for:

[0481] Get M first-step data groups sent by the terminal, wherein the M first-step data groups each include P second-step data groups reported by the wearable device for each of the M consecutive days, each second-step data group includes the plurality of first-step data, and M and P are both integers greater than or equal to 1;

[0482] Based on the M first step gait data sets, the gait data corresponding to the M days and / or the gait data corresponding to each day in the M days are checked to see if they are normal, and the detection results are obtained.

[0483] Of course, in practical applications, the various components in server 1700 are coupled together through bus system 1704. It can be understood that bus system 1704 is used to implement communication between these components. In addition to the data bus, bus system 1704 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 17 The general labeled all buses as Bus System 1704.

[0484] The third memory 1703 in this embodiment is used to store various types of data to support the operation of the server 1700. Examples of such data include any computer program used to operate on the server 1700.

[0485] The methods disclosed in the embodiments of this application can be applied to, or implemented by, the third processor 1702. The third processor 1702 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware or by instructions in the software form of the third processor 1702. The third processor 1702 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The third processor 1702 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, specifically a third memory 1703. The third processor 1702 reads information from the third memory 1703 and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0486] In an exemplary embodiment, server 1700 may be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic components to perform the aforementioned method.

[0487] It is understood that the memories (first memory 1503, second memory 1603, and third memory 1703) in the embodiments of this application can be volatile memories or non-volatile memories, or both. Non-volatile memories can be read-only memories (ROM), programmable read-only memories (PROM), erasable programmable read-only memories (EPROM), electrically erasable programmable read-only memories (EEPROM), magnetic random access memories (FRAM), flash memories, magnetic surface memories, optical discs, or compact disc read-only memories (CD-ROM); magnetic surface memories can be disk storage or magnetic tape storage. Volatile memories can be random access memories (RAM), which are used as external caches.By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM). The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memory.

[0488] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium. For example, it may include a first memory 1503 storing a computer program, which can be executed by a first processor 1502 of a wearable device 1500 to complete the steps described in the aforementioned wearable device-side method. Another example is a second memory 1603 storing a computer program, which can be executed by a second processor 1602 of a terminal 1600 to complete the steps described in the aforementioned terminal-side method. Yet another example is a third memory 1703 storing a computer program, which can be executed by a third processor 1702 of a server 1700 to complete the steps described in the aforementioned server-side method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0489] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0490] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0491] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A behavior recognition method based on gait data, characterized in that, Applications in wearable devices, including: Based on the peak values ​​in the gait data collected by the wearable device, the gait data collected by the wearable device is divided into multiple first gait data, and each first gait data represents the gait data within a gait cycle. If the defined first-step data meets the set conditions, the first-step data is reported to the terminal. The first-step data is used by the terminal to identify the behavior of the user wearing the wearable device.

2. The method according to claim 1, characterized in that, The setting conditions include: The multiple first-step state data represent the user continuously walking to reach a set number of steps. The set number of steps is represented as the product of 3 and N. N is determined based on the user's walking speed and the maximum number of steps required to complete the set turning path, and N is an integer greater than 1.

3. The method according to claim 1, characterized in that, The method further includes: Detect whether the user is walking and obtain the detection result; If the detection result indicates that the user is in a walking state, the collected gait data is saved to segment the first gait data.

4. The method according to claim 1, characterized in that, The method further includes: The first sensing data collected by the multiple sensors in the wearable device are normalized to the quaternion fusion coordinate system under the Earth coordinate system to obtain the second sensing data corresponding to the multiple sensors respectively. The second sensing data corresponding to the multiple sensors are fused and corrected to obtain the gait data collected by the wearable device.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Gait data acquisition is triggered at set time intervals; where... The set time interval is determined based on the user's medication time, gait data collection time, and corresponding gait data within a historical time period.

6. A behavior recognition method based on gait data, characterized in that, Applied to terminals, including: Receive multiple first step gait data reported by a wearable device, wherein each first step gait data represents gait data within a gait cycle; Determine whether the plurality of first step gait data contains a first data segment to obtain a first determination result. The first data segment represents the gait data of the user wearing the wearable device during the process of traversing a complete set turning path. If the first judgment result indicates that the first data segment is included in the plurality of first step gait data, then identify the normal gait data and / or abnormal gait data in the first data segment.

7. The method according to claim 6, characterized in that, The plurality of first step state data satisfy the set conditions, the set conditions include: the plurality of first step state data represent that the user has walked continuously to reach a set number of steps, the set number of steps is represented as the product of 3 and N, N is determined based on the user's walking speed and the maximum number of steps required to walk to complete the set turning path, and N is an integer greater than 1.

8. The method according to claim 7, characterized in that, The step of determining whether the plurality of first-step state data contains a first data segment includes: Based on the first step state data of the first stage and the first step state data of the second stage, it is determined whether the user walks in a straight line in the first stage and the second stage, and a second determination result is obtained. The first step state data of the first stage is the k-3Nth to k-2N-1th gait data among the plurality of first step state data, and the first step state data of the second stage is the k-2Nth to kN-1th gait data among the plurality of first step state data. k is the number of the plurality of first step state data, and k is an integer greater than 3N. When the second judgment result indicates that the user walks in a straight line in both the first stage and the second stage, based on the angle between the straight line the user walks in the first stage and the straight line the user walks in the second stage, it is determined that the first data segment is included in the first step state data of the third stage, wherein the first step state data of the third stage is the kNth to kth gait data among the plurality of first step state data.

9. The method according to claim 6, characterized in that, The method further includes: Determine the set time interval; where, The set time interval is used by the wearable device to trigger the collection of gait data; the set time interval is determined based on the user's medication time, gait data collection time and corresponding gait data in a historical period.

10. The method according to any one of claims 6 to 9, characterized in that, The method further includes: M first-step data groups are sent to the server, wherein each of the M first-step data groups includes P second-step data groups reported by the wearable device for each of the M consecutive days, and each second-step data group includes the plurality of first-step data groups, wherein M and P are both integers greater than or equal to 1.

11. A behavior recognition method based on gait data, characterized in that, Applied to servers, including: Get M first-step data groups sent by the terminal, wherein the M first-step data groups each include P second-step data groups reported by the wearable device for each of the M consecutive days, each second-step data group includes the plurality of first-step data, and M and P are both integers greater than or equal to 1; Based on the M first step gait data sets, the gait data corresponding to the M days and / or the gait data corresponding to each day in the M days are checked to see if they are normal, and the detection results are obtained.

12. The method according to claim 11, characterized in that, Based on the M first step gait data sets, detect whether the gait data for each day within the M days is normal, including: Calculate the first-order forward difference between gait data from adjacent collection times in the gait data for each day to obtain the first calculation result; The second calculation result is obtained by calculating the second-order forward difference between multiple first calculation results; Based on the obtained second calculation results, it is determined whether the gait data corresponding to each day of the M days is normal.

13. The method according to claim 11, characterized in that, Based on the M first step gait data sets, detect whether the gait data corresponding to the M days is normal, including: Calculate the mean of gait data at each identical collection time during the M days; Based on the deviation between the gait data collected at the corresponding time each day and the mean, it is determined whether the gait data for the M days is normal.

14. A behavior recognition device based on gait data, characterized in that, include: A segmentation unit is used to divide the gait data collected by the wearable device into multiple first-step gait data based on the peak value in the gait data collected by the wearable device, wherein each first-step gait data represents the gait data within a gait cycle. The first reporting unit is used to report the multiple first-step data to the terminal when the multiple first-step data meet the set conditions. The multiple first-step data are used by the terminal to perform behavior recognition on the user wearing the wearable device.

15. A behavior recognition device based on gait data, characterized in that, include: The first receiving unit is used to receive multiple first step gait data reported by the wearable device, wherein each first step gait data represents gait data within a gait cycle; The first judgment unit is used to determine whether the plurality of first step gait data contains a first data segment and obtain a first judgment result. The first data segment represents the gait data of the user wearing the wearable device during the process of walking a complete set turning path. The identification unit is configured to identify normal gait data and / or abnormal gait data in the first data segment when the first judgment result indicates that the first data segment is included in the plurality of first step gait data.

16. A behavior recognition device based on gait data, characterized in that, include: The acquisition unit is used to acquire M first-step data groups sent by the terminal, wherein the M first-step data groups respectively include P second-step data groups reported by the wearable device for each day in M ​​consecutive days, each second-step data group includes the plurality of first-step data, and M and P are both integers greater than or equal to 1; The detection unit is used to detect whether the gait data corresponding to the M days and / or the gait data corresponding to each day in the M days are normal based on the M first step gait data groups, and obtain the detection result.

17. A wearable device, characterized in that, include: A first processor and a first communication interface; wherein... The first processor is configured to divide the gait data collected by the wearable device into multiple first-step gait data based on the peak values ​​in the gait data collected by the wearable device, wherein each first-step gait data represents gait data within a gait cycle. The first communication interface is used to report the multiple first-step state data to the terminal when the multiple first-step state data meet the set conditions. The multiple first-step state data are used by the terminal to perform behavior recognition on the user wearing the wearable device.

18. A terminal, characterized in that, include: A second processor and a second communication interface; wherein... The second communication interface is used to receive multiple first step gait data reported by the wearable device, wherein each first step gait data represents gait data within a gait cycle; The second processor is used to determine whether the plurality of first step gait data contains a first data segment and obtain a first determination result. The first data segment represents the gait data of the user wearing the wearable device during the process of walking a complete set turning path. The second processor is configured to identify normal gait data and / or abnormal gait data within the first data segment when the first determination result indicates that the plurality of first step gait data includes the first data segment.

19. A server, characterized in that, include: A third processor and a third communication interface; wherein... The third communication interface is used to acquire M first-step data groups sent by the terminal, wherein the M first-step data groups each include P second-step data groups reported by the wearable device for each of the M consecutive days, and each second-step data group includes the plurality of first-step data groups, wherein M and P are both integers greater than or equal to 1. The third processor is used to detect whether the gait data corresponding to the M days and / or the gait data corresponding to each day in the M days are normal based on the M first step gait data groups, and obtain the detection result.

20. A wearable device, characterized in that, include: A first processor and a first memory for storing computer programs capable of running on the processor. Wherein, when the first processor is used to run the computer program, it performs the steps of the method according to any one of claims 1 to 5.

21. A terminal, characterized in that, include: A second processor and a second memory for storing computer programs that can run on the processor. Wherein, when the second processor is used to run the computer program, it performs the steps of the method according to any one of claims 6 to 10.

22. A server, characterized in that, include: A third processor and a third memory for storing computer programs that can run on the processor. When the third processor runs the computer program, it performs the steps of the method according to any one of claims 11 to 13.

23. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5, or the steps of the method according to any one of claims 6 to 10, or the steps of the method according to any one of claims 11 to 13.

24. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5, or the steps of the method according to any one of claims 6 to 10, or the steps of the method according to any one of claims 11 to 13.

Images