Gait parameter extraction method based on recurrent neural network combined with foot pressure sensor

Abstract

The application provides a gait parameter extraction method based on a recurrent neural network combined with a foot pressure sensor, and the gait parameter extraction method comprises the following steps: acquiring a walking video of a subject; inputting the walking video into a human key point detection model to obtain a human key point feature vector sequence; acquiring plantar pressure data of the subject; determining a time point of at least one gait event according to the plantar pressure data; and obtaining the gait parameter according to the time point of the gait event and / or the human key point feature vector sequence. The gait parameter extraction method based on the recurrent neural network combined with the foot pressure sensor is convenient to operate, low in cost, and high in precision.

Description

Technical Field

[0001] This application relates to the field of gait recognition technology, and in particular to a method for extracting gait parameters based on a recurrent neural network combined with a foot pressure sensor. Background Technology

[0002] Human gait is an important biological characteristic reflecting an individual's health status and is closely related to various physiological and pathological states. Gait parameter analysis can quantitatively and objectively characterize movement patterns during walking, which is of great significance in clinical medicine, especially for the diagnosis and rehabilitation assessment of neurological diseases such as Parkinson's disease and post-stroke sequelae.

[0003] Currently, gait information collection primarily relies on gait assistive devices, which typically include sensor components, exoskeleton structures, and data processing units. By collecting human motion data through sensors and analyzing it using pre-defined algorithms, gait parameters characterizing the user's posture and behavioral features can be extracted. Based on different data acquisition methods, existing gait parameter extraction methods are mainly divided into three categories: labeled methods, wearable sensor methods, and unlabeled methods.

[0004] Tagging method: Reflective markers are affixed to key joints or skeletal landmarks in the human body. A motion capture system consisting of multiple high-speed cameras records the spatial trajectory of the markers, and then calculates gait parameters. This method is highly accurate and widely used in scientific and clinical research, but it has significant limitations: subjects need to wear tight clothing and perform a cumbersome tagging operation, resulting in long preparation times; at the same time, the system requires a large experimental space and expensive equipment, making it costly, complex to use, and difficult to popularize.

[0005] Wearable sensor method: Inertial measurement units (such as accelerometers and gyroscopes) or pressure sensors are fixed to specific parts of the human body (such as the feet and lower limbs) to calculate gait parameters by collecting kinematic or mechanical signals. This method is highly portable and suitable for daily monitoring, but it still requires wearing the device, which may affect the natural walking state, and long-term wear is not very comfortable.

[0006] Tagless method: Based on computer vision technology, it automatically identifies key points of the human body using depth cameras or ordinary cameras without the need for additional tags or wearable devices. It has the advantages of simple operation, low cost, and small space requirements. However, due to the lack of direct physical contact feedback, its accuracy in determining the timing of key events (such as heel strike and toe lift) is easily affected by occlusion, changes in lighting, and differences in clothing, resulting in relatively low accuracy in gait parameter extraction.

[0007] Furthermore, traditional contour-based methods (such as the mean difference image method) generate gait energy maps (GEI) for recognition by performing centroid alignment, difference operations, and principal component analysis on human contours in video frames. While these methods are computationally efficient, they heavily rely on the consistency of human body contours. When subjects wear different clothing, carry backpacks, or change their hairstyles, the contours change significantly, leading to a decrease in recognition accuracy and insufficient robustness.

[0008] In summary, existing technologies struggle to balance high accuracy with ease of use: high-precision methods often involve complex equipment and are inconvenient to use; while low-cost, easily deployable label-free methods suffer from low accuracy. Therefore, effectively improving the gait parameter extraction accuracy of label-free systems without increasing the burden on users has become a key research focus.

[0009] Therefore, there is an urgent need for a gait analysis method that integrates multi-source information and combines high accuracy with practicality. Summary of the Invention

[0010] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a gait parameter extraction method based on recurrent neural network combined with foot pressure sensor, aiming to build a convenient, high-precision, and quantifiable gait parameter extraction model to improve the accuracy of gait recognition.

[0011] Firstly, this application provides a gait parameter extraction method based on a foot pressure sensor, the gait parameter extraction method comprising:

[0012] Obtain walking videos of the subjects;

[0013] The walking video is input into the human key point detection model to obtain a human key point feature vector sequence, which includes the coordinates of human key points at different time points.

[0014] Acquire plantar pressure data of the subject, and the plantar pressure data is synchronized with the walking video in time;

[0015] Determine the timing of at least one gait event based on the plantar pressure data;

[0016] The gait parameters are obtained based on the time points of the gait events and / or the sequence of feature vectors of key human body points.

[0017] In one implementation of the first aspect, the gait parameters include gait cycles, the plantar pressure data includes heel pressure data, the gait events include heel strike events, and the gait parameter extraction method further includes:

[0018] The timing of the heel strike event is determined based on the heel pressure data.

[0019] The gait cycle is obtained by calculating the time difference between two consecutive heel strike events on the same side of the body.

[0020] The gait parameters include stride length, the plantar pressure data includes heel pressure data, the gait events include heel strike events, and the gait parameter extraction method further includes:

[0021] The timing of the heel strike event is determined based on the heel pressure data.

[0022] Based on the human body key point feature vector sequence, obtain the horizontal coordinates of the ankle joint key point corresponding to the time point of the heel ground contact event;

[0023] The stride length is obtained by calculating the difference in horizontal coordinates of key ankle joint points on the same side of the body during two consecutive heel strike events.

[0024] In one implementation of the first aspect, the key points of the human body include ankle key points, knee key points, and hip key points, and determining the time point of the heel strike event based on the heel pressure data includes:

[0025] The heel pressure data is differentiated to obtain the rate of change of heel pressure.

[0026] The time point at which the rate of change of heel pressure is greater than or equal to a first threshold is selected as a candidate time point for the heel contact event.

[0027] Based on the human body key point feature vector sequence, calculate the vertical movement velocity of the ankle joint key point corresponding to a candidate time point of the heel-to-ground event;

[0028] The time point when the movement speed is less than or equal to the second threshold is selected as the secondary candidate time point of the heel contact event;

[0029] Based on the human body key point feature vector sequence, obtain the coordinates of the ankle joint key point, the knee joint key point, and the hip joint key point corresponding to the secondary candidate time point of the heel ground contact event;

[0030] Based on the coordinates of the ankle joint key point, the knee joint key point, and the hip joint key point, calculate the knee flexion angle corresponding to the secondary candidate time point of the heel strike event;

[0031] The secondary candidate time point corresponding to the largest knee flexion angle is selected as the time point of the heel strike event.

[0032] In one implementation of the first aspect, the gait parameters include the stance phase time, the plantar pressure data includes big toe pressure data, the gait events include toe-off events, and the gait parameter extraction method further includes:

[0033] The timing of the toe lift-off event was determined based on the big toe pressure data.

[0034] The support phase time is obtained by calculating the time difference between the toe lift-off event and the heel touch-down event.

[0035] In one implementation of the first aspect, determining the time point of the toe-off event based on the big toe pressure data includes:

[0036] The rate of change of big toe pressure is obtained by differentiating the big toe pressure data.

[0037] The time point at which the rate of change of pressure on the big toe is greater than or equal to the third threshold is selected as a candidate time point for the toe-off event.

[0038] Based on the human body key point feature vector sequence, calculate the vertical movement velocity of the ankle joint key point corresponding to a candidate time point of the toe off-ground event;

[0039] The time point at which the movement speed first reaches the fourth threshold is selected as the time point of the toe-off event.

[0040] In one implementation of the first aspect, the gait parameters include the swing phase time, and the gait parameter extraction method includes: calculating the difference between the gait period and the support phase time to obtain the swing phase time.

[0041] In one implementation of the first aspect, acquiring the plantar pressure data of the subject includes:

[0042] Obtain raw heel pressure data from the subjects;

[0043] The zero values ​​in the original heel pressure data are retained, and the non-zero values ​​in the original heel pressure data are subjected to Gaussian filtering to obtain the plantar pressure data.

[0044] Secondly, this application provides a gait recognition method based on a recurrent neural network, the gait recognition method comprising:

[0045] Using the gait parameter extraction method described above, the human body key point feature vector sequence and gait parameters are obtained;

[0046] A recurrent neural network is constructed, wherein the input of the recurrent neural network is the sequence of feature vectors of human key points, and the output of the recurrent neural network is the gait parameters and the disease category;

[0047] Using the sequence of human key point feature vectors within the same gait cycle as input and the subject's gait parameters as labels, the recurrent neural network is trained to obtain a trained gait recognition network.

[0048] The sequence of feature vectors of key human body points to be identified is input into the trained gait recognition network to obtain the disease category.

[0049] In one implementation of the second aspect, the recurrent neural network is a long short-term memory network.

[0050] As described above, the gait parameter extraction method based on recurrent neural networks combined with foot pressure sensors described in this application is not only easy to operate and low in cost, but also has high accuracy. Attached Figure Description

[0051] Figure 1 The flowchart shown is a gait parameter extraction method based on a foot pressure sensor according to an embodiment of this application.

[0052] Figure 2 The flowchart shown is a gait cycle extraction method based on a foot pressure sensor in one embodiment of this application.

[0053] Figure 3 The flowchart shown is a step length extraction method based on a foot pressure sensor in one embodiment of this application.

[0054] Figure 4 This is shown as an embodiment of the present application. Figure 3 The detailed flowchart of step S410.

[0055] Figure 5 The flowchart shown is a support phase time extraction method based on a foot pressure sensor in one embodiment of this application.

[0056] Figure 6 This is shown as an embodiment of the present application. Figure 5 The detailed flowchart of step S420.

[0057] Figure 7 This is shown as an embodiment of the present application. Figure 1 The detailed flowchart of step S300.

[0058] Figure 8 The flowchart shown is a gait recognition method based on a recurrent neural network in one embodiment of this application.

[0059] Figure 9The diagram shown is a schematic of a gait cycle extraction method based on a foot pressure sensor in one embodiment of this application. Detailed Implementation

[0060] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0061] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0062] This embodiment provides a gait parameter extraction method based on a foot pressure sensor, including:

[0063] Step S100: Acquire a walking video of the subject;

[0064] Step S200: Input the walking video into the human key point detection model to obtain the human key point feature vector sequence, which includes the coordinates of human key points at different time points.

[0065] Step S300: Acquire plantar pressure data of the subject, and synchronize the plantar pressure data with the walking video in time;

[0066] Step S400: Determine the time point of at least one gait event based on plantar pressure data;

[0067] Step S500: Gait parameters are obtained based on the time points of gait events and / or the sequence of feature vectors of human key points.

[0068] The gait parameters in this embodiment include gait cycle, stance phase time, swing phase time, cadence, stride length, stride length, average gait speed, joint angles, etc. This embodiment is used to extract one or more of these gait parameters.

[0069] In this embodiment, each frame of the walking video can be either a conventional two-dimensional image or a depth image. When the image in the walking video is a depth image, it can be obtained by capturing the subject's walking video with a depth camera, by capturing the subject's walking video with a binocular camera, or by simultaneously capturing multiple images from multiple angles using a multi-view camera system, and then calculating the three-dimensional coordinates of the depth image through triangulation.

[0070] The human key point detection model in this embodiment is used to automatically identify and locate the coordinate positions of key points of the human body from videos or images. For example, the coordinates of the left shoulder are (x1, y1, z1), the coordinates of the right knee are (x2, y2, z2), the coordinates of the left ankle are (x3, y3, z3), etc. For each frame of the video, the coordinate positions of key points of the human body are automatically identified.

[0071] The human keypoint detection model in this embodiment can be OpenPose, MediaPipe, HRNet, AlphaPose, etc.

[0072] The coordinates of each key point of the human body in each frame of the image are combined into a vector to obtain the human body key point feature vector of each frame. For example, the human body key point feature vector of a certain frame of the image is [0.85, 1.20, 3.10, 0.83, 0.90, 3.15, 0.80, 0.10, 3.20], which means that the coordinates of the hip joint key point are (0.85, 1.20, 3.10), the coordinates of the knee joint key point are (0.83, 0.90, 3.15), and the coordinates of the ankle joint key point are (0.80, 0.10, 3.20).

[0073] The human keypoint feature vectors from multiple frames of a video are arranged chronologically to obtain a sequence of human keypoint feature vectors. For example, frame 1 [0.85, 1.20, 3.10, ..., 3.20], frame 2 [0.86, 1.19, 3.11, ..., 3.21], and frame 3 [0.87, 1.18, 3.12, ..., 3.22]. This sequence of human keypoint feature vectors can be used to describe the movement patterns of the human body and further analyze gait parameters and gait types.

[0074] During the subject's walking process, the spatial coordinates of various key points on the subject's body are recorded using cameras and a human key point detection model. Simultaneously, plantar pressure sensors placed on the soles of the subjects' feet record plantar pressure data. Therefore, the plantar pressure data is synchronized with the walking video. Based on the time points corresponding to the plantar pressure data, the spatial coordinates of various key points on the subject at the same time point can be determined.

[0075] One or more foot pressure sensors can be used to acquire plantar pressure data of the subject. The types of foot pressure sensors can be thin-film pressure sensors, piezoresistive sensors, piezoelectric sensors, fiber optic sensors, etc.

[0076] In this embodiment, the gait events are selected for calculating specific gait parameters; for example, heel strike is selected as the gait event for calculating the gait cycle. When the heel strikes the ground, plantar pressure data changes. Therefore, the timing of the gait event can be determined based on the plantar pressure data.

[0077] The gait parameters in this embodiment can be of various types. Some can be obtained simply by the time point of the corresponding gait event, while others require a comprehensive judgment by combining the gait event and the human key point feature vector at the time of the gait event to increase accuracy.

[0078] The gait parameter extraction method in this embodiment uses plantar pressure data to provide real gait events as "time anchors," effectively overcoming the misjudgment of the timing of gait events caused by occlusion, lighting, and clothing changes in pure visual methods. The error can be controlled at the millisecond level, which is significantly better than methods that rely solely on image analysis.

[0079] The gait parameter extraction method in this embodiment requires no reflective markers, no tight clothing, and no large motion capture system. It can complete high-quality gait parameter extraction using only a depth camera and plantar pressure data. It is not only low-cost and easy to operate, but also has high accuracy.

[0080] Please refer to Figure 2 , Figure 9 In one embodiment of the present invention, gait parameters include gait cycle, plantar pressure data includes heel pressure data, gait events include heel strike events, and the gait parameter extraction method includes:

[0081] Step S100: Acquire the subject's walking video;

[0082] Step S200: Input the walking video into the human key point detection model to obtain the human key point feature vector sequence, which includes the coordinates of human key points at different time points.

[0083] Step S310: Obtain heel pressure data from the subject;

[0084] Step S410: Determine the time point of the heel strike event based on heel pressure data;

[0085] Step S510: Calculate the time difference between two adjacent heel touchdown events on the same side of the body to obtain the gait cycle.

[0086] Please refer to Figure 3In one embodiment of the present invention, gait parameters include stride length, plantar pressure data includes heel pressure data, gait events include heel strike events, and the gait parameter extraction method includes:

[0087] Step S100: Acquire the subject's walking video;

[0088] Step S200: Input the walking video into the human key point detection model to obtain the human key point feature vector sequence, which includes the coordinates of human key points at different time points.

[0089] Step S310: Obtain heel pressure data from the subject;

[0090] Step S410: Determine the time point of the heel strike event based on heel pressure data;

[0091] Step S521: Based on the human body key point feature vector sequence, obtain the coordinates of the ankle joint key point in the horizontal direction corresponding to the time point of the heel contact event.

[0092] Step S522: Calculate the difference in horizontal coordinates of key ankle joint points on the same side of the body for two consecutive heel strike events to obtain stride length.

[0093] In this embodiment, the human body key point feature vector sequence includes multiple human body key point feature vectors arranged in chronological order, used to record the human posture at each time point. Therefore, based on the time point of the heel-to-ground event, the human body key point feature vector corresponding to that time point can be obtained. This human body key point feature vector includes the three-dimensional coordinates of each key point of the human body, from which the horizontal coordinates of the ankle joint key point are obtained.

[0094] The horizontal direction represents the subject's walking direction. During the subject's walking, the sequence of heel strike events is as follows: first left heel strike, first right heel strike, second left heel strike, and second right heel strike. The first and second left heel strikes are two consecutive heel strikes on the left side of the body, and the first and second right heel strikes are two consecutive heel strikes on the right side of the body. The stride length is obtained by subtracting the horizontal coordinates of the ankle joint at the time of the first left heel strike from the horizontal coordinates of the ankle joint at the time of the second left heel strike. Stride length can also be calculated based on two consecutive right heel strikes, using a method analogous to the left heel strikes, which will not be elaborated further.

[0095] In one embodiment, the joint tracking system employs a markerless camera system to sample normal walking videos, ensuring synchronization between the foot pressure acquisition module and the data frames acquired by the markerless camera system. Multiple frames are obtained and assembled into a sampled image sequence. After the recorded video is split frame by frame, a coordinate system is established for the images to acquire the coordinates of each joint and keyframes of heel contact and toe lift-off, thereby obtaining gait time parameters and upper and lower limb joint angle parameters during the walking process.

[0096] In one embodiment, ground-marked scale blocks are used as stride measurement standards, and gait spatiotemporal information is extracted by combining heel-to-ground and toe-to-ground keyframes.

[0097] Please refer to Figure 4 In one embodiment of the present invention, the key points of the human body include the ankle joint key point, the knee joint key point, and the hip joint key point. Step S410 includes:

[0098] Step S411: Differentiate the heel pressure data to obtain the rate of change of heel pressure;

[0099] Step S412: Select a time point where the rate of change of heel pressure is greater than or equal to the first threshold as a candidate time point for the heel strike event;

[0100] Step S413: Based on the human body key point feature vector sequence, calculate the vertical movement velocity of the ankle joint key point corresponding to a candidate time point of the heel touchdown event.

[0101] Step S414: Select the time point when the movement speed is less than or equal to the second threshold as the secondary candidate time point for the heel contact event;

[0102] Step S415: Based on the human body key point feature vector sequence, obtain the coordinates of the ankle joint key point, knee joint key point, and hip joint key point corresponding to the secondary candidate time point of the heel touch-the-ground event.

[0103] Step S416: Calculate the knee flexion angle corresponding to the secondary candidate time point of the heel strike event based on the coordinates of the ankle joint key points, knee joint key points, and hip joint key points.

[0104] Step S417: Select the secondary candidate time point corresponding to the largest knee flexion angle as the time point of the heel strike event.

[0105] The first characteristic of a heel strike event is a change in heel pressure. This embodiment first performs a differential processing of the heel pressure data for initial screening. This initial screening eliminates time points where heel pressure is zero before heel strike. However, ground vibrations, dragging, and other factors may trigger false alarms from the heel pressure sensor, leading to inaccurate gait parameters. This embodiment then performs a second screening based on the vertical movement velocity of key ankle joint points, identifying time points where the downward ankle velocity is close to zero as secondary candidate time points. Since the subject's walking posture is variable, the downward ankle velocity may differ for each heel strike event. To further improve the accuracy of gait parameters, this embodiment also uses the maximum knee flexion angle for final determination.

[0106] This embodiment combines a sudden increase in heel pressure, a decrease in ankle joint velocity, and an increase in knee flexion angle to propose a three-level joint judgment mechanism. This mechanism overcomes the limitations of traditional single-modal detection, resists interference from abnormal signals such as sensor drift or key point recognition deviation, and provides a more stable time point for heel strike events, thereby improving the accuracy of gait parameters.

[0107] In one embodiment, the rate of change of heel pressure is obtained using the forward difference quotient method:

[0108]

[0109]

[0110] in, The rate of change of heel pressure; This refers to the current plantar pressure data; This is the plantar pressure data from the previous moment.

[0111] In one embodiment, the joint angle is calculated using the following formula:

[0112]

[0113] The vector from the elbow to the wrist ( The formula is as follows:

[0114]

[0115] in, For the shoulder's three-dimensional coordinates; The elbow's three-dimensional coordinates; Let the coordinates be the three-dimensional coordinates of the wrist. The joint angle α is extracted using the two vectors calculated above and the inverse cosine formula between the vectors, as follows:

[0116]

[0117] In one embodiment, the knee flexion angle is calculated using the following formula:

[0118]

[0119] In the formula , , This represents the lengths of the three sides of the triangle formed by the hip, knee, and ankle joints. , , Indicates the first Intermediate variables for calculating the angle at each joint and in the nth frame, where For the hip joint, For the knee joint, For the ankle joint, This represents the knee angle at frame n.

[0120] In one embodiment, the vertical velocity of the ankle joint key points is calculated using the following formula:

[0121]

[0122] in, Let n be the average displacement of the ankle joint in the nth frame. The average velocity of the ankle joint in the nth frame. , , In the nth frame, respectively, the ankle joint is... , , Coordinate values ​​on the axis This represents the direction of ankle joint movement in frame n, with negative values ​​indicating downward movement and positive values ​​indicating upward movement. The acquisition frequency is 100 frames per second, with speed measured in meters per second. Each frame is averaged over the preceding and following four frames. In the formula... The unit needs to be adjusted by multiplying by 100 and dividing by 4.

[0123] In one embodiment of the present invention, the gait parameters include stride length, which is calculated using the following formula:

[0124]

[0125] In the formula, For step length, The Z-axis position is the starting point of the nth gait cycle.

[0126] In one embodiment of the present invention, gait parameters include the swing phase time, which is calculated using the following formula:

[0127]

[0128] In the formula, This refers to the frame number corresponding to the start point of the swing phase (end point of the support phase) in the nth gait cycle. and These represent the velocity and direction of the toe point, respectively. This is the frame number corresponding to the start point of the nth gait cycle, which is also the end point of the previous gait cycle. and These are the support phase time and the oscillation phase time, respectively. Since the acquisition frequency is 100 frames per second and the time unit is seconds, it needs to be divided by 100.

[0129] Please refer to Figure 5 In one embodiment of the present invention, gait parameters include stance phase time, plantar pressure data includes big toe pressure data, gait events include toe-off events, and the gait parameter extraction method includes:

[0130] Step S100: Acquire the subject's walking video;

[0131] Step S200: Input the walking video into the human key point detection model to obtain the human key point feature vector sequence, which includes the coordinates of human key points at different time points.

[0132] Step S310: Obtain heel pressure data from the subject;

[0133] Step S320: Obtain the pressure data of the subject's big toe;

[0134] Step S410: Determine the time point of the heel strike event based on heel pressure data;

[0135] Step S420: Determine the time point of the toe lift-off event based on the big toe pressure data;

[0136] Step S530: Calculate the time difference between the toe-off event and the heel-on event to obtain the support phase time.

[0137] In one embodiment, at least two foot pressure sensors are installed on the sole of the subject's shoe, wherein the first foot pressure sensor is installed at a position corresponding to the subject's big toe, and the second foot pressure sensor is installed at a position corresponding to the center of the subject's heel.

[0138] In one embodiment, the gait period is denoted as The oscillation phase time is recorded as The supporting item time is recorded as The specific calculation formula is as follows:

[0139]

[0140] The coordinates of the start and end points of the walk are obtained using a markerless camera and denoted as follows: and The time of start of walking is obtained through a foot pressure sensor. and end time The gait space parameters are obtained by combining depth data and time parameters, as shown in the following formula:

[0141] average walking speed for:

[0142]

[0143] stride :

[0144]

[0145] Step length for:

[0146]

[0147] Let the total number of steps be... Step frequency for:

[0148]

[0149] Please refer to Figure 6 In one embodiment of the present invention, step S420 includes:

[0150] Step S421: Differentiate the big toe pressure data to obtain the rate of change of big toe pressure.

[0151] Step S422: Select a time point where the rate of change of pressure on the big toe is greater than or equal to the third threshold as a candidate time point for the toe-off event.

[0152] Step S423: Based on the human body key point feature vector sequence, calculate the vertical movement velocity of the ankle joint key point corresponding to a candidate time point of the toe-off-ground event.

[0153] Step S424: Select the time point when the movement speed first reaches the fourth threshold as the time point of the toe-off event.

[0154] In one embodiment of the present invention, the gait parameters include the swing phase time, and the gait parameter extraction method includes: calculating the difference between the gait period and the support phase time to obtain the swing phase time.

[0155] Please refer to Figure 7 In one embodiment of the present invention, step S300 includes:

[0156] Step S301: Obtain the subject's raw heel pressure data;

[0157] Step S302: The zero values ​​in the original heel pressure data are retained, and the non-zero values ​​in the original heel pressure data are subjected to Gaussian filtering to obtain plantar pressure data.

[0158] The gait cycle consists of two main phases: the standing phase and the swing phase. During data acquisition, the sensor is susceptible to external signal interference, resulting in numerous spikes in the acquired pressure data. Therefore, the data needs to be filtered first. Foot pressure sensors generate high-frequency errors during acquisition; based on the characteristics of these errors, Gaussian filtering is selected for data preprocessing. To avoid introducing errors due to filtering zero-value pressure data, the zero-value points are not filtered and their original values ​​are retained.

[0159] To address the issue of varying pressure signals among different subjects during walking, in one embodiment of the present invention, step S302 further includes: normalizing the data after Gaussian filtering to obtain plantar pressure data.

[0160] Please refer to Figure 8 This embodiment also provides a gait recognition method based on a recurrent neural network, characterized in that the gait recognition method includes:

[0161] Step S610: Using the gait parameter extraction method of the above embodiment, obtain the human body key point feature vector sequence and gait parameters;

[0162] Step S620: Construct a recurrent neural network. The input of the recurrent neural network is the sequence of human key point feature vectors, and the output of the recurrent neural network is gait parameters and disease category.

[0163] Step S630: Using the sequence of human key point feature vectors within the same gait cycle as input and the gait parameters of the subject as labels, train a recurrent neural network to obtain a trained gait recognition network.

[0164] Step S640: Input the sequence of human key point feature vectors to be identified into the trained gait recognition network to obtain the disease category.

[0165] In one embodiment of the present invention, the combination of the human body key point feature vector sequence and gait parameters corresponding to each pedestrian is used as a training sample, resulting in multiple training samples and forming a training set. Any training sample in the training set is input into the original recurrent neural network, and the loss value of the original recurrent neural network is calculated based on the output of the original recurrent neural network and the gait parameters in the training sample. If the loss value is less than a fifth threshold, the recurrent neural network is used as a trained gait recognition network. Specifically, the parameters of the left and right legs are considered independent, and two training samples are obtained for each walking test of the subject.

[0166] In one embodiment of the present invention, the recurrent neural network is a long short-term memory network.

[0167] Long Short-Term Memory (LSTM) networks utilize the temporal and spatial features of each feature vector in a human keypoint feature vector sequence to achieve long-term and short-term memory of the sequence. This allows for comprehensive consideration of the temporal (contextual features of multiple feature vectors) and spatial (global features of multiple feature vectors and individual features of a single feature vector) characteristics of each feature vector in the sequence, thus identifying the pedestrian identity corresponding to the human keypoint feature vector sequence. The output is the predicted probability of each disease category corresponding to the human keypoint feature vector sequence, and the disease category with the highest predicted probability is taken as the disease category corresponding to that human keypoint feature vector sequence.

[0168] The scope of protection of the gait parameter extraction method based on foot pressure sensor in this application is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting or replacing steps in the prior art based on the principle of this application is included within the scope of protection of this application.

[0169] This embodiment also provides an electronic device, which is a user's mobile device such as a mobile phone, PAD, wearable device, or smart AI device; the electronic device includes a memory for storing computer programs; and a processor for running the computer programs to implement the gait parameter extraction method based on foot pressure sensors in the above embodiments.

[0170] The memory is connected to the processor via the system bus and communicates with it. The memory stores computer programs, and the processor runs the computer programs to enable electronic devices to perform actions such as... Figures 1 to 9 The method for extracting gait parameters is shown.

[0171] It should also be noted that the system bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. The communication interface is used to enable communication between the database access device and other devices (such as clients, read-write databases, and read-only databases).

[0172] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0173] In addition, this embodiment also provides a storage medium storing program instructions, which, when executed by a processor, implement the gait parameter extraction method based on foot pressure sensor described in the above embodiment.

[0174] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof. The storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. This available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state drive (SSD)).

[0175] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0176] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for extracting gait parameters based on a foot pressure sensor, characterized in that, The gait parameter extraction method includes: Obtain walking videos of the subjects; The walking video is input into the human key point detection model to obtain a human key point feature vector sequence. The human key point feature vector sequence includes the coordinates of human key points at different time points. The human key points include ankle joint key points, knee joint key points, and hip joint key points. Obtain raw heel pressure data from the subjects; Zero values ​​in the original heel pressure data are retained, and Gaussian filtering is applied to the non-zero values ​​in the original heel pressure data to obtain plantar pressure data. The plantar pressure data is synchronized with the walking video in time. Determine the timing of at least one gait event based on the plantar pressure data; The gait parameters are obtained based on the time points of the gait events and / or the sequence of feature vectors of human key points. The gait events include heel strike events; Determining the time point of heel strike event based on the heel pressure data includes: The heel pressure data is differentiated to obtain the rate of change of heel pressure. The time point at which the rate of change of heel pressure is greater than or equal to a first threshold is selected as a candidate time point for the heel contact event. Based on the human body key point feature vector sequence, calculate the vertical movement velocity of the ankle joint key point corresponding to a candidate time point of the heel-to-ground event; The time point when the movement speed is less than or equal to the second threshold is selected as the secondary candidate time point of the heel contact event; Based on the human body key point feature vector sequence, obtain the coordinates of the ankle joint key point, the knee joint key point, and the hip joint key point corresponding to the secondary candidate time point of the heel ground contact event; Based on the coordinates of the ankle joint key point, the knee joint key point, and the hip joint key point, calculate the knee flexion angle corresponding to the secondary candidate time point of the heel strike event; The knee flexion angle is calculated using the following formula: ； In the formula , , This represents the lengths of the three sides of the triangle formed by the hip, knee, and ankle joints. , , Indicates the first Intermediate variables for calculating the angle at each joint and in the nth frame, where For the hip joint, For the knee joint, For the ankle joint, This represents the knee angle at frame n. The secondary candidate time point corresponding to the largest knee flexion angle is selected as the time point of the heel strike event; The gait parameters include the gait period; The step of obtaining the gait parameters based on the time points of the gait events and / or the sequence of feature vectors of the human body key points includes: calculating the time difference between two adjacent heel touchdown events on the same side of the human body to obtain the gait cycle.

2. The gait parameter extraction method based on a foot pressure sensor according to claim 1, characterized in that, The gait parameters include stride length, the gait events include heel strike events, and the gait parameter extraction method further includes: The timing of the heel strike event is determined based on the heel pressure data. Based on the human body key point feature vector sequence, obtain the horizontal coordinates of the ankle joint key point corresponding to the time point of the heel ground contact event; The stride length is obtained by calculating the difference in horizontal coordinates of key ankle joint points on the same side of the body during two consecutive heel strike events.

3. The gait parameter extraction method based on a foot pressure sensor according to claim 1, characterized in that, The gait parameters include the stance phase time, the plantar pressure data includes big toe pressure data, the gait events include toe-off events, and the gait parameter extraction method further includes: The timing of the toe lift-off event was determined based on the big toe pressure data. The support phase time is obtained by calculating the time difference between the toe lift-off event and the heel touch-down event.

4. The gait parameter extraction method based on a foot pressure sensor according to claim 3, characterized in that, Determining the time point of the toe-off event based on the big toe pressure data includes: The rate of change of big toe pressure is obtained by differentiating the big toe pressure data. The time point at which the rate of change of pressure on the big toe is greater than or equal to the third threshold is selected as a candidate time point for the toe-off event. Based on the human body key point feature vector sequence, calculate the vertical movement velocity of the ankle joint key point corresponding to a candidate time point of the toe off-ground event; The time point at which the movement speed first reaches the fourth threshold is selected as the time point of the toe-off event.

5. The gait parameter extraction method based on a foot pressure sensor according to claim 3, characterized in that, The gait parameters include the swing phase time, and the gait parameter extraction method includes: calculating the difference between the gait period and the support phase time to obtain the swing phase time.

6. A gait recognition method based on a recurrent neural network, characterized in that, The gait recognition method includes: Using the gait parameter extraction method described in any one of claims 1 to 5, a sequence of human key point feature vectors and gait parameters are obtained; A recurrent neural network is constructed, wherein the input of the recurrent neural network is the sequence of feature vectors of human key points, and the output of the recurrent neural network is the gait parameters and the disease category; Using the sequence of human key point feature vectors within the same gait cycle as input and the subject's gait parameters as labels, the recurrent neural network is trained to obtain a trained gait recognition network. The sequence of feature vectors of key human body points to be identified is input into the trained gait recognition network to obtain the disease category.

7. The gait recognition method based on a recurrent neural network according to claim 6, characterized in that, The recurrent neural network is a long short-term memory network.

Images