A knee angle prediction device and method based on physical topology characteristics

Abstract

The application belongs to the technical field of motion capture, and discloses a knee joint angle prediction device and method based on physical topology characteristics. The method comprises the following steps: collecting knee joint measurement data comprising muscle electrical signals and inertial sensing signals, performing filtering and normalization preprocessing on the knee joint measurement data to obtain normalized knee joint measurement data; performing feature extraction and feature screening on the normalized knee joint measurement data to obtain knee joint feature data; converting the knee joint feature data into graph data comprising nodes and edges; inputting the graph data into a graph neural network to output a predicted knee joint angle. The device relies on a leg ring and comprises a data acquisition device, a data preprocessing unit, a data feature processing unit, a graph data processing unit and a graph neural network unit. The knee joint angle prediction method improves the prediction accuracy of the knee joint angle. The device has the advantages of simple installation and small interference to human motion, and can accurately predict the knee joint angle.

Description

Technical Field

[0001] This invention relates to the field of human motion capture technology, and in particular to a knee joint angle prediction device and method based on physical topology characteristics. Background Technology

[0002] In recent years, with the development of technology, the demand for exoskeleton robots in fields such as medical rehabilitation, industry, and the military has been increasing, and the application scope of exoskeleton robots is also constantly expanding. Predicting the knee joint angle is a crucial step in the continuous and smooth control of exoskeletons. Especially in the field of medical rehabilitation, with the significant increase in hemiplegic patients due to population aging and cerebrovascular diseases, exoskeleton robots that can provide rehabilitation patients with functions such as motor assistance and rehabilitation training have become a research hotspot. Because accurate prediction of lower limb angles can significantly improve human-machine coordination, predicting the knee joint angle has become a critical technical aspect.

[0003] In existing technologies, surface electromyography (sEMG) signals are typically acquired using dispersed surface muscle electrodes, and neural network prediction models, such as BP, CNN, and Bi-LSTM neural networks, are used to predict the knee joint angle. However, existing detection devices suffer from problems such as complex and time-consuming deployment, difficulty in locating muscles in specific areas, and interference with normal human movement when worn.

[0004] In terms of data acquisition, most existing knee joint angle prediction devices use distributed electrodes, which have a large number of electrodes. Each electrode needs to be precisely positioned at the muscle location. However, due to individual differences, the location of the muscle cannot be accurately predicted, making the installation of the device very difficult. In addition, distributed electrodes make it difficult for subjects to adapt to the device quickly and cause great interference to human movement.

[0005] Regarding prediction accuracy, most existing studies employ continuous motion estimation models using principal component analysis and regularized extreme learning machines, with a root mean square error of 8.478. Using BP neural networks for angle estimation, the predicted knee joint angle error ranges from 3.25% to 11.65%. Therefore, existing technologies for predicting knee joint angles suffer from difficulties in equipment installation, significant interference with human movement, and low prediction accuracy.

[0006] Therefore, improving the accuracy of knee joint angle prediction while minimizing the difficulty of equipment installation and the complexity of methods has become an urgent technical problem to be solved. Summary of the Invention

[0007] The purpose of this invention is to address the problems of low accuracy and high system complexity in knee joint angle prediction. It proposes a knee joint angle prediction device and method based on physical topology characteristics, aiming to improve the accuracy of knee joint angle prediction while minimizing the complexity of system implementation.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A method for predicting knee joint angles based on physical topology characteristics, comprising:

[0010] Knee joint measurement data are collected, including electrical muscle signals and inertial sensing signals;

[0011] The knee joint measurement data were filtered and normalized preprocessed to obtain normalized knee joint measurement data.

[0012] Feature extraction and feature filtering were performed on normalized knee joint measurement data to obtain knee joint feature data;

[0013] Convert knee joint feature data into graph data including nodes and edges;

[0014] The graph data is input into the graph neural network, and the predicted knee angle is output.

[0015] The electromyographic signal is a signal from the surface of the knee joint muscles and is acquired by a configured electromyographic signal acquisition unit; the inertial sensing signal is acquired by an inertial sensor unit located at the knee joint; the electromyographic signal acquisition unit includes multiple surface electromyographic electrodes distributed on the surface of the knee joint muscles.

[0016] The surface muscle electrodes are distributed according to the physical topological characteristics of the knee joint muscles.

[0017] The step of filtering and normalizing the knee joint measurement data to obtain normalized knee joint measurement data includes: performing baseline removal processing on the electromyography (EMG) signal and then performing high-pass filtering; performing low-pass filtering on the inertial sensing signal; and performing normalization processing on the filtered EMG signal and inertial sensing signal respectively to obtain normalized EMG signal and normalized inertial sensing signal.

[0018] The feature extraction and filtering of normalized knee joint measurement data includes: extracting knee joint feature data, including time-domain features, transform-domain features, and time-leading features, from the normalized knee joint measurement data; evaluating the knee joint feature data using one or more weighting methods and ranking them by weight; and filtering multiple common features from one or more weighted ranking sequences according to weight thresholds as the filtered knee joint feature data. The weighting methods include weight calculation based on the NCFS method, weight calculation using the maximum correlation minimum redundancy method, and weight calculation based on Relief filtering.

[0019] The process of converting knee joint feature data into graph data including nodes and edges includes: determining a node feature matrix based on the number of sliding windows and feature data; calculating the correlation coefficient of adjacent nodes based on the node feature matrix; and obtaining the features of the edges between nodes based on the correlation coefficient of adjacent nodes.

[0020] The graph neural network includes a first graph neural subnetwork, a second graph neural subnetwork, a readout layer, a knockout layer, and a linear layer;

[0021] The process of inputting graph data into a graph neural network and outputting a predicted knee joint angle includes:

[0022] The first graph neural subnetwork is used to perform adjacency aggregation, activation, and pooling on each node of the graph data to form a new node feature vector.

[0023] The second neural subnetwork is used to perform adjacency aggregation, activation, and pooling on each new node feature vector to form a new node feature vector again.

[0024] The readout layer is used to obtain the first and second graph neural subnetworks to form new node feature vectors;

[0025] The culling layer is used to perform random deletion processing on the new node feature vectors;

[0026] The feature vector information of the undeleted nodes is integrated using a linear layer and then linearly processed to output the predicted knee joint angle.

[0027] Compared with existing technologies, the method for predicting knee joint angles based on physical topology proposed in this application has the following advantages:

[0028] 1. Using a leg ring with multiple surface muscle electrodes allows for the collection of more biological information;

[0029] 2. Introducing one or more inertial sensing signals can more accurately obtain displacement change information;

[0030] 3. The leg rings used are more convenient to wear and can wirelessly transmit data to the PC, reducing interference with human movement and making exercise more natural;

[0031] 4. The ring structure of the leg ring is matched with the physical topology of the leg. The leg muscles form a tendon that surrounds the patella and extends downward to form the patellar ligament, which terminates at the tibial tuberosity. The detection electrodes of the leg ring are distributed according to the physical topology, which makes the measurement data obtained based on the physical topology of the leg more accurate.

[0032] 5. For feature selection between muscle electroelectrodes, weights are obtained through multiple weight calculation methods. Features with higher weight values ​​are selected. Considering different feature conditions, the NCFS method, the maximum correlation minimum redundancy method, and the Relief method are used respectively. The selected node features are more significant and can significantly improve prediction accuracy.

[0033] 6. The GNN model used showed a superior match with the actual knee joint angle. The results showed that the statistical measure of goodness of fit increased by 6%, the deviation between the actual and predicted values ​​decreased by an average of 60.5%, and the linear relationship between the expected and observed values ​​improved by an average of 1.5%.

[0034] The present invention also provides a knee joint angle prediction device based on physical topology characteristics, comprising:

[0035] A data acquisition device is configured to acquire knee joint measurement data, which includes electrical muscle signals and inertial sensing signals.

[0036] The data preprocessing unit is configured to perform filtering and normalization preprocessing on the knee joint measurement data to obtain normalized knee joint measurement data.

[0037] The data feature processing unit is configured to extract and filter features from normalized knee joint measurement data to obtain knee joint feature data.

[0038] The graph data processing unit is configured to convert knee joint feature data into graph data including nodes and edges;

[0039] A graph neural network unit is configured to input graph data into a graph neural network, which outputs a predicted knee angle.

[0040] The data acquisition device is connected to the data preprocessing unit, the data preprocessing unit is connected to the data feature processing unit, the data feature processing unit is connected to the graph data processing unit, and the graph data processing unit is connected to the graph neural network unit.

[0041] The data acquisition device includes:

[0042] The electromyography (EMG) acquisition unit is configured to acquire EMG signals from the surface of the knee joint muscles.

[0043] An inertial sensor unit is configured to acquire inertial sensing signals at the knee joint;

[0044] The muscle electromyography (EMG) acquisition unit includes multiple surface EMG electrodes distributed on the surface of the knee joint muscles.

[0045] The surface muscle electrodes are distributed according to the physical topological characteristics of the knee joint muscles.

[0046] Compared with the prior art, the beneficial effects of the device for predicting knee joint angle based on physical topology provided by the present invention are the same as the beneficial effects of the method for predicting knee joint angle based on physical topology described in the above technical solutions, and will not be repeated here. Attached Figure Description

[0047] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0048] Figure 1 This is a flowchart of the knee joint angle prediction method based on physical topology in an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of a knee joint angle prediction device based on physical topology characteristics in an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the graph neural network structure in an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of the feature weight sorting results in an embodiment of the present invention;

[0052] Figure 5 This is the GraphSAGE1 framework diagram in an embodiment of the present invention;

[0053] Figure 6 This is a diagram showing the feature weight ranking results in an embodiment of the present invention. Detailed Implementation

[0054] To facilitate a clear description of the technical solutions in the embodiments of the present invention, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. For example, the first threshold and the second threshold are merely used to distinguish different thresholds and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" are not necessarily different.

[0055] It should be noted that in this invention, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0056] In this invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a combination of a and b, a combination of a and c, a combination of b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0057] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, illustrates a method for predicting knee joint angles based on physical topology, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more definite definition of the scope of protection of the present invention. The specific implementation of the method for predicting knee joint angles based on physical topology according to the present invention is described in detail with reference to the accompanying drawings and specific embodiments.

[0058] Example 1

[0059] like Figure 1 The flowchart shown is for a knee joint angle prediction method based on physical topology characteristics, including the following steps:

[0060] S1, collect knee joint measurement data; the knee joint measurement data includes electromyographic signals and inertial sensing signals; the electromyographic signals are surface signals of the knee joint muscles and are collected by a configured electromyographic signal acquisition unit; the inertial sensing signals are collected by an inertial sensor unit located at the knee joint; the electromyographic signal acquisition unit includes multiple surface electromyographic electrodes distributed on the surface of the knee joint muscles.

[0061] S2, perform filtering and normalization preprocessing on the knee joint measurement data to obtain normalized knee joint measurement data;

[0062] The filtering specifically includes: performing baseline removal processing on the electromyography signal followed by high-pass filtering; performing low-pass filtering on the inertial sensing signal to obtain filtered knee joint measurement data; the filtered knee joint measurement data includes filtered electromyography signal and filtered inertial sensing signal.

[0063] The normalization preprocessing specifically involves: normalizing the filtered electromyography signal and the filtered inertial sensing signal respectively to obtain normalized knee joint measurement data; the normalized knee joint measurement data includes normalized electromyography signal and normalized inertial sensing signal.

[0064] The normalization process includes, but is not limited to, row normalization, column normalization, norm normalization, max-min normalization, neural network normalization, logarithmic function normalization, and arctangent function normalization.

[0065] S3, extract and filter features from normalized knee joint measurement data to obtain knee joint feature data;

[0066] The feature extraction steps include extracting time-domain features, transform-domain features, and time-lead features from the preprocessed surface electromyography (SEMG) signal and inertial sensing signal. Specifically: the time-domain features include variance, root mean square (RMS) value, integral value, waveform length, and mean absolute value; the transform-domain features include the absolute mean, RMS value, Wilson amplitude, and waveform length of the decomposition coefficients extracted using wavelet transform and wavelet packet decomposition; the time-lead feature is extracted through a time advance window.

[0067] Preferably, the wavelet transform includes, but is not limited to, continuous wavelet transform, discrete wavelet transform, and stationary wavelet transform. The wavelet basis used in the wavelet transform includes, but is not limited to, db, bior, coif, and sym wavelet bases, and: db wavelet bases include: db2, db4, db5, db7, and db8; bior wavelet bases include: bior2.3, bior3.7, and bior6.8; coif wavelet bases include: coif5; and sym wavelet bases include sym2, sym4, and sym8.

[0068] In the feature selection step, specifically: the weights of the various features extracted by the multiple feature extraction methods in step S3 are calculated and sorted using the NCFS method, the maximum correlation minimum redundancy method and the Relief method respectively, to obtain the first, second and third weight sorting sequences;

[0069] The process of calculating the weights of multiple features and ranking them using the NCFS method is as follows:

[0070] The weights for each surface electromyography (EMG) signal node and inertial sensing feature are determined as follows:

[0071]

[0072] Where K is the number of nodes, and in specific implementation, K = 16; the w f w fi w adfi These are the weights of each feature, each node feature, and the preceding features of the response time; the weight value w for each feature. f Sort the sequences to obtain the first weighted sorted sequence.

[0073] The maximum correlation minimum redundancy algorithm is based on an mRMR evaluation algorithm, where mRMR is the maximum value of the difference between correlation and redundancy. Given two random variables x and y, with probability density functions p(x) and p(y) respectively, the mutual information I(x,y) is:

[0074]

[0075] p(x,y) is the joint probability density function of random variables x and y; the correlation between the feature set and the target value is calculated, and the correlation is the average of all mutual information between each feature in the feature set and the target value; the redundancy of all features in the feature set is calculated as the average of all mutual information between two features; finally, the weights of multiple features are calculated based on the maximum correlation and minimum redundancy method and the weights are sorted to obtain the second weight sorting sequence.

[0076] The third weight ranking sequence is obtained by calculating feature weights based on the Relief filter and sorting the weights. The Relief algorithm assigns different weights to features based on their relevance to the category, and features with weights below a certain threshold are removed. The relevance is based on a feature's ability to distinguish nearby samples. Specifically, a sample (R) is randomly selected from the training set. Then, the nearest neighbor sample (H) is selected from samples of the same category as R. The nearest neighbor sample (M) is found from samples of different categories than R. If the distance between R and H on a certain feature is less than the distance between R and M, then the feature is beneficial for distinguishing nearest neighbors of the same and different categories, and the weight of that feature is increased. Conversely, if the distance between R and H on a certain feature is greater than the distance between R and M, then the feature has a negative effect on distinguishing nearest neighbors of the same and different categories, and the weight of that feature is decreased. This process is repeated m times, and finally, the average weight of each feature is obtained. The third weight ranking sequence is then obtained after weight sorting.

[0077] The top N common features from the first, second, and third weighted sorting sequences are selected as the filtered knee joint feature data. The feature filtering method among the electrode nodes described above selects features with higher weight values ​​and comprehensively considers the use of the NCFS method, the maximum correlation minimum redundancy method, and the Relief method under different feature conditions, which can effectively improve prediction accuracy.

[0078] S4 converts knee joint feature data into graph data including nodes and edges;

[0079] S5 inputs the graph data into the graph neural network and outputs the predicted knee angle.

[0080] like Figure 2 The diagram shows a device for predicting knee joint angles based on physical topology, including:

[0081] A data acquisition device is configured to acquire knee joint measurement data, which includes electrical muscle signals and inertial sensing signals.

[0082] The data preprocessing unit is configured to perform filtering and normalization preprocessing on the knee joint measurement data to obtain normalized knee joint measurement data.

[0083] The data feature processing unit is configured to extract and filter features from normalized knee joint measurement data to obtain knee joint feature data.

[0084] The graph data processing unit is configured to convert knee joint feature data into graph data including nodes and edges;

[0085] A graph neural network unit is configured to input graph data into a graph neural network, which outputs a predicted knee angle.

[0086] The data acquisition device includes:

[0087] The electromyography (EMG) acquisition unit is configured to acquire EMG signals from the surface of the knee joint muscles.

[0088] An inertial sensor unit is configured to acquire inertial sensing signals at the knee joint;

[0089] The muscle electromyography (EMG) acquisition unit includes multiple surface EMG electrodes distributed on the surface of the knee joint muscles.

[0090] In step S1, knee joint measurement data is collected by using multiple surface electromyography (EMG) electrodes distributed on the surface of the knee joint muscles to acquire EMG signals from the muscle surface. The knee joint measurement data includes EMG signals and inertial sensing signals; inertial sensing signals at the knee joint are acquired using an inertial sensor unit. The data acquisition device is specifically a leg ring worn on the knee joint, a ring-shaped knee brace covering the knee joint, or a strap fixed to the knee joint with buckles.

[0091] In practical implementation, the data acquisition device uses a leg ring with 16 or 32 surface muscle electrodes to collect more biological information; the surface muscle electrodes have an ABS plastic substrate and stainless steel electrode plates. The inertial sensor unit includes one or more inertial sensors.

[0092] The data acquisition device also includes a wireless transmission unit for transmitting knee joint measurement data to a data processing terminal, such as a PC or a remote server connected via a network. Wireless data transmission reduces interference with human movement, making movement more natural and the device easier to wear. Wireless transmission methods include mobile communication networks, Wi-Fi networks, and Bluetooth.

[0093] like Figure 3 The diagram shows the physical topology of the data acquisition device for knee joint measurement data acquisition. Figure 3 The illustrated data measurement device includes seven surface electromyography electrodes and two inertial sensors.

[0094] Taking the leg ring device as an example, Figure 3 Electrodes 0 and 1 are inertial sensor electrodes, while electrodes 2 through 8 are surface muscle electrodes. Inertial sensors 0 and 1 are located directly above and below the knee joint, respectively, to detect and measure data such as thigh acceleration and tilt.

[0095] Figure 3The diagram also illustrates the muscle distribution of the knee joint, primarily the quadriceps femoris muscle group on the anterior side above the knee. Specifically, 'a' represents the rectus femoris, with electrodes 2 and 8 detecting signals at different locations within this muscle group; 'b' and 'c' represent the vastus lateralis, muscles on the lateral side above the knee joint, detected by electrodes 7 and 3, respectively; 'd' represents the semitendinosus, detected by electrodes 4 and 6; and 'e' represents the biceps femoris muscle group on the medial side above the knee joint, detected by electrode 5.

[0096] like Figure 4 As shown, another physical topology diagram of the data acquisition device for knee joint measurement data acquisition is given. Figure 4 The illustrated data measurement device includes four surface electromyography electrodes and two inertial sensors.

[0097] Taking the leg ring device as an example, Figure 4 Electrodes 0 and 1 are inertial sensor electrodes, while electrodes 2 through 5 are surface muscle electrodes. Inertial sensors 0 and 1 are located directly above and below the knee joint, respectively, to detect and measure data such as thigh acceleration and tilt. Figure 4 The diagram also illustrates the muscle distribution of the knee joint, mainly the quadriceps femoris group on the anterior side above the knee joint. Among them, a is the quadriceps femoris group composed of the rectus femoris and the vastus medialis and lateralis, and the second and fifth electrodes detect signals at different positions of this muscle group; b is the vastus medialis, and the fourth electrode detects signals at the corresponding position of this muscle group; c is the vastus lateralis, and the third electrode detects signals at the corresponding position of this muscle group.

[0098] Taking the leg band device as an example, each electrode is positioned corresponding to the knee. After the leg band is worn, the electrodes naturally contact the corresponding measurement points on the leg, allowing for more accurate data measurement. Furthermore, the leg band is easy to wear, simplifying the data measurement process and eliminating the problem of overly complex electrode connections.

[0099] In S2, the knee joint measurement data undergoes filtering and normalization preprocessing to obtain normalized knee joint measurement data. Specifically, this includes: performing baseline removal processing on the electromyography (EMG) signal followed by high-pass filtering; performing low-pass filtering on the inertial sensing signal; in practice, the baseline of the acquired EMG signal is removed, and the baseline-removed EMG signal is then subjected to high-pass filtering to remove low-frequency noise below 20Hz; the acquired inertial sensing signal is subjected to low-pass filtering to remove high-frequency noise above 10Hz; and the filtered EMG signal and inertial sensing signal are then normalized to obtain normalized EMG signal and normalized inertial sensing signal, respectively. The normalization includes, but is not limited to, row normalization, column normalization, norm normalization, max-min normalization, neural network normalization, log-log function normalization, and arctangent function normalization; in practice, L2 norm normalization is used.

[0100] In S3, feature extraction and feature filtering are performed on the normalized knee joint measurement data to obtain knee joint feature data.

[0101] In the feature extraction step, time-domain features, transform-domain features, and time-lead features of the preprocessed surface electromyography (sEMG) signal and inertial sensing signal are extracted. Specifically: the time-domain features are extracted using sEMG characteristics, including variance, root mean square value, electromyography integral value, waveform length, and mean absolute value; the transform-domain features include the absolute mean, root mean square value, Wilson amplitude, and waveform length of the decomposition coefficients; and the time-lead features are extracted using a time advance window.

[0102] In the feature selection step, after extracting multiple features, the weights are calculated and sorted according to one or more different weight determination methods to obtain a weight sorting sequence.

[0103] The weights for each surface electromyography (EMG) signal node and inertial sensing feature are generated using the NCFS method. The calculation method is as follows: Where K is the number of nodes, and w f w fi w adfi The weights of each feature, each node feature, and the temporal preceding features of the response are combined to obtain the first weighted sort sequence.

[0104] The feature data is processed and transformed, the joint distribution and marginal distribution of the two features are calculated, the feature data is normalized, and the results of each dimension are stored using reasonable feature data. Then, the distribution and mutual information between features and between features and response variables are calculated, and the features are sorted using the maximum correlation and minimum redundancy method to obtain the second weighted sorting sequence.

[0105] The feature weights are calculated based on the Relief filtering method, and the third weight sorting sequence is obtained.

[0106] The common features of the top N items in the first, second, and third weighted sorting sequences are used as the filtered knee joint feature data; the value of N is greater than or equal to 5, and in this embodiment, N is selected as 5;

[0107] The feature selection method between the above electrode nodes selects features with higher weight values ​​and comprehensively considers the use of NCFS method, maximum correlation minimum redundancy method and Relief method under different feature conditions, which effectively improves the prediction accuracy.

[0108] In S4, the knee joint feature data is converted into graph data including nodes and edges; specifically, the filtered knee joint feature data is converted into graph data, generating nodes and edges; the node features are represented by a matrix as follows:

[0109]

[0110] Among them, F node The feature matrix of the node, For each node's feature value, t is the number of sliding windows, N is the number of features, and L is the total number of nodes;

[0111] Based on the feature values ​​of nodes, the correlation coefficient between adjacent nodes is obtained as follows:

[0112] Where, p ij f is the correlation coefficient between adjacent nodes. i f is the eigenvalue of node i. j Let Cov(f) be the eigenvalue of node j. i ,f j Let be the covariance between two nodes i and j, and let D(f) be the covariance between them. i ) and D(f j ) are respectively f i and f j The variance.

[0113] Based on the correlation coefficient of nodes, the feature values ​​of outgoing edges are determined, as follows:

[0114] Among them, F edg Let N be the edge feature, and p be the number of nodes. i Let i be the i-th feature of each node based on the correlation coefficient.

[0115] In S5, the graph data is input into the graph neural network, and the predicted knee angle is output.

[0116] The graph neural network includes a first graph neural subnetwork, a second graph neural subnetwork, a readout layer, a culling layer, and a linear layer. The first graph neural subnetwork performs adjacency aggregation, activation, and pooling on each node of the graph data to form a new node feature vector. The second graph neural subnetwork performs adjacency aggregation, activation, and pooling on each new node feature vector to form another new node feature vector. The readout layer acquires the new node feature vectors formed by the first and second graph neural subnetworks. The culling layer performs random deletion on the acquired new node feature vectors. The linear layer integrates the information from the remaining new node feature vectors and performs linear processing to output the predicted knee joint angle.

[0117] The first and second graph neural subnetworks use the GraphSAGE framework.

[0118] like Figure 5As shown, in the first graph neural subnetwork GraphSAGE1 framework, aggregation is performed on the neighboring nodes of each node in the input graph data, and the aggregation result is concatenated with the node's feature vector to obtain the feature vector of each node for downstream use. The newly formed node feature vectors are then activated to complete the nonlinear transformation of the data; after activation, pooling is performed to compress the amount of data and parameters, reducing overfitting.

[0119] In the second graph neural subnetwork GraphSAGE2 framework, the new node feature vectors output by the pooling of the first graph neural subnetwork GraphSAGE1 framework are input into the GraphSAGE2 framework, and after activation and pooling processing, they are read out.

[0120] During reading, the new node feature vectors output by the first and second neural subnetworks are read out.

[0121] After reading, randomly hide half of the neuron nodes (randomly delete them).

[0122] For the new node feature vectors left after random deletion, the node information after graph convolution is integrated into the graph; the integrated information is then processed by two linear layers.

[0123] After steps S1 to S5, the knee joint angle model prediction operation was completed, and the predicted knee joint angle was obtained.

[0124] like Figure 5 As shown, the convolutional layer is SAGEConv. Furthermore, this model uses SAGEPooling within the GraphSAGE framework to reduce the graph size while preserving the most important information. After graph convolution, maintenance information from the nodes is aggregated into the entire graph through a readout function.

[0125] During the training of a graph neural network, for newly deleted nodes, the input is propagated forward through the modified network, and the resulting loss is propagated backward through the modified network. After this process is completed on the training samples, the corresponding parameters are updated on the neurons that were not deleted using the stochastic gradient descent method. Finally, the node information after graph convolution is integrated into the graph.

[0126] By employing the above-mentioned two-level graph neural network (GNN) model, the knee joint angle prediction method of this application shows a superior match with the actual knee joint angle. The results show that the statistical measure of goodness of fit increases by 6%, the deviation between the actual value and the predicted value decreases by an average of 60.5%, and the linear relationship between the expected value and the observed value improves by an average of 1.5%.

[0127] Example 2

[0128] Methods for predicting knee joint angles can be applied to assess athletes' posture and condition during training; in rehabilitation therapy, they can help patients understand the degree of recovery; and for people who stand or sit for long periods, they can help prevent knee joint strain.

[0129] In practice, test data were collected from four subjects. Subjects wore leg bands containing 16 electromyographic electrodes and one inertial sensor; the bands were positioned directly above and below the knee joint. Test data were collected from the subjects in various scenarios.

[0130] Data collection was conducted on subjects going up and down stairs: the first 2 seconds of data collection were spent standing, and the 2nd to 10th seconds were spent going up and down stairs.

[0131] Data collection for the standing and squatting experiments: The data collection for the standing and squatting experiments included the first 2 seconds of standing, the 2nd to 4th seconds of squatting, the 4th to 6th seconds of maintaining the squatting position, and the 6th to 8th seconds of standing up. S4. Data collection for the walking experiments: The data collection for the walking experiments included the first 2 seconds of standing, and the following 2 to 55 seconds of walking.

[0132] The baseline of the acquired surface electromyography (EMG) signal was removed, and the resulting EMG signal was then subjected to a high-pass filter to remove low-frequency noise below 20Hz. The acquired inertial sensor signal was subjected to a low-pass filter to remove high-frequency noise above 10Hz. The filtered EMG and inertial sensor signals were then normalized.

[0133] The preprocessed surface muscle electromyography (EMG) signal and inertial sensing signal are wirelessly transmitted to the host computer. The preprocessed motion state test data are divided into training set, validation set and test set in a ratio of 6:1:3.

[0134] During training, training set data is used. After the trained model is validated by the validation set, the performance can be tested using the test set.

[0135] Next, feature extraction was performed on the preprocessed surface muscle electrical signals and inertial sensing signals using various extraction methods.

[0136] The temporal features of surface muscle electromyography (EMG) signals and inertial sensing signals are extracted. The temporal features include variance, EMG integral value, root mean square (RMS), waveform length, mean absolute value, and Wilson amplitude.

[0137] DB7 wavelet was used to perform three-level signal decomposition to extract time-frequency features. The time sliding window used in the extraction process was 100ms in length and the window increment was 10ms. The start and end points of the time advance window were calculated as follows:

[0138] Among them, t adv This indicates the advance of the time, where T is the sampling period and n is the number of samples. ads n s These are the starting points of the advance time window and the time window, respectively; n a d e n e These represent the end point of the advance time window and the end point of the time window, respectively.

[0139] Based on time-frequency characteristics, a time lead feature is extracted using an advance window: the mean absolute value, root mean square, and waveform length of the decomposition coefficients are extracted, along with the time lead feature based on these characteristics. The start and end points of the time lead window are calculated, resulting in a lead time of 200ms.

[0140] After extracting node feature data through various extraction methods, feature filtering is performed. Specifically, by reducing feature dimensionality and selecting features, the NCFS method is used to generate the weights of each surface electromyography signal node and inertial sensing feature. The calculation method is as follows:

[0141] like Figure 6 The image shows the feature weight ranking results, with the top five features selected as the filtered features. The top five features in the feature weight ranking are waveform length, root mean square, third-order absolute mean, variance, and electromyographic integral value.

[0142] In this embodiment, 17 nodes and 16 edges are included. The nodes include 16 electromyography (EMG) electrode nodes and one inertial sensor node. Nodes 1-16 represent surface EMG signal electrodes, and node 0 represents the inertial sensor. The electrode nodes are connected to adjacent nodes according to the topological characteristics of the leg band. For example, node No.1 is connected to nodes No.2 and No.16.

[0143] In the process of converting the selected features into graph data, the dimension of each node is calculated, and the result is a dimension of 200; then, the correlation coefficient of related nodes is calculated; the attributes of adjacent edges are represented; and the attributes of edges and nodes are generated based on the calculation results to obtain graph data.

[0144] The graph data is used to form an image, which is then input into a graph neural network model. Specifically, nodes are input into the GraphSAGE1 framework, and aggregation is performed on the neighbors of each node. The aggregation result is then concatenated with the node's features to obtain the vectors of each node for downstream use. The newly formed node feature vectors are then activated to complete the nonlinear transformation of the data. After activation, pooling based on a self-attention mechanism is performed to compress the amount of data and parameters to 0.8 of the original number of nodes, thereby reducing overfitting.

[0145] The new node is read out as part of the process, while the other part is fed into the GraphSAGE2 framework. After activation and pooling are completed, it is read out again.

[0146] After reading from the two-level graph neural network, half of the neuron nodes are randomly hidden (randomly deleted), and then training and optimization are performed; the input is forward-propagated through the modified network, and then the resulting loss is back-propagated through the modified network.

[0147] At this point, after a small batch of training samples has completed the training process of this graph neural network, the corresponding parameters of the neurons that were not deleted are updated using stochastic gradient descent. Then, the node information from the graph convolution is integrated back onto the graph; the integrated information is then processed in two linear layers.

[0148] Figure 5 The convolutional layer used is SAGEConv. Furthermore, this model employs SAGEPooling within the GraphSAGE framework to reduce the graph size while preserving the most important information. After graph convolution, maintenance information from the nodes is aggregated into the entire graph through a readout function.

[0149] After training, a graph neural network model is obtained. When using the model to predict angles, the predicted knee joint angle can be directly output through the linear layer.

[0150] In the process of evaluating the performance of graph neural network models, the statistical measure Rfit is calculated to measure the goodness of fit. 2 The deviation between actual and predicted values ​​(RMSE) and the linear relationship (CC) between expected and observed values. The R... 2 The value of is between 0 and 1. The closer to 1, the better the model fits the data, and the closer to 0, the worse the model fits the data. RMSE is the root mean square value of the prediction error of the regression model. The smaller the RMSE, the higher the prediction accuracy of the model. CC is used to evaluate the linear correlation between the predicted value and the actual observed value. Its value is between -1 and 1, where -1 represents a perfect negative correlation, 0 represents no correlation, and 1 represents a perfect positive correlation.

[0151] The statistical value R of measuring goodness of fit based on graph neural network models, CNN models, and BiLSTM models. 2 The deviation between actual and predicted values ​​(RMSE) and the linear relationship (CC) between expected and observed values ​​are shown in Table 1. The performance results for comparison are presented in Table 1.

[0152] Table 1: Performance Comparison of Graph Neural Network Model, CNN Model, and BiLSTM Model

[0153]

[0154] As shown in Table 1, the average estimated RMSE of the graph neural network model is 3.23, which is 63% lower than that of the CNN model and 58% lower than that of the Bi-LSTM model. The estimated CC is 0.994, which is 2% higher than that of the CNN model and 1% higher than that of the Bi-LSTM model. The estimated R2 is 0.985, which represents a 7% and 5% performance improvement over the CNN and Bi-LSTM models, respectively.

[0155] The present invention also provides a computer storage medium storing instructions that, when executed, implement the method for predicting knee joint angles based on physical topology as described above.

[0156] The present invention also provides a computing device, including a processor and a communication interface coupled to the processor; the processor is used to run computer programs or instructions to implement the method for predicting knee joint angles based on physical topology as described above.

[0157] Those skilled in the art will readily recognize that, based on the units and method steps of the various examples described in conjunction with the embodiments disclosed herein, the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the present invention.

[0158] In this embodiment of the invention, the knee joint angle prediction device based on physical topology characteristics, as described in the above method example, can be divided into functional modules. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this embodiment is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used.

[0159] Although the invention has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made therein without departing from the spirit and scope of the invention. Accordingly, this specification and drawings are merely exemplary descriptions of the invention as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if such modifications and variations of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include such modifications and variations.

Claims

1. A method for predicting knee joint angles based on physical topology characteristics, characterized in that, include: Knee joint measurement data are collected, including electrical muscle signals and inertial sensing signals; The knee joint measurement data were filtered and normalized preprocessed to obtain normalized knee joint measurement data. Feature extraction and feature filtering were performed on normalized knee joint measurement data to obtain knee joint feature data; Convert knee joint feature data into graph data including nodes and edges; The graph data is input into the graph neural network, and the predicted knee angle is output. The electromyographic signal is a signal from the surface of the knee joint muscles and is acquired by a configured electromyographic signal acquisition unit; the inertial sensing signal is acquired by an inertial sensor unit located at the knee joint; the electromyographic signal acquisition unit includes multiple surface electromyographic electrodes distributed on the surface of the knee joint muscles. The graph neural network includes a first graph neural subnetwork, a second graph neural subnetwork, a readout layer, a knockout layer, and a linear layer; The process of inputting graph data into a graph neural network and outputting a predicted knee joint angle includes: The first graph neural subnetwork is used to perform adjacency aggregation, activation, and pooling on each node of the graph data to form a new node feature vector. The second neural subnetwork is used to perform adjacency aggregation, activation, and pooling on each new node feature vector to form a new node feature vector again. The readout layer is used to obtain the first and second graph neural subnetworks to form new node feature vectors; The culling layer is used to perform random deletion processing on the new node feature vectors; The feature vector information of the undeleted nodes is integrated using a linear layer and then linearly processed to output the predicted knee joint angle.

2. The method according to claim 1, characterized in that, The surface muscle electrodes are distributed according to the physical topological characteristics of the knee joint muscles.

3. The method according to claim 1, characterized in that, The step of filtering and normalizing the knee joint measurement data to obtain normalized knee joint measurement data includes: After baseline removal processing, the electromyography signal is high-pass filtered. Low-pass filtering is applied to the inertial sensing signal; The filtered electromyographic (EMG) signal and inertial sensor signal were normalized to obtain normalized EMG signal and normalized inertial sensor signal, respectively.

4. The method according to claim 1, characterized in that, The feature extraction and feature filtering of the normalized knee joint measurement data includes: Extract knee joint feature data, including time domain features, transform domain features, and time lead features, from normalized knee joint measurement data; One or more weighting evaluation methods are used to evaluate the knee joint feature data and then the weights are ranked accordingly. Multiple common features are selected from one or more weighted sorting sequences based on weight thresholds to form the filtered knee joint feature data.

5. The method according to claim 4, characterized in that, The weight evaluation methods include weight calculation methods based on the NCFS method, weight calculation methods using the maximum correlation minimum redundancy method, and weight calculation methods based on Relief filtering.

6. The method according to claim 1, characterized in that, The process of converting knee joint feature data into graph data including nodes and edges includes: Based on the number of sliding windows and feature data, determine the feature matrix of each node; The correlation coefficient between adjacent nodes is calculated based on the node feature matrix; The characteristics of the edges between nodes are obtained based on the correlation coefficient between adjacent nodes.

7. A knee joint angle prediction device based on physical topology characteristics, characterized in that, include: A data acquisition device is configured to acquire knee joint measurement data, which includes electrical muscle signals and inertial sensing signals. The data preprocessing unit is configured to perform filtering and normalization preprocessing on the knee joint measurement data to obtain normalized knee joint measurement data. The data feature processing unit is configured to extract and filter features from normalized knee joint measurement data to obtain knee joint feature data. The graph data processing unit is configured to convert knee joint feature data into graph data including nodes and edges; A graph neural network unit is configured to input graph data into a graph neural network, which outputs a predicted knee angle. The graph neural network includes a first graph neural subnetwork, a second graph neural subnetwork, a readout layer, a knockout layer, and a linear layer; The process of inputting graph data into a graph neural network and outputting a predicted knee joint angle includes: The first graph neural subnetwork is used to perform adjacency aggregation, activation, and pooling on each node of the graph data to form a new node feature vector. The second neural subnetwork is used to perform adjacency aggregation, activation, and pooling on each new node feature vector to form a new node feature vector again. The readout layer is used to obtain the first and second graph neural subnetworks to form new node feature vectors; The culling layer is used to perform random deletion processing on the new node feature vectors; The feature vector information of the undeleted nodes is integrated using a linear layer and then linearly processed to output the predicted knee joint angle.

8. The apparatus according to claim 7, characterized in that, The data acquisition device includes: The electromyography (EMG) acquisition unit is configured to acquire EMG signals from the surface of the knee joint muscles. An inertial sensor unit is configured to acquire inertial sensing signals at the knee joint; The muscle electromyography (EMG) acquisition unit includes multiple surface EMG electrodes distributed on the surface of the knee joint muscles.

9. The apparatus according to claim 8, characterized in that, The surface muscle electrodes are distributed according to the physical topological characteristics of the knee joint muscles.

Images