A method and device for identifying and gait assessment across patient phases, and a storage medium

Abstract

The application discloses a kind of across patient phase identification and gait assessment method, belong to rehabilitation medical technical field.The method includes: the friction electricity signal generated when the lower limb of the patient to be measured is collected, obtains multimodal data;Multimodal data is used as input information, and gait assessment result and gait phase identification result are obtained by multitask model;Multitask model includes representation learning base network, unsupervised domain adaptation layer, mixed multitask layer;Representation learning base network extracts high-level features from multimodal data;Unsupervised domain adaptation layer obtains features by domain adaptation layer with weight parameter, and the features are mapped to a higher-dimensional latent feature space to obtain shared features;Mixed multitask layer inputs shared features into the independent subnetwork of multitask, and the gait assessment result and gait phase identification result of the patient to be measured are correspondingly output.The lower limb motor dysfunction patient is better to carry out rehabilitation training.

Description

Technical Field

[0001] This invention belongs to the field of rehabilitation medicine technology, and more specifically, relates to a method, device and storage medium for cross-patient phase recognition and gait assessment. Background Technology

[0002] In recent years, with the increasing aging of the population, the incidence of lower limb motor dysfunction caused by diseases such as stroke, Parkinson's disease, and hemophilic arthritis has shown a significant upward trend. Statistics show that approximately 10 million new stroke cases are diagnosed globally each year, and more than 70% of these patients will have varying degrees of motor dysfunction. These patients often exhibit abnormal gait characteristics, severely impacting their quality of life, and some may even never regain normal walking ability. Modern rehabilitation medicine research and clinical practice indicate that, in addition to early surgical intervention and drug treatment, systematic and scientific rehabilitation training plays an irreplaceable and crucial role in the treatment of lower limb motor dysfunction.

[0003] Using exoskeletons and other lower limb rehabilitation robots for rehabilitation training can alleviate the workload of medical staff and improve the training efficiency and rehabilitation outcomes for patients. Gait recognition and gait assessment are two key technologies in lower limb rehabilitation: through gait assessment, doctors can comprehensively understand and track the patient's walking ability and rehabilitation progress to develop personalized rehabilitation plans; through gait recognition, rehabilitation robots can proactively respond to the patient's lower limb movement intentions and provide assistance, resulting in better outcomes compared to passive rehabilitation training.

[0004] In gait recognition, current methods typically utilize sensors to collect motion information and develop corresponding algorithms for gait identification. For example, vision-based motion capture systems can obtain temporal-space parameters of a patient's joints or skeleton while walking, providing motion information for gait recognition. However, motion capture systems are usually expensive, require specialized operators, and visual signals are easily affected by occlusion and lighting conditions, making them inconvenient for clinical applications. Acquiring motion information based on wearable sensors is more convenient, with inertial measurement units (IMUs), pressure sensors, and surface electromyography (sEMG) being widely used. In addition, novel sensors such as triboelectric nanogenerator sensors (TENG sensors) have recently been applied in the field of human-computer interaction. IMUs can capture the spatial motion information of lower limb joints, pressure sensors placed on the soles of the feet can reflect the force distribution between the foot and the ground, and sEMG and triboelectric sensors can extract muscle movement features. However, single-modal sensor signals are difficult to obtain comprehensive information and are easily affected by noise. By fusing multimodal information, the performance and robustness of the system can be effectively improved.

[0005] Currently, gait assessment relies primarily on the subjective judgment of physicians. Doctors observe the patient's walking process and score it using a rating scale to generate an assessment result. For example, the Tinetti Balance and Gait Scale tests stride length, gait symmetry, and gait continuity, with each item scored by the physician on an integer scale between 0 and 2. The final score is then calculated as the assessment result. Similar scales include the Brunel Balance Scale and the Stroke Posture Control Scale. This assessment method is highly subjective and lacks granular results, making it difficult to capture subtle progress during rehabilitation and adjust the rehabilitation training plan in a timely manner.

[0006] Existing gait phase recognition algorithms are weak in cross-patient (or cross-user) adaptability, struggling to accommodate the differences in gait characteristics among different patients / users. These differences can stem from various individual factors such as age, height, muscle strength, and health status, and are therefore often referred to as individual differences. Most algorithms require data collection and labeling for new patients for calibration or retraining; otherwise, the accuracy of phase recognition will decrease, limiting the practicality of rehabilitation robots. Furthermore, many existing methods focus solely on gait assessment or phase recognition. In reality, besides individual differences, a patient's gait characteristics also change as their walking ability improves with rehabilitation training. If doctors need to reassess and update the rehabilitation robot's recognition algorithm every time a patient's walking ability improves, it will significantly reduce the patient's willingness and efficiency in rehabilitation.

[0007] Therefore, there is an urgent need for a gait assessment and phase recognition method that can integrate multimodal data and has cross-patient adaptability to improve the efficiency and quality of rehabilitation training. Summary of the Invention

[0008] To address the shortcomings of related technologies, the present invention aims to provide a cross-patient phase recognition and gait assessment method, device, and storage medium, with the goal of realizing a gait assessment and phase recognition method that integrates multimodal data and has cross-patient adaptability, thereby improving the efficiency and quality of rehabilitation training.

[0009] To achieve the above objectives, in a first aspect, the present invention provides a method for cross-patient phase recognition and gait assessment, comprising:

[0010] Collect triboelectric signals generated during lower limb movement of the patient under test to obtain multimodal data;

[0011] Using the multimodal data as input, gait evaluation results are obtained through a multi-task model. Gait phase recognition results ;

[0012] The multi-task model includes a representation learning base network, an unsupervised domain adaptation layer, and a hybrid multi-task layer;

[0013] The representation learning base network is used to extract high-level feature representations from the input multimodal data to obtain features. ;

[0014] The unsupervised domain adaptation layer is used to adapt the features output by the representation learning layer. After a weight parameter is The domain adaptation layer obtains features Then fix its parameters. Random Fourier Feature Transform , will feature Mapping to a higher-dimensional latent feature space To obtain shared features ;

[0015] The hybrid multitasking layer is used to share features. The input is divided into independent sub-networks for multiple tasks, which perform gait assessment and phase recognition on the patient to be tested, and output the gait assessment results of the patient to be tested respectively. Gait phase recognition results .

[0016] Optionally, when assessing the gait of the patient to be tested, the independent sub-network includes a multi-classification task. ;

[0017] The multi-task model predicts the assessment score of the patient based on the input multimodal data, obtains the category label based on the assessment score, and determines the corresponding gait assessment result of the patient. Each score corresponds to a category label, and the category labels required for model training are... The assessment results are given by a professional physician based on the Tinetti scale; among them... It is the number of categories;

[0018] When performing gait phase recognition on the patient to be tested, the independent sub-network includes a regression task. ;

[0019] The multi-task model predicts the proportion of the current gait phase of the patient to the entire gait cycle based on the input multimodal data, and uses this as the gait phase recognition result. ,in, A gait cycle is the time period from the first heel strike to the next heel strike.

[0020] Optionally, the multi-task model employs a multi-task loss function. The expression for the multi-task loss function is:

[0021] ;

[0022] in, This represents the model's predicted labels for multi-class classification tasks or the predicted values ​​for regression tasks. Indicates category label or actual value. Assigning weights to tasks, when all tasks should be trained equally. Set as ; For the task The value of the loss function; This represents the model's prediction results for T tasks. This represents the actual values ​​of T tasks.

[0023] Optionally, the multi-classification task uses the cross-entropy loss function, and the regression task uses the mean squared error loss function;

[0024] The expression for the multi-task loss function is:

[0025] ;

[0026] in, It is the sample size. It is the number of categories. Samples in multi-class classification tasks The true label, if the sample Category ,but ,otherwise ; The model predicts the sample. Category The probability of; For the samples in the regression task The true label, These are predicted values.

[0027] Optionally, the will Mapping to a higher-dimensional latent feature space To obtain shared features ,include:

[0028] Obtain the source domain dataset in The average vector in The target domain dataset is The average vector in is Wherein, the source domain dataset is the training data used during model training, and the target domain dataset is the dataset of the patients to be tested used when the model is applied;

[0029] Using similarity loss function The similarity loss function, which measures the distributional difference between the source and target domains, is expressed as follows:

[0030] ;

[0031] By minimizing the similarity loss function Fine-tuning the parameters of the unsupervised domain adaptation layer This makes the average vector of the target domain The average vector of the source domain Aligning the distributions of the target domain with the average shared feature vector. As a shared feature .

[0032] Optionally, the step of acquiring triboelectric signals generated during lower limb movement of the patient under test to obtain multimodal data includes:

[0033] The triboelectric signals generated during the movement of the lower limbs of the patient under test are collected by triboelectric sensors arranged in the muscles of the lower limbs of the patient under test, and the plantar pressure distribution signals generated during the movement of the lower limbs of the patient under test are collected by pressure sensors arranged in the soles of the feet of the patient under test. The triboelectric signals and the plantar pressure distribution signals constitute a multimodal signal.

[0034] The acquired multimodal signals are downsampled at the target sampling frequency to unify their sampling frequency and maintain consistency in the time dimension of data acquired by different sensors; thus obtaining the preprocessed triboelectric sensor signal. and pressure sensor signal Among them, the target sampling frequency The scope is: .

[0035] Secondly, the present invention also provides a cross-patient phase recognition and gait assessment device, comprising:

[0036] The data acquisition module is used to collect the triboelectric signals generated during the lower limb movement of the patient under test and acquire multimodal data.

[0037] The gait evaluation module is used to take the multimodal data as input information and obtain gait evaluation results through a multi-task model. Gait phase recognition results ;

[0038] The multi-task model includes a representation learning base network, an unsupervised domain adaptation layer, and a hybrid multi-task layer;

[0039] The representation learning base network is used to extract high-level feature representations from the input multimodal data to obtain features. ;

[0040] The unsupervised domain adaptation layer is used to adapt the features output by the representation learning layer. After a weight parameter is The domain adaptation layer obtains features Then fix its parameters. Random Fourier Feature Transform , will feature Mapping to a higher-dimensional latent feature space To obtain shared features ;

[0041] The hybrid multitasking layer is used to share features. The input is divided into independent sub-networks for multiple tasks, which perform gait assessment and phase recognition on the patient to be tested, and output the gait assessment results of the patient to be tested respectively. Gait phase recognition results .

[0042] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cross-patient phase recognition and gait assessment method as described in any one of the first aspects.

[0043] Compared with the prior art, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0044] 1. This invention provides a cross-patient phase recognition and gait assessment method. By constructing a multi-task model, multimodal data of lower limb movement collected from patients are input to obtain gait assessment and gait phase recognition results. In the multi-task model, representation learning base networks and unsupervised adaptive layers are used for representation learning and distribution adaptation to extract features from the multimodal data. Shared features that can be applied simultaneously to gait assessment and phase recognition tasks are obtained. These shared features improve the system's efficiency and performance, enabling simultaneous gait assessment and phase recognition tasks. The system can also adapt to differences in gait characteristics among different patients, enhancing its practicality. This provides a new solution for key technologies in rehabilitation assessment and intention recognition in the field of lower limb rehabilitation assistance, helping patients with lower limb motor dysfunction to better conduct rehabilitation training.

[0045] 2. This invention provides a cross-patient phase recognition and gait assessment method. The multi-task transfer learning framework provided by this invention is not limited to two tasks (T=2) of phase recognition and gait assessment, but can also be applied to more tasks (T≥2). It can be flexibly modified, trained, and applied according to actual needs. Joint learning of multiple tasks helps the model learn more general features, improving the generalization ability of the entire model and the predictive performance of each task. Attached Figure Description

[0046] Figure 1 This is a structural diagram of a triboelectric sensor.

[0047] Figure 2 This is a schematic diagram of a triboelectric sensor.

[0048] Figure 3 It is a fixed position for gait multimodal data acquisition sensors.

[0049] Figure 4 This is an example of multimodal data from healthy individuals.

[0050] Figure 5 This is an example of patient multimodal data.

[0051] Figure 6 It is a multi-task transfer learning framework. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0053] The following description, in conjunction with a preferred embodiment, illustrates the content involved in the above embodiments.

[0054] Example 1

[0055] In the clinical diagnosis of certain diseases, artificial intelligence-assisted diagnostic technology has been widely applied, even achieving more accurate diagnostic results than human doctors, such as in lung cancer diagnosis and CT image diagnosis. Therefore, the field of lower limb rehabilitation medicine urgently needs to introduce sensing systems capable of collecting large-scale, high-quality gait data and develop corresponding gait assessment algorithms to utilize artificial intelligence technology for objective, fine-grained gait evaluation. This will allow patients to clearly understand their rehabilitation progress and facilitate timely adjustments to rehabilitation plans by doctors, ultimately improving the effectiveness of rehabilitation training.

[0056] This invention provides a method for cross-patient phase recognition and gait assessment, including:

[0057] Collect triboelectric signals generated during lower limb movement of the patient under test to obtain multimodal data;

[0058] Using the multimodal data as input, gait evaluation results are obtained through a multi-task model. Gait phase recognition results ;

[0059] The multi-task model includes a representation learning base network, an unsupervised domain adaptation layer, and a hybrid multi-task layer;

[0060] The representation learning base network is used to extract high-level feature representations from the input multimodal data to obtain features. ;

[0061] The unsupervised domain adaptation layer is used to adapt the features output by the representation learning layer. After a weight parameter is The domain adaptation layer obtains features Then fix its parameters. Random Fourier Feature Transform , will feature Mapping to a higher-dimensional latent feature space To obtain shared features ;

[0062] The hybrid multitasking layer is used to share features. The input is divided into independent sub-networks for multiple tasks, which perform gait assessment and phase recognition on the patient to be tested, and output the gait assessment results of the patient to be tested respectively. Gait phase recognition results .

[0063] The embodiments of the present invention include: 1) a multimodal data acquisition and fusion scheme to comprehensively capture gait information and improve the accuracy of subsequent gait assessment and phase recognition; 2) constructing a multi-task model, namely a multi-task transfer learning (MTTL) framework, which improves the efficiency and performance of the system by sharing features, can perform gait assessment and phase recognition simultaneously, and can adapt to the differences in gait characteristics of different patients, thereby improving the practicality of the system.

[0064] Optionally, the step of acquiring triboelectric signals generated during lower limb movement of the patient under test to obtain multimodal data includes:

[0065] The triboelectric signals generated during the movement of the lower limbs of the patient under test are collected by triboelectric sensors arranged in the muscles of the lower limbs of the patient under test, and the plantar pressure distribution signals generated during the movement of the lower limbs of the patient under test are collected by pressure sensors arranged in the soles of the feet of the patient under test. The triboelectric signals and the plantar pressure distribution signals constitute a multimodal signal.

[0066] Select a target sampling frequency This frequency is less than or equal to the lowest sampling frequency of all sensors, that is:

[0067] ;

[0068] At the same time, set To avoid aliasing of sensor signals;

[0069] The acquired multimodal signals are downsampled at the target sampling frequency to unify their sampling frequency and maintain consistency in the time dimension of data acquired by different sensors; thus obtaining the preprocessed triboelectric sensor signal. and pressure sensor signal .

[0070] To achieve comprehensive gait information acquisition, a multimodal data acquisition scheme is adopted. The sensor distribution locations in this scheme are as follows: Figure 3As shown, six sensors were placed on one side of the patient's lower limb, including four triboelectric sensors and two plantar pressure sensors.

[0071] Specifically, such as Figure 1 As shown in the attached figure, the labels are: 1: Natural latex, 2: PET (Polyethylene terephthalate), 3: PDMS / ST (PDMS: Polydimethylsiloxane; ST: SrTiO3, Strontium titanate), 4: COOH@MCNTs (Carboxylated carbon nanotubes), 5: PDMS (Polydimethylsiloxane). In the structure of the triboelectric sensor, natural latex serves as the positive friction layer; the cyclic PET acts as a spacer layer, allowing for better contact and separation between the positive and negative friction layers; PDMS / ST serves as the negative friction layer; COOH@MCNTs serve as the electrode; and PDMS serves as an encapsulation layer to protect the electrode layer. The working principle of the triboelectric sensor is based on the principles of friction devices and electrostatic induction, that is, when two materials with significantly different electronegativity come into contact and separate, a changing electric field is generated, which drives electrons to flow back and forth in the external circuit. In this embodiment, four triboelectric sensors are fixed to the surfaces of key muscle groups that are primarily activated during the gait cycle, including the rectus femoris (responsible for hip flexion), vastus lateralis (involved in knee extension), tibialis anterior (controls foot dorsiflexion), and gastrocnemius (dominantly plantar flexion), to collect triboelectric signals generated during lower limb movement; the specific principle is as follows. Figure 2 As shown, when a muscle contracts, the muscle bulges, causing deformation of the surrounding skin and bringing the two friction layers into contact. At this time, because the electronegativity of the upper friction layer (latex or nylon) is less than that of the lower friction layer (PDMS / ST or PDMS / CCTO composite film), electrons transfer from the less electronegative material to the more electronegative material. When the muscle relaxes, the two friction layers gradually separate, and electrons transfer from the more electronegative material to the less electronegative material. Therefore, the triboelectric sensor can output an electrical signal to reflect muscle movement information.

[0072] Meanwhile, a force-sensitive resistor (FSR), i.e., a plantar pressure sensor, is set in both the forefoot (metatarsal region) and the heel (calcaneal region) to monitor the plantar pressure distribution characteristics in real time, i.e. to collect the plantar pressure distribution signal generated when the patient's lower limbs move.

[0073] After acquiring multimodal signals, data preprocessing is required. This includes:

[0074] Let the original signal collected by the i-th triboelectric sensor "TENG i" be . Its sampling frequency is The raw signal collected by the j-th pressure sensor "FSR j" is Its sampling frequency is Since the maximum frequency of muscle (fast-twitch muscle fibers) contraction is approximately 50 Hz, and the frequency of walking is even lower, the signal is first targeted... and Cutoff frequency First, a low-pass filter is applied to remove the influence of high-frequency noise. Second, to ensure that the data collected by different sensors remain consistent over time, the acquired multimodal signals need to be downsampled to unify their sampling frequency. A target sampling frequency is then selected. This frequency should be less than or equal to the lowest sampling frequency of all sensors. That is:

[0075] (1)

[0076] To prevent aliasing, it is also necessary to ensure Let the pre-processed signals from the triboelectric sensor and the pressure sensor be respectively... and .

[0077] In this embodiment, multimodal data from triboelectric sensors (TENG) and pressure sensors (FSR) are fused to provide comprehensive input information for subsequent multi-task models of gait evaluation and gait phase recognition.

[0078] The goal of gait assessment is to generate fine-grained assessment results by analyzing the patient's gait characteristics. However, multi-task models require labeled information for supervised training, so assessment results from a professional physician are needed first. For gait phase recognition, the labels need to be manually labeled. Typically, pressure sensor signals can be observed, and threshold methods can be used to classify the phases into labels. To simplify the expression, the multi-channel triboelectric signal is denoted as:

[0079] (2)

[0080] The multi-channel pressure signal is:

[0081] (3)

[0082] The input to the multi-task model is a multi-modal signal composed of the fusion of triboelectric signals and plantar pressure distribution signals, i.e.:

[0083] (4)

[0084] Figure 4The figure shows an example of a segment of multimodal data from a healthy person, with the corresponding gait phase markers drawn as dashed lines. Figure 5 This is data from a specific patient. As shown in the figure, there is a large voltage signal output, and the signal exhibits a clear periodicity with the gait, demonstrating good consistency. In this embodiment, the triboelectric sensor is effective in acquiring the signal.

[0085] Furthermore, the multimodal signals of the patient to be tested... The input is processed in a multi-task model to obtain gait evaluation results. Gait phase recognition results The multi-task model provided in this embodiment has cross-patient generalization ability, and a schematic diagram of the model is shown below. Figure 6 As shown.

[0086] Establishing a multi-task model means establishing a multi-task transfer learning framework, specifically as follows: Figure 6 As shown, it includes three main modules: representation learning base network, unsupervised domain adaptation layer, and hybrid multi-task layer.

[0087] 1. Representation Learning Base Network: Its function is to extract high-level feature representations from input multimodal data to obtain features. Typically, convolutional neural networks or Transformers are used as basic building blocks, and relatively deep networks are designed to provide sufficient representation capabilities. Let the parameters of the representation learning base network be... .

[0088] 2. Unsupervised Domain Adaptation Layer: The main function of this layer is to address the data distribution differences between the source and target domains. This layer is responsible for the features output by the representation learning layer. After passing through a weight parameter of The domain adaptation layer obtains features. :

[0089] (5)

[0090] in, These are learnable parameters.

[0091] Then the features Set fixed parameters Random Fourier Feature Transform , will feature Mapping to a higher-dimensional latent feature space In the process, shared features are obtained. :

[0092] (6)

[0093] Furthermore, aligning the data distribution differences between the source and target domains specifically includes:

[0094] Describe the source domain dataset in The average vector in is The target domain dataset (or a batch of data during the domain adaptation phase) in The average vector in is Similarity loss function The difference in distribution between the source and target domains is defined as follows:

[0095] (7)

[0096] The source domain dataset is the training data used during model training, and the target domain dataset is the dataset of patients to be tested used when the model is applied. yes and The angle between them. After migrating to a new patient or after significant progress in recovery, domain adaptation is required to eliminate distributional discrepancies and shifts. This is achieved by minimizing... Fine-tuning the parameters of the domain adaptation layer This makes the average vector of the target domain The average vector of the source domain Alignment of distributions, averaging the shared feature vectors of the target domain. As a shared feature .

[0097] 3. Hybrid Multitasking Layer: This layer will share features. In the input multi-task model, each task has a different learning objective in its independent sub-network, so the sub-network can be designed individually according to requirements. Let the task be... The network is ,in If the parameters are network parameters, then the output of this task is represented as:

[0098] (8)

[0099] Record the task The real label is The loss function is Therefore, the network parameters can be updated using gradient descent. .

[0100] After the gradients of each task pass through the sub-network, they are first processed through the coefficients. A weighted summation is performed, followed by backpropagation. This is equivalent to the gradient propagating from the hybrid multi-task layer to the unsupervised adaptation layer originating from a single overall multi-task loss function. :

[0101] (9)

[0102] To minimize the loss across multiple tasks, the first half of the network will learn shared features among the tasks. This improves generalization ability. This represents the model's predicted labels for multi-class classification tasks or the predicted values ​​for regression tasks. Indicates category label or actual value. As the weight of the task, Values ​​can be assigned based on the importance of the task, with more weight allocated to important tasks. Generally speaking, Set as This indicates that all tasks should be trained equally. This represents the model's prediction results for T tasks. This represents the actual values ​​of T tasks.

[0103] The multi-task model is applied to two tasks: gait assessment and phase recognition, i.e., T=2.

[0104] Specifically, when assessing the gait of the patient being tested, the independent sub-network includes multi-classification tasks. , used for gait assessment;

[0105] The multi-task model predicts the assessment score of the patient based on the input multimodal data, obtains the category label based on the assessment score, and determines the corresponding gait assessment result of the patient. Each score corresponds to a category label, and the category labels required for model training are... The assessment results are given by a professional physician based on the Tinetti scale; among them... It is the number of categories;

[0106] When performing gait phase recognition on the patient to be tested, the independent sub-network includes a regression task. , used for phase identification;

[0107] The multi-task model predicts the proportion of the current gait phase of the patient to the entire gait cycle based on the input multimodal data, and uses this as the gait phase recognition result. ,in, A gait cycle is defined as the time period from the start of one heel strike (0×100%) to the next heel strike (1×100%). In a gait cycle, except for the start and end data, the intermediate gait phase values ​​are labeled using linear interpolation.

[0108] Multi-classification tasks Using the cross-entropy loss function:

[0109] (10)

[0110] in It is the sample size. It is the number of categories. It is a sample from Task 1 The true label (if the sample) Category ,but ,otherwise ), The model predicts the sample. Category The probability of.

[0111] Return mission Then the mean squared error loss function is used:

[0112] (11)

[0113] in, For the sample The true label, This is the predicted value. Therefore, for gait evaluation and phase recognition, the expression for the multi-task loss function is:

[0114] (12)

[0115] in, It is the sample size. It is the number of categories. Samples in multi-class classification tasks The true label (if the sample) Category ,but ,otherwise ), The model predicts the sample. Category The probability of; For the samples in the regression task The true label, These are predicted values.

[0116] In the pre-training phase of the multi-task model, the model is trained using a large-scale multimodal dataset. The goal of pre-training is to optimize the multi-task loss function. This approach aims to learn general features for gait assessment and phase recognition tasks. Specifically, the model can be optimized using mini-batch gradient descent, calculating the loss for the current batch of data in each iteration and updating the model parameters, including the network parameters of individual subnetworks, through backpropagation. Parameters of the domain adaptation layer and the parameters of the representation learning base network The pre-training process continues until the model's performance on the validation set stabilizes, indicating that the model has learned the general patterns and features of the gait data. The pre-trained model's parameters and the mean of the hidden layer features obtained from the training set are then saved. As source domain information.

[0117] After pre-training, the multi-task model enters the domain adaptation phase for new patients. Since gait patterns may differ among patients, directly using the pre-trained model may not achieve optimal performance on new patient data. Therefore, the goal of the domain adaptation phase is to quickly calibrate the pre-trained model to adapt to the gait characteristics of new patients through transfer learning. This process is unsupervised to avoid the need for labeling during calibration, thereby improving usability.

[0118] Specifically, by minimizing the loss function Fine-tuning of the multi-task model. During the domain adaptation phase, the model is trained with a small learning rate and only the parameters are updated. This approach preserves the general features learned during the pre-training phase while adapting to the specific gait patterns of new patients. In this way, the model can achieve higher assessment and recognition accuracy on new patient data, thereby meeting personalized needs.

[0119] Based on the above embodiments, the multi-task transfer learning framework provided by the present invention is not limited to two tasks (T=2) of phase recognition and gait evaluation, but can also be applied to more tasks (T≥2), and can be flexibly modified, trained and applied according to actual needs. For example, the multi-task transfer learning framework can also achieve the following tasks: (1) Fine-grained gait evaluation. It is regarded as a regression task. During training, the evaluation results given by the doctor are still used, but the loss function is no longer cross-entropy loss, but a loss function suitable for regression is used instead. After training, the model can perform fine-grained prediction and evaluation, no longer limited to integer scores, and can give decimal score evaluations. (2) User identity recognition. It is regarded as a classification task. The user's identity is used as a label. Based on the user's gait characteristics, the identity of the current user is identified, so that personalized adjustments can be made.

[0120] Furthermore, the hybrid multi-task layer of the model in this application, combined with a regression task, enables fine-grained evaluation. Through multi-task learning, coarse-grained evaluation tasks based on integer scores and other tasks can be used to assist in the training of fine-grained evaluation tasks.

[0121] This invention provides a cross-patient phase recognition and gait assessment method. By constructing a multi-task model, multimodal data collected during lower limb movement from patients is input to obtain gait assessment and gait phase recognition results. Within the multi-task model, features are extracted from the multimodal data through representation learning base networks and unsupervised adaptive layers. Shared features are obtained that can be applied simultaneously to both gait assessment and phase recognition tasks. These shared features improve the system's efficiency and performance, enabling simultaneous gait assessment and phase recognition tasks and adapting to the differences in gait characteristics among different patients, thus enhancing the system's practicality. This provides a new solution for key technologies in rehabilitation assessment and intention recognition in the field of lower limb rehabilitation assistance, helping patients with lower limb motor dysfunction to better conduct rehabilitation training.

[0122] Example 2

[0123] This invention provides a cross-patient phase recognition and gait assessment device, comprising:

[0124] The data acquisition module is used to collect the triboelectric signals generated during the lower limb movement of the patient under test and acquire multimodal data.

[0125] The gait evaluation module is used to take the multimodal data as input information and obtain gait evaluation results through a multi-task model. Gait phase recognition results ;

[0126] The multi-task model includes a representation learning base network, an unsupervised domain adaptation layer, and a hybrid multi-task layer;

[0127] The representation learning base network is used to extract high-level feature representations from the input multimodal data to obtain features. ;

[0128] The unsupervised domain adaptation layer is used to adapt the features output by the representation learning layer. After a weight parameter is The domain adaptation layer obtains features Then fix its parameters. Random Fourier Feature Transform , will feature Mapping to a higher-dimensional latent feature space To obtain shared features ;

[0129] The hybrid multitasking layer is used to share features. The input is divided into independent sub-networks for multiple tasks, which perform gait assessment and phase recognition on the patient to be tested, and output the gait assessment results of the patient to be tested respectively. Gait phase recognition results .

[0130] The cross-patient phase recognition and gait assessment device provided in this embodiment is used to perform the cross-patient phase recognition and gait assessment method in Embodiment 1, and has the same beneficial effects.

[0131] Example 3

[0132] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the cross-patient phase recognition and gait assessment method as described in any one of Embodiment 1.

[0133] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for gait assessment and identification across patient phases, comprising: Comprising The triboelectric signals generated during the movement of the lower limbs of the patient to be tested are collected by arranging triboelectric sensors on the muscles of the lower limbs of the patient to be tested, and the plantar pressure distribution signals generated during the movement of the lower limbs of the patient to be tested are collected by arranging pressure sensors on the soles of the feet of the patient to be tested; the triboelectric signals and the plantar pressure distribution signals constitute multi-modal data; The multi-modal data is taken as input information, and a gait assessment result is obtained through a multi-task model and a gait phase recognition result ; The multi-task model comprises a representation learning base network, an unsupervised domain adaptation layer, and a hybrid multi-task layer; The characteristic learning base network is used to extract high-level feature representation from the input multi-modal data, to obtain features ; The unsupervised domain adaptation layer is used for mapping the features output by the representation learning layer to features through a domain adaptation layer with a weight parameter , and then performing a random Fourier feature transformation with a fixed parameter to map the features to a higher-dimensional latent feature space to obtain shared features ; The mixed multi-task layer is configured to share features In the independent sub-networks of the input multi-task, gait assessment and phase recognition are performed on the to-be-tested patient respectively, and the gait assessment result of the to-be-tested patient is output And the gait phase recognition result .

2. The method of claim 1, wherein, when performing gait assessment on a patient under test, the independent sub-network comprises a multi-classification task ; The multi-task model predicts an evaluation score of a to-be-tested patient according to inputted multi-modal data, acquires a category label according to the evaluation score, and determines a corresponding gait evaluation result of the to-be-tested patient; wherein each score corresponds to a category label, and the category label required during model training is given according to the Tinetti scale evaluation; wherein, is the number of categories. The independent sub-network includes a regression task when performing gait phase recognition on a patient under test ; The multi-task model predicts, according to input multi-modal data, a proportion of a current gait phase of a to-be-tested patient in an entire gait cycle as a gait phase recognition result wherein, One gait cycle is a time period from once heel strike to the next heel strike.

3. The method of claim 2, wherein, The multi-task model adopts a multi-task loss function An expression of the multi-task loss function is: ； wherein, represents a predicted label of a model for a multi-classification task or a predicted value of a regression task, represents a class label or a true value, is a weight for a task, when each task should be equally trained, is set to ; is a value of a loss function for a task ; represents a prediction result of a model for T tasks, represents a true value of T tasks.

4. The method of claim 3, wherein, The multi-classification task adopts a cross-entropy loss function, and the regression task adopts a mean square error loss function; The expression of the multi-task loss function is: ； where, is the number of samples, is the number of classes, is the true label of sample in a multi-classification task, if sample belongs to class , , otherwise ; is the probability that model predicts sample belongs to class ; is the true label of sample in a regression task, is the predicted value.

5. The method of claim 1, wherein, Said mapping to a higher dimensional latent feature space , obtaining shared features , comprising: An average vector of a source domain data set in is , and an average vector of a target domain data set in is ; wherein the source domain data set is training data used in model training, and the target domain data set is a data set of a patient to be measured used in model application; using a similarity loss function measuring the distribution difference between the source domain and the target domain, the similarity loss function expression being: ； by minimizing the similarity loss function , fine-tuning parameters of the unsupervised domain adaptation layer such that the average shared feature vector of the target domain is aligned with the distribution of the average shared feature vector of the source domain , the average shared feature vector of the target domain is aligned with the distribution of the average shared feature vector of the source domain .

6. The method of claim 1, wherein, The triboelectric signals generated during the movement of the lower limbs of the patient to be tested are collected by arranging triboelectric sensors on the muscles of the lower limbs of the patient to be tested, and the plantar pressure distribution signals generated during the movement of the lower limbs of the patient to be tested are collected by arranging pressure sensors on the soles of the feet of the patient to be tested; the triboelectric signals and the plantar pressure distribution signals constitute multi-modal data; Comprising: The collected multi-modal signals are down-sampled at a target sampling frequency to make the sampling frequencies uniform and keep the data collected by different sensors consistent in time dimension; and the pre-processed triboelectric sensor signals and pressure sensor signals are obtained; wherein the target sampling frequency ranges from .

7. A cross-patient phase recognition and gait assessment device, comprising: The data acquisition module is configured to collect triboelectric signals generated during the movement of the lower limbs of the patient to be tested by arranging triboelectric sensors on the muscles of the lower limbs of the patient to be tested, and collect plantar pressure distribution signals generated during the movement of the lower limbs of the patient to be tested by arranging pressure sensors on the soles of the feet of the patient to be tested; the triboelectric signals and the plantar pressure distribution signals constitute multi-modal data; The multi-task model comprises a representation learning base network, an unsupervised domain adaptation layer, and a hybrid multi-task layer; A gait evaluation module is configured to input the multi-modal data as input information into a multi-task model to obtain a gait evaluation result and a gait phase recognition result ; The computer program, when executed by a processor, implements the cross-patient phase recognition and gait assessment method according to any one of claims 1 to 6. The characteristic learning base network is used to extract high-level feature representation from the input multi-modal data, to obtain features ; The unsupervised domain adaptation layer is used for mapping the features output by the representation learning layer through a domain adaptation layer with a weight parameter to obtain features , and then performing a random Fourier feature transformation with a fixed parameter to map the features to a higher-dimensional latent feature space to obtain shared features ; The mixed multi-task layer is configured to share features In the independent sub-networks of the input multi-task, gait assessment and phase recognition are respectively performed on the to-be-tested patient, and the gait assessment result of the to-be-tested patient is output and the gait phase recognition result .

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, ​

Images