A method, device and equipment for detecting abnormal states of a lower limb exoskeleton based on VAE

Abstract

The present invention provides a method, device and equipment for detecting abnormal states of a lower limb exoskeleton based on VAE. The method includes: collecting walking data of a human wearing an exoskeleton robot; inputting the walking data into a pre-trained VAE network for data reconstruction to obtain reconstructed data corresponding to the walking data; calculating an anomaly score based on the walking data and the reconstructed data; and determining that the interaction between the human and the exoskeleton robot is abnormal when the anomaly score exceeds a preset threshold. In the present invention, the VAE network is trained only using a small amount of normal walking data without requiring machine abnormal usage data, thus reducing the data acquisition cost while ensuring high accuracy and real-time performance of anomaly detection. By directly inputting the walking data into the VAE network, it is possible to detect anomalies such as faults and running errors during the interaction between the human and the exoskeleton robot, and to track continuously changing abnormal states, and to quantitatively evaluate the abnormal states by calculating the anomaly score.

Description

Technical Field

[0001] The present invention relates to the field of exoskeleton robot state detection, and particularly to a method, device, and equipment for detecting abnormal states of a lower limb exoskeleton based on VAE. Background Art

[0002] Anomaly detection methods can detect outliers in monitoring data, thereby discovering abnormal situations such as machine failures and operation errors during operation. Compared with model-based methods, deep anomaly detection (DAD) technology can capture the relationship between multimodal data and the execution task state, thereby learning complex features to process large-scale data.

[0003] Current detection methods are mostly used for exoskeleton state classification, and it is difficult to track continuously changing abnormal states and quantitatively evaluate abnormal states; moreover, existing anomaly detection requires abnormal data such as collisions for training, and anomalies may damage the equipment, making data acquisition difficult and costly. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention provide a method, device, and equipment for detecting abnormal states of a lower limb exoskeleton based on VAE, so as to overcome the above problems or at least partially solve the above problems.

[0005] In a first aspect of embodiments of the present invention, a method for detecting abnormal states of a lower limb exoskeleton based on VAE is disclosed, and the method includes:

[0006] Collect walking data of a human wearing an exoskeleton robot, where the data includes: human limb angles, exoskeleton robot joint angles, and human-machine interaction torques;

[0007] Input the walking data into a pre-trained VAE network for data reconstruction to obtain reconstructed data corresponding to the walking data, where the VAE network is trained with normal walking data as training samples;

[0008] Calculate an anomaly score based on the walking data and the reconstructed data;

[0009] Determine that the interaction between the human and the exoskeleton robot is abnormal when the anomaly score exceeds a preset threshold.

[0010] Optionally, the inputting the walking data into a pre-trained VAE network for data reconstruction to obtain reconstructed data corresponding to the walking data includes:

[0011] Use the encoder in the VAE network to map the walking data to the latent space to obtain a deep representation of the walking data;

[0012] Based on the deep representation of the walking data, the decoder in the VAE network is used to reconstruct the data, and the reconstructed data is obtained.

[0013] Optionally, the encoder in the VAE network is used to map the walking data to the latent space to obtain the deep representation of the walking data, including:

[0014] The first fully connected layer in the encoder is used to process the walking data into a feature vector;

[0015] The second fully connected layer in the encoder is used to process the feature vector to obtain the deep representation of the walking data, and the deep representation characterizes the distribution of the walking data in the latent space.

[0016] Optionally, based on the deep representation of the walking data, the decoder in the VAE network is used to reconstruct the data, and the reconstructed data is obtained, including:

[0017] Reparameterization is performed based on the deep representation of the walking data to obtain distribution parameters, and sampling is performed on the latent space distribution according to the distribution parameters to obtain an intermediate variable encoding;

[0018] Two fully connected layers in the decoder are used to process the intermediate variable encoding to obtain reconstructed data with the same dimension as the walking data.

[0019] Optionally, when the anomaly score exceeds a preset threshold, it further includes: smoothly mapping the anomaly score to a corresponding weight according to a weight function;

[0020] Based on the weight, the parameters of the variable impedance model of the exoskeleton robot are adjusted, and the parameters of the variable impedance model characterize the degree of assistance of the exoskeleton robot to the human body.

[0021] Optionally, the weight function ω(s) is expressed as:

[0022]

[0023] where λ 1 and λ 2 respectively represent two constants greater than zero, m represents a constant for normalizing the anomaly score to a specified small range, and h represents the offset of the weight function from the origin of the coordinate.

[0024] Optionally, the variable impedance model of the exoskeleton robot is expressed as:

[0025]

[0026] where C d and Kd represent the desired damping and desired stiffness matrices respectively, and q and q d represent the robot joint angles and the reference trajectory in the joint space respectively, and represent the first-order derivatives of q and q d with respect to time respectively, and τ e represents the interaction torque between the human body and the exoskeleton robot.

[0027] Optionally, the VAE network is trained in the following manner:

[0028] Input the normal walking data of the human body wearing the exoskeleton robot into the VAE network for training, where the normal walking data is the data collected when the human wears the exoskeleton robot and walks normally following a preset trajectory;

[0029] The VAE network performs data reconstruction based on the normal walking data to obtain the normal reconstruction data corresponding to the normal data;

[0030] Calculate the loss function used to describe the difference between the normal walking data and the normal reconstruction data. The loss function includes the reconstruction error and the KL divergence, and the loss function is expressed as:

[0031]

[0032] where, represents the normal reconstruction data, x t represents the normal walking data, KL represents the divergence, represents the distribution that the normal reconstruction data follows, represents the normal distribution;

[0033] Optimize the parameters of the VAE network based on the loss function. After the training is completed, the trained VAE network is obtained.

[0034] In the second aspect of the embodiments of the present invention, a lower limb exoskeleton abnormal state detection device based on VAE is disclosed. The device includes:

[0035] A data acquisition module for acquiring the walking data of the human body wearing the exoskeleton robot. The data includes: human body limb angles, exoskeleton robot joint angles, and human-machine interaction torques;

[0036] A data reconstruction module for inputting the walking data into a pre-trained VAE network for data reconstruction to obtain the reconstruction data corresponding to the walking data, where the VAE network is trained with normal walking data as the training samples;

[0037] Anomaly calculation module, configured to calculate an anomaly score according to the walking data and the reconstructed data;

[0038] Anomaly determination module, configured to determine that the interaction between the human body and the exoskeleton robot is abnormal when the anomaly score exceeds a preset threshold.

[0039] In a third aspect of the embodiments of the present invention, an electronic device is disclosed, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes, it implements the VAE-based lower limb exoskeleton anomaly state detection method described in the first aspect of the embodiments of the present invention.

[0040] The embodiments of the present invention have the following advantages:

[0041] In the embodiments of the present invention, a VAE-based lower limb exoskeleton anomaly state detection method is proposed. First, the walking data of a human body wearing an exoskeleton robot is collected, and then the walking data is input into a pre-trained VAE network for data reconstruction to obtain reconstructed data corresponding to the walking data. Then, an anomaly score is calculated according to the walking data and the reconstructed data. When the anomaly score exceeds a preset threshold, it is determined that the interaction between the human body and the exoskeleton robot is abnormal. Since the VAE network is trained only with a small amount of normal walking data and does not require machine anomaly usage data, the data acquisition cost is reduced, and at the same time, high accuracy and real-time performance of anomaly detection can be ensured. By directly inputting the walking data into the VAE network, it is possible to detect anomalies such as faults and operation errors during the interaction between the human body and the exoskeleton robot, and to track continuously changing anomaly states, and to quantitatively evaluate the anomaly states by calculating the anomaly score. Description of the Drawings

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 It is a flowchart of the steps of a VAE-based lower limb exoskeleton anomaly state detection method provided by the embodiments of the present invention;

[0044] Figure 2 It is a schematic diagram of the overall architecture of a VAE-based lower limb exoskeleton anomaly state detection network provided by the embodiments of the present invention;

[0045] Figure 3 It is a schematic diagram of the relationship curve between the anomaly score and the weight in a weight function provided by the embodiments of the present invention;

[0046] Figure 4 It is a schematic structural diagram of a lower limb exoskeleton abnormal state detection device based on VAE provided by an embodiment of the present invention. Specific implementation manners

[0047] To make the above objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0048] An embodiment of the present invention provides a method for detecting abnormal states of a lower limb exoskeleton based on VAE, as Figure 1 shown Figure 1 It is a flowchart of the steps of a method for detecting abnormal states of a lower limb exoskeleton based on VAE provided by an embodiment of the present invention, including steps S101 to S104:

[0049] Step S101: Collect the walking data of a human body wearing an exoskeleton robot, and the data includes: human body limb angles, exoskeleton robot joint angles, and human-machine interaction torques.

[0050] In this embodiment, the exoskeleton robot refers to a lower limb exoskeleton robot for assisting human walking. The lower limb exoskeleton robot has four joints (i.e., two hip joints and two knee joints), and all use serial elastic actuators (SEA). The spring stiffness of the SEA is 635 N·m, and the output torque of the SEA (i.e., the human-machine interaction torque) can be calculated through the spring deflection and the known stiffness.

[0051] In this embodiment, the walking data of a human body wearing an exoskeleton robot observed by sensors is sampled at a fixed frequency. Specifically, two types of proprioceptive sensors, namely inertial measurement units (IMUs, JY901) and joint angle sensors (2048-line encoders), are configured in the exoskeleton robot to collect walking data. Among them, the IMUs are used to detect the human body limb angles, and the 2048-line encoders are used to measure the exoskeleton robot joint angles and human-machine interaction torques. In addition, the walking data of a human body wearing an exoskeleton robot is time series data, and the collected data is divided into data segments of a fixed length by introducing a sliding window as the input of the VAE network.

[0052] In this embodiment, the walking data includes normal walking data and abnormal walking data. Normal data refers to the data collected when a human wears an exoskeleton robot and walks normally following a preset trajectory. At this time, the conflict between the human body and the exoskeleton robot is relatively small. Abnormal walking data is the data with a relatively intense conflict between the human body and the exoskeleton robot. Among them, the abnormal walking data includes data on unstable standing and falling, and data on limb fatigue and weakness states.

[0053] In this embodiment, by measuring the angles of human limbs and the angles of exoskeleton robot joints, the gait phase of the walking cycle can be identified to judge the position difference between the human and robot joints. By measuring the human-robot interaction torque, the physical interaction between the human and the exoskeleton robot can be measured. This embodiment can achieve data collection only relying on the proprioceptive sensors of the exoskeleton robot. Compared with existing anomaly detection technologies, it reduces the data collection cost and is easier to implement.

[0054] Step S102: Input the walking data into a pre-trained VAE network for data reconstruction to obtain reconstruction data corresponding to the walking data. Among them, the VAE network is trained with normal walking data as training samples.

[0055] Among them, the VAE (Variational auto-encoder) network is a generative model, including two parts: an encoder and a decoder. In the VAE network, the input data x passes through the encoder p θ (z|x) to obtain the latent feature z, and z is input into the decoder q φ (x|z) to generate the reconstruction data x'. Generally, the reconstruction error is used to evaluate the difference between the reconstruction data and the original input data. When the distribution of the input data is significantly different from that of the training set data, the trained VAE network cannot generate data similar to the new input data, and the reconstruction error increases. It can be considered that the input data is abnormal data at this time.

[0056] In this embodiment, the walking data collected in step S101 is input into the VAE network for data reconstruction to obtain reconstruction data corresponding to the walking data. In subsequent steps, the difference between the two is judged by calculating the error between the input walking data and the reconstruction data.

[0057] In an alternative embodiment, the step of inputting the walking data into a pre-trained VAE network for data reconstruction to obtain reconstruction data corresponding to the walking data includes step A1 and step A2:

[0058] Step A1: Use the encoder in the VAE network to map the walking data into the latent space to obtain the deep representation of the walking data.

[0059] In this step, as Figure 2 shown, the encoder consists of two fully connected layers, using the rectified linear unit (ReLu) as the activation function. Among them, the first fully connected layer of the encoder is equivalent to a flatten layer, and the number of neurons in the second fully connected layer of the encoder is less than that of the first fully connected layer. The walking data is projected into the latent space through the encoder of the VAE network to obtain a deep representation, from which the pattern features of the exoskeleton robot driving the human body to walk can be captured.

[0060] Specifically, using the encoder in the VAE network to map the walking data into the latent space to obtain the deep representation of the walking data includes:

[0061] Using the first fully connected layer in the encoder to process the walking data into a feature vector; using the second fully connected layer in the encoder to process the feature vector to obtain the deep representation of the walking data, and the deep representation characterizes the distribution of the walking data in the latent space.

[0062] Among them, the first fully connected layer flattens the input 100×10 - dimensional walking data into a 1000×1 feature vector, and the second fully connected layer then processes this feature vector to obtain the deep representation of the walking data, that is, a 400×1 feature vector. The deep representation characterizes the distribution of the walking data in the latent space, and the distribution l of the deep representation in the latent space can be expressed as:

[0063] l = μ + σ⊙∈

[0064] where μ and σ are parameters obtained from the encoder, representing the mean and standard deviation respectively, ∈ represents the distribution of the input data noise, and the noise is set to follow the standard normal distribution, that is, ∈~N(0, I).

[0065] Step A2: Based on the deep representation of the walking data, use the decoder in the VAE network to reconstruct the data to obtain the reconstructed data.

[0066] In this step, as Figure 2 shown, the decoder also consists of two fully connected layers, and the number of neurons in the first fully connected layer of the decoder is less than that of the second fully connected layer.

[0067] Specifically, based on the deep representation of the walking data, using the decoder in the VAE network to reconstruct the data to obtain the reconstructed data includes:

[0068] Reparameterize based on the deep representation of the walking data to obtain distribution parameters, and sample the latent space distribution according to the distribution parameters to obtain an intermediate variable encoding; use two fully connected layers in the decoder to process the intermediate variable encoding to obtain reconstructed data with the same dimension as the walking data.

[0069] Among them, reparameterizing based on the deep representation of the walking data means sampling the deep representation, that is, performing parameter sampling on the deep representation based on the mean and variance of the deep representation to obtain an intermediate variable encoding. A feature vector with a smaller dimension (i.e., the intermediate variable encoding) is obtained after parameter sampling of the deep representation. For example, for the 400×1 feature vector in step A1, the mean of 50×1 and the standard deviation of 50×1 of the deep representation are obtained through two separate fully connected layers, that is, the distribution parameters of the deep representation. The intermediate variable encoding is sampled to obtain a 50×1 feature vector, and then two fully connected layers are used to process the 50×1 feature vector to obtain reconstructed data with the same dimension as the input walking data.

[0070] In this embodiment, the walking data is directly input into the VAE network, and the VAE network is used for data reconstruction. In subsequent steps, by analyzing the reconstructed data and the input walking data, abnormal detections such as faults and operation errors during the interaction between the human body and the exoskeleton robot can be realized.

[0071] Step S103: Calculate the anomaly score according to the walking data and the reconstructed data.

[0072] In this embodiment, the anomaly score is used to characterize the reconstruction error between the input walking data and the reconstructed data. Since the VAE network is trained using normal walking data as training samples, for normal walking data, the reconstruction error between the reconstructed data and the input normal walking data is smaller, and for abnormal walking data, the reconstruction error between the reconstructed data and the input abnormal walking data is larger. Therefore, it can be determined whether the input data is normal walking data or abnormal walking data through the anomaly score.

[0073] Specifically, the mean square error between the walking data and the reconstructed data is used to characterize the anomaly score, and the anomaly score s is defined as:

[0074]

[0075] where x t represents the collected walking data, that is, the input data of the VAE network, represents the reconstructed data, that is, the output data of the VAE network, and MSE(·) represents the mean square error.

[0076] When the physical conflict between the human body and the exoskeleton robot is more significant, the reconstruction error between the walking data and the reconstructed data is larger, that is, the anomaly score is larger. When the conflict between the human body and the exoskeleton robot is smaller, the reconstruction error decreases, and the corresponding anomaly score is smaller.

[0077] Step S104: When the anomaly score exceeds a preset threshold, it is determined that the interaction between the human body and the exoskeleton robot is abnormal.

[0078] In this embodiment, the anomaly score threshold is set according to the normal conflict limit during the interaction between the human body and the exoskeleton robot. During the interaction between the human body and the exoskeleton robot, there is always a certain interaction force and conflict. That is, in the case of normal interaction, there is also a certain interaction force and conflict between the human body and the exoskeleton robot, but at this time, both the interaction force and the conflict are relatively small and do not affect the health of the human body. Therefore, there is an anomaly score threshold during the interaction between the human body and the exoskeleton robot.

[0079] During the interaction between the human body and the exoskeleton robot, when the anomaly score oscillates repeatedly near a lower threshold, the exoskeleton robot should ignore the minor conflict at this time (regard it as a normal interaction situation) to avoid overreacting to the conflict; when the anomaly score exceeds the threshold, it indicates that the current interaction mode between the exoskeleton robot and the human body is significantly different from the normal state, and there may be a serious conflict between the human body and the exoskeleton robot. At this time, it is determined that the interaction between the human body and the exoskeleton robot is abnormal.

[0080] In this embodiment, by directly inputting the walking data into the VAE network, it is possible to detect anomalies such as faults and operation errors during the interaction between the human body and the exoskeleton robot, and it is possible to track continuously changing abnormal states, and quantitatively evaluate the abnormal states by calculating the anomaly score.

[0081] In an alternative embodiment, when the anomaly score exceeds a preset threshold, the exoskeleton robot can adjust its degree of assistance to the human body according to the anomaly score, so as to reduce the conflict with the wearer, specifically including the following steps B1 and B2:

[0082] Step B1: Smoothly map the anomaly score to a corresponding weight according to a weight function;

[0083] Specifically, the anomaly score s is mapped to a smooth curve through the weight function ω(s) for adjusting the variable impedance model parameters of the exoskeleton robot, and the weight function ω(s) is expressed as:

[0084]

[0085] where λ 1 and λ 2respectively represent two constants greater than zero, which determine the range and the intermediate value of the weight function, m represents a constant for normalizing the anomaly score to a specified small range, and h represents the offset of the weight function from the origin of coordinates.

[0086] Exemplarily, when λ 1 = 2, λ 2 = 2.3, m = 1, h = 6, the weight function is as shown in Figure 3 where 4 < s < 8 is a transition smoothing region from the normal walking state to the abnormal walking state. At this time, the exoskeleton robot has a buffer region to reduce the impedance and switch to the human-dominated mode to reduce conflicts; when s > 8, the weight function saturates and only maintains a minimal level of assistance.

[0087] Step B2: Adjust the parameters of the variable impedance model of the exoskeleton robot based on the weight, and the parameters of the variable impedance model characterize the degree of assistance of the exoskeleton robot to the human body.

[0088] In this step, the parameters of the variable impedance model characterizing the degree of assistance of the exoskeleton robot to the human body mean that when the parameters of the variable impedance model are larger, it indicates that the exoskeleton robot has a greater degree of assistance to the human body. At this time, it is mainly the exoskeleton robot that dominates the human walking; when the parameters of the variable impedance model are smaller, it indicates that the exoskeleton robot has a smaller degree of assistance to the human body. At this time, the exoskeleton robot controls more passively and compliantly, and it is mainly the human body that dominates the walking.

[0089] The variable impedance model of the exoskeleton robot refers to the force-position model of the exoskeleton robot, which depicts the force-position hybrid relationship. This model describes a first-order system without inertial parameters and matches the lower limb model of the human body. Specifically, the variable impedance model of the exoskeleton robot is expressed as:

[0090]

[0091] where C d and K d respectively represent the desired damping and the desired stiffness matrix, both of which are diagonal constant matrices, (n is the number of degrees of freedom of the robot), q and q d respectively represent the robot joint angle and the reference trajectory in the joint space, and respectively represent the first-order derivatives of q and q d with respect to time, and τ e represents the interaction torque between the human body and the exoskeleton robot.

[0092] Multiplying both sides of the above variable impedance model of the exoskeleton robot by the weight ω(s), the variable impedance model of the exoskeleton robot can be expressed as:

[0093]

[0094] Among them, Both of them change with the weight function. By changing the value of the weight function ω(s), the parameters of the variable impedance model can be changed, continuously adjusting the assistance degree of the exoskeleton to the human body, smoothly switching the assistance mode, and avoiding human-machine conflict. Specifically, when the anomaly score s increases and enters the transition smoothing region, ω(s) shrinks (refer to Figure 3 ), so that the parameters of the variable impedance model of the exoskeleton robot decrease, making the robot become more passively compliant and slowing down the conflict. Otherwise, the parameters of the variable impedance model of the exoskeleton robot are larger to maintain the dominant state of the robot.

[0095] In this embodiment, by using the anomaly score of the VAE network to adjust the model parameters of the variable impedance model of the exoskeleton robot, continuous adjustment of the impedance of the exoskeleton robot is achieved, and the interaction mode between the exoskeleton robot and the human body is smoothly switched to weaken the human-machine conflict.

[0096] In an alternative embodiment, the VAE network for anomaly detection of the interaction between the human body and the exoskeleton robot is pre-trained, and the VAE network is trained in the following manner, including steps C1 to C4:

[0097] Step C1: Input the normal walking data of the human body wearing the exoskeleton robot into the VAE network for training. The normal walking data is the data collected when a human wears the exoskeleton robot and walks normally following a preset trajectory.

[0098] In this step, without abnormal data, only the normal walking data is used to train the VAE network. Furthermore, the VAE network trained with the normal walking data can accurately reconstruct the normal walking data, making the reconstructed data have a small reconstruction error with the normal walking data to achieve anomaly detection.

[0099] Step C2: The VAE network reconstructs the data based on the normal walking data to obtain the normal reconstructed data corresponding to the normal data.

[0100] Step C3: Calculate the loss function for describing the difference between the normal walking data and the normal reconstructed data. The loss function includes the reconstruction error and the KL divergence, and the loss function is expressed as:

[0101]

[0102] Among them, represents the normal reconstructed data, x t represents the normal walking data, KL represents the divergence, Indicates the distribution that the normal reconstructed data follows. Indicates a normal distribution.

[0103] In this step, the reconstruction error between the normal data and the reconstructed normal data is represented by calculating the mean square error between the normal data and the reconstructed normal data, and the KL divergence between the calculated distribution that the normal data follows and the theoretically distribution of the normal data.

[0104] Step C4: Optimize the parameters of the VAE network based on the loss function. After the training is completed, a trained VAE network is obtained.

[0105] In this step, the parameters of the VAE network are optimized using the constructed loss function to reduce the reconstruction error between the data reconstructed by the VAE network and the normal walking data. When the constructed loss function reaches a preset value, the training ends, and the VAE network parameters at this time are used as the final VAE network parameters.

[0106] In this embodiment, a method for detecting abnormal states of a lower limb exoskeleton based on VAE is proposed. First, the walking data of a human wearing an exoskeleton robot is collected, and then the walking data is input into a pre-trained VAE network for data reconstruction to obtain reconstructed data corresponding to the walking data. Then, an anomaly score is calculated based on the walking data and the reconstructed data. When the anomaly score exceeds a preset threshold, it is determined that the interaction between the human and the exoskeleton robot is abnormal. Since the VAE network is trained using only a small amount of normal walking data and does not require machine abnormal usage data, the data acquisition cost is reduced, and at the same time, high accuracy and real-time performance of anomaly detection can be ensured; by directly inputting the walking data into the VAE network, it is possible to detect anomalies such as faults and running errors during the interaction between the human and the exoskeleton robot, and it is possible to track continuously changing abnormal states, and quantitatively evaluate the abnormal states by calculating the anomaly score.

[0107] In addition, compared with the existing anomaly detection that requires abnormal data such as collisions to train the network, and the anomalies may cause damage to the equipment, it is difficult to obtain training data and the cost is relatively high. This embodiment only relies on the proprioceptive sensors of the exoskeleton robot, so it is easier to implement and has a lower cost.

[0108] The embodiment of the present invention also provides a device for detecting abnormal states of a lower limb exoskeleton based on VAE, as Figure 4 shown. Figure 4 FIG. is a schematic structural diagram of a device for detecting abnormal states of a lower limb exoskeleton based on VAE provided by an embodiment of the present invention. The device includes:

[0109] The data acquisition module 41 is used to acquire the walking data of the human-worn exoskeleton robot, and the data includes: human limb angles, exoskeleton robot joint angles, and human-machine interaction torques;

[0110] The data reconstruction module 42 is used to input the walking data into a pre-trained VAE network for data reconstruction to obtain reconstructed data corresponding to the walking data, where the VAE network is trained with normal walking data as training samples;

[0111] The anomaly calculation module 43 is used to calculate an anomaly score based on the walking data and the reconstructed data;

[0112] The anomaly determination module 44 is used to determine that the interaction between the human body and the exoskeleton robot is abnormal when the anomaly score exceeds a preset threshold.

[0113] In an optional embodiment, the data reconstruction module includes:

[0114] The data mapping module is used to map the walking data to the latent space by using the encoder in the VAE network to obtain a deep representation of the walking data;

[0115] The data generation module is used to perform data reconstruction by using the decoder in the VAE network based on the deep representation of the walking data to obtain reconstructed data.

[0116] In an optional embodiment, the data mapping module includes:

[0117] The first data mapping sub-module is used to process the walking data into a feature vector by using the first fully connected layer in the encoder;

[0118] The second data mapping sub-module is used to process the feature vector by using the second fully connected layer in the encoder to obtain a deep representation of the walking data, and the deep representation characterizes the distribution of the walking data in the latent space.

[0119] In an optional embodiment, the data generation module includes:

[0120] The first data generation sub-module is used to perform reparameterization based on the deep representation of the walking data to obtain distribution parameters, and sample the latent space distribution according to the distribution parameters to obtain an intermediate variable encoding;

[0121] The second data generation sub-module is used to process the intermediate variable encoding by using two fully connected layers in the decoder to obtain reconstructed data with the same dimension as the walking data.

[0122] In an alternative embodiment, the device further comprises:

[0123] A parameter adjustment module, configured to adjust parameters of the variable impedance model of the exoskeleton robot based on the weights, where the parameters of the variable impedance model characterize the degree of assistance of the exoskeleton robot to the human body.

[0124] In an alternative embodiment, the device further comprises:

[0125] A training data input module, configured to input normal walking data of a human wearing the exoskeleton robot into the VAE network for training, where the normal walking data is data collected when a human wears the exoskeleton robot and normally follows a preset trajectory;

[0126] A training reconstruction module, configured to reconstruct data by the VAE network based on the normal walking data to obtain normal reconstruction data corresponding to the normal data;

[0127] A loss calculation module, configured to calculate a loss function for describing the difference between the normal walking data and the normal reconstruction data, where the loss function includes a reconstruction error and a KL divergence;

[0128] A parameter optimization module, configured to optimize parameters of the VAE network based on the constructed loss function. After the training is completed, a trained VAE network is obtained.

[0129] An embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes to implement the method for detecting abnormal states of a lower limb exoskeleton based on VAE according to the embodiments of the present invention.

[0130] Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other.

[0131] Embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of methods, devices, and equipment according to the embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, and the combination of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general computer, a special computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0132] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction means that implements the functions specified in one or more of the processes and / or blocks Figure 1 one or more of the processes and / or blocks Figure 1 specified in one or more of the blocks or blocks.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one or more of the processes and / or blocks Figure 1 one or more of the processes and / or blocks Figure 1 specified in one or more of the blocks or blocks.

[0134] Although the preferred embodiments of the embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0135] Finally, it should also be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variation thereof are intended to cover non-exclusive inclusion, such that a process, method, article or terminal device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising the element.

[0136] The above has introduced in detail a method, apparatus and device for detecting abnormal states of a lower limb exoskeleton based on VAE. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A method for detecting abnormal states of a lower limb exoskeleton based on VAE, characterized in that, the method includes: Collect the walking data of a human wearing an exoskeleton robot, and the data includes: human limb angles, exoskeleton robot joint angles, and human-machine interaction torques; Input the walking data into a pre-trained VAE network for data reconstruction to obtain reconstructed data corresponding to the walking data, where the VAE network is trained with normal walking data as training samples; Calculate an abnormal score s based on the walking data and the reconstructed data, and the abnormal score s is defined as: Among them, x t represents walking data, represents reconstructed data, and MSE(·) represents mean squared error; In the case where the abnormal score exceeds a preset threshold, it is determined that the interaction between the human and the exoskeleton robot is abnormal, and the abnormal score is smoothly mapped to a corresponding weight according to a weight function, and the parameters of the variable impedance model of the exoskeleton robot are adjusted based on the weight. The parameters of the variable impedance model characterize the degree of assistance of the exoskeleton robot to the human, and the weight function ω(s) is expressed as: where λ 1 and λ 2 respectively represent two constants greater than zero, m represents a constant for normalizing the anomaly score to a specified small range, and h represents the offset of the weight function from the coordinate origin; The variable impedance model of the exoskeleton robot is expressed as: Among them, C d and K d respectively represent the desired damping and the desired stiffness matrices, q and q d respectively represent the robot joint angles and the reference trajectory in the joint space, and respectively represent the first-order derivatives of q and q d with respect to time, and τ e represents the interaction torque between the human body and the exoskeleton robot.

2. The method according to claim 1, characterized in that, the inputting the walking data into a pre-trained VAE network for data reconstruction to obtain reconstructed data corresponding to the walking data includes: Using the encoder in the VAE network to map the walking data to the latent space to obtain a deep representation of the walking data; Based on the deep representation of the walking data, using the decoder in the VAE network for data reconstruction to obtain reconstructed data.

3. The method according to claim 2, characterized in that, the using the encoder in the VAE network to map the walking data to the latent space to obtain a deep representation of the walking data includes: Using the first fully connected layer in the encoder to process the walking data into a feature vector; Using the second fully connected layer in the encoder to process the feature vector to obtain a deep representation of the walking data, and the deep representation characterizes the distribution of the walking data in the latent space.

4. The method according to claim 2, characterized in that, the based on the deep representation of the walking data, using the decoder in the VAE network for data reconstruction to obtain reconstructed data includes: Performing reparameterization based on the deep representation of the walking data to obtain distribution parameters, and sampling the latent space distribution according to the distribution parameters to obtain an intermediate variable encoding; Using the two fully connected layers in the decoder to process the intermediate variable encoding to obtain reconstructed data with the same dimension as the walking data.

5. The method according to any one of claims 1-4, characterized in that, the VAE network is trained in the following manner: Input the normal walking data of a human wearing an exoskeleton robot into the VAE network for training, and the normal walking data is the data collected when a human wears an exoskeleton robot and walks normally following a preset trajectory; The VAE network performs data reconstruction based on the normal walking data to obtain normal reconstructed data corresponding to the normal walking data; Calculate a loss function for describing the difference between the normal walking data and the normal reconstruction data, where the loss function includes a reconstruction error and a KL divergence, and the loss function is expressed as: Among them, represents the normal reconstructed data, and x t represents the normal walking data, and KL represents the divergence, represents the distribution that the normal reconstructed data follows, represents the normal distribution; Optimize the parameters of the VAE network based on the loss function. After the training is completed, a trained VAE network is obtained.

6. A lower limb exoskeleton abnormal state detection device based on VAE, characterized in that the device includes: a data acquisition module for acquiring the walking data of a human wearing an exoskeleton robot, where the data includes: human limb angles, exoskeleton robot joint angles, and human-machine interaction torques; a data reconstruction module for inputting the walking data into a pre-trained VAE network for data reconstruction to obtain reconstructed data corresponding to the walking data, where the VAE network is trained with normal walking data as training samples; an abnormality calculation module for calculating an abnormality score s based on the walking data and the reconstructed data, where the abnormality score s is defined as: Among them, x t represents walking data, represents reconstructed data, and MSE(·) represents mean squared error; an abnormality determination module for determining that the interaction between the human and the exoskeleton robot is abnormal when the abnormality score exceeds a preset threshold, and smoothly mapping the abnormality score to a corresponding weight according to a weight function, and adjusting the parameters of the variable impedance model of the exoskeleton robot based on the weight, where the parameters of the variable impedance model characterize the degree of assistance of the exoskeleton robot to the human, and the weight function ω(s) is expressed as: where λ 1 and λ 2 respectively represent two constants greater than zero, m represents a constant for normalizing the anomaly score to a specified small range, and h represents the offset of the weight function from the coordinate origin; The variable impedance model of the exoskeleton robot is expressed as: Among them, C d and K d represent the desired damping and the desired stiffness matrices respectively, q and q d represent the robot joint angles and the reference trajectory in the joint space respectively, and represent the first-order derivatives of q and q d with respect to time respectively, and τ e represents the interaction torque between the human body and the exoskeleton robot.

7. An electronic device, characterized in that it includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes, it implements the VAE-based lower limb exoskeleton abnormal state detection method according to any one of claims 1-5.

Images