Lower limb prosthetic terrain recognition system and method based on multi-source signals

Abstract

The present invention provides a lower limb prosthetic terrain recognition system and method based on multi-source signals, comprising: fixing the residual limb on a fixed wear module; connecting the fixed wear module with a support connection module; connecting the support connection module with a first rigid shell; connecting the first rigid shell with a first drive module; connecting the first drive module with a second rigid shell; connecting the second rigid shell with a second drive module through a connecting rod; connecting the second drive module with a prosthetic foot; forming a receiving cavity inside the first rigid shell; an inertial measurement module and a data integration module being arranged in the receiving cavity; fixing a distance measurement module on the side surface of the first rigid shell; binding a strap above the knee on one side of a healthy limb, fixing the distance measurement module on the strap; and measuring the distance between the residual limb and the healthy limb end sensor and the ground respectively. A camera module is fixed on the side surface of the second rigid shell to obtain information on the ground in front of the knee.

Description

Technical Field

[0001] The present invention relates to the technical field of wearable medical treatment devices, and in particular, to a lower limb prosthetic terrain recognition system and method based on multi-source signals, and more particularly, to a lower limb electric prosthetic design that can automatically recognize terrain. Background Art

[0002] As the only compensation for residual limbs, the emergence and development of prostheses have brought hope for amputees to resume normal life. At present, lower limb prostheses can be divided into two types: unpowered prostheses and powered prostheses. The joints of unpowered prostheses do not provide power, and they mainly rely on pure mechanical structures to imitate healthy human limbs. For patients, their walking will consume about 20%-30% more energy than healthy people, which puts more stringent requirements on the patient's physical strength. However, because of its relatively low cost, its development is relatively rapid. At present, many patients use this type of unpowered prosthesis. In order to make up for this defect, powered prostheses have gradually been developed. At present, powered prostheses can be divided into two types according to their different driving methods. One is pneumatic artificial muscle. Its principle is to use an air pump to inject high-pressure air into the cavity to imitate muscle contraction, but its shortcomings are very obvious. The prosthesis is large in size and noisy, which brings great mental pressure to patients. At the same time, due to the compressibility of air, there is a certain gap between its theoretical control effect and actual use. As another power source, the motor has higher energy conversion efficiency, and its driving process is relatively quiet and the control is more precise than pneumatic. Therefore, the current mainstream view is that it is more reasonable to use motors as the driving source of powered prostheses. At present, some companies and research institutions use motors in conjunction with spring energy storage and release, which further improves the efficiency of energy use.

[0003] However, powered prostheses do have shortcomings, the two most obvious of which are the walking asymmetry between the healthy limb and the affected side and the stability problem in complex environments. The first shortcoming exists not only in powered prostheses, but also in unpowered prostheses, while the second shortcoming is more obvious in powered prostheses. In order to solve the stability problem in complex environments, scholars have proposed various solutions and developed human motion intention recognition. The current mainstream methods are human motion intention recognition based on mechanical signals, bio-neural signals and biomechanical signals. In response to this shortcoming of powered prostheses, the present invention has designed a lower limb powered prosthesis design that can automatically identify terrain, which can be applied to the five main motion scenarios in daily life, including flat-bottom walking, uphill and downhill, and up and down stairs.

[0004] Patent document CN113499173A (application number: 202110780493.5) discloses a lower limb prosthesis terrain recognition and motion prediction system based on real-time instance segmentation, including: a binocular camera collects terrain information around the lower limb prosthesis, an image acquisition and information transmission hardware platform collects the information obtained by the binocular camera and uploads it to a cloud server, the image acquisition and information transmission hardware platform is electrically connected to the lower limb prosthesis controller, and the lower limb prosthesis controller controls the movement mode of the lower limb prosthesis; a target terrain monitoring module, an instance segmentation module, a feature matching module and a position and size calculation module are deployed on the cloud server, and the terrain information is processed by the above modules in turn and fed back to the image acquisition and information transmission hardware platform. Summary of the invention

[0005] In view of the defects in the prior art, the object of the present invention is to provide a lower limb prosthetic terrain recognition system and method based on multi-source signals.

[0006] A lower limb prosthetic terrain recognition system based on multi-source signals provided by the present invention comprises:

[0007] A fixed wear module 1, a support connection module 2, a first rigid shell 3, a first drive module 4, a distance measurement module 5, a strap 6, an inertial measurement module 7, a camera module 8, a second rigid shell 9, a second drive module 10, a plantar pressure sensing module 11, a data integration module 14 and a prosthetic foot 18;

[0008] The residual limb is fixed on the fixed wearable module 1; the fixed wearable module 1 is connected to the supporting connection module 2; the supporting connection module 2 is connected to the first rigid shell 3; the first rigid shell 3 is connected to the first driving module 4; the first driving module 4 is connected to the second rigid shell 9; the second rigid shell 9 is connected to the second driving module 10 through a connecting rod; the second driving module 10 is connected to the artificial foot 18;

[0009] A receiving cavity 13 is formed inside the first rigid shell 3; the inertial measurement module 7 and the data integration module 14 are arranged in the receiving cavity 13; the inertial measurement module 7 is used to measure the acceleration and angular velocity information of the quadriceps femoris muscle on the residual limb side during movement; the data integration module 14 determines the current terrain based on the collected data;

[0010] The distance measuring module 5 is fixed to the side surface of the first rigid shell 3; the strap 6 is tied above the knee of the healthy limb, and the distance measuring module 5 is fixed to the strap 6; the distance measuring module 5 measures the distance between the residual limb and the healthy limb end sensor and the ground respectively;

[0011] The camera module 8 is fixed to the side surface of the second rigid shell 9, and is used to obtain information of the ground in front of the knee;

[0012] The first driving module 4 and the second driving module 10 replace the residual limb knee joint and ankle joint; drive the lower limb prosthesis to imitate the walking of a normal healthy limb;

[0013] The plantar pressure sensing module 11 is disposed at the bottom of the prosthetic foot 18 to obtain the pressure caused by the prosthetic foot 18 during walking;

[0014] The EEG cap 20 is worn on the human head and uses the electrode sheet 19 to collect EEG signals. The patient will subconsciously make movements when looking at the road ahead, generating EEG signals. Different EEG signals will be generated for different gaits. The EEG signals are input into the data integration module 14 as one of the judgment bases.

[0015] Based on the data collected by the inertial measurement module 7 , the distance measurement module 5 , the camera module 8 , the plantar pressure sensor module 11 and the electrode sheet 19 , the data integration module 14 determines the current terrain based on the collected data using the trained classifier.

[0016] Preferably, the fixed wearing module 1 comprises a prosthesis receiving cavity 15 and an air pressure valve 16 ; the prosthesis is placed in the prosthesis receiving cavity 15 and fixed by means of the air pressure valve 16 .

[0017] Preferably, the limb receiving cavity 15 is made of silicone, polypropylene sheet or cast by ABS resin.

[0018] Preferably, the first driving module 4 and the second driving module 10 are Maxon micromotors or von Hubble motors.

[0019] Preferably, the plantar pressure sensing module 11 includes a pad 12 and a preset number of pressure sensors 17; the preset number of pressure sensors 17 are arranged on the pad 12; the preset number of pressure sensors 17 are respectively distributed on the phalanges, metatarsals and calcaneus; the prosthetic foot 18 is placed on the pad 12, and when walking, gait analysis is performed on the pressure changes caused by the prosthetic foot 18 on the pressure sensors 17.

[0020] Preferably, a magic sticker is attached to the outer side of the strap 6, and the distance measurement module 5 is fixed on the strap 6 by the magic sticker.

[0021] According to a lower limb prosthetic terrain recognition system based on multi-source signals provided by the present invention, the following steps are performed using the lower limb prosthetic terrain recognition system based on multi-source signals:

[0022] Step S1: collecting data of various dimensions based on the inertial measurement module 7, the distance measurement module 5, the camera module 8, the plantar pressure sensor module 11 and the electrode sheet 19;

[0023] Step S2: preprocessing the collected data to obtain preprocessed data;

[0024] Step S3: training the neural network model using the preprocessed data to obtain a trained neural network;

[0025] Step S4: Use the trained neural network to perform terrain recognition, and drive the motor through the drive module to adopt the corresponding gait output.

[0026] Preferably, the step S2 adopts: filtering the collected data and extracting time-frequency domain features.

[0027] Preferably, the neural network includes: a BP network and / or a fuzzy neural network.

[0028] Preferably, step S3 adopts: due to the limitations of the linear regression algorithm, a ridge regression method is adopted, a regularization term paradigm is introduced to solve the irreversible problem of ill-conditioned matrices, and a minimum mean square error method is used to solve it, thereby obtaining a trained ridge regression classifier.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The invention designs a method for terrain recognition of lower limb prostheses based on multi-source signals, which collects multi-dimensional signals such as mechanical signals, biomechanical signals and visual signals as inputs, and can effectively overcome the shortcomings of the hysteresis of mechanical signals and the time-varying nature of biological neural signals, so that the processor can more accurately judge the current terrain and ensure that the patient can use the prosthesis more comfortably;

[0031] 2. The invention designs a method for terrain recognition of lower limb prostheses based on multi-source signals. In the process of terrain judgment, gait information is taken into consideration, and the difference in pressure on different parts of the sole of the foot caused by dorsiflexion and plantar flexion when going up and down slopes is used to provide a new idea for motion scene judgment. At the same time, the distance measurement module is used to measure the height difference between the prosthetic side and the prosthetic side, and the information of the healthy side is extracted to improve the accuracy of the processing module in judging the scene of going up and down stairs.

[0032] 3. The invention designs a method for terrain recognition of lower limb prostheses based on multi-source signals. The lower limb shell is made of ABS engineering plastics, which is lighter in weight than other powered prostheses and can alleviate the burden on patients when using them.

[0033] 4. Aiming at the existing problems in the use of lower limb powered prostheses, the present invention designs a powered lower limb prosthesis that can automatically identify terrain. The implementation of this project is conducive to promoting the research and development and application of related products, improving the comfort of patients, and ensuring the normal life of patients as much as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0035] Figure 1 Illustration of a wearer of a powered prosthetic limb climbing stairs.

[0036] Figure 2 Schematic diagram of the plantar pressure module.

[0037] Figure 3 Schematic diagram of the EEG cap.

[0038] Figure 4 Schematic diagram of network training.

[0039] Figure 5 A schematic diagram for network usage.

[0040] Among them, 1-fixed wear module; 2-support connection module; 3-first rigid shell; 4-first drive module; 5-distance measurement module; 6-strap; 7-inertial measurement module; 8-camera module; 9-second rigid shell 9; 10-second drive module; 11-plantar pressure sensing module; 12-pad 12; 13-accommodating cavity; 14-data integration module; 15-prosthesis receiving cavity; 16-air pressure valve; 17-pressure sensor; 18-artificial foot; 19-electrode sheet; 20-EEG cap. DETAILED DESCRIPTION

[0041] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0042] When patients with lower limb amputation use powered prostheses, due to different terrains, when going up and down stairs or walking up and down slopes and on flat ground, the prostheses cannot form effective coordination with the healthy limbs, resulting in poor use effects. When patients use the present invention, the trained classifier can effectively identify the current terrain through the data obtained by each sensor module, so that the prostheses and healthy limbs can cooperate with each other, improving the use effect.

[0043] Example 1

[0044] A lower limb prosthetic terrain recognition system based on multi-source signals provided by the present invention comprises:

[0045] A fixed wear module 1, a support connection module 2, a first rigid shell 3, a first drive module 4, a distance measurement module 5, a strap 6, an inertial measurement module 7, a camera module 8, a second rigid shell 9, a second drive module 10, a plantar pressure sensing module 11, a data integration module 14 and a prosthetic foot 18;

[0046] The residual limb is fixed on the fixed wearable module 1; the fixed wearable module 1 is connected to the supporting connection module 2; the supporting connection module 2 is connected to the first rigid shell 3; the first rigid shell 3 is connected to the first driving module 4; the first driving module 4 is connected to the second rigid shell 9; the second rigid shell 9 is connected to the second driving module 10 through a connecting rod; the second driving module 10 is connected to the artificial foot 18; the first driving module 4 acts as a knee joint of the human body; the second driving module 10 acts as an ankle joint of the residual limb;

[0047] A receiving cavity 13 is formed inside the first rigid shell 3; the inertial measurement module 7 and the data integration module 14 are arranged in the receiving cavity 13; the inertial measurement module 7 is used to measure the acceleration and angular velocity information of the quadriceps femoris muscle on the residual limb side during movement; the data integration module 14 transmits the collected data to the data integration module via wired and / or wireless means to determine the current terrain;

[0048] The distance measuring module 5 is fixed to the side surface of the first rigid shell 3; the strap 6 is tied above the knee of the healthy limb, and the distance measuring module 5 is fixed to the strap 6; the distance measuring module 5 measures the distance between the residual limb and the healthy limb end sensor and the ground respectively;

[0049] The camera module 8 is fixed to the side surface of the second rigid shell 9, and is used to obtain information of the ground in front of the knee;

[0050] The first driving module 4 and the second driving module 10 form a two-degree-of-freedom lower limb power prosthesis to replace the residual limb knee joint and ankle joint; drive the lower limb prosthesis to imitate the walking of a normal healthy limb;

[0051] The plantar pressure sensing module 11 is disposed at the bottom of the prosthetic foot 18 to obtain the pressure caused by the prosthetic foot 18 during walking;

[0052] The EEG cap 20 is worn on the human head and uses the electrode sheet 19 to collect EEG signals. The patient will subconsciously make movements while looking at the road ahead, generating EEG signals. Different EEG signals will be generated for different gaits. The EEG signals are input into the data integration module 14 as one of the judgment bases. The EEG data acquisition device can use ActiveTwo 32-channel EEG instrument;

[0053] Based on the data collected by the inertial measurement module 7 , the distance measurement module 5 , the camera module 8 , the plantar pressure sensor module 11 and the electrode sheet 19 , the data integration module 14 determines the current terrain based on the collected data using the trained classifier.

[0054] Specifically, the wearable module 1 is fixed to the residual limb of the lower limb; the distance between the residual limb and the knee is in the range of 150mm to 250mm, and the minimum circumference of the outer diameter of the residual limb of the lower limb is 400mm and the maximum circumference is 600mm;

[0055] Specifically, a thin, smooth nylon stocking can be placed over the residual limb;

[0056] Specifically, the supporting connection module 2 is made of polyethylene sheet or polypropylene sheet;

[0057] Specifically, the first rigid shell 3 and the second rigid shell 9 can be made of titanium alloy or ultra-light aluminum alloy material;

[0058] Specifically, the camera module 8 adopts a binocular camera.

[0059] Specifically, the inertial measurement module adopts an IMU inertial sensor, and a 9-axis attitude sensor-HI229 is optional;

[0060] The distance measurement module uses two laser ranging radars;

[0061] The plantar pressure sensing module 11 is made of carbon fiber material, and the sensor uses a micro pressure sensor.

[0062] Specifically, the fixed wearing module 1 includes a prosthesis receiving cavity 15 and an air pressure valve 16; the prosthesis is placed in the prosthesis receiving cavity 15 and fixed by the air pressure valve 16; the air pressure valve adopts a negative pressure valve.

[0063] The limb receiving cavity 15 is connected to the first rigid shell 3 via the supporting connection module 2, and acts as the femoral head of the residual limb of the human body;

[0064] Specifically, the limb receiving cavity 15 is cast by silicone or plate, and the air pressure valve 16 and related structural parts are installed on the receiving cavity 15; resin casting is adopted, and the receiving cavity 15 and structural parts such as the air pressure valve 16 are sealed and connected by bonding.

[0065] Specifically, the first driving module 4 and the second driving module 10 adopt Maxon micro motors or von Hubble motors.

[0066] Specifically, the plantar pressure sensing module 11 includes a pad 12 and a preset number of pressure sensors 17; the preset number of pressure sensors 17 are arranged on the pad 12; the preset number of pressure sensors 17 are respectively distributed at the phalanges, metatarsals and calcaneus; the artificial foot 18 is placed on the pad 12, and when walking, gait analysis is performed on the pressure changes caused by the artificial foot 18 on the pressure sensors 17. The pad 12 is made of polyethylene or polypropylene sheet; the artificial foot 18 is cast and molded with silicone; the pad 12 is placed in the shoe, and the artificial limb 18 is placed in the shoe, on the pad 12, simulating a normal human foot.

[0067] Specifically, a magic sticker is attached to the outer side of the strap 6, and the distance measurement module 5 is fixed on the strap 6 by the magic sticker.

[0068] The binding strap 6 can be made by sewing polyester Oxford cloth;

[0069] According to a lower limb prosthetic terrain recognition system based on multi-source signals provided by the present invention, the following steps are performed using the lower limb prosthetic terrain recognition system based on multi-source signals:

[0070] Step S1: collecting data of various dimensions based on the inertial measurement module 7, the distance measurement module 5, the camera module 8, the plantar pressure sensor module 11 and the electrode sheet 19;

[0071] Step S2: preprocessing the collected data to obtain preprocessed data; specifically, filtering is first performed on the collected data. Since the interference of the data of each module is different, the data of each module must be filtered. The EEG signal is preprocessed using the EEGLAB toolbox, and the time-frequency domain features are extracted to obtain preprocessed data. For the data collected by other modules, a similar process is adopted. After filtering to remove noise interference, the corresponding time-frequency domain features are extracted to obtain preprocessed data;

[0072] Step S3: using the preprocessed data to train the neural network model to obtain a trained neural network; wherein the neural network model includes a BP network model and / or a fuzzy neural network model.

[0073] The neural network integrates the feature data from the camera module 8, the inertial measurement module 7, the distance measurement module 5, the plantar pressure sensor module 11 and the EEG module 20. The activation function uses the Sigmoid function, and the weight of the neuron is converted by the weight transformation amount △W. ij Adjust W ij , through continuous iteration, repeatedly adjusting W ij Weight.

[0074] Specifically, the step S3 adopts: due to the limitations of the linear regression algorithm, the ridge regression method is adopted, the regularization term paradigm is introduced to solve the irreversible problem of the ill-conditioned matrix, and the minimum mean square error method is used to solve it, so as to obtain the trained ridge regression classifier.

[0075] Step S4: Use the trained neural network to perform terrain recognition, and drive the motor through the drive module to adopt the corresponding gait output.

[0076] Example 2

[0077] Embodiment 2 is a preferred embodiment of Embodiment 1

[0078] The present invention provides a method for terrain recognition of lower limb prosthesis based on multi-source signals, such as Figures 1 to 5 As shown, it includes a fixed wear module 1, a support connection module 2, a data integration module 14, a first rigid shell 3, a second rigid shell 9, a distance measurement module 5, an inertial measurement module 7, a plantar pressure sensor module 11, a camera module 8, an EEG signal measurement module 20 and several structural parts.

[0079] Furthermore, the structural member includes a first rigid shell 3 and a second rigid shell 9, the strap 6 is tied above the knee of the healthy limb, a magic sticker can be attached to the outside of the strap 6, and the distance measurement module 5 is fixed to the strap 6 by the magic sticker.

[0080] Specifically, the prosthetic receiving cavity 15 can be molded by casting resin material, and the air pressure valve 16 is fixed to the posterior side of the femur. The patient places the residual limb in the prosthetic receiving cavity 15 and fixes it using the air pressure valve 16 .

[0081] Furthermore, the powered lower limb prosthesis is set to have two degrees of freedom, and the first drive module 4 and the second drive module 10 use maxon micromotors, which are fixed to the lower ends of the first rigid shell 3 and the second rigid shell 9 in sequence to serve as compensation for the patient's residual limb knee and ankle joints.

[0082] Furthermore, the artificial foot 18 is fixed at the lower end of the second rigid shell 9 as a substitute for the residual limb of the patient. The pad 12 acts as an insole and is placed at the bottom of the shoe. There are 5 pressure sensors 17, which are respectively distributed at the phalanges, metatarsals and calcaneus. The artificial foot 18 is placed on the pad 12, and the pressure change caused by the artificial foot 18 on the pressure sensor 17 on the pad during walking is used for gait analysis. The camera 8 is installed at the front upper end of the second rigid shell 9, specifically, it is placed at the connection between the first rigid shell 3 and the second rigid shell 9, in front of the first driving module 4, and is used to obtain information about the ground in front of the knee, and use this information as one of the bases for judging the terrain. The distance measurement module 5 is composed of a pair of laser radars, one of which is fixed at the side end of the first rigid shell, and the other is fixed at the corresponding position on the healthy limb side through the strap 6. They respectively measure the distance between the residual limb and the healthy limb end sensor and the ground, and use the distance difference as one of the bases for judging the terrain. The inertial measurement module 7 is built into the accommodating cavity 13 to measure the acceleration, angular velocity and other information of the quadriceps femoris muscle on the residual limb side during movement, and obtains an angle signal by integration. The difference in the angle range under different movement states is used as evidence for judging the terrain.

[0083] Specifically, the following five terrains may appear in actual application scenarios. The following five terrains may appear in the daily life of amputees, namely, walking on flat ground, walking up stairs, walking down stairs, walking uphill and walking downhill. First, the sensors of each module collect data and preprocess the received data. The main operations are filtering and denoising the data and framing the data. When filtering and denoising, for the data collected by the IMU inertial sensor, a low-pass filter is used to filter out high-frequency noise, and the bandwidth of the filter is set to around 30Hz. For the signals collected by other sensors, a bandpass filter is used to filter out high-frequency noise and low-frequency motion artifacts. If there is a large deviation in the number of data samples between classes, a sample balance operation is performed to extract and interpolate the data so that the number of samples of each class remains roughly balanced. Secondly, since the data collected by the sensor is continuous, in order to extract useful information from it, the data must be divided into data windows. Here, a sliding window method is used to divide the data window length every 100ms, and then 50ms is moved on the original data as the next frame of data. After obtaining the data window, in order to obtain effective information of the window, feature extraction is performed again. Each frame is taken as a time window, and the time-frequency domain features are extracted by short-time Fourier transform. The most discriminative features are selected from the large number of extracted features, and the features that contribute the most to the performance of the classifier are established to establish an existing feature subset. The features after dimensionality reduction are finally subjected to pattern classification, and a classification method based on pattern recognition is adopted. Here, linear discriminant analysis LDA is adopted. Before use, the classifier must first be trained, and the actual data is classified using the trained model parameters. After preprocessing, the data collected by each sensor is passed to the data integration module 14, which receives instructions from the host PC through the WiFi5.8G module and sends the integrated data to the host PC.

[0084] More specifically, the present invention utilizes the camera module 8 and the inertial measurement module 7 to identify and reconstruct terrain information, utilizes the plantar pressure sensing module 11 to verify the correctness of the terrain, and utilizes the distance measurement module 5 and the plantar pressure sensing module 11 to adopt a gait adapted to the terrain according to the identified terrain. The camera module 8 is used to locate the initial point at a reference position, and the height and distance information of the obstacles measured later are all converted into the initial reference coordinate system, thereby forming a terrain map.

[0085] The camera module forms 3D point cloud data through a set of ordered 3D coordinate discrete points arranged in a regular network. According to the data corresponding to different terrains, the corresponding terrain can be matched with the point cloud data collected in real time after training and comparison, thereby realizing the terrain reconstruction function; the inertial data collected by the inertial measurement module includes angular velocity information and acceleration information, which are solved by the strapdown inertial navigation algorithm SINS, including coordinate transformation, integral transformation, zero-speed correction, etc.

[0086] The plantar pressure module receives different time-amplitude characteristics of the pressure distribution received by the five pressure sensors at the bottom when the plantar pressure module adopts different gaits according to the terrain. Based on the data reading of different gaits of residual limb patients in advance, the pressure data of the corresponding gait is labeled for training. After that, during use, the corresponding terrain gait can be judged according to the pressure information. The key lies in the weight ratio when making decisions.

[0087] The distance measurement module realizes the corresponding function by using a laser ranging radar near the knee position on the prosthesis and residual limb, because the height difference of the knees of the patients is different when walking on a flat road, going up and down a slope, and going up and down stairs. The height difference is the largest when going up and down stairs, while the flat road is relatively gentle relative to the slope, so this can be used as one of the criteria of the classifier;

[0088] Since the main movement scene of the lower limbs is movement in the sagittal plane, there is no need to consider left and right movement. Taking a specific obstacle as an example, when the human body moves from time T1 to time T2, the inertial measurement module 7 can be used to obtain the horizontal acceleration ax and the vertical acceleration ay, and the horizontal displacement L1 and the vertical displacement H1 can be obtained by integration:

[0089]

[0090] At time T2, the measurement distance of the camera module 8 is L2, and the angle between the camera module 8 and the horizontal plane is α. Then, the horizontal distance L3 and the vertical distance H2 between the obstacle and the camera module 8 at time T2 are respectively:

[0091] L3=L2*cosα

[0092] H2=L2*sinα

[0093] Then, it can be obtained that at time T1, i.e., the initial time, the horizontal distance L and the vertical distance H between the specific obstacle and the camera module 8 are:

[0094] L=L1+L3

[0095] H=H2-H1

[0096] By using the above method, subsequent scenes can be continuously scanned and calculated, thereby realizing reconstruction of the terrain scene ahead.

[0097] The plantar pressure sensing modules 11 are distributed in turn at the phalanges, metatarsals and calcaneus, forming 3 rows, with the number of rows being 2, 1 and 2 respectively. Taking walking on flat ground as an example, when the patient is walking on flat ground, when the plantar pressure sensor located at the calcaneus measures a pressure rise and exceeds a set threshold, it can be calibrated that the heel touches the bottom at this time, and the bottoming event is marked as FS. When the pressure sensor located at the metatarsal measures that the pressure rises to a certain threshold and then begins to decrease, accompanied by the pressure of the pressure sensor located at the phalanges beginning to show a similar periodic change in the time domain after the metatarsal pressure sensor measures, it can be calibrated that the toes are off the ground at this time, and the off-the-ground event is marked as FO. In addition, since the distribution of plantar pressure when the patient's foot is in dorsiflexion or toe flexion when going up and down stairs and up and down slopes is also different, different plantar pressure change information can be obtained according to different terrains, and the terrain information constructed by the camera module 8 and the inertial measurement module 7 can be verified, and corresponding gait cycles can be used to cope with different terrains. The distance measurement module 5 measures the height information of the healthy limb side and the residual limb side from the ground, respectively, and is recorded as H p and H s ,At the initial time of T1, the distance information is recorded as reference information. When the patient is walking, H p and H s The change is periodic, and the height difference Δ=H p -H S , according to the threshold variation range of Δ when a person walks, it can be divided into two situations, namely, low threshold and high threshold states, wherein the low threshold state corresponds to the gait when walking on flat ground, and the high threshold state corresponds to the two states of going up and down stairs and going up and down slopes. More specifically, when the fluctuation of Δ is in the high threshold range, the PC host computer calculates its partial derivative with respect to time according to the height information. If the partial derivative is in a continuous state, the corresponding height time curve is smooth, and it can be concluded that the high threshold height difference Δ measured and calculated by the distance measurement module 8 in this case corresponds to going up and down slopes, and if the partial derivative is discontinuous, the corresponding height time curve is not smooth, and it can be concluded that the high threshold height difference Δ measured and calculated by the distance measurement module 8 in this case corresponds to going up and down stairs. In addition, combined with the reconstructed terrain information and plantar pressure information, the PC host computer can further determine the corresponding terrain and take the corresponding gait, and transmit the gait information to the data integration module 14 through WiFi, and the data integration module 14 sends a signal to the first driving module 4 and the second driving module 10 to drive the action of the prosthesis.

[0098] The EEG signal measurement module 20 collects the original EEG signal of the patient through a flexible electrode cap, amplifies it through an amplifier, records the EEG signal at a sampling frequency of 500Hz, filters the original signal through a 0.5-50Hz bandpass filter, eliminates high-frequency signal interference, and extracts time-frequency features through discrete wavelet transform. By using the diagonalization of the spatial matrix, a set of optimal projection space filters is found, so that the variance of one signal is the largest and the variance of the other is the smallest, maximizing and minimizing the variance difference between the two signals, thereby obtaining a high-resolution feature vector and extracting the spatial distribution characteristics of various types of signals. The obtained features are transmitted to the data integration module 14 via WiFi, and the data integration module 14 transmits the data to the PC host computer using WiFi (5.8G).

[0099] The five sets of feature data from the data integration module 14 are input into the PC host computer. The data fusion adopts the feature layer fusion of the intermediate level. The typical neural network is used to process the multi-sensor target tracking. The neural network adopts the BP neural network. The BP network adopts a three-layer structure. The number of input nodes is initially set to 5, the number of output nodes is 1, and the learning rate is adaptive. The number of hidden layer nodes is unknown. According to experience, it can be assumed that the number of hidden layer nodes is H. Let m and n be the number of nodes in the input and output layers respectively, and a is an adjustment parameter from 1 to 10. Then In order to obtain the most appropriate number of hidden layer nodes, the initial learning rate is set to 0.3. The appropriate adjustment parameters are selected through continuous iteration of training to obtain the appropriate number of hidden layer nodes. After the number of hidden layer nodes is obtained, the learning rate can be set to adaptive to make the network converge as quickly as possible. After the network is trained, the weight components corresponding to each group of nodes are obtained. When used, the feature data obtained by each sensor is input into the node corresponding to the weight component to obtain the corresponding output, which is then passed to the drive module to further drive the motor to adopt the corresponding gait.

[0100] The present invention can determine the current terrain through the five groups of information measured by the measurement module and the classification by the classifier, so that amputees can use the prosthesis more conveniently and comfortably.

[0101] In the description of the present application, it should be understood that the terms "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0102] Those skilled in the art know that, in addition to implementing the system, device and its various modules provided by the present invention in a purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, the system, device and its various modules provided by the present invention can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing the method and structures within the hardware component.

[0103] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A lower limb prosthetic terrain recognition system based on multi-source signals, characterized in that: include: A fixed wearable module (1), a supporting connection module (2), a first rigid shell (3), a first driving module (4), a distance measurement module (5), a strap (6), an inertial measurement module (7), a camera module (8), a second rigid shell (9), a second driving module (10), a plantar pressure sensing module (11), a data integration module (14), and a prosthetic foot (18); The residual limb is fixed on the fixed wearable module (1); the fixed wearable module (1) is connected to the supporting connection module (2); the supporting connection module (2) is connected to the first rigid shell (3); the first rigid shell (3) is connected to the first driving module (4); the first driving module (4) is connected to the second rigid shell (9); the second rigid shell (9) is connected to the second driving module (10) via a connecting rod; the second driving module (10) is connected to the artificial foot (18); A receiving cavity (13) is formed inside the first rigid shell (3); the inertial measurement module (7) and the data integration module (14) are arranged in the receiving cavity (13); the inertial measurement module (7) is used to measure the acceleration and angular velocity information of the quadriceps femoris muscle on the residual limb side during movement; the data integration module (14) determines the current terrain based on the collected data; The distance measurement module (5) is fixed on the side surface of the first rigid shell (3); the strap (6) is tied above the knee of the healthy limb, and the distance measurement module (5) is fixed on the strap (6); the distance measurement module (5) respectively measures the distance between the residual limb and the healthy limb end sensor and the ground; The camera module (8) is fixed to the side surface of the second rigid shell (9) and is used to obtain information about the ground in front of the knee; The first driving module (4) and the second driving module (10) replace the residual limb knee joint and ankle joint; drive the lower limb prosthesis to imitate the walking of a normal healthy limb; The plantar pressure sensing module (11) is arranged at the bottom of the artificial foot (18) to obtain the pressure caused by the artificial foot (18) when walking; It also includes an electroencephalogram cap (20), which is worn on the human head and uses an electrode sheet (19) to collect electroencephalogram signals. When the patient looks at the road ahead, he or she will subconsciously make movements to generate electroencephalogram signals. Different electroencephalogram signals will be generated for different gaits. The electroencephalogram signals are input into the data integration module (14) as one of the judgment bases. Based on the data collected by the inertial measurement module (7), the distance measurement module (5), the camera module (8), the plantar pressure sensor module (11) and the electrode sheet (19), the data integration module (14) determines the current terrain based on the collected data using a trained classifier.

2. The lower limb prosthetic terrain recognition system based on multi-source signals according to claim 1 is characterized in that: The fixed wearable module (1) comprises a prosthesis receiving cavity (15) and an air pressure valve (16); the prosthesis is placed in the prosthesis receiving cavity (15) and fixed using the air pressure valve (16).

3. The lower limb prosthetic terrain recognition system based on multi-source signals according to claim 2 is characterized in that: The limb receiving cavity (15) is formed by casting with silica gel, polypropylene plate or ABS resin.

4. The lower limb prosthetic terrain recognition system based on multi-source signals according to claim 1, characterized in that: The first driving module (4) and the second driving module (10) adopt Maxon micro motors or von Hubble motors.

5. The lower limb prosthetic terrain recognition system based on multi-source signals according to claim 1, characterized in that: The plantar pressure sensing module (11) comprises a pad (12) and a preset number of pressure sensors (17); the preset number of pressure sensors (17) are arranged on the pad (12); the preset number of pressure sensors (17) are respectively distributed at the phalanges, metatarsals and calcaneus; the artificial foot (18) is placed on the pad (12), and when walking, gait analysis is performed on the pressure changes caused by the artificial foot (18) on the pressure sensors (17).

6. The lower limb prosthetic terrain recognition system based on multi-source signals according to claim 1, characterized in that: A magic sticker is attached to the outer side of the binding strap (6), and the distance measurement module (5) is fixed to the binding strap (6) by the magic sticker.

7. A method for terrain recognition of lower limb prosthesis based on multi-source signals, characterized in that: The lower limb prosthetic terrain recognition system based on multi-source signals described in any one of claims 1 to 6 is used to perform the following steps: Step S1: collecting data of various dimensions based on the inertial measurement module (7), the distance measurement module (5), the camera module (8), the plantar pressure sensing module (11) and the electrode sheet (19); Step S2: preprocessing the collected data to obtain preprocessed data; Step S3: training the neural network model using the preprocessed data to obtain a trained neural network; Step S4: Use the trained neural network to perform terrain recognition, and drive the motor through the drive module to adopt the corresponding gait output.

8. The method for lower limb prosthetic terrain recognition based on multi-source signals according to claim 7, characterized in that: The step S2 adopts: filtering the collected data and extracting time-frequency domain features.

9. The method for lower limb prosthetic terrain recognition based on multi-source signals according to claim 7, characterized in that: The neural network includes: BP network and / or fuzzy neural network.

10. The method for lower limb prosthetic terrain recognition based on multi-source signals according to claim 7, characterized in that: The step S3 adopts: due to the limitations of the linear regression algorithm, the ridge regression method is adopted, the regularization term paradigm is introduced to solve the irreversible problem of the ill-conditioned matrix, and the minimum mean square error method is used to solve it, so as to obtain the trained ridge regression classifier.

Images