Novel multi-sensory upper limb movement measuring system

Abstract

The invention discloses a novel multi-sense upper limb motion measuring system which comprises a soft felt garment (10), a motion measuring subsystem (12) and a sense measuring subsystem (13), wherein the motion measuring subsystem (12) and the sense measuring subsystem (13) are used for respectively acquiring motion and sensed multi-mode signals and transmitting the motion and sensed multi-mode signals to a multi-mode signal processing module (14) to be amplified and filtered; the wireless signal concentrator (15) collects data from the multimode signal processing module (14); the computer client (16) integrates multi-modal signals to construct an upper limb motion sensing state information system, and the system comprises an upper limb motion calculation model (11) and a multi-sensing information real-time state; the upper limb movement calculation model (11) calculates the data signal and forms a movement model having real-time position coordinate data of each joint. According to the invention, simple, accurate and comprehensive upper limb movement monitoring real-time support is provided, and the accuracy and comprehensiveness of upper limb movement data acquisition and analysis can be effectively improved for the fields of virtual reality interaction and the like.

Description

Technical Field

[0001] The invention relates to the technical field of medical rehabilitation equipment and is a novel multi-sensory upper limb movement measurement system. Background Art

[0002] In the field of rehabilitation medicine, rehabilitation treatment for stroke patients has always been a research focus and difficulty. With the advancement of medical technology, rehabilitation training methods have continued to develop, but they still face many challenges. Traditional rehabilitation training often focuses on simple limb movement training, such as relying on rehabilitation therapists to manually assist patients in joint movements or using simple mechanical devices to guide limb movements. This method lacks accurate monitoring and quantitative evaluation of the patient's overall status during training, making it difficult to gain an in-depth understanding of the patient's neuromuscular control and subtle changes in rehabilitation progress.

[0003] In recent years, virtual reality (VR) technology has gradually been introduced into multisensory training, aiming to provide patients with a richer and more immersive training environment. However, there are obvious deficiencies in the combination of existing VR multisensory training equipment and patients' actual motion data. On the one hand, when obtaining the patient's upper limb posture information, the methods used by most devices are not accurate and comprehensive enough. For example, the position information obtained by the IMU will have cumulative errors and cannot accurately reflect the complex movement of the upper limbs in three-dimensional space, including subtle angle changes of each joint and the overall spatial position and posture changes of the limbs. This results in poor synchronization between the movement of the virtual upper limb and the patient's real upper limb in the VR scene, which is prone to delays, deviations and other problems, seriously affecting the patient's depth perception and training effect.

[0004] On the other hand, there are also defects in the collection of multi-sensory data. The collection of sensory information such as touch and temperature of the hands is not perfect, and the sensory stimulation in the real environment is not fully simulated, making it difficult for patients to obtain a real interactive experience during VR training. For example, when grasping a virtual object, it is impossible to accurately perceive the object's texture, weight, temperature and other properties, which is extremely unfavorable for the patient's motor relearning and neural function remodeling, and may even cause the patient to form a maladaptive movement pattern, limiting the potential for functional recovery.

[0005] At the same time, with the growing demand for personalized treatment plans in rehabilitation medicine, existing rehabilitation equipment is difficult to meet the requirements of precise rehabilitation training based on individual differences of patients. The lack of in-depth analysis and effective use of multi-dimensional data of patients makes it impossible to formulate optimized training strategies for each patient's unique rehabilitation condition, thereby reducing the efficiency and success rate of rehabilitation treatment. In summary, it is extremely important to develop a data glove that can accurately obtain upper limb posture information based on forward kinematics modeling methods and realize multi-sensory data acquisition. It is expected to fill the gap in existing rehabilitation technology, provide more efficient, accurate and personalized rehabilitation training solutions for stroke patients and other upper limb rehabilitation needs, and promote the further development of rehabilitation medical technology. Summary of the invention

[0006] Purpose of the invention: The purpose of the present invention is to provide a novel multi-sensory upper limb motion measurement system to solve a series of problems such as the lack of integration of existing limb movement and perception training with VR training, resulting in a lack of multi-sensory stimulation and inaccurate motion feedback in patients, which in turn leads to difficulty in depth perception, easy formation of bad movement habits and limited functional recovery. Through innovative technical means, the deep interaction and precise coordination between the two can be achieved, the effectiveness and personalization of rehabilitation training can be improved, and patients can be helped to better recover their limb function and neural perception ability.

[0007] Technical solution: The novel multi-sensory upper limb motion measurement system described in the present invention comprises a soft felt clothing integrated multi-modal data acquisition system and a multi-mode signal processing and transmission module; the motion measurement subsystem and the sensory measurement subsystem respectively obtain the multi-mode signals of motion and perception and transmit them to the multi-mode signal processing module for amplification and filtering; the wireless signal concentrator collects the processed data from the multi-mode signal processing module and sends it to the computer client, and this process can be transmitted via WiFi; the computer client integrates the multi-modal signals to construct a complete, accurate and real-time updated upper limb motion perception status information system, which includes an upper limb motion calculation model and real-time status of multi-sensory information; the upper limb motion calculation model is calculated through the multi-mode signals collected by the motion measurement subsystem to form a motion model with real-time position coordinate data of each joint.

[0008] Furthermore, the system is composed of a soft felt garment for integrating and fixing the motion measurement subsystem, the sensory measurement subsystem, the multi-mode signal processing module, the data Bluetooth transmission module and the wireless signal concentrator into a wearable device;

[0009] The upper limb motion calculation model calculates the real-time position of the upper limb joints through motion multi-mode data, including calculations of trigonometric functions, vector rotations, and coordinate transformations;

[0010] The motion measurement subsystem collects angle data of adduction, abduction, extension, flexion, internal and external rotation of the upper limb joint during exercise, the position of the head relative to the sternoclavicular joint, is fixed at the corresponding acquisition position of the measurement corresponding data, and collects the corresponding original electrical signal and outputs it to the multi-mode signal processing module;

[0011] The sensory measurement subsystem collects hand temperature and pressure signals, obtains temperature sensation, touch sensation and pain sensation, and outputs original electrical signals;

[0012] The multi-modal signal processing module receives the original electrical signals from the motion measurement subsystem and the sensory measurement subsystem, filters and amplifies them, and extracts features for specific joint motion data, and then transmits the processed multi-modal data to the data Bluetooth transmission module and the wireless signal concentrator respectively;

[0013] After completing the Bluetooth data aggregation and integration, the data Bluetooth transmission module and the wireless signal concentrator transmit all data to the computer client;

[0014] The computer terminal constructs an upper limb movement perception state information system for the received data, including an upper limb movement calculation model and a real-time state of multi-sensory information.

[0015] Furthermore, the upper limb motion calculation model includes connections from the sternoclavicular joint O0 to the shoulder joint rotation center O2, the elbow joint rotation center O3, the wrist joint rotation center O4, the metacarpophalangeal joint, the proximal finger joint, the distal finger joint, and the fingertip in sequence. In addition, O1 is the origin of the local coordinate system where the ultrasonic receiving clock synchronization device is located. The calculation process of the upper limb motion calculation model is as follows:

[0016] The sternoclavicular joint is the origin of the global coordinate system. The sternoclavicular joint is the origin of the global coordinate system. It is stipulated that the forward direction is the positive direction of the x-axis (directly in front of the human body), the left direction is the positive direction of the y-axis (left side of the human body), and the upward direction is the positive direction of the z-axis (vertical to the ground). A right-handed rectangular coordinate system is established. The local coordinate system directions of the remaining joints are the same;

[0017] The coordinate transformation of the shoulder joint rotation center O2 relative to the sternoclavicular joint O0, assuming that the measured shoulder width is d shoulder , so half of the shoulder width is L1. According to anatomical knowledge, it is assumed to be in the x-axis direction of the global coordinate system and on the horizontal line. Then the coordinates of the shoulder joint rotation center relative to the sternoclavicular joint are (x2, y2, z2) = (0, L1, 0) (here, the slight vertical deviation caused by individual differences in the human body is ignored, taking the left upper limb as an example);

[0018] The coordinates of the elbow joint rotation center O3 relative to the shoulder joint rotation center are calculated, assuming that the arm length is L upperarm, the rotation angle of the upper arm around the shoulder joint rotation center O2 of the local coordinate system x2 axis is θ1, the rotation angle of the upper arm around the shoulder joint rotation center O2 of the local coordinate system z axis is θ2, then the coordinates of O3 in the global coordinate system are (x3, y3, z3):

[0019]

[0020] The wrist joint rotation center is O4, and the upper arm vector is Its length is L upperarm , the forearm vector is Its length is L forearm， ∠O2O3O4=α1, vector around the upper arm The rotation angle is α2, After elbow flexion The angle is α1, Expressed as (Δx, Δy, Δz), according to the relationship between vector projection and trigonometric function, we can get:

[0021]

[0022] Conduct a winding The rotation α2 of is calculated by the Rodrigues rotation formula:

[0023] R=I+(sinα2)K+(1-sinα2)K 2

[0024] Where I is the identity matrix K is the axis of rotation The corresponding antisymmetric matrix is ​​of the form:

[0025]

[0026] Calculate K first 2 :

[0027]

[0028] Then substitute it into the Rodriguez rotation formula to calculate the rotation matrix R:

[0029]

[0030] Multiply the coordinates (Δx, Δy, Δz) at the initial position by the rotation matrix R to obtain the rotated coordinates (x4, y4, z4), that is:

[0031]

[0032] By expanding the matrix multiplication, we can get the specific expressions of x4, y4, z4;

[0033] The coordinate transformation from the wrist joint to the metacarpal joint obtains the internal and external rotation angles of the elbow joint through the tension sensor device, and the coordinates of the metacarpophalangeal joint with respect to the global coordinate system are calculated using the same method as that for calculating the wrist joint coordinates;

[0034] The metacarpophalangeal joint, the proximal interphalangeal joint, and the distal interphalangeal joint only consider their flexion movement in one plane, ignoring the movement in other directions and the soft tissue deformation of the finger. Assume that the rotation angle of the interphalangeal joint is γ i , translated by d along x i distance, i is the knuckle number, and the homogeneous transformation matrix between adjacent knuckles is:

[0035]

[0036] Thus, the coordinates of each finger joint and fingertip relative to the global coordinate system are obtained.

[0037] Furthermore, the motion measurement subsystem is collaboratively composed of a variety of high-precision sensors, including a tension sensor, a uniaxial flexible bending strain sensor, an ultrasonic low-power ultrasonic transmitter module, an ultrasonic receiving device, a clock synchronization module, a flexible printed circuit board and electrical connecting wires, wherein:

[0038] The tension sensing device is composed of a strap, a tension sensor, and an elastic rope, wherein the strap is fixed at the upper arm near the shoulder joint and elbow joint, and at the forearm near the elbow joint and wrist joint; the tension sensor and the elastic rope are connected to each other, and when the arm hangs naturally with the palm facing one side of the body, the two ends are vertically fixed on the strap;

[0039] The uniaxial flexible bending strain sensor is placed on the shoulder, wrist and finger joints, closely fitting the limbs, and outputting the continuous change of angles during joint movement, which is used as a parameter for upper limb forward kinematics modeling to help construct real-time upper limb movement postures;

[0040] The low-power ultrasonic transmitter module serves as the global coordinate system of the forward kinematics upper limb modeling, and transmits ultrasonic signals to the ultrasonic receiving device installed on the head;

[0041] The ultrasonic receiving clock synchronization device is fixed to the head by a strap, accurately records the propagation time of the ultrasonic wave, and realizes the accurate synchronization of the ultrasonic transmitter and receiver clocks;

[0042] The electrical connection line and the flexible printed circuit board are designed into a special shape to provide displacement compensation for joint bending, and connect the flexible bending strain sensor and the multi-mode signal processing module.

[0043] Furthermore, the sensory measurement subsystem is composed of a strip-shaped flexible piezoelectric film sensor, a circular pressure film sensor and a high-precision thermistor temperature sensor.

[0044] The strip-shaped flexible piezoelectric film sensor is placed in the glove from the base of the palm to the fingertips. It has three sensitive units corresponding to the fingertips, the palm and the base of the palm. It simultaneously obtains pressure information of multiple joints and identifies different gripping gestures. When the pressure value is not 0, it indicates contact. When the pressure value exceeds the threshold, it indicates pain.

[0045] The circular pressure film sensor is placed on the tip of the thumb to obtain the tactile and pain information of the thumb. When the pressure value is not 0, it indicates contact, and when the pressure value exceeds the threshold, it indicates pain.

[0046] The high-precision thermistor temperature sensor is placed on the fingertips to monitor the temperature changes of the fingertip contact environment at all times, providing key data for restoring the real tactile experience and in-depth analysis of the patient's perception feedback on temperature.

[0047] Furthermore, the tension sensor module obtains the rotation angle n of the shoulder joint and the elbow joint by: considering the upper arm as a bottom surface with a circumference of c upperarm , height is h upperarm When the shoulder joint rotates internally and externally, the elastic rope fixed on the strap at both ends will produce elastic strain x. According to Hu Ke's law, the elastic force F of the spring is proportional to the elongation or compression x of the spring. Its expression is F = kx, where k is the elastic coefficient. Then, given the elastic coefficient k and the tension F upperarm , then the elongation x=F upperarm / k. From this, we can get the base length of the triangle formed by the stretched elastic rope, its initial state and the bottom of the cylinder is The calculation formula of the internal and external rotation angle of the shoulder joint is obtained as follows:

[0048]

[0049] Similarly, the perimeter of the base is c forearm , height is h forearm The internal and external rotation angles of the forearm cylinder n forearm for:

[0050]

[0051] Furthermore, the low-power ultrasonic transmitter module (242) and the ultrasonic receiving clock synchronization device (243) are used to determine the coordinates (x0, y0, z0) of the sternoclavicular joint O0 relative to the ultrasonic receiving clock synchronization device (243) O1, assuming that the propagation speed of the ultrasonic wave in the current environment is v (which is calculated according to the ambient temperature T as v=331.4+0.6T), and the three ultrasonic transmitters A, B, and C at the sternoclavicular joint O0 form a triangle, and assuming that the flight times of the three points A, B, and C to the corresponding ultrasonic receiving clock synchronization device (243) fixed on the top of the head are t A ,t B ,t C , the position coordinates of the sternoclavicular joint O0 can be calculated based on the spatial geometric relationship:

[0052]

[0053] Where (X A ,y A , z A )、(x B ,y B , z B )、(x C ,y C , z C ) are the known coordinate positions of the three ultrasonic transmitters A, B, and C at the sternoclavicular joint in the global coordinate system. By solving the above equations, the coordinates (x0, y0, z0) of the sternoclavicular joint O0 relative to the ultrasonic receiving clock synchronization device (243) O1 on the top of the head can be obtained. Then, by the coordinate transformation of the two points, the coordinates of the ultrasonic receiving clock synchronization device (243) O1 on the top of the head relative to the sternoclavicular joint O0 in the global coordinate system can be obtained as (-x0, -y0, -z0).

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] 1. In view of the fact that the traditional upper limb motion measurement system has a single sensor type and limited data modality in data collection, the new multi-sensory upper limb motion measurement system constructed by the present invention integrates a variety of high-precision sensors such as tension sensing devices, strip-shaped flexible piezoelectric film sensors, high-precision thermistor temperature sensors, flexible bending strain sensors, ultrasonic transmitters and receivers, etc. These sensors work together to synchronously collect multi-modal data such as touch, temperature, angle, tension and position, thereby providing a rich and accurate data foundation for a comprehensive and in-depth analysis of the upper limb motion state, greatly making up for the shortcomings of traditional technologies in data comprehensiveness and accuracy.

[0056] 2. Taking into account the problems that the prior art often faces in the process of data processing and transmission, such as poor filtering effect, unstable transmission and difficulty in data integration, the data processing and Bluetooth transmission module of the present invention adopts a programmable filter chip, which can flexibly adjust the filtering parameters according to the characteristics of different sensor signals to ensure the purity of the signal. At the same time, the Bluetooth transmission unit has a signal strength detection function, which can dynamically adjust the transmission power or execute a retransmission strategy according to the signal condition to ensure stable data transmission. In addition, the wireless data concentrator can efficiently aggregate and integrate multi-source data, and use the WiFi module to achieve fast and reliable communication with the computer terminal, realizing smooth data processing and seamless transmission, and effectively improving the data processing efficiency and transmission reliability of the entire system.

[0057] 3. Aiming at the defect that the previous upper limb motion measurement system is difficult to construct an accurate and real-time updated kinematic model, the present invention builds a global coordinate system based on human anatomy and metrology knowledge, takes the sternoclavicular joint as the origin, uses a flexible bending strain sensor to measure the joint angle, and combines ultrasonic positioning technology to determine the key joint position, thereby establishing a complete upper limb forward kinematic model. This model not only covers the key parts from the sternoclavicular joint to the shoulder joint, elbow joint, wrist joint and finger joint, but also can be dynamically updated according to the real-time feedback data from the sensor, accurately restore the upper limb movement posture in space, and provide extremely accurate motion information support for application scenarios such as medical rehabilitation and virtual reality interaction, which has strongly promoted the technical development and application expansion in related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of the system structure of a preferred embodiment of the present invention;

[0059] Figure 2 is a schematic diagram of the upper limb movement calculation model of the present invention;

[0060] Figure 3 It is a schematic diagram of the structure of a motion measurement subsystem of a preferred embodiment of the present invention;

[0061] Figure 4 is a schematic diagram of a sensory measurement subsystem and an index finger cross section of a preferred embodiment of the present invention;

[0062] Figure 5 It is a schematic diagram of the upper limb joint internal and external rotation model of the present invention;

[0063] Among them: 10-soft felt clothing, 11-upper limb motion calculation model, 12-motion measurement subsystem, 13-sensory measurement subsystem, 14-multi-mode signal processing module, 15-data Bluetooth transmission module and wireless signal concentrator, 16-computer client; 240-tension sensing device, 241-single-axis flexible bending strain sensor, 242-ultrasonic low-power ultrasonic transmitter module, 243-ultrasonic receiving clock synchronization device, 244-flexible printed circuit board and electrical connection wires;

[0064] 300-strip flexible piezoelectric film sensor, 301-circular pressure film sensor, 302-high-precision thermistor temperature sensor. DETAILED DESCRIPTION

[0065] The following describes several preferred embodiments of the present invention with reference to the drawings in the specification, so that the technical content is clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned in the text.

[0066] In the drawings, components with the same structure are indicated by the same numerical reference numerals, and components with similar structures or functions are indicated by similar numerical reference numerals. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present invention does not limit the size and thickness of each component. In order to make the illustration clearer, the thickness of the components is appropriately exaggerated in some places in the drawings.

[0067] like Figure 1 As shown, the novel wearable upper limb-hand sensation and motion measurement system of this embodiment is composed of a soft felt garment 10, an upper limb motion calculation model 11, a motion measurement subsystem 12, a sensation measurement subsystem 13, a multi-mode signal processing module 14, a data Bluetooth transmission module 14, a wireless signal concentrator 15 and a computer client 16, wherein:

[0068] The soft felt garment 10 is used to integrate the motion measurement subsystem 12, the sensory measurement subsystem 13, the multi-mode signal processing module 14, the data Bluetooth transmission module 14 and the wireless signal concentrator 15 into one wearable device, which is convenient for wearing and fixing each module;

[0069] The upper limb motion calculation model 11 calculates the real-time position of the upper limb joints through motion multi-modal data, including the calculation of trigonometric functions, vector rotations, and coordinate transformations;

[0070] The motion measurement subsystem 12 collects the angle data of the adduction, abduction, extension, flexion, internal and external rotation of the upper limb joints during exercise, the position relationship of the head relative to the sternoclavicular joint, and is fixed at the corresponding acquisition position of the corresponding data with hook-surface Velcro, and collects the corresponding original electrical signal and outputs it to the multi-mode signal processing module 14;

[0071] The sensory measurement subsystem (13) collects hand temperature and pressure signals, obtains temperature sensation, touch sensation and pain sensation, and outputs original electrical signals;

[0072] The multi-mode signal processing module 14 receives the original electrical signals from the motion measurement subsystem 12 and the sensory measurement subsystem 13, performs simple filtering, amplification and feature extraction for specific joint motion data, and then transmits the processed multi-modal data such as touch, temperature, vibration, angle, tension and position to the wireless data concentrator 15 via Bluetooth.

[0073] After completing the Bluetooth data aggregation and integration, the wireless data concentrator 15 transmits all the data to the computer client 16 using the WiFi module;

[0074] The computer terminal 16 constructs a complete, accurate and real-time updated upper limb movement perception state information system for the data received via WIFI, which includes the upper limb movement calculation model 11 and the real-time state of multi-sensory information.

[0075] like Figure 2 As shown, the upper limb motion calculation model 11 includes the connection from the sternoclavicular joint O0 to the shoulder joint rotation center O2, the elbow joint rotation center O3, the wrist joint rotation center O4, the metacarpophalangeal joint, the proximal finger joint, the distal finger joint, and the fingertip in sequence. In addition, O1 is the origin of the local coordinate system where the ultrasonic receiving clock synchronization device 243 is located. The calculation process of the upper limb motion calculation model 11 is as follows:

[0076] The sternoclavicular joint is the origin of the global coordinate system. The sternoclavicular joint is the origin of the global coordinate system. It is stipulated that the forward direction is the positive direction of the x-axis (the front of the human body), the left direction is the positive direction of the y-axis (the left side of the human body), and the upward direction is the positive direction of the z-axis (vertical to the ground). The right-handed rectangular coordinate system is established. The local coordinate system directions of the remaining joints are the same;

[0077] Coordinate transformation of the shoulder joint rotation center O2 relative to the sternoclavicular joint O0, assuming the measured shoulder width is d shoulder , so half of the shoulder width is L1. According to anatomical knowledge, it is assumed to be in the x-axis direction of the global coordinate system and on the horizontal line. Then the coordinates of the shoulder joint rotation center relative to the sternoclavicular joint are (x2, y2, z2) = (0, L1, 0) (here, the slight vertical deviation caused by individual differences in the human body is ignored, taking the left upper limb as an example);

[0078] The coordinates of the elbow joint rotation center O3 relative to the shoulder joint rotation center are calculated, assuming that the length of the upper arm is L upperarm, the rotation angle of the upper arm around the shoulder joint rotation center O2 of the local coordinate system x2 axis is θ1, the rotation angle of the upper arm around the shoulder joint rotation center O2 of the local coordinate system z axis is θ2, then the coordinates of O3 in the global coordinate system are (x3, y3, z3):

[0079]

[0080] The wrist rotation center is O4, and the upper arm vector is Its length is L upperarm , the forearm vector is Its length is L forearm , ∠O2O3O4=α1, around the upper arm vector The rotation angle is α2, After elbow flexion The angle is α1, Expressed as (Δx, Δy, Δz), according to the relationship between vector projection and trigonometric function, we can get:

[0081]

[0082] Rewind The rotation α2 is calculated using the Rodrigues' rotation formula:

[0083] R=I+(sinα2)K+(1-sinα2)K 2

[0084] Where I is the identity matrix K is the axis of rotation The corresponding antisymmetric matrix is ​​of the form:

[0085]

[0086] Calculate K first 2 :

[0087]

[0088] Then substitute it into the Rodriguez rotation formula to calculate the rotation matrix R:

[0089]

[0090] Multiply the coordinates (Δx, Δy, Δz) at the initial position by the rotation matrix R to obtain the rotated coordinates (x4, y4, z4), that is:

[0091]

[0092] By expanding the matrix multiplication, we can get the specific expressions of x4, y4, z4;

[0093] The coordinate transformation from the wrist joint to the metacarpal joint is to obtain the internal and external rotation angles of the elbow joint through the tension sensor device, and the coordinates of the metacarpophalangeal joint with respect to the global coordinate system are calculated using the same method as the wrist joint coordinates;

[0094] The metacarpophalangeal joint, proximal interphalangeal joint, and distal interphalangeal joint only consider their flexion movement in one plane, ignoring the movement in other directions and the soft tissue deformation of the finger. The rotation angle of the interphalangeal joint is γ i , translated by d along x i distance, i is the knuckle number, and the homogeneous transformation matrix between adjacent knuckles is:

[0095]

[0096] Thus, the coordinates of each finger joint and fingertip relative to the global coordinate system are obtained.

[0097] Depend on Figure 3 As shown, the motion measurement subsystem 12 is composed of a variety of high-precision sensors, including a tension sensor 240, a uniaxial flexible bending strain sensor 241, an ultrasonic low-power ultrasonic transmitter module 242, an ultrasonic receiving device 243, a clock synchronization module 244, a flexible printed circuit board 245 and an electrical connection line 245, wherein:

[0098] The tension sensing device 240 is composed of a strap, a tension sensor, and an elastic rope, wherein the strap is fixed at the upper arm near the shoulder joint and elbow joint, and at the forearm near the elbow joint and wrist joint; the tension sensor and the elastic rope are connected to each other, and when the arm hangs naturally with the palm facing one side of the body, the two ends are vertically fixed on the strap;

[0099] The uniaxial flexible bending strain sensors 241 are placed on the shoulders, wrists and finger joints, closely fitting the limbs, and outputting the continuous changes in angles during joint movement, which are used as parameters for upper limb forward kinematics modeling to help construct real-time upper limb movement postures;

[0100] The low-power ultrasonic transmitter module 242 serves as the global coordinate system of the forward kinematic upper limb modeling, and transmits ultrasonic signals to the ultrasonic receiving device 243 installed on the head;

[0101] The ultrasonic receiving device 243 is fixed to the head by a strap to accurately record the propagation time of the ultrasonic wave. A head mounted display with this function can also be used as a signal receiving end.

[0102] The clock synchronization module 243 realizes accurate synchronization of the ultrasonic transmitter and receiver clocks through a specially designed synchronization signal transmission protocol;

[0103] The electrical connection line 244 and the flexible printed circuit board 244 are designed to be in a special shape. In this embodiment, they are designed to be S-shaped, so that when the finger is bent, displacement compensation of joint bending is provided to connect the flexible bending strain sensor 241 and the multi-mode signal processing module 14.

[0104] like Figure 4 As shown, the sensory measurement subsystem 13 is composed of a strip-shaped flexible piezoelectric film sensor 300, a circular pressure film sensor 301 and a high-precision thermistor temperature sensor 302, which collects the sensory information of the hand, wherein:

[0105] The strip-shaped flexible piezoelectric film sensor 300 is placed in the glove from the base of the palm to the fingertips. It has three sensitive units, corresponding to the fingertips, the palm and the base of the palm. It can obtain pressure information of multiple joints at the same time and identify different gripping gestures. When the pressure value is not 0, it means there is contact, and when the pressure value exceeds the threshold, it means there is pain.

[0106] The circular pressure film sensor 301 is placed on the tip of the thumb to obtain the tactile information of the thumb. When the pressure value is not 0, it indicates contact, and when the pressure value exceeds the threshold, it indicates pain.

[0107] The high-precision thermistor temperature sensor 302 is placed on the fingertips to monitor the temperature changes of the fingertip contact environment at all times, providing key data for restoring the real touch experience and in-depth analysis of the patient's perception feedback on temperature;

[0108] like Figure 5 As shown in the figure, the method to obtain the rotation angle n of the shoulder joint and the elbow joint is: regard the upper arm as a circle with a bottom circumference of c upperarm , height is h upperarm When the shoulder joint rotates internally and externally, the elastic ropes fixed on the straps at both ends will produce elastic strain x. According to Hu Ke's law, the elastic force F of the spring is proportional to the elongation or compression x of the spring. Its expression is F = kx, where k is the elastic coefficient. Then, given the elastic coefficient k and the tension F upperarm , then the elongation x=F upperarm / k. From this, we can get the base length of the triangle formed by the stretched elastic rope, its initial state and the bottom of the cylinder is The calculation formula of the internal and external rotation angle of the shoulder joint is obtained as follows:

[0109]

[0110] Similarly, the perimeter of the base is C forearm , height is h forearm The internal and external rotation angle of the forearm cylinder is 360n forearm for:

[0111]

[0112] like Figure 3 As shown, the low-power ultrasonic transmitter module 242 and the ultrasonic receiving clock synchronization device 243 are used to determine the coordinates (x0, y0, z0) of the sternoclavicular joint O0 relative to the ultrasonic receiving clock synchronization device 243O1. Assume that the propagation speed of the ultrasonic wave in the current environment is v (which is calculated according to the ambient temperature T as v=331.4+0.6T). The three ultrasonic transmitters A, B, and C at the sternoclavicular joint O0 form a triangle. Assume that the flight time from the three points A, B, and C to the corresponding ultrasonic receiving clock synchronization device 243 fixed on the top of the head is t respectively. A ,t B ,t C , the position coordinates of the sternoclavicular joint O0 can be calculated based on the spatial geometric relationship:

[0113]

[0114] Where (x A , Y A ,z A )、(x B ,y B , z B )、(x C ,y C , z C ) are the known coordinate positions of the three ultrasonic transmitters A, B, and C at the sternoclavicular joint in the global coordinate system. By solving the above equations, the coordinates (x0, y0, z0) of the sternoclavicular joint O0 relative to the ultrasonic receiving clock synchronization device 243O1 on the top of the head can be obtained. Then, the coordinates of the ultrasonic receiving clock synchronization device 243O1 on the top of the head relative to the sternoclavicular joint O0 in the global coordinate system can be obtained by the coordinate transformation of the two points as (-x0, -y0, -z0).

[0115] The above is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any technician in the relevant technical field, without departing from the scope of the technical solution of the present invention, makes any form of equivalent replacement or modification to the technical solution and technical content disclosed in the present invention, which does not depart from the content of the technical solution of the present invention and still falls within the protection scope of the present invention.

Claims

1. A novel multi-sensory upper limb movement measurement system, characterized in that: The invention comprises a soft felt garment (10) integrated with a multi-modal data acquisition system and a multi-modal signal processing and transmission module (14); a motion measurement subsystem (12) and a sensory measurement subsystem (13) respectively acquire motion and sensory multi-modal signals and transmit them to a multi-modal signal processing module (14) for amplification and filtering; a wireless signal concentrator (15) collects data processed by the multi-modal signal processing module (14) and sends the data to a computer client (16), and this process can be transmitted via WiFi; the computer client (16) integrates the multi-modal signals to construct a complete, accurate and real-time updated upper limb motion perception state information system, which comprises an upper limb motion calculation model (11) and a real-time state of multi-sensory information; the upper limb motion calculation model (11) is calculated through the multi-modal signals collected by the motion measurement subsystem (12) to form a motion model with real-time position coordinate data of each joint.

2. The novel multisensory upper limb movement measurement system according to claim 1, characterized in that: The soft felt garment (10) is used to integrate and fix the motion measurement subsystem (12), the sensory measurement subsystem (13), the multi-mode signal processing module (14), the data Bluetooth transmission module and the wireless signal concentrator (15) into a wearable device; The upper limb motion calculation model (11) calculates the real-time position of the upper limb joints through motion multi-mode data, including calculations of trigonometric functions, vector rotations, and coordinate transformations; The motion measurement subsystem (12) collects angle data of adduction, abduction, extension, flexion, internal and external rotation of the upper limb joint during exercise, the position of the head relative to the sternoclavicular joint, is fixed at a corresponding acquisition position for measuring corresponding data, and collects corresponding original electrical signals and outputs them to the multi-mode signal processing module (14); The sensory measurement subsystem (13) collects hand temperature and pressure signals, obtains temperature sensation, touch sensation and pain sensation, and outputs original electrical signals; The multi-mode signal processing module (14) receives the original electrical signals from the motion measurement subsystem (12) and the sensory measurement subsystem (13), filters and amplifies the original electrical signals, and extracts the features of the specific joint motion data, and then transmits the processed multi-mode data to the data Bluetooth transmission module and the wireless signal concentrator (15) respectively; After completing the Bluetooth data aggregation and integration, the data Bluetooth transmission module and the wireless signal concentrator (15) transmit all the data to the computer client (16); The computer terminal (16) constructs an upper limb movement perception state information system based on the received data, including an upper limb movement calculation model and a real-time state of multi-sensory information.

3. The novel multisensory upper limb movement measurement system according to claim 2, characterized in that: The upper limb motion calculation model (11) includes connections from the sternoclavicular joint O0 to the shoulder joint rotation center O2, the elbow joint rotation center O3, the wrist joint rotation center O4, the metacarpophalangeal joint, the proximal finger joint, the distal finger joint, and the fingertip in sequence, O1 is the origin of the local coordinate system where the ultrasonic receiving clock synchronization device (243) is located, and the calculation process of the upper limb motion calculation model (11) is as follows: The sternoclavicular joint O0 is the origin of the global coordinate system. It is stipulated that the positive direction of the x-axis is forward, the front of the human body, the positive direction of the y-axis is to the left, the left side of the human body, and the positive direction of the z-axis is upward. The right-handed rectangular coordinate system is established vertically to the ground, and the local coordinate system directions of the remaining joints are the same; The coordinate transformation of the shoulder joint rotation center O2 relative to the sternoclavicular joint O0, assuming that the measured shoulder width is d shoulder , so half of the shoulder width is L1. According to anatomical knowledge, it is assumed to be in the x-axis direction of the global coordinate system and on the horizontal line. Then the coordinates of the shoulder joint rotation center relative to the sternoclavicular joint are (x2, y2, z2) = (0, -L1, 0). Here, the slight vertical deviation caused by individual differences in the human body is ignored. Take the left upper limb as an example; The coordinates of the elbow joint rotation center O3 relative to the shoulder joint rotation center O2 are calculated, assuming that the arm length is L upperarm , the rotation angle of the upper arm around the shoulder joint rotation center O2 of the local coordinate system x2 axis is θ1, the rotation angle of the upper arm around the shoulder joint rotation center O2 of the local coordinate system z axis is θ2, then the coordinates of O3 in the global coordinate system are (x3, y3, z3): The wrist joint rotation center is O4, and the upper arm vector is Its length is L upperarm , the forearm vector is Its length is L forearm , ∠O2O3O4=α1, around the upper arm vector The rotation angle is α2, After elbow flexion The angle is α1, Expressed as (Δx, Δy, Δz), according to the relationship between vector projection and trigonometric function, we can get: Conduct a winding The rotation α2 of is calculated by the Rodrigues′ rotation formula: R=I+(sinα2)K+(1-sinα2)K 2 Where I is the identity matrix K is the axis of rotation The corresponding antisymmetric matrix is ​​of the form: Calculate K first 2 : Then substitute it into the Rodriguez rotation formula to calculate the rotation matrix R: Multiply the coordinates (Δx, Δy, Δz) at the initial position by the rotation matrix R to obtain the rotated coordinates (x4, y4, z4), that is: By expanding the matrix multiplication, we can get the specific expressions of x4, y4, z4; The coordinate transformation from the wrist joint to the metacarpal joint obtains the internal and external rotation angles of the elbow joint through the tension sensor (240), and the coordinates of the metacarpophalangeal joint with respect to the global coordinate system are calculated using the same method as that for calculating the wrist joint coordinates; The metacarpophalangeal joint, the proximal interphalangeal joint, and the distal interphalangeal joint only consider their flexion movement in one plane, ignoring the movement in other directions and the soft tissue deformation of the finger. Assume that the rotation angle of the interphalangeal joint is γi, and the translation along x is d i distance, i is the knuckle number, and the homogeneous transformation matrix between adjacent knuckles is: Thus, the coordinates of each finger joint and fingertip relative to the global coordinate system are obtained.

4. The novel multisensory upper limb movement measurement system according to claim 2, characterized in that: The motion measurement subsystem (12) is collaboratively composed of a variety of high-precision sensors, including a tension sensor, a uniaxial flexible bending strain sensor, an ultrasonic low-power ultrasonic transmitter module, an ultrasonic receiving device, a clock synchronization module, a flexible printed circuit board and electrical connecting wires, wherein: The tension sensing device (240) is composed of a strap, a tension sensor, and an elastic rope, wherein the strap is respectively fixed at the upper arm near the shoulder joint and elbow joint, and at the forearm near the elbow joint and wrist joint; the tension sensor and the elastic rope are connected to each other, and when the arm hangs naturally with the palm facing one side of the body, the two ends are vertically fixed on the strap; The uniaxial flexible bending strain sensor (241) is placed on the shoulder, wrist and finger joints, closely fits the limbs, and outputs the continuous change of the angle during joint movement, which is used as a parameter for upper limb forward kinematics modeling to help construct real-time upper limb movement posture; The low-power ultrasonic transmitter module (242) serves as a global coordinate system for forward kinematic upper limb modeling, transmitting ultrasonic signals to an ultrasonic receiving device installed on the head; The ultrasonic receiving clock synchronization device (243) is fixed on the head, accurately records the propagation time of the ultrasonic wave, and realizes the accurate synchronization of the ultrasonic transmitter and receiver clocks; The electrical connection line (244) and the flexible printed circuit board (244) are designed to be in a special shape to provide displacement compensation for joint bending, and connect the uniaxial flexible bending strain sensor (241) and the multi-mode signal processing module (14).

5. The novel multisensory upper limb movement measurement system according to claim 2, characterized in that: The sensory measurement subsystem (13) is composed of a strip-shaped flexible piezoelectric film sensor (300), a circular pressure film sensor (301), and a high-precision thermistor temperature sensor (302); The strip-shaped flexible piezoelectric film sensor (300) is placed in the glove from the base of the palm to the fingertips, and has three sensitive units corresponding to the finger pulp, the palm and the base of the palm, and simultaneously obtains pressure information of multiple joints to identify different grasping gestures; The circular pressure film sensor (301) is placed on the tip of the thumb to obtain tactile information of the thumb; The high-precision thermistor temperature sensor (302) is placed on the fingertips to constantly monitor the temperature changes of the fingertip contact environment, providing key data for restoring the real tactile experience and in-depth analysis of the patient's perceived feedback on temperature.

6. According to the novel multi-sensory upper limb motion measurement system of claim 2, the method of the tension sensor module (240) to obtain the rotation angle n of the shoulder joint and the elbow joint is: regard the upper arm as a circle with a bottom circumference of c upperarm , height is h upperarm When the shoulder joint rotates internally and externally, the elastic ropes fixed on the straps at both ends will produce elastic strain x. According to Hu Ke's law, the elastic force F of the spring is proportional to the elongation or compression x of the spring. Its expression is F = kx, where k is the elastic coefficient. Then, given the elastic coefficient k and the tension F upperarm , then the elongation x=F upperarm / k, from which we can get the base length of the triangle formed by the stretched elastic rope, its initial state and the bottom of the cylinder is The calculation formula of the internal and external rotation angle of the shoulder joint is obtained as follows: Similarly, the perimeter of the base is c forearm , height is h forearm The internal and external rotation angles of the forearm cylinder n forearm for:

7. The novel multisensory upper limb movement measurement system according to claim 2, characterized in that: The low-power ultrasonic transmitter module (242) and the ultrasonic receiving clock synchronization device (243) are used to determine the coordinates (x0, y0, z0) of the sternoclavicular joint O0 relative to the ultrasonic receiving clock synchronization device (243) O1. Assuming that the propagation speed of the ultrasonic wave in the current environment is v (which is calculated according to the ambient temperature T as v=331.4+0.6T), the three ultrasonic transmitters A, B, and C at the sternoclavicular joint O0 form a triangle. Assuming that the flight time from the three points A, B, and C to the corresponding ultrasonic receiving clock synchronization device (243) fixed on the top of the head is t respectively. A ,t B ,t C , the position coordinates of the sternoclavicular joint O0 can be calculated based on the spatial geometric relationship: Where (X A ,y A , z A )、(x B ,y B , z B )、(x C ,y C , z C ) are the known coordinate positions of the three ultrasonic transmitters A, B, and C at the sternoclavicular joint in the global coordinate system. By solving the above equations, the coordinates (x0, y0, z0) of the sternoclavicular joint O0 relative to the ultrasonic receiving clock synchronization device (243) O1 on the top of the head can be obtained. Then, by the coordinate transformation of the two points, the coordinates of the ultrasonic receiving clock synchronization device (243) O1 on the top of the head relative to the sternoclavicular joint O0 in the global coordinate system can be obtained as (-x0, -y0, -z0).

Images