A multi-sensory upper limb motor measurement system

Abstract

This invention discloses a novel multi-sensory upper limb motion measurement system, comprising a soft felt garment (10), a motion measurement subsystem (12), and a sensory measurement subsystem (13) that respectively acquire multi-modal signals of motion and perception and transmit them to a multi-modal signal processing module (14) for amplification and filtering; a wireless signal concentrator (15) collects data from the multi-modal signal processing module (14); a computer client (16) integrates the multi-modal signals to construct an upper limb motion perception state information system, including an upper limb motion calculation model (11) and real-time multi-sensory information; the upper limb motion calculation model (11) calculates the data signals to form a motion model with real-time position coordinate data of each joint. This invention provides simple, accurate, and comprehensive real-time support for upper limb motion monitoring, which can effectively improve the accuracy and comprehensiveness of upper limb motion data acquisition and analysis in fields such as virtual reality interaction.

Description

Technical Field

[0001] This invention relates to the field of medical rehabilitation equipment technology, specifically a multi-sensory upper limb motion measurement system. Background Technology

[0002] In the field of rehabilitation medicine, the rehabilitation treatment of stroke patients has always been a key focus and a challenge. With advancements in medical technology, rehabilitation training methods have continuously evolved, but many challenges remain. Traditional rehabilitation training often emphasizes simple limb movement exercises, such as relying on therapists to manually assist patients in joint movements or using simple mechanical devices to guide limb movements. This approach lacks precise monitoring and quantitative assessment of the patient's overall condition during training, making it difficult to gain a deep understanding of the patient's neuromuscular control and subtle changes in rehabilitation progress.

[0003] In recent years, virtual reality (VR) technology has been increasingly incorporated into multisensory training, aiming to provide patients with a richer and more immersive training environment. However, existing VR multisensory training devices have significant shortcomings in integrating with patients' actual movement data. On the one hand, when acquiring upper limb posture information, most devices employ methods that are not precise or comprehensive enough. For example, the positional information acquired by the IMU (Integrated Mutual Acupuncture Unit) can accumulate errors, failing to accurately reflect the complex movements of the upper limbs in three-dimensional space, including subtle angular changes in various joints and the overall spatial position and posture changes of the limbs. This results in poor synchronization between the virtual upper limb and the patient's real upper limb in the VR scene, easily leading to delays and deviations, severely impacting the patient's depth perception and training effectiveness.

[0004] On the other hand, there are also shortcomings in multi-sensory data acquisition. The acquisition of sensory information such as touch and temperature in the hands is not perfect, failing to fully simulate sensory stimulation in the real environment, making it difficult for patients to obtain a realistic interactive experience during VR training. For example, when grasping virtual objects, it is impossible to accurately perceive the texture, weight, and temperature of the objects, which is extremely detrimental to the patient's motor relearning and neural function remodeling, and may even lead to the development of maladaptive movement patterns, limiting the potential for functional recovery.

[0005] Meanwhile, with the increasing demand for personalized treatment plans in rehabilitation medicine, existing rehabilitation equipment is insufficient to meet the requirements of precise rehabilitation training based on individual patient differences. The lack of in-depth analysis and effective utilization of multi-dimensional patient data prevents the development of optimized training strategies tailored to each patient's unique rehabilitation condition, thus reducing the efficiency and success rate of rehabilitation treatment. In conclusion, developing a data glove capable of accurately acquiring upper limb posture information and achieving multi-sensory data acquisition based on positive kinematic modeling methods is of paramount importance. It holds the potential to fill gaps in existing rehabilitation technologies, providing more efficient, precise, and personalized rehabilitation training solutions for stroke patients and others requiring upper limb rehabilitation, and promoting the further development of rehabilitation medical technology. Summary of the Invention

[0006] Purpose of the invention: The purpose of this invention is to provide a multi-sensory upper limb motor measurement system to address the shortcomings in the integration of existing limb motor and sensory training with VR training. This results in a lack of multi-sensory stimulation and inaccurate motor feedback for patients, leading to difficulties in depth perception, the formation of bad movement habits, and limitations on functional recovery. Through innovative technical means, this invention achieves deep interaction and precise synergy between the two, improving the effectiveness and personalization of rehabilitation training, and helping patients better recover limb function and neurosensory abilities.

[0007] Technical Solution: The multi-sensory upper limb motion measurement system of the present invention includes a soft felt garment integrated with a multimodal data acquisition system and a multimodal signal processing and transmission module; the motion measurement subsystem and the sensory measurement subsystem respectively acquire multimodal signals of motion and perception and transmit them to the multimodal signal processing module for amplification and filtering; a wireless signal concentrator collects the data processed by the multimodal signal processing module and sends it to a computer client, which can be transmitted via WiFi; the computer client integrates the multimodal signals to construct a complete, accurate and real-time updated upper limb motion perception state information system, which includes an upper limb motion calculation model and real-time status of multisensory information; the upper limb motion calculation model is calculated by the multimodal signals acquired by the motion measurement subsystem to form a motion model with real-time position coordinate data of each joint.

[0008] Furthermore, the system uses soft felt clothing to integrate and fix the motion measurement subsystem, sensory measurement subsystem, multimode signal processing module, data Bluetooth transmission module, and wireless signal concentrator into a wearable device;

[0009] The upper limb motion calculation model calculates the real-time position of the upper limb joints using multimodal motion data, including calculations of trigonometric functions, vector rotation, and coordinate transformation.

[0010] The motion measurement subsystem collects the angle data of adduction, abduction, extension, flexion and internal / external rotation of the upper limb joints during movement, as well as the positional relationship of the head relative to the sternoclavicular joint. It is fixed at the corresponding acquisition position for measuring the corresponding data and collects the corresponding raw electrical signals and outputs them to the multimode signal processing module.

[0011] The sensory measurement subsystem collects hand temperature and pressure signals, acquires temperature, touch, and pain sensations, and outputs raw electrical signals.

[0012] The multimodal signal processing module receives raw electrical signals from the motion measurement subsystem and the sensory measurement subsystem, filters and amplifies them, extracts features from specific joint motion data, and then transmits the processed multimodal data to the data Bluetooth transmission module and the wireless signal concentrator, respectively.

[0013] After completing the Bluetooth data aggregation and integration, the data Bluetooth transmission module and the wireless signal concentrator transmit all data to the computer client.

[0014] The computer terminal constructs an upper limb motion perception state information system based on the received data, including an upper limb motion calculation model and real-time status of multi-sensory information.

[0015] Furthermore, the upper limb motion calculation model includes sequential connections from the sternoclavicular joint O0 to the shoulder joint rotation center O2, the elbow joint rotation center O3, the wrist joint rotation center O4, the metacarpophalangeal joints, the proximal interphalangeal joints, the distal interphalangeal joints, and the fingertips. Additionally, O1 is the origin of the local coordinate system at the location of the ultrasonic receiving device. The calculation process of the upper limb motion calculation model is as follows:

[0016] The sternoclavicular joint is the origin of the global coordinate system. The sternoclavicular joint is the origin of the global coordinate system. The positive x-axis is defined as forward (directly in front of the human body), the positive y-axis is defined as left (left side of the human body), and the positive z-axis is defined as upward (vertically upward from the ground). A right-handed rectangular coordinate system is established. The local coordinate system of the other joints has the same direction.

[0017] The coordinate transformation of the shoulder joint rotation center O2 relative to the sternoclavicular joint O0, assuming the measured shoulder width is d. shoulder Therefore, half of the shoulder width is L1. According to anatomical knowledge, let it be in the x-axis direction of the global coordinate system and on the horizontal line. Then the coordinates of the shoulder joint rotation center relative to the sternoclavicular joint are (x2, y2, z2) = (0, L1, 0). (Here, we ignore the slight vertical offset caused by individual differences in the human body and take the left upper limb as an example).

[0018] The coordinates of the elbow joint rotation center O3 relative to the shoulder joint rotation center are calculated, assuming the upper arm length is L. upperarmLet θ1 be the rotation angle of the upper arm around the shoulder joint rotation center O2 on the x2 axis of the local coordinate system, and θ2 be the rotation angle of the upper arm around the shoulder joint rotation center O2 on the z axis of the local coordinate system. Then, the coordinates of O3 in the global coordinate system are x3, y3, z3.

[0019]

[0020] The wrist joint rotation center is O4, and the upper arm vector is... Its length is L upperarm Forearm vector is Its length is L forearm , ∠O2O3O4=α1, around the upper arm vector The angle of rotation is α2. After elbow flexion and The included angle is α1. Represented as (Δx, Δy, Δz), it can be obtained from vector projection and trigonometric function relationships:

[0021]

[0022] Perform circling The rotation α2 is calculated using Rodrigues' rotation formula:

[0023] R = I + (sinα²)K + (1 - sinα²)K 2

[0024] Where I is the identity matrix K is related to the axis of rotation. The corresponding antisymmetric matrix has the following form:

[0025]

[0026] First calculate K 2 :

[0027]

[0028] Then substitute it into the Rodriguez rotation formula to calculate the rotation matrix R:

[0029]

[0030] Multiplying the initial coordinates (Δx, Δy, Δz) by the rotation matrix R yields the rotated coordinates (x4, y4, z4), i.e.:

[0031]

[0032] The specific expressions for x4, y4, and z4 can be obtained by matrix multiplication expansion;

[0033] The coordinate transformation from the wrist joint to the metacarpophalangeal joint is obtained by acquiring the internal and external rotation angles of the elbow joint through a tension sensing device, and the coordinates of the metacarpophalangeal joint relative to the global coordinate system are calculated using the same method as the calculation of the wrist joint coordinates.

[0034] The metacarpophalangeal joint, the proximal interphalangeal joint, and the distal interphalangeal joint only consider their flexion movement within one plane, ignoring movements in other directions and factors such as soft tissue deformation of the fingers. Let the rotation angle of the interphalangeal joint be γ. i It was translated by d along x. i Distance, where i is the phalanx number, and the homogeneous transformation matrix between adjacent phalanges is:

[0035]

[0036] This allows us to obtain the coordinates of each finger joint and fingertip relative to the global coordinate system.

[0037] Furthermore, the motion measurement subsystem is composed of multiple high-precision sensors working together, including a tension sensing device, a uniaxial flexible bending strain sensor, an ultrasonic low-power ultrasonic transmitter module, an ultrasonic receiver, a clock synchronization module, a flexible printed circuit board, and electrical connection wires.

[0038] The tension sensing device consists of a strap, a tension sensor, and an elastic cord. The strap is fixed at the upper arm near the shoulder and elbow joints, and at the forearm near the elbow and wrist joints. The tension sensor and the elastic cord are connected to each other. When the arm hangs naturally with the palm facing one side of the body, both ends are fixed vertically to the strap.

[0039] The uniaxial flexible bending strain sensor is placed in the shoulder, wrist and finger joints, closely fitting the limbs, and outputs the continuous change of angle during joint movement, which serves as a parameter for positive kinematic modeling of the upper limbs, helping to construct real-time upper limb movement postures.

[0040] The low-power ultrasonic transmitter module serves as the global coordinate system for modeling the positive kinematics of the upper limb and transmits ultrasonic signals to the ultrasonic receiver device mounted on the head.

[0041] The ultrasonic receiving device is fixed to the head with a strap, accurately records the propagation time of the ultrasonic waves, and achieves precise synchronization between the clocks of the ultrasonic transmitter and receiver.

[0042] The electrical connection lines and the flexible printed circuit board are designed with a special shape to provide displacement compensation for joint bending, connecting the flexible bending strain sensor and the multimode signal processing module.

[0043] Furthermore, the sensory measurement subsystem consists of a strip-shaped flexible piezoelectric thin film sensor, a circular pressure thin film sensor, and a high-precision thermistor temperature sensor.

[0044] The strip-shaped flexible piezoelectric film sensor is placed on the glove from the base of the palm towards the fingertips. It has three sensitive units corresponding to the fingertips, palm, and base of the palm. It can simultaneously acquire pressure information from multiple joints, identify different gripping gestures, and indicate contact when the pressure value is not 0. If the pressure value exceeds the threshold, it indicates pain.

[0045] The circular pressure film sensor is placed on the tip of the thumb to acquire tactile and pain information of the thumb. When the pressure value is not 0, it indicates that there is contact. When the pressure value exceeds the threshold, it indicates that there is pain.

[0046] The high-precision thermistor temperature sensor is placed on the fingertip to monitor the temperature changes of the environment in which the fingertip touches the ground, providing key data for restoring a realistic tactile experience and for in-depth analysis of patients' temperature perception feedback.

[0047] Furthermore, the method by which the tension sensing module obtains the rotation angle n of the shoulder and elbow joints is: to consider the upper arm as a circle with a base circumference of c. upperarm The height is h upperarm Imagine a cylinder. When the shoulder joint rotates internally or externally, the elastic ropes fixed to the straps at both ends will experience elastic strain *x*. According to Huke's law, the spring force *F* is directly proportional to the spring's elongation or compression *x*, expressed as *F = kx*, where *k* is the elastic coefficient. Given the elastic coefficient *k* and the tension *F*... upperarm Then the elongation x = F upperarm / k. Therefore, the length of the base of the triangle formed by the stretched elastic rope, its initial state, and the base of the cylinder is... The formula for calculating the internal and external rotation angles of the shoulder joint is as follows:

[0048]

[0049] Similarly, the perimeter of the base is c. forearm The height is h forearm The internal and external rotation angle n of the forearm cylinder forearm for:

[0050]

[0051] Furthermore, the low-power ultrasonic transmitter module and the ultrasonic receiver are used to determine the coordinates (x0, y0, z0) of the sternoclavicular joint O0 relative to the ultrasonic receiver O1. Let the speed of ultrasonic wave propagation in the current environment be v (calculated based on the ambient temperature T according to v = 331.4 + 0.6T). The three ultrasonic transmitters A, B, and C at the sternoclavicular joint O0 form a triangle. Let the flight times from points A, B, and C to the corresponding ultrasonic receiver fixed above the head be t. A t B t C Then, the coordinates of the sternoclavicular joint O0 can be calculated based on spatial geometric relationships:

[0052]

[0053] Where (x) A ,y A ,z A ), (x B ,y B ,z B ), (x C ,y C ,z C Let O0 be the known coordinates of the three ultrasonic transmitters A, B, and C at the sternoclavicular joint in the global coordinate system. By solving the above system of equations, we can obtain the coordinates (x0, y0, z0) of the sternoclavicular joint O0 relative to the ultrasonic receiving device O1 above the head. Then, by transforming the coordinates of the two points, we can obtain the coordinates (-x0, -y0, -z0) of the ultrasonic receiving device O1 above the head relative to the sternoclavicular joint O0 in the global coordinate system.

[0054] Compared with the prior art, the present invention has the following advantages:

[0055] 1. Addressing the limitations of traditional upper limb motion measurement systems in terms of sensor type and data modality, this invention constructs a multi-sensory upper limb motion measurement system that integrates multiple high-precision sensors, including a tension sensor, a strip-shaped flexible piezoelectric film sensor, a high-precision thermistor temperature sensor, a flexible bending strain sensor, and an ultrasonic transmitter and receiver. These sensors work collaboratively to simultaneously acquire multi-modal data such as touch, temperature, angle, tension, and position, providing a rich and accurate data foundation for comprehensive and in-depth analysis of upper limb movement states, significantly compensating for the shortcomings of traditional technologies in terms of data comprehensiveness and accuracy.

[0056] 2. Considering the problems often encountered in existing technologies during data processing and transmission, such as poor filtering effects, unstable transmission, and difficulties in data integration, the data processing and Bluetooth transmission module of this invention adopts a programmable filter chip, which can flexibly adjust filtering parameters according to the characteristics of different sensor signals to ensure signal purity. Simultaneously, the Bluetooth transmission unit has a signal strength detection function, which can dynamically adjust the transmission power or execute retransmission strategies according to signal conditions to ensure stable data transmission. Furthermore, the wireless data concentrator can efficiently aggregate and integrate multi-source data and achieve fast and reliable communication with the computer terminal via the WiFi module, realizing smooth data processing and seamless transmission, effectively improving the data processing efficiency and transmission reliability of the entire system.

[0057] 3. Addressing the shortcomings of previous upper limb motion measurement systems in constructing accurate and real-time updated kinematic models, this invention, based on human anatomy and metrology, establishes a global coordinate system with the sternoclavicular joint as the origin. It utilizes flexible bending strain sensors to measure joint angles and combines this with ultrasonic positioning technology to determine the positions of key joints, thereby establishing a complete upper limb forward kinematic model. This model not only covers all key areas from the sternoclavicular joint to the shoulder, elbow, wrist, and even finger joints, but also dynamically updates based on real-time sensor feedback data, accurately reproducing the upper limb's movement posture in space. This provides extremely precise motion information support for applications such as medical rehabilitation and virtual reality interaction, powerfully promoting technological development and application expansion in related fields. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the system structure of a preferred embodiment of the present invention;

[0059] Figure 2 This is a schematic diagram of the upper limb movement calculation model of the present invention;

[0060] Figure 3 This is a schematic diagram of the motion measurement subsystem structure according to a preferred embodiment of the present invention;

[0061] Figure 4 This is a preferred embodiment of the sensory measurement subsystem and a cross-sectional view of the index finger of the present invention;

[0062] Figure 5 This is a schematic diagram of the upper limb joint internal and external rotation model of the present invention;

[0063] Among them: 10-Soft felt clothing, 11-Upper limb motion calculation model, 12-Motion measurement subsystem, 13-Sensory measurement subsystem, 14-Multi-mode signal processing module, 15-Data Bluetooth transmission module and wireless signal concentrator, 16-Computer client;

[0064] 240 - Tensile sensing device; 241 - Uniaxial flexible bending strain sensor; 242 - Ultrasonic low-power ultrasonic transmitter module; 243 - Ultrasonic receiving device; 244 - Flexible printed circuit board and electrical connection wire; 300 - Strip-shaped flexible piezoelectric film sensor; 301 - Circular pressure film sensor; 302 - High-precision thermistor temperature sensor. Detailed Implementation

[0065] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0066] In the accompanying drawings, components with the same structure are indicated by the same numerical designation, and components with similar structures or functions are indicated by similar numerical designations. The dimensions and thicknesses of each component shown in the drawings are arbitrary, and the present invention does not limit the dimensions and thicknesses of each component. To make the illustrations clearer, the thickness of some components has been appropriately exaggerated in the drawings.

[0067] like Figure 1 As shown, the wearable upper limb-hand sensory and motor measurement system of this embodiment consists of a soft felt garment 10, an upper limb motor calculation model 11, a motor measurement subsystem 12, a sensory measurement subsystem 13, a multi-mode signal processing module 14, a Bluetooth data transmission module 15, a wireless signal concentrator 15, and a computer client 16.

[0068] The soft felt garment 10 is used to integrate the motion measurement subsystem 12, the sensory measurement subsystem 13, the multimode signal processing module 14, the data Bluetooth transmission module 14, and the wireless signal concentrator 15 into a wearable device, which is convenient for wearing and fixing the various modules.

[0069] The upper limb motion calculation model 11 calculates the real-time position of the upper limb joints through multi-modal motion data, including the calculation of trigonometric functions, vector rotation, and coordinate transformation.

[0070] The motion measurement subsystem 12 collects the angle data of adduction, abduction, extension, flexion and internal / external rotation of the upper limb joints during movement, as well as the positional relationship of the head relative to the sternoclavicular joint. It is fixed to the corresponding acquisition position for measuring the corresponding data using hook and loop fasteners, and the corresponding raw electrical signals are collected and output to the multimode signal processing module 14.

[0071] The sensory measurement subsystem (13) collects hand temperature and pressure signals, acquires temperature, touch and pain sensations, and outputs raw electrical signals;

[0072] The multi-mode signal processing module 14 receives the raw electrical signals from the motion measurement subsystem 12 and the sensory measurement subsystem 13, performs simple filtering, amplification and feature extraction for specific joint motion data, and then transmits the processed multi-modal data such as touch, temperature, vibration, angle, tension and position to the wireless data concentrator 15 via Bluetooth.

[0073] After completing the Bluetooth data aggregation and integration, the wireless data concentrator 15 uses the WiFi module to transmit all the data to the computer client 16.

[0074] The computer terminal 16 constructs a complete, accurate and real-time updated upper limb motion perception state information system based on the data received via WIFI, which includes the upper limb motion calculation model 11 and the real-time status of multi-sensory information.

[0075] like Figure 2 As shown, the upper limb motion calculation model 11 includes the following components connected sequentially from the sternoclavicular joint O0 to the shoulder joint rotation center O2, the elbow joint rotation center O3, the wrist joint rotation center O4, the metacarpophalangeal joints, the proximal interphalangeal joints, the distal interphalangeal joints, and the fingertips. Additionally, O1 is the origin of the local coordinate system where the ultrasonic receiving device is located. The calculation process of the upper limb motion calculation model 11 is as follows:

[0076] The sternoclavicular joint is the origin of the global coordinate system. The sternoclavicular joint is the origin of the global coordinate system. The positive x-axis is defined as forward (directly in front of the human body), the positive y-axis is defined as left (left side of the human body), and the positive z-axis is defined as upward (vertically upward from the ground). A right-handed rectangular coordinate system is established. The local coordinate systems of the other joints are in the same direction.

[0077] Coordinate transformation of the shoulder joint rotation center O2 relative to the sternoclavicular joint O0, assuming the measured shoulder width is d. shoulder Therefore, half of the shoulder width is L1. According to anatomical knowledge, let it be in the x-axis direction of the global coordinate system and on the horizontal line. Then the coordinates of the shoulder joint rotation center relative to the sternoclavicular joint are (x2, y2, z2) = (0, L1, 0) (here we ignore the slight vertical offset caused by individual differences in the human body, taking the left upper limb as an example).

[0078] Calculation of the coordinates of the elbow joint rotation center O3 relative to the shoulder joint rotation center, assuming the upper arm length is L. upperarm Let θ1 be the rotation angle of the upper arm around the shoulder joint rotation center O2 on the x2 axis of the local coordinate system, and θ2 be the rotation angle of the upper arm around the shoulder joint rotation center O2 on the z axis of the local coordinate system. Then, the coordinates of O3 in the global coordinate system are x3, y3, z3.

[0079]

[0080] The wrist joint rotation center is O4, and the upper arm vector is... Its length is L upperarm Forearm vector is Its length is L forearm , ∠O2O3O4=α1, around the upper arm vector The angle of rotation is α2. After elbow flexion and The included angle is α1. Represented as (Δx, Δy, Δz), it can be obtained from vector projection and trigonometric function relationships:

[0081]

[0082] Go around again Rotation α2 is calculated using Rodrigues' rotation formula:

[0083] R = I + (sinα²)K + (1 - sinα²)K 2

[0084] Where I is the identity matrix K is related to the axis of rotation. The corresponding antisymmetric matrix has the following form:

[0085]

[0086] First calculate K 2 :

[0087]

[0088] Then substitute it into the Rodriguez rotation formula to calculate the rotation matrix R:

[0089]

[0090] Multiplying the initial coordinates (Δx, Δy, Δz) by the rotation matrix R yields the rotated coordinates (x4, y4, z4), i.e.:

[0091]

[0092] The specific expressions for x4, y4, and z4 can be obtained by matrix multiplication expansion;

[0093] The coordinate transformation from the wrist joint to the metacarpophalangeal joint is achieved by obtaining the internal and external rotation angles of the elbow joint through a tension sensor, and then calculating the coordinates of the metacarpophalangeal joints relative to the global coordinate system using the same method as the wrist joint coordinates.

[0094] The metacarpophalangeal joints, proximal interphalangeal joints, and distal interphalangeal joints are considered only for flexion within a single plane, neglecting movements in other directions and soft tissue deformation of the fingers. Let the rotation angle of the interphalangeal joints be γ. i It was translated by d along x. i Distance, where i is the phalanx number, and the homogeneous transformation matrix between adjacent phalanges is:

[0095]

[0096] This allows us to obtain the coordinates of each finger joint and fingertip relative to the global coordinate system.

[0097] Depend on Figure 3 As shown, the motion measurement subsystem 12 is composed of multiple high-precision sensors, including a tension sensing device 240, a uniaxial flexible bending strain sensor 241, an ultrasonic low-power ultrasonic transmitter module 242, an ultrasonic receiver 243, a clock synchronization module, a flexible printed circuit board, and electrical connection lines 244.

[0098] The tension sensing device 240 consists of a strap, a tension sensor, and an elastic cord. The strap is fixed at the upper arm near the shoulder and elbow joints, and at the forearm near the elbow and wrist joints. The tension sensor and the elastic cord are connected to each other. When the arm hangs naturally with the palm facing one side of the body, the two ends are fixed vertically to the strap.

[0099] The uniaxial flexible bending strain sensor 241 is placed in the shoulder, wrist and finger joints, closely fitting the limbs, and outputs the continuous change of angle during joint movement. This serves as a parameter for positive kinematic modeling of the upper limbs, helping to construct real-time upper limb movement postures.

[0100] The low-power ultrasonic transmitter module 242 serves as the global coordinate system for the positive kinematic upper limb modeling and transmits ultrasonic signals to the ultrasonic receiver 243 mounted on the head.

[0101] The ultrasonic receiving device 243 is fixed to the head by a strap to accurately record the propagation time of ultrasonic waves. Alternatively, a head-mounted display with this function can be used as the signal receiving end.

[0102] The clock synchronization module achieves precise synchronization between the clocks of the ultrasonic transmitter and receiver through a specially designed synchronization signal transmission protocol;

[0103] The flexible printed circuit board and electrical connection line 244 are designed in a special shape, in this embodiment, an S-shape, so as to provide displacement compensation for joint bending when the finger is bent, and connect the flexible bending strain sensor 241 to the multimode signal processing module 14.

[0104] like Figure 4As shown, the sensory measurement subsystem 13 consists of a strip-shaped flexible piezoelectric thin-film sensor 300, a circular pressure thin-film sensor 301, and a high-precision thermistor temperature sensor 302, which collects sensory information from the hand, wherein:

[0105] A strip-shaped flexible piezoelectric film sensor 300 is placed on the glove from the base of the palm towards the fingertips. It has three sensitive units corresponding to the fingertips, palm, and base of the palm. It can simultaneously acquire pressure information from multiple joints, identify different gripping gestures, and indicate that there is contact when the pressure value is not 0. If the pressure value exceeds the threshold, it indicates that there is pain.

[0106] A circular pressure film sensor 301 is placed on the tip of the thumb to acquire tactile information of the thumb. When the pressure value is not 0, it indicates that there is contact. When the pressure value exceeds the threshold, it indicates that there is pain.

[0107] The high-precision thermistor temperature sensor 302 is placed on the fingertip to monitor the temperature changes of the environment in which the fingertip is in contact at all times, providing key data for restoring the real tactile experience and for in-depth analysis of the patient's temperature perception feedback.

[0108] like Figure 5 As shown, the method for obtaining the rotation angle n of the shoulder and elbow joints is: consider the upper arm as a circle with a base circumference of c. upperarm The height is h upperarm Imagine a cylinder. When the shoulder joint rotates internally or externally, the elastic ropes fixed to the straps at both ends will experience elastic strain *x*. According to Huke's law, the spring force *F* is directly proportional to the spring's elongation or compression *x*, expressed as *F = kx*, where *k* is the elastic coefficient. Given the elastic coefficient *k* and the tension *F*... upperarm Then the elongation x = F upperarm / k. Therefore, the length of the base of the triangle formed by the stretched elastic rope, its initial state, and the base of the cylinder is... The formula for calculating the internal and external rotation angles of the shoulder joint is as follows:

[0109]

[0110] Similarly, the perimeter of the base is c. forearm The height is h forearm The internal and external rotation angles of the forearm cylinder are 360°. forearm for:

[0111]

[0112] like Figure 3As shown, the low-power ultrasonic transmitter module 242 and ultrasonic receiver 243 are used to determine the coordinates (x0, y0, z0) of the sternoclavicular joint O0 relative to the ultrasonic receiver 243O1. Let the speed of ultrasonic wave propagation in the current environment be v (calculated based on the ambient temperature T according to v = 331.4 + 0.6T). The three ultrasonic transmitters A, B, and C at the sternoclavicular joint O0 form a triangle. Let the flight times from points A, B, and C to the corresponding ultrasonic receiver 243 fixed above the head be t, respectively. A t B t C Then, the coordinates of the sternoclavicular joint O0 can be calculated based on spatial geometric relationships:

[0113]

[0114] Where (x) A ,y A ,z A ), (x B ,y B ,z B ), (x C ,y C ,z C Let O0 be the known coordinates of the three ultrasonic transmitters A, B, and C at the sternoclavicular joint in the global coordinate system. By solving the above system of equations, we can obtain the coordinates (x0, y0, z0) of the sternoclavicular joint O0 relative to the ultrasonic receiver 243O1 on the top of the head. Then, by transforming the coordinates of the two points, we can obtain the coordinates (-x0, -y0, -z0) of the ultrasonic receiver 243O1 on the top of the head relative to the sternoclavicular joint O0 in the global coordinate system.

[0115] The above are merely preferred embodiments of the present invention and do not constitute any limitation on the present invention. Any equivalent substitutions or modifications made by those skilled in the art to the technical solutions and content disclosed in the present invention without departing from the scope of the present invention shall be deemed to have remained within the scope of protection of the present invention.

Claims

1. A multi-sensory upper limb motor measurement system, characterized in that, The system includes a soft felt garment integrated with a multimodal data acquisition system and a multimodal signal processing module. A motion measurement subsystem and a sensory measurement subsystem acquire multimodal signals of motion and perception, respectively, and transmit them to the multimodal signal processing module for amplification and filtering. A wireless signal concentrator collects the processed data from the multimodal signal processing module and sends it to a computer client via WiFi. The computer client integrates the multimodal signals to construct an upper limb motion perception state information system, which includes an upper limb motion calculation model and real-time multisensory information. The upper limb motion calculation model is calculated using the multimodal signals acquired by the motion measurement subsystem to form a motion model with real-time position coordinate data of each joint. The motion measurement subsystem collects the angle data of adduction, abduction, extension, flexion and internal / external rotation of the upper limb joints during movement, as well as the positional relationship of the head relative to the sternoclavicular joint. It is fixed at the corresponding acquisition position for measuring the corresponding data and collects the corresponding raw electrical signals and outputs them to the multimode signal processing module. The sensory measurement subsystem collects hand temperature and pressure signals, acquires temperature, touch, and pain sensations, and outputs raw electrical signals.

2. The multi-sensory upper limb motor measurement system according to claim 1, characterized in that, The soft felt garment is used to integrate and fix the motion measurement subsystem, sensory measurement subsystem, multimode signal processing module, data Bluetooth transmission module and wireless signal concentrator into a wearable device; The upper limb motion calculation model calculates the real-time position of the upper limb joints using multimodal motion data, including calculations of trigonometric functions, vector rotation, and coordinate transformation. The multimodal signal processing module receives raw electrical signals from the motion measurement subsystem and the sensory measurement subsystem, filters and amplifies them, extracts features from specific joint motion data, and then transmits the processed multimodal data to the data Bluetooth transmission module and the wireless signal concentrator, respectively. After completing the Bluetooth data aggregation and integration, the data Bluetooth transmission module and the wireless signal concentrator transmit all data to the computer client. The computer client constructs an upper limb motion perception state information system based on the received data, including an upper limb motion calculation model and real-time status of multi-sensory information.

3. The multi-sensory upper limb motor measurement system according to claim 2, characterized in that, The upper limb motion calculation model includes the movement from the sternoclavicular joint. To the center of shoulder joint rotation Elbow joint rotation center Wrist joint rotation center The metacarpophalangeal joints, proximal interphalangeal joints, distal interphalangeal joints, and fingertips are connected in sequence. The calculation process of the upper limb motion calculation model is as follows, with the origin of the local coordinate system at the location of the ultrasonic receiving device: The sternoclavicular joint Let the origin of the global coordinate system be defined, and let the direction forward be... The positive axis is directly in front of the human body, and to the left is... The positive axis direction, the left side of the human body, upwards is... Establish a right-handed rectangular coordinate system with the positive axis pointing vertically upwards and perpendicular to the ground. The local coordinate systems of all other joints are in the same direction. The shoulder joint rotation center relative to the sternoclavicular joint The coordinate transformation, assuming the measured shoulder width is... Therefore, half the shoulder width is Based on anatomical knowledge, let the center of rotation of the shoulder joint lie on the x-axis of the global coordinate system, and on the horizontal line. Then the coordinates of the center of rotation of the shoulder joint relative to the sternoclavicular joint are: Here, we ignore the slight vertical deviation caused by individual differences in the human body, taking the left upper limb as an example; The elbow joint rotation center Relative to the center of rotation of the shoulder joint The coordinate calculation is performed, assuming the length of the upper arm is... The center of rotation of the upper arm around the shoulder joint Local coordinate system The rotation angle of the axis is The upper arm rotates around the center of the shoulder joint. The rotation angle of the z-axis of the local coordinate system is In the global coordinate system The coordinates are : ； The wrist joint rotation center is The upper arm vector is Its length is Forearm vector is Its length is , = , around the upper arm vector The angle of rotation is , After elbow flexion and The included angle is , Represented as Based on vector projection and trigonometric function relationships, we can obtain: ； Perform circling rotation It is calculated using Rodrigues' rotation formula: ； Where I is the identity matrix , It is with the axis of rotation The corresponding antisymmetric matrix has the following form: ； First calculate : ； Then substitute it into the Rodriguez rotation formula to calculate the rotation matrix R: ； Coordinates of the initial position Multiplying by the rotation matrix R yields the rotated coordinates. ,Right now: ； We can obtain the following by matrix multiplication expansion: The specific expression; The coordinate transformation from the wrist joint to the metacarpophalangeal joint is achieved by obtaining the internal and external rotation angles of the elbow joint through a tension sensing device, and then calculating the coordinates of the metacarpophalangeal joint relative to the global coordinate system using the same method as calculating the wrist joint coordinates. The metacarpophalangeal joint, the proximal interphalangeal joint, and the distal interphalangeal joint only consider their flexion movement within one plane, ignoring movements in other directions and factors such as soft tissue deformation of the fingers. Let the joint rotation angle be... ,along Translated distance, Let the phalanges be numbered, and the homogeneous transformation matrix between adjacent phalanges be: ； This allows us to obtain the coordinates of each finger joint and fingertip relative to the global coordinate system.

4. The multi-sensory upper limb motor measurement system according to claim 2, characterized in that, The motion measurement subsystem is composed of multiple high-precision sensors, including a tension sensor, a uniaxial flexible bending strain sensor, a low-power ultrasonic transmitter module, an ultrasonic receiver, a clock synchronization module, a flexible printed circuit board, and electrical connection wires. The tension sensing device consists of a strap, a tension sensor, and an elastic cord. The strap is fixed at the upper arm near the shoulder and elbow joints, and at the forearm near the elbow and wrist joints. The tension sensor and the elastic cord are connected to each other. When the arm hangs naturally with the palm facing one side of the body, both ends are fixed vertically to the strap. The uniaxial flexible bending strain sensor is placed in the shoulder, wrist and finger joints, closely fitting the limb, and outputs the continuous change of angle during joint movement, which serves as a parameter for positive kinematic modeling of the upper limb, helping to construct real-time upper limb movement posture. The low-power ultrasonic transmitter module serves as the global coordinate system for positive kinematic upper limb modeling and transmits ultrasonic signals to the ultrasonic receiving device installed on the head. The ultrasonic receiver is fixed to the head and accurately records the propagation time of the ultrasonic waves; the clock synchronization module enables precise synchronization between the clocks of the ultrasonic transmitter and receiver. The flexible printed circuit board and electrical connection lines are designed in a special shape to provide displacement compensation for joint bending, connecting the uniaxial flexible bending strain sensor to the multimode signal processing module.

5. The multi-sensory upper limb motor measurement system according to claim 2, characterized in that, The sensory measurement subsystem consists of a strip-shaped flexible piezoelectric thin film sensor, a circular pressure thin film sensor, and a high-precision thermistor temperature sensor. The strip-shaped flexible piezoelectric film sensor is placed on the glove from the base of the palm towards the fingertips. It has three sensitive units corresponding to the fingertips, palm, and base of the palm, and can simultaneously acquire pressure information from multiple joints to identify different gripping gestures. The circular pressure film sensor is placed on the tip of the thumb to acquire tactile information from the thumb. The high-precision thermistor temperature sensor is placed on the fingertip to monitor the temperature changes of the environment in which the fingertip touches the ground, providing key data for restoring a realistic tactile experience and for in-depth analysis of patients' temperature perception feedback.

6. The multi-sensory upper limb motion measurement system according to claim 3, wherein the tension sensing device acquires the rotation angles of the shoulder and elbow joints. The method is as follows: consider the upper arm as a circle with a base circumference of... Gao Wei The cylindrical object, when the shoulder joint rotates internally or externally, will experience elastic strain due to the elastic ropes fixed to the straps at both ends. Huke's Law states that the elastic force of a spring... With the elongation or compression of the spring Proportional, its expression is ,in Let be the elastic coefficient. Then, given the elastic coefficient... and tension Then the elongation Therefore, the length of the base of the triangle formed by the stretched elastic rope, its initial state, and the base of the cylinder is: The formula for calculating the internal and external rotation angles of the shoulder joint is as follows: ； Similarly, the perimeter of the base is Gao Wei Internal and external rotation angles of the forearm cylinder for: 。 7. The multi-sensory upper limb motor measurement system according to claim 4, characterized in that, The low-power ultrasonic transmitter module and ultrasonic receiver are used to determine the sternoclavicular joint. Relative to the ultrasonic receiving device coordinates Let the speed of ultrasonic wave propagation in the current environment be... It depends on the ambient temperature according to Calculations show that the sternoclavicular joint The three ultrasonic transmitters A, B, and C form a triangle. Let the flight times from points A, B, and C to the corresponding ultrasonic receivers fixed overhead be... , , The sternoclavicular joint can then be calculated based on spatial geometric relationships. Location coordinates: ； in , , Given the known coordinates of the three ultrasonic transmitters A, B, and C at the sternoclavicular joint in the global coordinate system, the sternoclavicular joint can be obtained by solving the above system of equations. Relative to the ultrasonic receiving device mentioned above the head coordinates The ultrasonic receiving device located above the head can then be obtained through coordinate transformation of the two points. Sternoclavicular joint relative to global coordinate system The coordinates are .

Images