Finger pose capture system and method fusing visual solving and scalable line module

Abstract

The application relates to the technical field of motion capture, and discloses a finger posture motion capture system fusing visual calculation and a scalable wire module, which comprises a glove body, a fingertip connecting assembly, an integrated control module, a scalable wire module, an electromagnetic coil tension control unit, a monocular vision acquisition module and a main control unit; high-frequency and high-precision measurement of finger flexion and extension actions is realized through the scalable wire module, dynamic adjustment of cable tension is realized through the electromagnetic coil tension control unit, the global pose of a hand is acquired through the monocular vision acquisition module, and precise calculation of the finger posture is realized through a multi-modal fusion algorithm. The application can solve the problems that the existing motion capture scheme has poor anti-shielding capability and is prone to large accumulated drift errors, and simultaneously considers global pose acquisition and force feedback capability, and is suitable for VR / AR interaction, sign language recognition and other scenes.

Description

Technical Field

[0001] This application relates to the field of motion capture technology, and in particular to a finger posture motion capture system that integrates visual computation and a scalable line module. Background Technology

[0002] Currently, the mainstream finger posture capture technologies are mainly divided into two categories: pure vision solutions and motion capture solutions based on inertial / bending sensors.

[0003] Pure vision-based solutions, represented by algorithms such as HaMeR and MediaPipeHands, rely on monocular or multi-view cameras to capture hand images for 3D pose reconstruction. Their advantage lies in their low wearing burden and lack of complex wearable devices. However, they have inherent drawbacks that cannot be overcome: monocular vision suffers from scale ambiguity, making it impossible to accurately obtain the absolute displacement of finger joints, and depth measurement errors are large; when fingers are self-occluded, interactively occluded, or under poor lighting conditions, the confidence of joint detection drops sharply, 3D reconstruction completely fails, and the continuity of motion capture cannot be guaranteed; they have high computing power requirements, and the frame rate and latency are difficult to meet the real-time interaction needs when deployed on edge devices; at the same time, they can only achieve pose reconstruction and cannot provide force feedback interaction capabilities, thus limiting application scenarios.

[0004] Traditional data gloves based on inertial measurement units (IMUs) and bending sensors have the advantage of being unaffected by light or occlusion, enabling continuous attitude measurement. However, they have significant drawbacks: IMUs suffer from zero-bias drift, and joint angle errors accumulate over time, requiring frequent calibration; bending sensors exhibit significant hysteresis, resulting in poor output consistency during repeated flexion and extension movements, making it impossible to accurately reproduce minute finger movements; they can only acquire the relative bending angle of the fingers, not the global 6DoF pose of the palm relative to the world coordinate system, thus failing to achieve spatial positioning; and the integration of multiple sensors leads to complex wiring, poor glove comfort, and the risk of sensor detachment and poor contact over long-term use.

[0005] In summary, existing finger motion capture solutions cannot simultaneously achieve high-precision drift-free measurement, continuous motion capture with anti-occlusion capabilities, global pose acquisition, and force feedback interaction capabilities, making it difficult to meet the high-precision and high-reliability motion capture requirements of scenarios such as VR / AR interaction, sign language recognition, and remote collaboration. Summary of the Invention

[0006] To address the aforementioned technical issues, this application provides a finger posture motion capture system that integrates visual computation and a scalable line module.

[0007] Firstly, this application provides a finger posture motion capture system that integrates visual computation and a scalable line module, employing the following technical solution: The finger posture motion capture system, which integrates visual calculation and a retractable wire module, includes a glove body, fingertip connection components, an integrated control module, a retractable wire module, an electromagnetic coil tension control unit, a monocular vision acquisition module, and a main control unit. The fingertip connection assembly is fixed to each fingertip of the glove body; The integrated control module is fixed to the back of the hand of the glove body; One end of the cable of the retractable cable module is fixedly connected to the fingertip connection component, and the other end is stored in the electromagnetic coil tension control unit in the integrated control module, which is used to extend and retract synchronously with the flexion and extension of the finger. The electromagnetic coil tension control unit is located inside the integrated control module and is used to house the retractable wire module, detect the extension length of the retractable wire module, and dynamically adjust the tension of the retractable wire module. The monocular vision acquisition module is fixed to the side of the integrated control module facing the fingers and is used to acquire real-time images of the hand. The main control unit is located inside the integrated control module and is electrically connected to the electromagnetic coil tension control unit and the monocular vision acquisition module, respectively, and is used to perform finger posture calculation by multimodal data fusion.

[0008] Optionally, the cable of the retractable cable module is a silicone-nylon composite cable. The electromagnetic coil tension control unit is equipped with a winding wheel, a multi-pole magnetic ring, and a linear Hall sensor corresponding to a single cable. The multi-pole magnetic ring is coaxially fixed to the axis of the winding wheel, and the linear Hall sensor is fixed to the side of the multi-pole magnetic ring to detect the rotation angle of the winding wheel to calculate the extension length of the cable.

[0009] Optionally, the electromagnetic coil tension control unit further includes a permanent magnet and a winding coil coaxially arranged with the winding reel. The main control unit controls the magnitude and direction of the input current of the winding coil through a pulse width modulation signal to generate a corresponding electromagnetic torque, thereby realizing dynamic tension adjustment of the stretchable wire module.

[0010] Optionally, the monocular vision acquisition module uses a global shutter monocular camera, the acquisition frame rate of which is not less than 30Hz, and the field of view covers the entire active area of ​​the hand.

[0011] Optionally, the main control unit has a built-in multimodal fusion calculation module. The multimodal fusion calculation module uses the cable extension length data of the retractable cable module to predict high-frequency finger joint postures and uses the image data of the monocular vision acquisition module to perform low-frequency cumulative error correction and global pose calculation.

[0012] Secondly, this application provides a finger posture motion capture method that integrates visual computation and a scalable line module, employing the following technical solution: A finger pose motion capture method that integrates visual computation and a scalable line module includes the following steps: S1. Cable data calculation steps: The change in the extension length of the retractable cable module is obtained in real time through the electromagnetic coil tension control unit, and the local bending angle data of each joint of the finger is calculated based on the inverse kinematics model. S2. Visual data processing steps: Acquire real-time images of the hand through a monocular vision acquisition module, input the images into a pre-trained 3D hand mesh reconstruction network, and output the 3D mesh data of the hand and the 6DoF global pose data of the wrist relative to the world coordinate system. S3. Fusion Calibration Step: The local bending angle data obtained in step S1 and the global pose data obtained in step S2 are spatiotemporally aligned and fused using the Kalman filter algorithm to output the final finger full-pose motion capture data.

[0013] Optionally, in step S1, the refresh rate for calculating the local bending angle data is not less than 1000Hz.

[0014] Optionally, in step S3, the confidence level of the 3D hand mesh reconstruction network output is obtained in real time. When the confidence level is lower than a preset threshold, the fusion weight of the visual data is reduced, and the posture output is maintained only based on the local bending angle data calculated from the cable data.

[0015] Optionally, it also includes a dynamic tension adjustment step: the main control unit adjusts the duty cycle of the pulse width modulation signal in real time according to the finger movement speed and the cable extension and retraction state of the retractable cable module, so as to keep the retractable cable module in a taut state, or output corresponding resistance to realize force feedback interaction.

[0016] Optionally, a pre-calibration step is also included: upon first use, extreme data of cable length are collected for user's clenched fist and fully extended hand gestures, and a baseline mapping model of cable length and finger joint bending angle is established.

[0017] In summary, this application includes at least the following beneficial technical effects: This application achieves high-frequency, high-precision detection of finger flexion and extension movements through the combination of a retractable cable module and an electromagnetic coil tension control unit. Dynamic tension adjustment avoids measurement errors caused by cable slack, improving the stability and consistency of motion capture data. The combination of a monocular vision acquisition module and a multimodal fusion algorithm not only utilizes visual calculation to obtain the global pose of the palm, eliminating the cumulative drift of physical sensors, but also uses physical measurement data to compensate for the failure of the visual solution in obstructed or poorly lit scenarios, ensuring the continuity and reliability of the motion capture process. It also simplifies the overall hardware structure of the system, reduces the wearing burden, and enables force feedback interaction through electromagnetic coil torque adjustment, expanding the system's application scenarios and enhancing the user's interactive experience. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the structure and principle of the glove body in the finger posture motion capture system of this application.

[0019] Figure 2 This is a schematic diagram of the finger posture motion capture method in this application. Detailed Implementation

[0020] The embodiments of this application are described in detail below, and examples of the embodiments are shown in the accompanying drawings.

[0021] In the description of this specification, the references to "certain embodiments," "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples" refer to specific features, structures, materials, or characteristics described in connection with the described embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0022] The finger posture motion capture system that integrates visual computation and a scalable line module provided in this embodiment, such as Figure 1 As shown, the overall hardware architecture includes the glove body, fingertip connection components, integrated control module, retractable cable module, electromagnetic coil tension control unit, monocular vision acquisition module, and main control unit. The specific implementation methods of each component are as follows: The glove body is made of high-elastic Lycra fabric in one piece, which can fit different sizes of hands. The fabric has an elongation rate of ≥200%, so there is no obvious feeling of restriction when wearing it. The fingertips of the five fingers of the glove body are reserved with fixing positions for fingertip connection components, and the back of the hand is reserved with an installation position for an integrated control module. The fabric surface has invisible wiring channels to hide the exposed section of the retractable wire module and avoid the cable tangling affecting wearing and movement.

[0023] The fingertip connection component uses a food-grade rigid ABS material for the fingertip pressure plate. Each pressure plate measures 10mm x 15mm and is fixed to the inside of the fingertip of the glove body. The pressure plate surface has a 1mm diameter cable fixing hole. The end of the retractable cable module is fixed in the fixing hole with a stainless steel cable clip. After fixing, there is no relative slippage, ensuring that the retractable cable module moves synchronously with the fingertip when the finger flexes and extends, thus ensuring the accuracy of length measurement.

[0024] The integrated control module uses a lightweight 3D-printed ABS shell with overall dimensions of 60mm×40mm×15mm and a weight of ≤20g. It is fixed to the center of the back of the hand with Velcro, and there is no noticeable weight when wearing it. The shell has isolated mounting cavities to accommodate five sets of electromagnetic coil tension control units, the main control unit and the power supply module. On the side of the shell facing the fingers, there is an embedded mounting position for a monocular vision acquisition module. The angle of the mounting position is tilted to ensure that the camera's field of view completely covers the entire active area of ​​the hand.

[0025] The retractable cable module uses a silicone-nylon composite cable with a single cable diameter of 0.8mm. The nylon inner core has a diameter of 0.4mm, providing a tensile strength of ≥200N to ensure long-term use without breakage or plastic deformation. The outer silicone coating is 0.2mm thick with a Shore hardness of 30A, providing a smooth touch and appropriate damping to prevent high-frequency vibration and noise when the cable retracts quickly, while also improving the cable's abrasion resistance. Each finger corresponds to an independent retractable cable module, for a total of 5, corresponding to the thumb, index finger, middle finger, ring finger, and little finger. The cable runs along the back of the finger, without affecting the normal flexion and extension movements of the finger.

[0026] Each finger corresponds to an independent electromagnetic coil tension control unit, totaling 5 units, integrated within the integrated control module. Each unit is controlled independently and does not interfere with others. Each unit includes a winding reel, a multi-pole magnetic ring, a linear Hall sensor, a permanent magnet, and a winding coil. Specific implementation details are as follows: Cable reel: 8mm in diameter, with a spiral groove on the surface of the reel body. The groove is 0.9mm deep and wide, which can completely accommodate the retractable cable module. The inner end of the retractable cable module is fixed to the bottom of the groove by adhesive. The cable is pulled out and rewound as the reel rotates in both directions. Multi-pole magnetic ring and linear Hall sensor: The multi-pole magnetic ring is an 8-pole radially magnetized neodymium iron boron magnetic ring with an outer diameter of 3mm. It is coaxially fixed to the rotation axis of the winding reel and rotates synchronously with the reel. The linear Hall sensor is model AH49E, fixed to the radial side of the multi-pole magnetic ring with a distance of 0.5mm from the surface of the magnetic ring. The sampling frequency is 1000Hz, which can detect the magnetic field change during the rotation of the multi-pole magnetic ring in real time and output the corresponding analog voltage signal. The main control unit reads the voltage signal through a 12-bit AD conversion channel, calculates the rotation angle and number of rotations of the winding reel, and then converts it into the real-time extension length of the retractable cable module. The detection resolution can reach 0.1mm. Permanent magnet and winding coil: The permanent magnet is a radially magnetized neodymium iron boron permanent magnet, coaxially fixed with the winding reel. The winding coil is wound around the outside of the permanent magnet, using 0.1mm enameled wire with 200 turns. The input terminal of the winding coil is electrically connected to the PWM output pin of the main control unit. The main control unit outputs a 20kHz PWM signal. By adjusting the duty cycle of the PWM signal, the magnitude and direction of the input current to the winding coil are controlled, thereby generating an electromagnetic torque that interacts with the permanent magnet. This electromagnetic torque can drive... The rotating reel enables the rewinding and dynamic tension adjustment of the retractable cable module. In the normal motion capture mode, the main control unit outputs a PWM signal with a 10% duty cycle to generate a constant, weak pull force, ensuring that the retractable cable module is always taut and avoiding measurement errors caused by cable slack. In the force feedback interaction mode, the main control unit adjusts the duty cycle of the PWM signal to 30%-80% according to the needs of the interaction scenario, increasing the electromagnetic torque and generating corresponding pull resistance to simulate the force feedback effect when a virtual object is touched or grasped.

[0027] The monocular vision acquisition module uses a global shutter monocular camera, model OV7251, with a resolution of 640×480 and a frame rate of 30Hz. The global shutter mode can effectively avoid the rolling shutter effect caused by rapid hand movements. The camera is equipped with a wide-angle lens with a field of view of 120°, which can completely cover the entire active area of ​​the hand, ensuring that all flexion and extension movements of the fingers are within the camera's acquisition range. The camera is electrically connected to the main control unit via a MIPI interface, transmitting the acquired real-time RGB images of the hand to the main control unit for processing.

[0028] The main control unit uses an STM32H743 microcontroller as the main control chip, with a built-in ARM Cortex-M7 core and a main frequency of 400MHz. It has 16 channels of 12-bit AD acquisition, 8 channels of PWM output, and a MIPI interface. It can simultaneously complete data acquisition from 5 linear Hall sensors, PWM control of 5 winding coils, reception and preprocessing of camera image data, and operation of multi-modal fusion algorithms. The main control unit also integrates a Bluetooth 5.0 module, which can transmit the calculated finger posture motion capture data to the host computer, VR / AR devices, and other terminals in real time with a transmission latency of ≤50ms. The power supply module uses a 3.7V, 300mAh lithium polymer battery, which is integrated into the integrated control module and can support continuous system operation for ≥4 hours, meeting daily use needs.

[0029] The finger posture motion capture data processing method in this embodiment, such as Figure 2 As shown, the core process consists of three main steps: cable data processing, visual data processing, and multimodal fusion calibration. It also includes pre-calibration and dynamic tension adjustment steps. The specific implementation details of each step are as follows: Pre-calibration steps When a user wears the system for the first time, an initial pre-calibration process is executed, specifically as follows: a. The user keeps all five fingers fully extended and remains still for 3 seconds. The system records the length of each retractable cable module at this time as the baseline length. ; b. The user makes a full fist and holds the position for 3 seconds. The system records the maximum extension of each retractable cable module at this time. And the maximum bending angle of the corresponding finger joints; c. Based on the recorded extreme value data, the system fits a baseline mapping model of the cable length and joint bending angle of each finger, and determines the rotation radius of each finger joint. With coupling coefficient The calibration value is obtained, and the initial calibration is completed.

[0030] During system operation, a dynamic calibration is automatically performed every 5 minutes: the system detects the user's hand movements, and when the user holds a still gesture for more than 1 second, it collects the cable measurement data and visual reconstruction data at this time. Based on the joint angle data obtained from visual reconstruction, the system fine-tunes the mapping model parameters of cable length and joint angle to compensate for minor measurement deviations caused by cable deformation and temperature changes. At the same time, it updates the process noise covariance matrix of the Kalman filter to ensure the long-term stability of the fusion algorithm.

[0031] Cable data processing steps: This step uses an electromagnetic coil tension control unit to obtain the change in the extension length of the stretchable wire module in real time. Based on the inverse kinematics model, it calculates the local bending angle data of each joint of the finger. The calculation refresh rate is 1000Hz, which is unaffected by light or occlusion and has no delay.

[0032] For the index, middle, ring, and little fingers, each finger contains three rotational joints: the metacarpophalangeal joint (MCP), the proximal interphalangeal joint (PIP), and the distal interphalangeal joint (DIP). A mapping model between the joint angles and the cable extension / retraction length is established, as shown in the following formula: ; in: This refers to the change in the telescopic length of the telescopic module, in mm. The reference length of the retractable wire module when the finger is fully extended is in mm and is determined through pre-calibration. The real-time length of the retractable wire module after the finger is bent is shown in mm. The radius of rotation of the finger joint is the perpendicular distance between the telescopic wire module wiring and the joint rotation axis, expressed in mm, and is determined through pre-calibration. The bending angle of the corresponding joint is expressed in rad. Corresponding to the MCP joint, Corresponding to the PIP joint, Corresponding DIP joint; Based on the physiological movement constraints of the human finger, there is a fixed coupling relationship between the bending angles of the PIP joint and the DIP joint, as shown in the following formula: ; in The coupling coefficient is set to 2 / 3, which conforms to the physiological movement characteristics of human fingers. It can be finely adjusted through pre-calibration.

[0033] Combining the two formulas above, the main control unit can determine the data based on real-time acquisition. The unique solution obtains the bending angles of the three joints MCP, PIP, and DIP, enabling high-frequency measurement of local finger posture.

[0034] For the thumb, which contains two rotational joints, namely the metacarpophalangeal joint (MCP) and the interphalangeal joint (IP), a corresponding mapping model is established, and the formula is as follows: ; in: The change in the telescopic length of the thumb-related telescopic wire module is expressed in mm. The radius of rotation of the thumb joint, in mm, is determined through pre-calibration; The angle of flexion of the thumb metacarpophalangeal joint is expressed in rad. The angle of flexion of the thumb's interphalangeal joint is expressed in rad.

[0035] The visual data processing steps acquire real-time images of the hand through a monocular vision acquisition module, input the images into a pre-trained 3D hand mesh reconstruction network, output the 3D mesh data of the hand and the 6DoF global pose data of the wrist relative to the world coordinate system, and output the reconstruction confidence.

[0036] In this embodiment, the 3D hand mesh reconstruction network adopts a Transformer-based HaMeR architecture. The network input is the RGB hand image output by the monocular vision acquisition module, which is preprocessed and standardized to a size of 224×224×3. The network output consists of the parameters of the hand MANO model, including pose parameters. Shape parameters And the 6DoF global pose of the wrist relative to the world coordinate system, including the 3D translation vector. With 3D rotation matrix .

[0037] Among them, the MANO model is a general parametric hand model that can generate a corresponding 3D hand mesh by inputting posture and shape parameters. The mesh contains 778 vertices, corresponding to the 3D coordinates of 21 hand joints, and can be directly used for calculating joint bending angles and calibrating global pose.

[0038] The training and deployment steps for the network are as follows: a. Dataset preparation: The FreiHAND, HO3D, and RHD public hand datasets are used. The datasets include monocular RGB hand images, corresponding hand 3D joint coordinates, and MANO model parameter annotations. The total sample size is no less than 1 million. At the same time, hand image samples with different lighting and different occlusion levels are collected to improve the model's anti-interference ability. b. Data preprocessing: The images in the dataset are uniformly scaled to 224×224 resolution, and data augmentation operations such as random flipping, brightness adjustment, Gaussian blur, and random occlusion are performed to improve the generalization ability of the model. c. Model training: The AdamW optimizer was used, with an initial learning rate of 1e-4, a batch size of 32, and 100 training epochs. The loss function was a weighted sum of the mean square error loss of 3D key coordinates, the L1 loss of MANO parameters, and the 2D reprojection error loss, with weighting coefficients of 0.6, 0.2, and 0.2, respectively. d. Model quantization and deployment: The trained floating-point model is quantized into an INT8 fixed-point model to reduce computing power requirements and memory usage, enabling it to be stably deployed and run at the edge of the main control unit. The inference time for a single frame image is ≤30ms, meeting real-time requirements.

[0039] During network operation, the 3D coordinates of 21 joints of the hand 3D mesh are output in real time, and the reconstruction confidence is also output. The confidence range is 0-1. When the hand is unobstructed and well lit, the confidence is ≥0.8. When there is severe occlusion, the confidence is <0.3.

[0040] The multimodal fusion calibration step employs a linear Kalman filter algorithm, which can perform spatiotemporal alignment and fusion calibration of the local bending angle data obtained from cable data and the global pose data obtained from visual data. It adopts a strategy of sensor-driven high-frequency motion and vision-driven low-frequency calibration. Based on the defined state vector, it performs 1000Hz high-frequency state prediction according to the joint angle data obtained from cable data. When the visual data is available and the reconstruction confidence is ≥ a preset threshold (the preset threshold is 0.6 in this embodiment), it uses the 3D coordinates of the joint points obtained from vision and the global pose of the wrist to perform 30Hz low-frequency measurement update, thereby clearing the accumulated error and calibrating the global pose.

[0041] The main control unit collects real-time data on the extension speed and length changes of the retractable cable module to determine the finger's movement state and adjusts the duty cycle of the PWM signal in real time: when the finger bends rapidly, the PWM signal duty cycle is reduced to decrease the pull force and prevent the cable from pulling and affecting the finger's movement; when the finger straightens rapidly, the PWM signal duty cycle is increased to increase the pull force, ensuring that the cable rewinds quickly and remains taut, avoiding measurement errors caused by cable slack; in force feedback interaction mode, the main control unit receives force feedback commands from the host computer, adjusts the PWM signal duty cycle to the corresponding value, generates matching pull resistance, simulates the physical properties of virtual objects such as hardness and weight, and achieves immersive force feedback interaction.

[0042] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A finger posture motion capture system integrating visual computation and a scalable line module, characterized in that, It includes the glove body, fingertip connection components, integrated control module, retractable thread module, electromagnetic coil tension control unit, monocular vision acquisition module, and main control unit; The fingertip connection assembly is fixed to each fingertip of the glove body; The integrated control module is fixed to the back of the hand of the glove body; One end of the cable of the retractable cable module is fixedly connected to the fingertip connection component, and the other end is stored in the electromagnetic coil tension control unit in the integrated control module, which is used to extend and retract synchronously with the flexion and extension of the finger. The electromagnetic coil tension control unit is located inside the integrated control module and is used to house the retractable wire module, detect the extension length of the retractable wire module, and dynamically adjust the tension of the retractable wire module. The monocular vision acquisition module is fixed to the side of the integrated control module facing the fingers and is used to acquire real-time images of the hand. The main control unit is located inside the integrated control module and is electrically connected to the electromagnetic coil tension control unit and the monocular vision acquisition module, respectively, and is used to perform finger posture calculation by multimodal data fusion.

2. The finger posture motion capture system integrating visual computation and a scalable line module according to claim 1, characterized in that, The retractable cable module uses silicone-nylon composite cable. The electromagnetic coil tension control unit is equipped with a winding wheel, a multi-pole magnetic ring, and a linear Hall sensor corresponding to a single cable. The multi-pole magnetic ring is coaxially fixed to the axis of the winding wheel, and the linear Hall sensor is fixed to the side of the multi-pole magnetic ring to detect the rotation angle of the winding wheel to calculate the extension length of the cable.

3. The finger posture motion capture system integrating visual computation and a scalable line module according to claim 2, characterized in that, The electromagnetic coil tension control unit also includes a permanent magnet and a winding coil coaxially arranged with the winding reel. The main control unit controls the magnitude and direction of the input current of the winding coil through a pulse width modulation signal to generate a corresponding electromagnetic torque, thereby realizing dynamic tension adjustment of the stretchable wire module.

4. The finger posture motion capture system integrating visual computation and a scalable line module according to claim 1, characterized in that, The monocular vision acquisition module uses a global shutter monocular camera with an acquisition frame rate of no less than 30Hz and a field of view covering the entire active area of ​​the hand.

5. The finger posture motion capture system integrating visual computation and a scalable line module according to claim 1, characterized in that, The main control unit has a built-in multimodal fusion calculation module. The multimodal fusion calculation module uses the cable extension length data of the retractable cable module to predict the high-frequency finger joint posture, and uses the image data of the monocular vision acquisition module to perform low-frequency cumulative error correction and global pose calculation.

6. A method for processing finger posture motion capture data based on the system described in any one of claims 1 to 5, characterized in that, Includes the following steps: S1. Cable data calculation steps: The change in the extension length of the retractable cable module is obtained in real time through the electromagnetic coil tension control unit, and the local bending angle data of each joint of the finger is calculated based on the inverse kinematics model. S2. Visual data processing steps: Acquire real-time images of the hand through a monocular vision acquisition module, input the images into a pre-trained 3D hand mesh reconstruction network, and output the 3D mesh data of the hand and the 6DoF global pose data of the wrist relative to the world coordinate system. S3. Fusion Calibration Step: The local bending angle data obtained in step S1 and the global pose data obtained in step S2 are spatiotemporally aligned and fused using the Kalman filter algorithm to output the final finger full-pose motion capture data.

7. The finger posture motion capture method integrating visual computation and a scalable line module according to claim 6, characterized in that, In step S1, the refresh rate for calculating the local bending angle data is not less than 1000Hz.

8. The finger posture motion capture method integrating visual computation and a scalable line module according to claim 6, characterized in that, In step S3, the confidence level of the 3D hand mesh reconstruction network output is obtained in real time. When the confidence level is lower than a preset threshold, the fusion weight of the visual data is reduced, and the posture output is maintained only based on the local bending angle data calculated from the cable data.

9. The finger posture motion capture method integrating visual computation and a scalable line module according to claim 6, characterized in that, It also includes a dynamic tension adjustment step: the main control unit adjusts the duty cycle of the pulse width modulation signal in real time according to the speed of finger movements and the extension and retraction status of the retractable cable module, so as to keep the retractable cable module in a taut state, or output corresponding resistance to achieve force feedback interaction.

10. The finger posture motion capture method integrating visual computation and a scalable line module according to claim 6, characterized in that, It also includes a pre-calibration step: when used for the first time, extreme data of cable length are collected for the user's clenched fist and fully extended hand gestures, and a benchmark mapping model of cable length and finger joint bending angle is established.

Images