A human-computer interaction data glove

Abstract

This application discloses a human-computer interaction data glove, comprising a glove body, a finger sensor on the finger area of ​​the glove body, a data acquisition device on the outside of the wrist area of ​​the glove body, and an inner lining with a flexible strain sensor on the inner lining. The data acquisition device is electrically connected to the finger sensor and the flexible strain sensor respectively. This invention uses the flexible strain sensor on the wrist to sense wrist stress and deformation in real time, inversely solves the wrist posture angle and uses it as the dynamic base coordinate of the finger kinematic model, performing inverse compensation for finger joint angles, effectively solving the finger positioning error problem caused by changes in wrist posture.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, and in particular to a human-computer interaction data glove. Background Technology

[0002] Data gloves, as important devices in the field of human-computer interaction, are widely used in virtual reality, teleoperated robots, rehabilitation training, and other scenarios. Existing data gloves typically place flexion sensors at the finger joints and pressure sensors at the fingertips to collect information on finger movement posture and force, thereby driving the virtual hand to achieve synchronized movements.

[0003] Traditional data gloves primarily concentrate their sensor placement on the fingers, neglecting the crucial role of the wrist as the pivot point for hand movements. Wrist flexion, adduction / abduction, and rotation directly affect the absolute position of the fingers in space. When the operator's wrist moves, even if the finger joint angles remain constant, the actual spatial position of the fingertips will shift. Traditional finger kinematic models typically fix the base coordinate system on the palm, failing to detect and compensate for positional deviations caused by wrist posture changes. This leads to fingertip positioning errors during data glove-virtual hand interaction, reducing the accuracy of human-computer interaction. Summary of the Invention

[0004] The purpose of this invention is to provide a human-computer interaction data glove to solve the technical problems of virtual fingertip positioning errors and poor interaction accuracy caused by wrist posture in traditional data gloves.

[0005] In a first aspect, the present invention provides a human-computer interaction data glove, specifically comprising: The glove body has a finger sensor on its finger section; The main body of the glove is equipped with a collection device on the outside of the wrist area; The glove body has an inner lining folded edge inside the wrist area, and a flexible strain sensor is installed on the inner lining folded edge; The acquisition device is electrically connected to the finger sensor and the flexible strain sensor, respectively.

[0006] Furthermore, the finger sensor includes a pressure sensor located at the thumb pad and an ion skin sensor located at the bent joint of the finger.

[0007] Furthermore, the inner liner fold edge adopts an embedded fold structure, and the flexible strain sensor is embedded in the folded interlayer of the inner liner fold edge.

[0008] Furthermore, the acquisition device includes a circuit board on which a microprocessor, an attitude sensor, and a wireless communication device are integrated; The microprocessor is electrically connected to the finger sensor, flexible strain sensor, posture sensor, and wireless communication device, respectively.

[0009] Secondly, the present invention provides an interaction method for a human-computer interaction data glove, comprising the following steps: a) Collect pressure signals, bending signals, and tactile signals from the finger area using a finger sensor; b) Acquire stress and deformation signals from the wrist area using a flexible strain sensor; c) Acquire overall hand movement posture signals based on posture sensors; d) Using the microprocessor of the acquisition device, the acquired pressure signal, bending signal, tactile signal, posture signal, stress signal and deformation signal are converted from analog to digital and filtered; e) Utilize a microprocessor to perform at least one interactive control operation among wrist posture dynamic compensation, grasping intention reverse recognition, virtual hand mapping dynamic compensation, force feedback feedforward-feedback composite control, and force feedback gain dynamic adjustment. f) The signal processed and calculated by the microprocessor is sent to an external terminal via a wireless communication device. The external terminal analyzes the received signal and reconstructs the real-time movement and force state of the hand.

[0010] Furthermore, the wrist posture dynamic compensation includes: Based on the stress and deformation signals of the wrist collected by the flexible strain sensor, the real-time posture angles (α, β, γ) of the wrist in three-dimensional space are solved by inverse stress-strain constitutive relations. Where α is the wrist flexion angle, β is the wrist adduction / abduction angle, and γ is the wrist rotation angle; The calculated real-time wrist posture angles are used as the dynamic base coordinates of the finger kinematics model. The finger joint angles θ1, θ2, θ3, and θ4 are inversely compensated to obtain the corrected joint angles θ′1, θ′2, θ′3, and θ′4. Wherein, θ1 is the flexion angle of the metacarpophalangeal joint of the finger, θ2 is the flexion angle of the proximal interphalangeal joint of the finger, θ3 is the flexion angle of the distal interphalangeal joint of the finger, and θ4 is the abduction angle of the carpometacarpal joint of the thumb. The fingertip coordinates are recalculated based on the corrected joint angles and compared with the initial fingertip coordinates obtained through the finger kinematics model. When the error between the two exceeds a preset threshold, the sensor calibration process or multi-source data fusion algorithm is triggered.

[0011] Furthermore, the reverse recognition of the grasping intent includes: Based on the pressure signal from the pressure sensor, the bending signal from the ion skin sensor at the finger joint, and the stress signal and rate of change from the flexible strain sensor at the wrist, a multimodal feature vector is constructed. This multimodal feature vector satisfies... ; Where F is the multimodal feature vector; P thumb B is the real-time pressure signal of the thumb. finger For real-time bending signals of finger joints; S wrist This is a real-time stress signal for the wrist. The rate of change of wrist stress; The multimodal feature vectors are input into a preset reverse inference model, which outputs the operator's grasping intention based on preset logical rules. Based on the grasping intent obtained from the reverse reasoning model, the virtual hand in the virtual environment is automatically controlled to perform the corresponding action.

[0012] Furthermore, the virtual hand mapping dynamic compensation includes: The finger joint length ratio between the main body of the glove and the virtual hand is obtained in advance, and the ratio satisfies... l′ ᵢ,ⱼ =k ᵢ,ⱼ ×l ᵢ,ⱼ ; Among them, l′ ᵢ,ⱼ k is the mapping length of the virtual hand's corresponding knuckles. ᵢ,ⱼ This is the length proportionality coefficient for the corresponding finger joint; l ᵢ,ⱼ The actual length of the j-th phalanx of the i-th finger corresponding to the main body of the glove; Introducing the wrist stress compensation coefficient η(S) wrist The above proportional relationship is dynamically adjusted, and the adjusted relationship satisfies... l′ ᵢ,ⱼ =η(S wrist )×k ᵢ,ⱼ ×l ᵢ,ⱼ ; Wherein, η(S) wrist )=1+μ×(S wrist -S0) / S0, where S0 is the reference stress value of the wrist at rest, μ is the pre-calibrated compensation coefficient, and S wrist The wrist stress signal is collected in real time by a flexible strain sensor; The compensation coefficient η(S) is dynamically adjusted based on real-time wrist stress signals. wrist When wrist stress increases, η(S) wrist When η(S) > 1, the mapping ratio of the virtual finger joint is increased accordingly; when the wrist stress decreases, η(S) > 1. wrist If ) < 1, the mapping ratio of the virtual finger joints will be reduced accordingly; Substitute the dynamically compensated virtual finger joint length into the fingertip mapping equation based on the DH parameter method to resolve the joint angle information of the virtual hand.

[0013] Furthermore, the force feedback-precursor composite control includes: Construct and pre-train a deep learning-based inverse reasoning network, taking pressure and bending signals collected by finger sensors and wrist stress signals collected by flexible strain sensors as inputs, and output the operator's expected grip force. In the virtual environment, before the virtual hand corresponding to the glove body comes into contact with the interactive object, the expected gripping force is preloaded into the microprocessor as a feedforward control quantity, and the corresponding virtual output force is pre-established. When the virtual hand interacts with the interactive object, the real-time force feedback information from the virtual environment is fused with the expected gripping force to obtain the final output force, which satisfies... F out =λ1×F pred +(1-λ1)×F fb ; Among them, F out λ1 is the final output force; λ1 is the fusion coefficient, dynamically adjusted based on the wireless communication delay and the rate of change of wrist stress; F pred For expected grip strength; F fb This provides real-time force information.

[0014] Furthermore, the dynamic adjustment of the force feedback gain includes: A mapping relationship between wrist stress signal and feedback force gain coefficient is established in advance, and the mapping relationship satisfies Among them, K f The feedback force gain coefficient; f(·) is the function symbol; S wrist S0 is the wrist stress signal; S0 is the reference stress value of the wrist at rest, and λ2 is the pre-calibrated adjustment coefficient. The feedback force gain is dynamically adjusted based on the real-time collected wrist stress signal. wrist When K increases, f >1 corresponds to an increase in actual output feedback force, simulating the real force feeling of gripping a heavy object; when S wrist When K decreases, f <1 corresponds to a reduction in the actual output feedback force, simulating the force felt when grasping or releasing a light object.

[0015] Compared with the prior art, the present invention has at least one of the following technical effects: 1. This invention uses a flexible strain sensor in the wrist to sense wrist stress and deformation in real time, solves the wrist posture angle and uses it as the dynamic base coordinate of the finger kinematic model, and performs inverse compensation for the finger joint angle to improve accuracy and effectively solve the problem of fingertip positioning error caused by changes in wrist posture.

[0016] 2. This invention integrates multimodal features such as thumb pressure, finger joint bending angle, and wrist stress and its rate of change to construct a reverse reasoning model based on preset logical rules. It can accurately distinguish four intentions: precise grasping, continuous holding, release, and hovering preparation. This overcomes the limitation of traditional methods that rely solely on finger bending signals and cannot distinguish subtle intentions, making human-computer interaction more intelligent and natural.

[0017] 3. This invention introduces a wrist stress compensation coefficient, which dynamically adjusts the mapping ratio of the virtual finger joints according to the operator's force application state. When the wrist stress increases, the mapping ratio is enlarged, and when the stress decreases, the mapping ratio is reduced, making the movement of the virtual hand more closely match the operator's actual force application state, thus solving the problem of unnatural movement under a fixed ratio mapping.

[0018] 4. This invention employs force feedback-force composite control. It predicts the operator's expected gripping force through a deep learning inverse reasoning network and preloads it as a feedforward before the virtual hand contacts the object. This is dynamically fused with the real-time force feedback information from the virtual environment, effectively compensating for the impact of wireless communication delay. At the same time, the force feedback gain is dynamically adjusted according to wrist stress, so that the feedback force increases when gripping heavy objects and decreases when gripping light objects or releasing them, simulating the force sensation under different force states in the real world.

[0019] 5. This invention constructs a complete technology chain from physical perception to semantic understanding to tactile feedback, from multi-source sensor data acquisition of the wrist, fingers, and thumb, to dynamic compensation of wrist posture, intention inverse reasoning, and mapping dynamic compensation, and then to force feedback-feedback composite control and gain dynamic adjustment. This significantly improves the immersion, naturalness, and intelligence level of human-computer interaction. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the overall structure of a human-computer interaction data glove provided in an embodiment of the present invention; Figure 2This is a schematic diagram of the ion skin sensor structure of a human-computer interaction data glove according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the pressure sensor structure of a human-computer interaction data glove according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the interaction method of a human-computer interaction data glove provided in an embodiment of the present invention.

[0022] Reference numerals: 1. Glove body; 2. Ion skin sensor; 3. Pressure sensor; 4. Data acquisition device; 5. Inner lining fold; 6. Flexible strain sensor; 7. Circuit board; 8. Microprocessor; 9. Attitude sensor; 10. Wireless communication device. Detailed Implementation

[0023] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0024] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0025] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0026] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0027] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0028] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0029] In this application embodiment, the entity executing the process includes a terminal device. This terminal device includes, but is not limited to, devices capable of executing the methods disclosed in this application, such as servers, computers, smartphones, and tablets.

[0030] refer to Figures 1-3 This embodiment provides a human-computer interaction data glove, specifically including: The glove body 1 has a finger sensor on its finger area; A collection device 4 is provided on the outside of the wrist area of ​​the glove body 1; The glove body 1 has an inner lining 5 inside the wrist area, and a flexible strain sensor 6 is installed on the inner lining 5. The data acquisition device 4 is electrically connected to the finger sensor and the flexible strain sensor 6, respectively.

[0031] The finger sensor includes a pressure sensor 3 located at the pad of the thumb and an ion skin sensor 2 located at the joint of the finger.

[0032] In this embodiment, the pressure sensor 3 is a thin-film piezoresistive sensor, model FlexiForceA201. The sensor is connected to the data acquisition device 4 via a flexible FPC cable, and the connection pins include VCC (3.3V), GND, and signal output (AO).

[0033] The ion skin sensor 2 uses a commercially available flexible capacitive sensor to sense the bending angle of the finger joints. This sensor has a three-layer structure: the top and bottom layers are PET flexible substrates, the middle layer is an elastic dielectric layer, and the electrodes are conductive silver paste printed layers. One sensor is placed at each finger joint, for a total of 14 (two joints for the thumb and three joints for each of the other four fingers). The sensor capacitance changes with the bending angle. The flexible strain sensor 6 uses a commercially available metal foil strain gauge, embedded in the folded layer of the inner lining edge 5. The flexible strain sensor 6 is located on the dorsal, ulnar, and radial sides of the wrist. This strain gauge operates based on the principle of metal resistance strain effect; the sensitive grid material is constantan foil, the substrate is a polyimide film, and it is connected to the acquisition device 4 via enameled wire.

[0034] The inner lining edge 5 adopts an embedded folded structure, and the flexible strain sensor 6 is embedded in the folded interlayer of the inner lining edge 5. The flexible strain sensor 6 is respectively located on the dorsal, ulnar, and radial sides of the wrist. The inner lining edge 5 adopts an embedded folded structure, and this folded interlayer is formed by heat-pressing two layers of fabric. The flexible strain sensor 6 is encapsulated between the two layers of fabric, and the sensor surface is fixed to the fabric with medical-grade silicone adhesive. The flexible strain sensor 6 is respectively located on the dorsal, ulnar, and radial sides of the wrist, and the sensor's sensing direction is parallel to the wrist's bending direction.

[0035] The data acquisition device 4 includes a circuit board 7, on which a microprocessor 8, an attitude sensor 9, and a wireless communication device 10 are integrated. The microprocessor 8 is electrically connected to the finger sensor, the flexible strain sensor 6, the posture sensor 9, and the wireless communication device 10, respectively.

[0036] In this implementation, the data acquisition device 4 uses an injection-molded shell and is fixed to the back of the wrist with Velcro. The microprocessor 8 is an STM32F407VET6 based on an ARM Cortex-M4 core with a 168MHz clock speed. It integrates three 12-bit ADCs, two DACs, multiple timers, and communication interfaces such as USART, SPI, and I2C. Pins PA0-PA3 of the microprocessor 8 are configured as ADC inputs, connecting to the finger sensor; pin PA4 is configured as an ADC input, connecting to the flexible strain sensor 6; pins PB6-PB9 are configured as I2C interfaces, connecting to the attitude sensor 9; and pins PC10-PC11 are configured as USART interfaces, connecting to the wireless communication device 10. The attitude sensor 9 uses a nine-axis inertial measurement unit (MPU9250), integrating a 3-axis accelerometer (±16g), a 3-axis gyroscope (±2000° / s), and a 3-axis magnetometer (±4800μT). The attitude sensor 9 is connected to the microprocessor 8 via an I2C interface, with a device address of 0x68. The wireless communication device 10 uses a Bluetooth 5.2 module, model nRF52832, supporting BLE and ANT protocols. The wireless communication device 10 connects to the microprocessor 8 via a UART interface at a baud rate of 115200bps. The communication protocol uses a custom data frame format: frame header (2 bytes 0xAA55) + data length (1 byte) + data payload (N bytes) + CRC16 checksum (2 bytes). The acquisition device 4 is powered by a 3.7V / 500mAh lithium battery, regulated to 3.3V by an LDO to supply power to the various chips.

[0037] refer to Figure 4 This embodiment provides an interaction method for a human-computer interaction data glove, executed by a microprocessor, specifically including the following steps: a) Collect pressure signals, bending signals, and tactile signals from the finger area using a finger sensor; b) Acquire stress and deformation signals from the wrist area using a flexible strain sensor; c) Acquire overall hand movement posture signals based on posture sensors; d) Using the microprocessor of the acquisition device, the acquired pressure signal, bending signal, tactile signal, posture signal, stress signal and deformation signal are converted from analog to digital and filtered; e) Utilize a microprocessor to perform at least one interactive control operation among wrist posture dynamic compensation, grasping intention reverse recognition, virtual hand mapping dynamic compensation, force feedback feedforward-feedback composite control, and force feedback gain dynamic adjustment. f) The signal processed and calculated by the microprocessor is sent to an external terminal via a wireless communication device. The external terminal analyzes the received signal and reconstructs the real-time movement and force state of the hand.

[0038] In this implementation, the microprocessor acquires the voltage signal from the pressure sensor at a sampling rate of 1kHz via an ADC interface and converts it into a pressure value P. thumb (Unit: N); The capacitance value of the ion skin sensor 2 is converted into the bending angle B by looking up a table or fitting a curve. finger (Unit: °); The resistance value of the flexible strain sensor 6 is converted into a stress value S. wrist (Unit: kPa) Meanwhile, the raw data (acceleration, angular velocity, magnetic force) of the attitude sensor is read at 200Hz through the I2C interface, and the overall attitude angles of the hand (roll angle φ, pitch angle θ, yaw angle ψ) are obtained through quaternion attitude calculation. Furthermore, the microprocessor performs analog-to-digital conversion and filtering on the acquired raw signal. The analog-to-digital conversion uses a 12-bit ADC resolution with a reference voltage of 3.3V. The filtering process uses moving average filtering and first-order low-pass filtering to remove high-frequency noise. Furthermore, the microprocessor, according to a preset configuration, executes at least one of the following interactive control operations: dynamic wrist posture compensation, inverse recognition of grasping intention, dynamic compensation of virtual hand mapping, force feedback-force feedback composite control, and dynamic adjustment of force feedback gain. The microprocessor packages the processed data into data frames and sends them to an external terminal via a wireless communication device. After receiving the data frames, the external terminal parses and reconstructs the real-time motion and force states of the hand to drive the virtual hand. These steps achieve real-time synchronization between the virtual hand and the operator's glove-based hand movements, and reconstruct a realistic grasping force sensation based on multimodal sensor information, thereby enhancing the naturalness and immersion of human-computer interaction.

[0039] Furthermore, wrist posture dynamic compensation includes: Based on the stress and deformation signals of the wrist collected by the flexible strain sensor, the real-time attitude angle (α,β,γ) of the wrist in three-dimensional space is solved by inverse stress-strain constitutive relation. Where α is the wrist flexion angle, β is the wrist adduction / abduction angle, and γ is the wrist rotation angle; The calculated real-time wrist posture angles are used as the dynamic base coordinates of the finger kinematics model. The finger joint angles θ1, θ2, θ3, and θ4 are inversely compensated to obtain the corrected joint angles θ′1, θ′2, θ′3, and θ′4. Wherein, θ1 is the flexion angle of the metacarpophalangeal joint of the finger, θ2 is the flexion angle of the proximal interphalangeal joint of the finger, θ3 is the flexion angle of the distal interphalangeal joint of the finger, and θ4 is the abduction angle of the carpometacarpal joint of the thumb. The fingertip coordinates are recalculated based on the corrected joint angles and compared with the initial fingertip coordinates obtained through the finger kinematics model. When the error between the two exceeds a preset threshold, the sensor calibration process or multi-source data fusion algorithm is triggered.

[0040] Specifically, the microprocessor uses the wrist stress signal S acquired by the flexible strain sensor. wrist The real-time attitude angles (α, β, γ) of the wrist in three-dimensional space are obtained by inversely solving the stress-strain constitutive relation using the deformation signal (calculated by strain).

[0041] A linear constitutive model σ = E·ε is adopted; Where σ is the stress at the wrist, E is the equivalent elastic modulus, and ε is the strain at the wrist.

[0042] ε was obtained by using three flexible strain sensors placed on the dorsal, ulnar, and radial sides of the wrist. dorsal ε ulnar ε radial； Where, ε dorsal Real-time strain on the dorsal side of the wrist; ε ulnar Real-time strain on the ulnar side of the wrist; ε radial Real-time strain on the radial side of the wrist; After wearing the device, an initial posture calibration is performed first. The system collects readings from three strain sensors when the wrist is in a neutral position (i.e., fingers naturally extended, wrist not bent, and without adduction or abduction), which are denoted as ε. dorsal0 ε ulnar0 ε radial0 It serves as a reference value for zero-point calibration.

[0043] Substituting the following empirical formula, the attitude angles are calculated to satisfy... α=K α ×(ε dorsal -ε dorsal0 )+α0; Where α is the wrist flexion angle; K α ε is the buckling angle calibration coefficient; dorsal Real-time strain on the dorsal side of the wrist; ε dorsal0 α0 represents the baseline strain on the dorsal side of the wrist (measured in a neutral position); α0 is the baseline value of the flexion angle. β=K β ×(ε ulnar -ε radial )+β0; Where β is the wrist adduction angle / abduction angle; K β For the inward / outward angle calibration coefficient; ε ulnar Real-time strain on the ulnar side of the wrist; ε radial β0 represents the real-time strain on the radial side of the wrist (measured in a neutral position); β0 is the baseline value for the adduction / abduction angle. γ=K γ ×(ε radial -ε radial0 )+γ0; Where γ is the pronation-supination angle of the wrist joint around the longitudinal axis of the forearm, pronation is defined as positive and supination as negative, and γ0 = 0° in the neutral position (no pronation / supination), which is completely consistent with the neutral position reference rules of α0 and β0; K γ ε is the rotation angle calibration coefficient; radial Real-time strain on the radial side of the wrist; ε radial0 γ0 is the reference strain on the radial side of the wrist; γ0 is the reference value of the rotation angle. Specifically, the calibration methods include: calibration coefficient K α K β K γ Pre-determined angles were achieved using a dedicated angle calibration fixture. During calibration, the wearable device was placed on the subject's hand, and the wrist was driven sequentially to preset angles for each degree of freedom using the calibration fixture (flexion: 0°, 10°, 20°, ..., 90°; adduction / abduction: -30°, -20°, -10°, 0°, 10°, 20°, 30°; pronation / supination: -80°, -70°, ..., 0°, ..., 70°, 80°), and the corresponding sensor strain values ​​at each angle were recorded. The least squares method was used to linearly fit the three sets of angle-strain data (flexion angle - dorsal strain difference, adduction / abduction angle - ulnar-radial strain difference, rotation angle - radial strain difference) to obtain the corresponding calibration coefficient K. α K β K γ The calibration sample size was no less than 10 subjects, and each subject's angle sequence for all degrees of freedom was measured three times. After calibration, the accuracy of the buckling angle measurement was ±2°, the accuracy of the adduction / abduction angle measurement was ±1.5°, and the accuracy of the rotation angle measurement was ±2°. All calibration coefficients were pre-stored in the memory of the acquisition device.

[0044] Furthermore, the calculated real-time wrist posture angle is used as the dynamic base coordinates of the finger kinematics model. In traditional methods, the base coordinates of the finger kinematics model are fixed at the palm; in this embodiment, the base coordinates are updated in real time with the wrist posture. Define a base coordinate system {B} fixed at the wrist, and the transformation matrix of the finger coordinate system {F} relative to {B} is T. BF The rotation matrix R corresponding to the wrist posture angle (α,β,γ) BF satisfy

[0045] Among them, R BF R is the rotation matrix from {B} to {F}. z (γ) is the fundamental rotation matrix for rotation γ about the Z-axis; R y (β) is the fundamental rotation matrix for rotation β about the Y-axis; R x(α) is the basic rotation matrix of rotation α about the X-axis; γ is the wrist rotation angle (about the Z-axis); β is the wrist adduction / abduction angle (about the Y-axis); α is the wrist flexion angle (about the X-axis). Simultaneously, the ZYX Euler angle rotation sequence is adopted, that is, first rotate α (bending angle) around the X-axis, then rotate β (adduction / extension angle) around the Y-axis, and finally rotate γ (rotation angle) around the Z-axis to form a complete homogeneous transformation matrix.

[0046] In this embodiment, the wrist base coordinate system {B} is used as the "root" of the finger kinematics model, and the rotation matrix R is used. BF =R z (γ)·R y (β)·R x (α) describes the effect of wrist posture changes on subsequent finger coordinates. The calculation of fingertip coordinates automatically incorporates real-time wrist movements, avoiding finger positioning errors caused by wrist movements.

[0047] In this embodiment, the finger kinematic model is established based on the DH parameter method. Taking the index finger as an example, the coordinate systems of each joint and the DH parameters are defined. The homogeneous transformation matrix of the fingertip relative to the base coordinate system {B} satisfies...

[0048] Among them, T Btip T is the total transformation matrix; BF For wrist-finger root transformation; T F1 For finger root-joint 1 transformation; T 12 For joint 1 to joint 2 transformation; T 23 For joint 2 to joint 3 transformation; T 3tip For joint 3 - fingertip transformation; It should be noted that in traditional methods, the object is typically fixed to the palm, T tip =T 手掌 ·T F1 ·T 12 ·T 23 ·T 3tip Ignoring wrist movement leads to inaccurate fingertip positioning during wrist movement. In this implementation, based on the dynamic wrist coordinate T... Btip =T BF ·T F1 ·T 12 ·T 23 ·T 3tip A method is introduced to update the base coordinates in real time based on wrist posture, achieving dynamic compensation. BF The overall wrist movement (flexion, adduction / abduction, rotation) is transmitted to the entire finger, consisting of the calculated real-time wrist posture angles (α, β, γ), realizing the core of the "dynamic base coordinates"—the base coordinates of the finger kinematic model are updated in real time with the wrist posture. First, through T...BF Considering the overall changes in wrist posture, then sequentially through T F1 T 12 T 23 By considering the bending of each joint of the fingers and finally taking into account the geometric offset of the fingertip using T3tip, the precise position and orientation of the fingertip relative to the fixed wrist coordinate system are obtained, taking into account the dynamic posture compensation of the wrist.

[0049] Furthermore, substituting the corrected wrist posture T... BF Solve for the initial fingertip coordinates (x) tip ,y tip ,z tip ); Recalculate the corrected fingertip coordinates (x′) based on the corrected joint angles. tip ,y′ tip ,z′ tip ), and compared with the initial fingertip coordinates (x) obtained through the traditional finger kinematics model (fixed base coordinates). tip ,y tip ,z tip Compare them.

[0050] The position error is defined to satisfy

[0051] in, For positional error; (x′) tip -x tip (y′) represents the x-axis direction error component; tip -y tip (z′) represents the y-axis direction error component; tip -z tip ) represents the z-axis direction error component; when When the threshold is exceeded, it indicates that the impact of wrist posture changes on fingertip positioning has exceeded the allowable range, triggering sensor calibration or multi-source data fusion algorithms to correct the deviation in wrist posture calculation; the two fingertip coordinates (x′) tip ,y′ tip ,z′ tip ) and (x tip ,y tip ,z tip All calculations are based on the same finger joint angles θ1, θ2, θ3, θ4, the only difference being whether or not a wrist posture transformation matrix T is introduced. BF By comparing the differences between the two, the degree of influence of wrist movement on finger positioning can be quantified, providing a basis for dynamic compensation.

[0052] Specifically, the multi-source data fusion algorithm uses extended Kalman filtering (EKF) to fuse the wrist stress inverse solution attitude with the attitude sensor measurement values; Extended Kalman Filter (EKF) satisfies X=[α,β,γ,ω α ,ω β ,ω γ ]ᵀ Where X is the state vector; α is the wrist flexion angle; β is the wrist adduction / abduction angle; γ is the wrist rotation angle; ω α ω is the buckling angular velocity; β ω represents the angular velocity of retraction / extension. γ It is the rotational angular velocity; Z=[α imu ,β imu ,γ imu ]ᵀ Where Z is the observation vector; α imu Buckling angle measured by IMU; β imu The inward / outward angle measured by the IMU; γ imu Rotation angle measured by the IMU; In this implementation, the purpose of the extended Kalman filter (EKF) fusion is to fuse the wrist stress inverse solution of the posture (α). strain ,β strain ,γ strain ) and attitude measured by IMU (α) imu ,β imu ,γ imu By fusing these data, a more accurate and stable wrist posture estimate can be obtained.

[0053] In this embodiment, unlike the traditional forward kinematics based solely on finger sensors, dynamic compensation for finger joint angles is achieved by introducing wrist information, thus solving the problem of finger positioning errors caused by changes in wrist posture.

[0054] Furthermore, grasping intent and reverse identification includes: The microprocessor simultaneously acquires pressure signals from a pressure sensor, bending signals from an ion skin sensor at the finger joint, and stress signals and rates of change from a flexible strain sensor at the wrist, constructing a multimodal feature vector. This multimodal feature vector satisfies... ; Where F is the multimodal feature vector; P thumb B is the real-time pressure signal of the thumb. finger For real-time bending signals of finger joints; S wrist This is a real-time stress signal for the wrist. The rate of change of wrist stress; The multimodal feature vectors are input into a preset inverse reasoning model, which is a classifier based on preset logical rules to identify the operator's grasping intention from sensor signals. The inverse reasoning model outputs the operator's grasping intention based on preset logical rules. In this embodiment, the grasping intent is divided into four types: precise grasping intent I precision Continuing to hold intention I grasp Release Intent I release Hovering / Preparing Intent I hover ; The preset logical judgment rules are: If P thumb >2.5 N, B finger ∈[30∘,90∘] and If the pressure is >5 kPa / s, it is classified as Iprecision; If P thumb ≤2.5 N, B finger ∈[30∘,90∘] and S wrist If the pressure is >15 kPa, then it is determined to be I. grasp ; If B finger ∉[30∘,90∘]and S wrist If the pressure is less than 8 kPa, it is determined to be I. release ; If only B is satisfied finger If ∈[30∘,90∘] and does not satisfy any of the above conditions, then it is determined as I. hover ; Among them, P thumb B is the real-time pressure signal of the thumb. finger For real-time bending signals of finger joints; S wrist This represents the real-time stress signal of the wrist; dS wrist / dt The wrist stress change rate; T1, T2, T3, and T4 are pre-calibrated judgment thresholds, θ min θ max To preset the joint angle range under the grasping posture; Specifically, in this embodiment, the threshold is set as follows: T1 = 2.5 N (thumb pressure threshold); T2 = 5 kPa / s (wrist force rate threshold); T3 = 15 kPa (wrist sustained grip stress threshold); T4 = 8 kPa (wrist release stress threshold); θ min =30° (lower limit of finger flexion); θ max =90° (upper limit of finger bending); Based on the grasping intent obtained from the reverse reasoning model, the virtual hand in the virtual environment is automatically controlled to perform corresponding actions; The microprocessor sends control commands to the virtual environment via a wireless communication device. The command format is: command code (1 byte) + intent code (1 byte). The intent code is defined as follows: 0x01: Precise grip intention; 0x02: Continuous grip intention; 0x03: Release intention; 0x04: Hover / preparation intention; The corresponding actions include: When a precise grasping intention is recognized, the virtual hand is controlled to enter force feedback mode and force feedback adjustment is enabled. When an intention to hold is recognized, the virtual hand is controlled to maintain the current gripping state and enter force-holding mode; When a release intention is recognized, the virtual hand is controlled to perform a release action and enters position follow mode; When the system detects a hovering or preparation intention, it controls the virtual hand to only track the real-time position and posture of the glove body without applying a gripping force.

[0055] In this embodiment, unlike traditional intention recognition based solely on finger bending signals, reverse reasoning and accurate recognition of the operator's intention are achieved by introducing thumb pressure information and wrist stress information.

[0056] Furthermore, the virtual hand mapping dynamic compensation includes: The finger joint length ratio between the main body of the glove and the virtual hand is obtained in advance, and the ratio satisfies... l′ ᵢ,ⱼ =k ᵢ,ⱼ ×l ᵢ,ⱼ ; Among them, l′ ᵢ,ⱼ k is the mapping length of the virtual hand's corresponding knuckles. ᵢ,ⱼ This is the length proportionality coefficient for the corresponding finger joint; l ᵢ,ⱼ The actual length of the j-th phalanx of the i-th finger corresponding to the main body of the glove; In this embodiment, the virtual hand uses the ShadowHand dexterous hand model, with a fixed finger joint length. ShadowHand is a dexterous hand manufactured by ShadowRobot Company in the UK. The operator's finger joint length is obtained through one of the following methods: Method 1: Actual measurement (using calipers to measure the length of each finger joint of the operator); Method 2: Based on statistical data of human anatomy, estimate according to the operator's hand length; proportionality coefficient k i,j It can be calculated using the following formula: ; Where, k ᵢ,ⱼ This is the length proportionality coefficient for the corresponding finger joint; l′ ᵢ,ⱼ The mapping length of the virtual hand's corresponding knuckles; lᵢ,ⱼ The actual length of the j-th phalanx of the i-th finger corresponds to the main body of the glove; "fixed" means that the phalanx length of the virtual hand is a predetermined constant value that does not change with the operator and is the basis for realizing the mapping compensation from human hand to virtual hand. For example, for the index finger, if the length of the metacarpophalangeal joint in the virtual hand model is 40mm and the operator's measurement is 50mm, then k 1,1 =0.8; If the virtual hand length of the proximal interphalangeal joint is 25mm, and the operator's measurement is 30mm, then k 1,2 =0.85; If the virtual hand length of the distal interphalangeal joint is 22mm, and the operator's measurement is 25mm, then k 1,3 =0.9. The coefficients for the middle finger and other fingers are calculated similarly. It should be noted that the above virtual hand model is only an illustrative example and does not constitute a limitation of the present invention. In practical applications, the knuckle length parameters can be obtained in advance according to the specific virtual hand model selected, and the proportional coefficients can be calculated using the same method.

[0057] Introducing the wrist stress compensation coefficient η(S) wrist The above proportional relationship is dynamically adjusted, and the adjusted relationship satisfies... l′ ᵢ,ⱼ =η(S wrist )×k ᵢ,ⱼ ×l ᵢ,ⱼ ; Wherein, η(S) wrist )=1+μ×(S wrist -S0) / S0, where S0 is the reference stress value of the wrist at rest, μ is the pre-calibrated compensation coefficient, and S wrist The wrist stress signal is collected in real time by a flexible strain sensor; The compensation coefficient η(S) is dynamically adjusted based on real-time wrist stress signals. wrist When wrist stress increases, η(S) wrist When η(S) > 1, the mapping ratio of the virtual finger joint is increased accordingly; when the wrist stress decreases, η(S) > 1. wrist If ) < 1, the mapping ratio of the virtual finger joints will be reduced accordingly; Substitute the dynamically compensated virtual finger joint length into the fingertip mapping equation based on the DH parameter method to resolve the joint angle information of the virtual hand.

[0058] Specifically, inverse kinematics is used to solve the problem, and the objective function satisfies ; Where, min represents the joint angle combination that minimizes the total error; P target P represents the target position of the operator's fingertip. forward The fingertip position calculated using the forward kinematics of the virtual hand; This refers to the positional error of a single fingertip; Let be the length of the virtual finger joint; θ be the angle of the virtual hand joint. The physical meaning of the objective function is to find a set of virtual hand joint angles θ such that the virtual fingertip positions P of all fingers are... forward As close as possible to the operator's actual fingertip target position P target The algorithm aims to minimize the total squared error between the two. The solution employs a cyclic coordinate descent (CCD) method, where each iteration sequentially optimizes the angle of each joint while fixing the other joints, gradually decreasing the objective function value. The algorithm terminates when the maximum number of iterations is reached or the change in fingertip position between two consecutive iterations is less than the convergence accuracy. Using this method, the virtual hand joint angles that best match the operator's hand posture can be calculated in real time.

[0059] In this embodiment, unlike traditional force control that relies solely on feedback force from the glove itself, feedforward-feedback composite control is achieved by inferring the operator's expected grip force in reverse. This effectively compensates for communication delays in teleoperation and improves the real-time nature and naturalness of force interaction.

[0060] Furthermore, the force feedback-precursor composite control includes: Construct and pre-train a deep learning-based inverse reasoning network, taking pressure and bending signals collected by finger sensors and wrist stress signals collected by flexible strain sensors as inputs, and output the operator's expected grip force. The whole is represented as "the fixed length of the j-th segment of the i-th finger of the virtual hand", which is the benchmark constant in the mapping compensation algorithm.

[0061] In this embodiment, the network structure is a three-layer fully connected neural network (input layer, hidden layer, output layer): Input layer: 6 neurons (P) thumb, B finger ,S wrist (and the angle of the three fingers). Hidden layer 1: 64 neurons, ReLU activation function; Hidden layer 2: 32 neurons, ReLU activation function; Output layer: 1 neuron, linearly activated, outputs the expected gripping force F. pred (Unit: N); Furthermore, the training dataset consisted of 50,000 sensor data sets collected from 10 volunteers across 50 different grasping tasks. These data sets were divided into training and testing sets in an 8:2 ratio. The mean squared error (MSE) loss function was used, the Adam optimizer was employed, the learning rate was 0.001, and the training lasted for 100 epochs. In a virtual environment, before the virtual hand corresponding to the glove body comes into contact with the interactive object, the expected gripping force is used as a feedforward control quantity and preloaded into the microprocessor via a wireless communication device to pre-establish the corresponding virtual output force. Among them, force feedback actuators include, but are not limited to, miniature cylinders; Specifically, the microprocessor is based on F pred The DAC outputs a control voltage to drive a force feedback actuator, thus pre-establishing a corresponding virtual output force. When the virtual hand interacts with the interactive object, the real-time force feedback information from the virtual environment is fused with the expected gripping force to obtain the final output force, which satisfies... F out =λ1×F pred +(1-λ1)×F fb ; Among them, F out λ1 is the final output force; λ1 is the fusion coefficient, dynamically adjusted based on the wireless communication delay and the rate of change of wrist stress; F pred For expected grip strength; F fb For real-time force information; In this embodiment, the fusion coefficient is dynamically adjusted based on the wireless communication delay time and the wrist stress change rate to satisfy... ; Where λ1 is the fusion coefficient; t delay Let τ be the wireless communication delay time, τ be the time constant, e be the natural constant, and σ(·) be the Sigmoid function. λ1 represents the rate of change of wrist stress. When the communication delay is large or the operator exerts force quickly, λ1 increases, and feedforward control dominates, compensating for the delay effect in advance. When the communication delay is small or the operator exerts force slowly, λ1 decreases, and feedback control dominates, ensuring the accuracy of force perception interaction. Through this dynamic adjustment mechanism, a smooth transition and adaptive optimization of feedforward-feedback composite control are achieved.

[0062] Furthermore, dynamic adjustment of the force feedback gain includes: A mapping relationship between wrist stress signal and feedback force gain coefficient is established in advance, and the mapping relationship satisfies ; Among them, K f The feedback force gain coefficient; f(·) is the function symbol; S wrist S0 is the wrist stress signal; S0 is the reference stress value of the wrist at rest, and λ2 is the pre-calibrated adjustment coefficient. In this embodiment, the actual output force of the adjustment mechanism satisfies F actual =Kf ×F base , of which F actual This represents the actual output force; K f F is the feedback force gain coefficient; base The fundamental feedback force for virtual environment computing; Specifically, through the above adjustment mechanism, the feedback force gain is dynamically adjusted based on the real-time collected wrist stress signal. When S wrist When K increases, f >1 corresponds to an increase in actual output feedback force, simulating the real force feeling of gripping a heavy object; when S wrist When K decreases, f <1 corresponds to a reduction in the actual output feedback force, simulating the force sensation of grasping or releasing a light object; In this embodiment, the force feedback actuator uses a miniature cylinder (FESTOEG-4-10-PK-2). Its piston diameter D = 4 mm. The microprocessor adjusts the output force F based on the actual force. actual Calculate the required gas supply pressure P, and ensure that the pressure value P satisfies... ; Among them, F actual The actual output force is given by π; D is the cylinder piston diameter (unit: mm); π is pi; f is the sliding friction resistance (unit: N). The microprocessor converts the calculated pressure value P into a corresponding analog voltage or PWM duty cycle, which in turn controls a proportional pressure valve (SMCITV0030-3N) to regulate the cylinder's intake pressure. The input voltage of this proportional pressure valve is linearly related to the output air pressure, enabling precise air pressure regulation. Ultimately, the cylinder generates a target force F. actual A consistent thrust is applied to the operator's fingers, simulating a realistic feeling of force.

[0063] In this embodiment, unlike traditional fixed-gain force feedback, dynamic adjustment of the feedback force is achieved by introducing wrist stress information, making the force interaction more natural and realistic.

[0064] For example: The operator first puts on data gloves and grabs a glass in a virtual environment; Initially, the operator's hand is relaxed, and the wrist is in a neutral position. A flexible strain sensor acquires the stress signal S. wrist =5kPa (S0), the hand posture angle measured by the attitude sensor is (0°, 0°, 0°); When the operator reaches out, they bring their hand close to the virtual glass. The microprocessor performs dynamic wrist posture compensation, updating the dynamic base coordinates in real time to ensure the virtual hand's position accurately follows. When hovering is about to start, the operator's hand hovers above the glass, with the fingers in a grasping posture (B finger ∈[30°, 90°]), but the thumb does not exert pressure (P thumb = 0.5N < T1), and the wrist stress does not increase significantly (S wrist = 6kPa < T3). The reverse inference model determines it as "hovering / preparing intention", and controls the virtual hand to only track the position without applying grasping force; When precisely grasping, the operator holds the glass and the thumb exerts pressure (P thumb = 3.2N > T1), and the wrist generates force ( ). The reverse inference model determines it as "precise grasping intention", and the virtual hand enters the force feedback mode. At the same time, the virtual hand mapping dynamic compensation calculates the compensation coefficient η = 1 + 0.2×(18 - 5) / 5 = 1.52 according to the wrist stress S wrist = 18kPa, and increases the virtual finger joint mapping ratio to make the grasping posture more natural; During force feedback, when the virtual hand touches the glass, the virtual environment feedbacks force sense information F<00了0231>= 2N. At the same time, the reverse inference network outputs the expected grasping force F pred = 2.5N. According to the communication delay t delay = 15ms, the fusion coefficient λ1 = 0.6 is calculated, and the final output force F out = 0.6×2.5 + 0.4×2 = 2.3N. The force feedback gain is dynamically adjusted according to the current wrist stress S wrist = 18kPa, and the gain K f = 1 + 0.3×(18 - 5) / 5 = 1.78 is calculated, and the actual output force is amplified to F actual = 1.78×2.3 ≈ 4.1N, simulating the real force feeling of holding the glass; When releasing, the operator loosens the glass, the thumb pressure drops to 0.5N (< T1), the wrist stress drops to 6kPa (< T4), and the fingers straighten (B finger ∉[30°, 90°]). The reverse inference model determines it as "release intention", and controls the virtual hand to perform the release action and enter the position following mode.

[0065] It can be understood that the content in an embodiment of a human - computer interaction data glove as Figure 1-3 shown is applicable to an embodiment of a human - computer interaction data glove interaction method. The functions specifically implemented in this embodiment of the human - computer interaction data glove interaction method are the same as those in an embodiment of a human - computer interaction data glove as Figure 1-3 shown, and the beneficial effects achieved are also the same as those in an embodiment of a human - computer interaction data glove as Figure 1-3 shown.

[0066] It should be noted that the information interaction and execution process between the above systems are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0067] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0068] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0069] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0070] In the embodiments disclosed in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0071] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

Claims

1. A human-computer interaction data glove, comprising a glove body (1), characterized in that, The glove body (1) is equipped with a finger sensor on the finger part; The glove body (1) has a collection device (4) on the outside of the wrist. The glove body (1) has an inner lining fold (5) inside the wrist area, and a flexible strain sensor (6) is provided on the inner lining fold (5). The acquisition device (4) is electrically connected to the finger sensor and the flexible strain sensor (6) respectively.

2. The human-computer interaction data glove according to claim 1, characterized in that, The finger sensor includes a pressure sensor (3) located at the thumb pad of the finger and an ion skin sensor (2) located at the bent joint of the finger.

3. The human-computer interaction data glove according to claim 1, characterized in that, The inner liner fold (5) adopts an embedded fold structure, and the flexible strain sensor (6) is embedded in the folded interlayer of the inner liner fold (5).

4. The human-computer interaction data glove according to claim 1, characterized in that, The acquisition device (4) includes a circuit board (7), on which a microprocessor (8), an attitude sensor (9), and a wireless communication device (10) are integrated. The microprocessor (8) is electrically connected to the finger sensor, the flexible strain sensor (6), the posture sensor (9), and the wireless communication device (10), respectively.

5. An interaction method for a human-computer interaction data glove, characterized in that, Includes the following steps: a) Collect pressure signals, bending signals, and tactile signals from the finger area using a finger sensor; b) Acquire stress and deformation signals from the wrist area using a flexible strain sensor; c) Acquire overall hand movement posture signals based on posture sensors; d) Using a microprocessor, the collected pressure signals, bending signals, tactile signals, posture signals, stress signals, and deformation signals are converted from analog to digital and filtered. e) Utilize a microprocessor to perform at least one interactive control operation among wrist posture dynamic compensation, grasping intention reverse recognition, virtual hand mapping dynamic compensation, force feedback feedforward-feedback composite control, and force feedback gain dynamic adjustment. f) The signal processed and calculated by the microprocessor is sent to an external terminal via a wireless communication device. The external terminal analyzes the received signal and reconstructs the real-time movement and force state of the hand.

6. The interaction method of a human-computer interaction data glove according to claim 5, characterized in that, The wrist posture dynamic compensation includes: Based on the stress and deformation signals of the wrist collected by the flexible strain sensor, the real-time posture angles (α, β, γ) of the wrist in three-dimensional space are solved by inverse stress-strain constitutive relations. Where α is the wrist flexion angle, β is the wrist adduction / abduction angle, and γ is the wrist rotation angle; The calculated real-time wrist posture angles are used as the dynamic base coordinates of the finger kinematics model. The finger joint angles θ1, θ2, θ3, and θ4 are inversely compensated to obtain the corrected joint angles θ′1, θ′2, θ′3, and θ′4. Wherein, θ1 is the flexion angle of the metacarpophalangeal joint of the finger, θ2 is the flexion angle of the proximal interphalangeal joint of the finger, θ3 is the flexion angle of the distal interphalangeal joint of the finger, and θ4 is the abduction angle of the carpometacarpal joint of the thumb. The fingertip coordinates are recalculated based on the corrected joint angles and compared with the initial fingertip coordinates obtained through the finger kinematics model. When the error between the two exceeds a preset threshold, the sensor calibration process or multi-source data fusion algorithm is triggered.

7. The interaction method of a human-computer interaction data glove according to claim 6, characterized in that, The reverse recognition of grasping intent includes: Based on the pressure signal from the pressure sensor, the bending signal from the ion skin sensor at the finger joint, and the stress signal and rate of change from the flexible strain sensor at the wrist, a multimodal feature vector is constructed. This multimodal feature vector satisfies... ； Where F is the multimodal feature vector; P thumb B is the real-time pressure signal of the thumb. finger For real-time bending signals of finger joints; S wrist This is a real-time stress signal for the wrist. The rate of change of wrist stress; The multimodal feature vectors are input into a preset reverse inference model, which outputs the operator's grasping intention based on preset logical rules. Based on the grasping intent obtained from the reverse reasoning model, the virtual hand in the virtual environment is automatically controlled to perform the corresponding action.

8. The interaction method of a human-computer interaction data glove according to claim 7, characterized in that, The virtual hand mapping dynamic compensation includes: The finger joint length ratio between the main body of the glove and the virtual hand is obtained in advance, and the ratio satisfies... l′ ᵢ,ⱼ =k ᵢ,ⱼ ×l ᵢ,ⱼ ； Among them, l′ ᵢ,ⱼ k represents the mapping length of the virtual hand's corresponding knuckles. ᵢ,ⱼ This is the length proportionality coefficient for the corresponding finger joint; l ᵢ,ⱼ The actual length of the j-th phalanx of the i-th finger corresponding to the main body of the glove; Introducing the wrist stress compensation coefficient η(S) wrist The above proportional relationship is dynamically adjusted, and the adjusted relationship satisfies... l′ ᵢ,ⱼ =η(S wrist )×k ᵢ,ⱼ ×l ᵢ,ⱼ ; Wherein, η(S) wrist )=1+μ×(S wrist -S0) / S0, where S0 is the reference stress value of the wrist at rest, μ is the pre-calibrated compensation coefficient, and S wrist The wrist stress signal is collected in real time by a flexible strain sensor; The compensation coefficient η(S) is dynamically adjusted based on real-time wrist stress signals. wrist When wrist stress increases, η(S) wrist When η(S) > 1, the mapping ratio of the virtual finger joint is increased accordingly; when the wrist stress decreases, η(S) > 1. wrist If ) < 1, the mapping ratio of the virtual finger joints will be reduced accordingly; Substitute the dynamically compensated virtual finger joint length into the fingertip mapping equation based on the DH parameter method to resolve the joint angle information of the virtual hand.

9. The interaction method of a human-computer interaction data glove according to claim 8, characterized in that, The force feedback-precursor composite control includes: Construct and pre-train a deep learning-based inverse reasoning network, taking pressure and bending signals collected by finger sensors and wrist stress signals collected by flexible strain sensors as inputs, and output the operator's expected grip force. In the virtual environment, before the virtual hand corresponding to the glove body comes into contact with the interactive object, the expected gripping force is preloaded into the microprocessor as a feedforward control quantity, and the corresponding virtual output force is pre-established. When the virtual hand interacts with the interactive object, the real-time force feedback information from the virtual environment is fused with the expected gripping force to obtain the final output force, which satisfies... F out =λ1×F pred +(1-λ1)×F fb ; Among them, F out λ1 is the final output force; λ1 is the fusion coefficient, dynamically adjusted based on the wireless communication delay and the rate of change of wrist stress; F pred For expected grip strength; F fb This provides real-time force information.

10. The interaction method of a human-computer interaction data glove according to claim 9, characterized in that, The dynamic adjustment of the force feedback gain includes: A mapping relationship between wrist stress signal and feedback force gain coefficient is established in advance, and the mapping relationship satisfies ； Among them, K f The feedback force gain coefficient; f(·) is the function symbol; S wrist S0 is the wrist stress signal; S0 is the reference stress value of the wrist at rest, and λ2 is the pre-calibrated adjustment coefficient. The feedback force gain is dynamically adjusted based on the real-time collected wrist stress signal. wrist When K increases, f >1 corresponds to an increase in actual output feedback force, simulating the real force feeling of gripping a heavy object; when S wrist When K decreases, f <1 corresponds to a reduction in the actual output feedback force, simulating the force felt when grasping or releasing a light object.

Images