Multi-modal feedback bionic hand and control method thereof

Abstract

The invention provides a multi-modal feedback bionic hand and a control method thereof, and relates to the technical field of bionic hands, the multi-modal feedback bionic hand comprises a skeleton structure layer, a driving execution layer, a bionic skin layer, a sensing monitoring layer and a processing module; the skeleton structure layer comprises a metacarpal bone and at least one phalanx; the driving execution layer comprises at least one phalanx driving unit, and each phalanx driving unit comprises a driving wire and a shape memory alloy spring; the bionic skin layer is used for wrapping the skeleton structure layer and the driving execution layer; the sensing monitoring layer comprises angle sensing units and fiber bragg grating sensing units, the angle sensing units are arranged at all joints of the phalanges, and the fiber bragg grating sensing units are arranged in the bionic skin layer; and the processing module is used for receiving the multi-mode sensing signal output by the sensing monitoring layer and controlling the driving signal output to the driving execution layer according to the multi-mode sensing signal. The bionic hand has high response speed and motion stability.

Description

Technical Field

[0001] This application relates to the field of bionic hand technology, and in particular to a multimodal feedback bionic hand and its control method. Background Technology

[0002] Worldwide, tens of millions of people who have suffered traumatic unilateral upper limb amputations face significant challenges in their daily lives. Wearable prosthetic hands are considered the ultimate solution for restoring hand function.

[0003] Currently, most existing mechanical prostheses and bionic hands use electric motors, hydraulic systems, or pneumatic devices as their driving sources. While these traditional driving methods can achieve a certain degree of hand movement, they suffer from problems such as complex structure, heavy weight, high power consumption, slow response, and insufficient bionicity, resulting in insufficient comfort and limited functionality of the bionic hand. Summary of the Invention

[0004] In view of this, this application proposes a multimodal feedback bionic hand and its control method.

[0005] In a first aspect, this application provides a multimodal feedback bionic hand, comprising: a skeletal structure layer, a drive execution layer, a bionic skin layer, a sensing and monitoring layer, and a processing module; The skeletal structure layer includes metacarpals and at least one phalanx, with each phalanx movably connected to the metacarpal. The drive execution layer includes at least one finger bone drive unit. Each finger bone drive unit includes a drive line and a shape memory alloy spring. The drive line is arranged along the extension direction of the finger bone. The shape memory alloy spring is connected to the drive line. The finger bone drive unit is used to stretch the drive line through the energized phase change contraction characteristics of the shape memory alloy spring to drive the flexion and extension of the corresponding finger bone. The bionic skin layer is used to cover the skeletal structure layer and the drive execution layer; The sensing and monitoring layer includes an angle sensing unit and a Bragg fiber grating sensing unit. The angle sensing unit is disposed at each joint of the phalanx, and the Bragg fiber grating sensing unit is disposed inside the bionic skin layer. The measuring points of the Bragg fiber grating sensing unit cover the metacarpals and each of the phalanges. The processing module is used to receive the multimodal sensing signal output by the sensing and monitoring layer, and control the driving signal output to the driving execution layer according to the multimodal sensing signal.

[0006] In one embodiment, the multimodal sensing signal includes an angle monitoring signal and a grating monitoring signal; the processing module is further configured to determine phalanx bending information based on the angle monitoring signal, determine stress information of each of the measuring points based on the grating monitoring signal, and control the driving signal output to the driving execution layer based on the phalanx bending information and the stress information of each of the measuring points.

[0007] In one embodiment, the processing module is further configured to determine whether the bionic hand is in contact with a foreign object based on the stress information of each of the measuring points, and when it is determined that the hand is in contact with a foreign object, to judge the object characteristics based on the grating monitoring signals of each of the measuring points, and to determine the target pressure sequence and the target angle sequence based on the object characteristics; to acquire in real time the first error information between each joint angle and the target angle sequence, and the second error information between the stress information of each measuring point and the target pressure sequence, and to adjust the driving current and heating time of each of the shape memory alloy springs using a fuzzy PID algorithm based on the first error information and the second error information.

[0008] In one embodiment, each measuring point is provided with a set of Bragg fiber gratings. Each set of Bragg fiber gratings includes two Bragg fiber gratings corresponding to different center wavelengths. One Bragg fiber grating is directly disposed in the bionic skin layer to sense the superposition information of strain and temperature, and the other Bragg fiber grating is placed in a strain isolation yarn tube to achieve strain isolation.

[0009] In one embodiment, the number of finger bones is five, and the Bragg fiber grating sensing unit includes multiple measuring optical fibers; Each of the finger bones is equipped with a corresponding measuring optical fiber, and the metacarpal bone region is equipped with four measuring optical fibers. Each optical fiber in the finger bone region is connected in series with three sets of Bragg fiber gratings, and each optical fiber in the palm region is connected in series with three sets of Bragg fiber gratings.

[0010] In one embodiment, the phalanx includes a proximal phalanx, a middle phalanx, and a distal phalanx, wherein the middle phalanx and the distal phalanx are connected by a distal interphalangeal joint, the proximal phalanx and the middle phalanx are connected by a proximal interphalangeal joint, and the proximal phalanx and the metacarpal bone are connected by a metacarpophalangeal joint. The bending angle of the metacarpophalangeal joint is 0°-90°, the swing angle of the metacarpophalangeal joint is -20°-20°, the bending angle of the proximal interphalangeal joint is 0°-100°, and the bending angle of the distal interphalangeal joint is 0°-80°.

[0011] In one embodiment, the finger bone driving unit includes two shape memory alloy springs, which are respectively arranged on the back and the pad of the finger bone. The shape memory alloy spring at the pad is used to drive the finger bone to bend towards the palm, and the shape memory alloy spring at the back is used to drive the finger bone to extend.

[0012] In one embodiment, a guide path is provided inside or on the side of the finger bone, and the drive line is arranged along the guide path, wherein the guide path is a micro-groove or a conduit.

[0013] In one embodiment, the drive execution layer further includes a temperature sensing element for real-time monitoring of the operating temperature of the shape memory alloy; The processing module is also used to reduce the amplitude of the drive signal or stop outputting the drive signal when the operating temperature of the shape memory alloy exceeds a set threshold.

[0014] Secondly, this application also provides a control method for a multimodal feedback bionic hand, wherein the control method for the multimodal feedback bionic hand is applied to the multimodal feedback bionic hand as described in the first aspect; the control method for the multimodal feedback bionic hand includes: Acquire multimodal sensing signals output by the sensing and monitoring layer, wherein the multimodal sensing signals include angle monitoring signals and grating monitoring signals; The finger bone bending information is determined based on the angle monitoring signal, and the stress information of each measuring point is determined based on the grating monitoring signal. The drive signal output to the drive execution layer is controlled based on the phalanx bending information and the stress information of each measuring point.

[0015] The multimodal feedback bionic hand proposed in this application has the following advantages over related technologies: 1. The multimodal feedback bionic hand of this application simulates the structure of the human hand by simulating the active connection between the metacarpals and phalanges in the skeletal structure layer, thus building an ergonomic motion foundation. The drive execution layer uses shape memory alloy springs in conjunction with drive lines to drive the flexion and extension of the phalanges by using the phase change contraction characteristics of energized phase change, replacing the traditional motor and gear transmission system. This achieves miniaturization and simplification of the drive module. Moreover, the high power density of the shape memory alloy springs meets the high output requirements, and its reversible phase change characteristics can better simulate the contraction and extension behavior of human muscles. Combined with the structural design adapted to the proportion of human hand joints, the movement trajectory of the phalanges is smooth and the force output is stable, achieving adaptive gripping and cushioning protection effects.

[0016] 2. By setting up a sensing and monitoring layer and placing angle sensing units at each joint of the phalanges, and placing Bragg fiber grating sensing units inside the bionic skin layer, the angle sensing units at the joints can collect rotation angle signals in real time. The Bragg fiber grating sensing units covering the metacarpals and phalanges inside the bionic skin layer achieve pressure detection through light wavelength drift. The two types of sensing signals form a multimodal sensing input, which is received by the processing module and accurately controlled to output the driving signal to the drive execution layer. This not only achieves real-time precise control and coordinated movement of the phalanges angle, but also corrects the gripping force through stress feedback, significantly improving the response speed and movement stability of the bionic hand. It comprehensively addresses the multiple needs of flexible control, precise sensing and reliable execution, thereby improving the comfort and functional diversity of the bionic hand.

[0017] 3. By arranging a set of Bragg fiber gratings at each of the measurement points, since each set of Bragg fiber gratings includes two Bragg fiber gratings corresponding to different center wavelengths, one Bragg fiber grating is directly set in the bionic skin layer to sense the superposition information of strain and temperature, and the other Bragg fiber grating is placed in the strain isolation yarn tube to achieve strain isolation, the Bragg fiber gratings can be used to realize the detection of dual parameters of temperature and pressure. With the processing module, tactile and thermal information can be fed back in real time, and fused with the angle sensor signal to form a multimodal perception closed loop, which greatly improves the intelligent perception capability of the bionic hand. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the skeletal structure of a multimodal feedback bionic hand in one embodiment of this application; Figure 2 This is a schematic diagram of the driving structure of a multimodal feedback bionic hand in one embodiment of this application; Figure 3 This is a schematic diagram of the sensing and monitoring structure of a multimodal feedback bionic hand in one embodiment of this application; Figure 4 This is a schematic diagram of the preparation process of the biomimetic skin layer in one embodiment of this application; Figure 5 This is a schematic diagram of the fiber Bragg fiber Bragg grating structure of a Bragg fiber Bragg grating sensing unit in one embodiment of this application; Figure 6 This is a schematic diagram of the kinematic coordinate system of a finger bone in one embodiment of this application; Figure 7 This is a schematic diagram of the movement space of the phalanx in one embodiment of this application; Figure 8 This is a flowchart illustrating the control method of a multimodal feedback bionic hand in one embodiment of this application.

[0020] Explanation of reference numerals in the attached figures: 1-Skeleton structure layer, 11-Metacarpal bone, 12-Phalangeal bone, 121-Proximal phalanx, 122-Middle phalanx, 123-Distal phalanx, 2-Phalangeal bone actuation unit, 21-Actuation line, 22-Shape memory alloy spring, 3-Bionic skin layer, 4-Sensing and monitoring layer, 41-Angle sensing unit, 42-Bragg fiber optic grating sensing unit. Detailed Implementation

[0021] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0022] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0023] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0024] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0025] In some embodiments, such as Figures 1 to 3 As shown, this application provides a multimodal feedback bionic hand, including: a skeleton structure layer 1, a drive execution layer, a bionic skin layer 3, a sensing and monitoring layer 4, and a processing module (not shown in the figure).

[0026] like Figure 1 As shown, the skeletal structure layer 1 includes metacarpals 11 and at least one phalanx 12, with each phalanx 12 movably connected to the metacarpal 11. The shape and size of the metacarpals 11 and phalanx 12 can be replicated according to the parameters of real human hand joints to ensure biomechanical simulation. The phalanges can be connected by micro-hinges as joints to form a mechanical skeleton with multiple degrees of freedom.

[0027] like Figure 2 As shown, the drive execution layer includes at least one phalanx drive unit 2. Each phalanx drive unit 2 includes a drive wire 21 and a shape memory alloy (SMA) spring 22. The drive wire 21 is arranged along the extension direction of the phalanx 12, and the shape memory alloy spring 22 is connected to the drive wire 21. The phalanx drive unit 2 is used to stretch the drive wire 21 through the energized phase change contraction characteristic of the shape memory alloy spring 22 to drive the flexion and extension of the corresponding phalanx 12. It should be noted that the drive wire 21 can be made of high-strength, low-friction PE wire to ensure tensile force transmission efficiency and fatigue resistance. The shape memory alloy drive units of each phalanx are distributed longitudinally along the skeleton, and coordinated movement of multiple joints can be achieved through hierarchical control.

[0028] The drive actuation layer uses a shape memory alloy spring 22 as the core drive element, which achieves bending and extension movements of the finger joints through its energized phase change contraction effect. The shape memory alloy spring 22 is made of nickel-titanium alloy material, which has high power density and reversible deformation characteristics. Each finger bone 12 can have an independent finger bone drive unit 2 built in. In the finger bone drive unit 2, one end of the shape memory alloy spring 22 is fixed to the arm support, and the other end is connected to the drive line 21. A guide path is provided inside or on the side of the finger bone 12, and the drive line 21 is arranged along the guide path (the guide path can be a micro-groove or a conduit), that is, the drive line 21 is guided along the inside or side of the finger bone 12, and its end is fixed to the corresponding finger bone 12 skeleton. When energized, the shape memory alloy spring 22 heats up under Joule heating, transforms into an austenitic structure and undergoes axial contraction, thereby tractioning the drive line 21 to contract synchronously, driving the phalanx 12 skeleton to achieve bending action; after power is cut off and cooling occurs, the shape memory alloy spring 22 returns to its original length, the drive line 21 relaxes, and the phalanx 12 returns to its initial extended state under the elastic restoring force of the bionic skin layer 3 or the action of the built-in bias spring.

[0029] The bionic skin layer 3 is used to cover the skeleton structure layer 1 and the drive execution layer to achieve external flexible protection and multimodal sensing functions. This bionic skin layer 3 can use flexible silicone material as the substrate, as shown in the schematic diagram below. Figure 2 and Figure 3 As shown. Figure 4 As shown, the skin layer is formed by casting using a mold, which consists of a core mold and an outer mold. The palm and finger parts are prepared separately. The palm mold consists of an outer shell and a palm core mold that fit together to form the overall cavity of the palm. The finger bone 12 skin mold structure consists of a finger core mold and a finger outer shell mold, with each finger mold corresponding to an independent finger bone 12 skin structure. During molding, a cavity is formed inside the mold that matches the bionic hand skeleton to facilitate subsequent encapsulation and assembly.

[0030] like Figure 3 As shown, the sensing and monitoring layer 4 includes an angle sensing unit 41 and a Bragg fiber grating (FBG) sensing unit 42. The angle sensing unit 41 is disposed at each joint of the phalanges 12, and the Bragg fiber grating sensing unit 42 is disposed inside the bionic skin layer 3. The measuring points of the Bragg fiber grating sensing unit 42 cover the metacarpals 11 and each phalanx 12. In applications, the angle sensor can be a Hall effect angle sensor, a photoelectric encoder, or a MEMS attitude sensor. A drive channel and sensing wire groove can be reserved inside the skeleton for embedding the shape memory alloy spring 22, the angle sensing unit 41, and the Bragg fiber grating sensing unit 42.

[0031] The Bragg fiber grating sensing unit 42 can be pre-embedded during the molding process of the bionic skin layer 3 to at least achieve real-time sensing and feedback of pressure. During the fabrication process of the bionic skin layer 3, in order to achieve multi-point distributed sensing of the skin, a Bragg fiber grating (FBG) array can be pre-embedded inside the silicone during the casting stage to form the Bragg fiber grating sensing unit 42.

[0032] The processing module is used to receive the multimodal sensing signals output by the sensing and monitoring layer 4, and control the driving signals output to the driving execution layer based on the multimodal sensing signals.

[0033] In the application, the processing module can include a main control unit and a drive control unit. The main control unit adopts a multi-threaded structure, which may include threads for data acquisition, signal fusion, control output, and safety monitoring. The real-time running cycle of the processing module can be set to 5–10ms to ensure high response speed during dynamic grasping and subtle movements. Fiber optic signals and angle signals can be sampled synchronously with timestamps, and then multi-source signal fusion is achieved through Kalman filtering to improve system stability and anti-interference capability. Data can be transmitted wirelessly or via wired interfaces and can be used for external host computer display and intelligent learning algorithm training.

[0034] The drive control unit may include a PWM constant current source, a power amplifier circuit, and a current sampling resistor. The drive control unit can control each finger bone drive unit 2 through independent channels. The main control unit controls the drive control unit to output a pulse width modulation (PWM) signal based on real-time sensor feedback. By adjusting the input current using PWM, precise control of the shape memory alloy temperature and contraction stroke can be achieved. The PWM signal can then be output through a MOSFET array, enabling independent adjustment and dynamic time-sharing control of multiple finger bone drive units 2. The power supply can be a constant voltage DC power supply (e.g., an adjustable 12V–24V power supply) and may include built-in temperature protection circuitry and current overload protection module.

[0035] The aforementioned multimodal feedback bionic hand simulates the structure of the human hand by connecting the metacarpals 11 and phalanges 12 in the skeletal structure layer 1, thus establishing an ergonomic motion foundation. The drive execution layer uses shape memory alloy springs 22 in conjunction with drive lines 21 to drive the phalanges 12 to flex and extend by using the phase change contraction characteristics of energized phase change, replacing the traditional motor and gear transmission system. This achieves miniaturization and simplification of the drive module. Furthermore, the high power density of the shape memory alloy springs 22 meets the high output requirements, and their reversible phase change characteristics can better simulate the contraction and extension behavior of human muscles. Combined with a structural design that adapts to the proportions of human hand joints, the movement trajectory of the phalanges is smooth and the force output is stable, achieving adaptive gripping and cushioning protection effects. In the sensing and monitoring layer 4, the angle sensing unit 41 at the joint can collect rotation angle signals in real time. The Bragg fiber grating sensing unit 42 covering the metacarpals 11 and phalanges 12 in the bionic skin layer 3 realizes pressure detection through optical wavelength drift. The two types of sensing signals form a multimodal sensing input. After being received by the processing module, the output is precisely controlled and sent to the driving execution layer as a driving signal. This not only realizes the real-time precise control and coordinated movement of the knuckle angle, but also corrects the gripping force through stress feedback, significantly improving the response speed and movement stability of the bionic hand. It comprehensively takes into account the multiple needs of flexible control, precise sensing and reliable execution, thereby improving the comfort and functional diversity of the bionic hand.

[0036] In some embodiments, the multimodal sensing signal includes an angle monitoring signal and a grating monitoring signal; the processing module is further configured to determine the bending information of the finger bone 12 based on the angle monitoring signal, determine the stress information of each measuring point based on the grating monitoring signal, and control the driving signal output to the driving execution layer based on the bending information of the finger bone 12 and the stress information of each measuring point.

[0037] The grating monitoring signal output from sensing and monitoring layer 4 can be demodulated simultaneously using wavelength division multiplexing to obtain the reflection wavelength changes of each optical fiber. Then, the center wavelength offset (Δλ) of each grating is extracted, and the corresponding stress (σ) data is obtained by the algorithm of the main control unit.

[0038] It is understandable that angle monitoring signals can accurately reflect the rotation angle, bending rate, and movement range of each phalanx. After signal analysis and data processing, the bending information of the phalanx 12 is clarified, which can include the bending angle of each joint. Based on the bending information of the phalanx 12, it can be determined whether the degree of bending matches the expected movement trajectory, providing a basis for subsequent drive control of the movement posture.

[0039] Simultaneously, the processing module receives the grating monitoring signal output from the Bragg fiber grating sensing unit 42. The output grating monitoring signal can be demodulated simultaneously using wavelength division multiplexing to obtain the reflection wavelength changes of each fiber. Then, the center wavelength offset (Δλ) of each grating is extracted, and the stress information of each measuring point is obtained through algorithm conversion. Based on this, the processing module performs collaborative fusion analysis of the bending information of the finger bone 12 and the stress information of each measuring point. It combines motion posture data to determine whether the flexion and extension of the finger bone 12 reaches the target position and whether the movement is smooth and coordinated. It also uses stress data to determine whether the gripping force is appropriate and whether there is excessive or insufficient local force. This avoids the one-sided control caused by relying solely on motion posture or force signals. It then accurately generates drive signals adapted to the current state and outputs them to the drive execution layer. By adjusting the energizing parameters of the shape memory alloy spring 22 (such as the magnitude of the drive current and the heating time), it corrects the bending angle and gripping force of the finger bone 12 in real time. This ensures that the movement of the finger bone 12 not only conforms to the preset trajectory but also dynamically adapts according to force feedback. It achieves coordinated and precise control of motion posture and gripping force, making the bionic hand's movements smoother and its grip more reliable, significantly improving its adaptive adjustment capability and control accuracy.

[0040] In some embodiments, the processing module is further configured to determine whether the bionic hand is in contact with a foreign object based on the stress information of each measuring point, and when it is determined that the hand is in contact with a foreign object, to judge the object characteristics based on the grating monitoring signal of each measuring point, and to determine the target pressure sequence and the target angle sequence based on the object characteristics; to acquire in real time the first error information of each joint angle and the target angle sequence, and the second error information of each measuring point and the target pressure sequence, and to adjust the driving current and heating time of each shape memory alloy spring 22 using a fuzzy PID algorithm based on the first error information and the second error information.

[0041] This system can pre-establish a database based on computer vision solutions for different object features. After determining the object features, it retrieves data from the database to determine the target pressure sequence and target angle sequence. The core purpose of the multi-point Bragg fiber grating distributed measurement points is to identify the object shape, control the grasping force, and ensure grasping safety. By collecting pressure distribution data from multi-point Bragg fiber gratings on the fingertips and palm, the processing module can determine the object's contact area, force location, and geometric shape characteristics. Based on this, the processing module automatically adjusts the shape memory alloy driving force of the corresponding joints according to the real-time changes in multi-point pressure values, keeping the grasping force within a set safe range, thereby achieving stable and safe grasping control. When a sudden increase in pressure or uneven force occurs at a certain measurement point, the driving current of each finger can be adjusted according to the local and overall pressure difference to achieve compliant wrapping and adaptive force balance of the object.

[0042] The stress σ at each measuring point can be monitored in real time using Bragg fiber grating sensing units 42 (e.g., 15 fingertips + 12 palms).i and temperature T i The processing module fuses the signals from each measuring point and calculates the control quantity.

[0043] First, it can be determined whether a measuring point is in contact with an object; when the contact stress... When the pressure exceeds the trigger value, the finger bone 12 corresponding to the measuring point enters contact mode. Using the signals from each measuring point, the system determines the object's characteristics (size, hardness, softness, eccentric position) based on the distribution, and calculates the pressure distribution P(x,y) of the fingertip.

[0044]

[0045] The drive signal is output to each finger bone drive unit 2, and the grasping force is closed-loop controlled using PID control:

[0046] in, The actual current acting on the SMA spring. k p , k i and k d These are the proportional system, integral coefficient, and differential coefficient, respectively, and the pressure error. The system measures the difference between the target gripping pressure and the current actual pressure. If the pressure at a certain point suddenly increases, the controller immediately reduces the SMA current at the corresponding joint to prevent damage to the object or skin. If the pressure is too low and slippage is possible, the system increases the SMA current to increase the gripping force and prevent the object from falling.

[0047] The angle monitoring signal is filtered, amplified, and calibrated to obtain the angle change of each joint. θ 1 , θ 2 , θ 3 Similarly, for the corresponding angle adjustment, the control algorithm adopts a closed-loop fuzzy PID structure, with the input being the target angle (…). θ * and measured angle ( θ The feedback quantity is the angle deviation. e ( t )= θ *- θThe system uses an angle sensor to correct the SMA's energizing duration and current amplitude in real time, allowing the flexion angle of the finger joint to dynamically follow the target trajectory. Combined with the stress feedback signal provided by the FBG array, it can achieve compliance adjustment and adaptive compensation of gripping force. When external contact pressure increases, the control system reduces input power to buffer the action. The processing module simultaneously receives fiber optic and angle data, using a fusion algorithm to comprehensively judge the posture and force conditions of the finger 12, achieving state recognition and action determination. When the system detects that the finger 12 is in contact with a foreign object and generates a stress signal, it determines whether the contact force exceeds a safety threshold; if it does, it automatically adjusts the SMA drive current to reduce gripping force, preventing clamping overload or damage to the object.

[0048] In this embodiment, the processing module accurately determines the contact state between the bionic hand and the external object through the stress information of the measuring points, avoiding ineffective control during the non-contact stage and improving energy utilization efficiency. After the bionic hand contacts the external object, it accurately identifies the object's characteristics based on the grating monitoring signals of each measuring point, making the formulation of target pressure sequences and target angle sequences more targeted, adapting to the gripping needs of different objects, and avoiding blindly applying force that could lead to object damage or unstable gripping. By acquiring the first error of the joint angle and target angle sequence and the second error of the measuring point stress and target pressure sequence in real time, the deviation information in the control process is fully captured. Combined with the fuzzy PID algorithm, which has the advantages of both robustness and accuracy, the driving current and heating time of each shape memory alloy spring 22 are dynamically adjusted. This not only utilizes the precise adjustment capability of the PID algorithm to reduce errors, but also uses the fuzzy algorithm to adapt to the nonlinearity of the phase transition characteristics of the shape memory alloy spring 22 and the uncertainty of external interference. This effectively compensates for the adjustment lag problem of the simple PID algorithm under complex working conditions, and realizes real-time dynamic correction of angle and pressure errors. As a result, the knuckle movement trajectory is more in line with the target requirements, and the gripping force is more accurately matched to the object's bearing range. This ensures the firmness of the grip and avoids damage to the object due to excessive pressure or slippage due to insufficient pressure. This significantly improves the accuracy, dynamic response speed and stability of the bionic hand's grip control, and realizes adaptive and reliable gripping of objects with different characteristics.

[0049] In some embodiments, such as Figure 3 As shown, each measuring point is equipped with a set of Bragg fiber gratings. Each set of Bragg fiber gratings includes two Bragg fiber gratings corresponding to different center wavelengths. One Bragg fiber grating is directly placed in the bionic skin layer 3 to sense the superposition information of strain and temperature. The other Bragg fiber grating is placed in the strain isolation yarn tube to achieve strain isolation.

[0050] It is understandable that by arranging a set of Bragg fiber gratings at each measuring point, the local strain and temperature changes at each measuring point can be sensed based on the grating monitoring signals. For example, Figure 5As shown, fiber Bragg gratings can be made using single-mode PI-coated optical fibers, with periodic grating structures etched in the fiber core to reflect optical signals of specific wavelengths. Single-mode PI-coated optical fibers possess high flexibility and high-temperature resistance, capable of withstanding temperature and stress changes during the silicone curing process. After incident light propagates through the fiber into the grating region, some light is reflected. The reflected wavelength drifts with changes in external strain or temperature, and the amount of change in the reflected wavelength is proportional to the stress and temperature changes at that location. The central reflected wavelength of the Bragg fiber grating is:

[0051]

[0052] In the above formula, λ B For the reflected center wavelength, n eff The effective refractive index of the fiber core, This represents the spatial period of the grating. When the optical fiber is subjected to external stress or temperature changes... n eff and All of these will change, causing a shift in the reflected wavelength. The amount of wavelength change can be expressed as:

[0053]

[0054] The axial strain experienced by the optical fiber. The change in temperature P e The effective elastic-optical coefficient of the optical fiber. α The coefficient of thermal expansion of optical fiber. ξ The thermo-optic coefficient of the optical fiber. To achieve simultaneous monitoring and separation of temperature and stress, this embodiment employs a dual-grating decoupling design: two FBGs with different center wavelengths are connected in series on the same optical fiber. One is directly bonded to the silicone skin to sense the superposition effect of strain and temperature; the other can be placed in a PTFE capillary to achieve strain isolation, responding only to temperature changes. The wavelength changes of the two are as follows:

[0055]

[0056] The reflection wavelength shift of the two gratings is measured in real time using a spectral demodulation system. and This allows us to determine the stress and temperature changes at that location.

[0057] In the application, during the fabrication of the bionic skin layer 3, fiber optic slots and lead-out holes can be pre-reserved in the corresponding mold. The optical fibers are arranged in shallow grooves etched on the surface of the mold core and fixed with silicone adhesive. Before casting, a thin silicone layer is laid to form the substrate, and then the FBG optical fiber is placed. A second casting is then performed to cover and form the entire skin. The optical fiber is positioned 0.3–0.5 mm from the skin surface at the fingertip to ensure strain transmission sensitivity. Part of the temperature reference grating is placed inside a PTFE capillary to achieve separation and decoupling of strain and temperature. The lead-out ends of the optical fibers are concentrated at the wrist and connected to the external demodulation module through a flexible protective sleeve. A stress relief cavity is provided at the wrist to prevent fiber bending damage.

[0058] In some embodiments, such as Figure 3 As shown, there are five finger bones 12, and the Bragg fiber grating sensing unit 42 includes multiple measurement optical fibers.

[0059] Each finger bone 12 is equipped with a corresponding measuring optical fiber, and the metacarpal bone 11 area is equipped with four measuring optical fibers. Each optical fiber in the finger bone 12 area is connected in series with three sets of Bragg fiber gratings, and each optical fiber in the palm area is connected in series with three sets of Bragg fiber gratings.

[0060] It is understandable that the number of finger bones 12 is set to five, which perfectly matches the structure of the five fingers of the human hand, and can maximize the reproduction of the human hand's grasping posture and range of motion, providing a structural basis for diverse grasping actions. The Bragg fiber grating sensing unit 42 is equipped with multiple measurement optical fibers. The measurement optical fibers corresponding to the five finger bones 12 can independently collect the stress signals of each finger, avoiding signal interference between different finger bones 12 and ensuring the accuracy of single finger stress monitoring. The four measurement optical fibers arranged in the metacarpal bone 11 area can fully cover key contact areas such as the palm and palmar edge, making up for the deficiency of incomplete coverage by a single optical fiber.

[0061] Meanwhile, the design of connecting three sets of Bragg fiber gratings in each fiber in the phalanx 12 region and three sets of Bragg fiber gratings in each fiber in the palm region significantly increases the measurement point density, enabling independent and continuous stress monitoring data to be obtained from the proximal, middle, and distal ends of the phalanx 12 and different sections of the metacarpal 11. This not only achieves accurate capture of stress changes in each movement segment of the phalanx 12 and each contact area of ​​the metacarpal 11, but also effectively avoids the one-sidedness and error of single-point monitoring through the collaborative sensing of multiple sets of gratings. This high-density, zoned sensor arrangement allows the Bragg fiber grating sensor unit 42 to output more comprehensive and detailed grating monitoring signals. Combined with the angle monitoring signals from the angle sensor unit 41, it provides the processing module with multi-dimensional, comprehensive sensing data support. This enables the processing module to accurately determine the bending information of each finger bone 12 and the stress distribution in different areas. Consequently, it can adjust the drive signals of the drive execution layer accordingly, ensuring that the bionic hand can correct its movement trajectory based on the bending feedback of the finger bones 12 and accurately control the gripping force based on the zoned stress feedback during the grasping process. This significantly improves the adaptability to objects of different shapes and sizes and the precision of gripping control, making the movements more in line with the natural movement patterns of the human hand and the gripping more stable and reliable.

[0062] In some embodiments, such as Figure 1 and Figure 6 As shown, the phalanx 12 includes a proximal phalanx 121, a middle phalanx 122, and a distal phalanx 123. The middle phalanx 122 and the distal phalanx 123 are connected by a distal interphalangeal joint. The proximal phalanx 121 and the middle phalanx 122 are connected by a proximal interphalangeal joint. The proximal phalanx 121 is connected to the metacarpal bone 11 by a metacarpophalangeal joint.

[0063] Based on the structural design and the physiological range of human finger bones, the constraints are set as follows: the bending angle of the metacarpophalangeal joint is 0°-90°, the swing angle of the metacarpophalangeal joint is -20°-20°, the bending angle of the proximal interphalangeal joint is 0°-100°, and the bending angle of the distal interphalangeal joint is 0°-80°.

[0064] In applications, the metacarpophalangeal joint has two degrees of freedom, with a spatial offset between its bending axis and its pivoting axis. l 1. The flexion angle and swing angle of the metacarpophalangeal joint are respectively θ 1 and α 1. The flexion angles of the proximal and distal interphalangeal joints are respectively θ 2 and θ 3. The lengths of the proximal phalanx 121, the middle phalanx 122, and the distal phalanx 123 are respectively l 2. l 3. l 4. Based on the Denavit-Hartenberg method, the position of the fingertip in the base coordinate system can be obtained through four homogeneous transformation matrices:

[0065]

[0066] in:

[0067] After multiplication, we obtain the homogeneous transformation matrix of the fingertips relative to the base of the palm:

[0068] Where R is the attitude matrix. This is the fingertip position vector. After setting constraints based on the structural design and the physiological range of the human phalanx 12, the motion space of the phalanx 12 is obtained as follows: Figure 7 As shown, the above limitations ensure that the bionic hand conforms to the bone parameters of a real human hand, thus guaranteeing bionic accuracy.

[0069] In some embodiments, the finger bone driving unit 2 includes two shape memory alloy springs 22, which are respectively arranged on the back and the pad of the finger bone 12. The shape memory alloy spring 22 at the pad is used to drive the finger bone 12 to bend towards the palm, and the shape memory alloy spring 22 at the back is used to drive the finger bone 12 to extend.

[0070] It can be understood that two shape memory alloy springs 22 are respectively arranged on the back and the pad of the finger bone 12. The shape memory alloy spring 22 at the pad is used to drive the finger bone 12 to bend towards the palm, and the shape memory alloy spring 22 at the back is used to drive the finger bone 12 to extend. The flexion and extension movements can be completed by the pair of shape memory alloy springs 22 working together to achieve an antagonistic drive arrangement at the back and pad positions, thereby obtaining a flexibility similar to that of human finger bones 12.

[0071] In some embodiments, the drive execution layer further includes a temperature sensing element for real-time monitoring of the operating temperature of the shape memory alloy. The processing module is also configured to reduce the amplitude of the drive signal or stop outputting the drive signal when the operating temperature of the shape memory alloy exceeds a set threshold.

[0072] It is understandable that, since the phase transformation characteristics of shape memory alloys are closely related to their operating temperature, excessively high temperatures can not only lead to a decline in key properties such as elastic modulus and resilience, but may also cause material fatigue, permanent deformation, or even burnout, thereby affecting the reliability and service life of the drive actuator layer. The processing module, by monitoring temperature data in real time, promptly takes protective measures such as reducing the drive signal amplitude or stopping the output drive signal when it determines that the shape memory alloy's operating temperature exceeds a set threshold. This effectively avoids irreversible damage to the shape memory alloy caused by high temperatures, extending its service life, and also prevents problems such as decreased drive accuracy and abnormal knuckle movement caused by temperature runaway. In applications, thermally conductive silicone grease can also be filled into the shape memory alloy mounting cavity, and heat dissipation holes can be created to improve heat dissipation efficiency.

[0073] In some embodiments, this application also provides a control method for a multimodal feedback bionic hand, which is applied to a multimodal feedback bionic hand as described above. Figure 8 As shown, the control method of the multimodal feedback bionic hand includes the following steps S801 to S803.

[0074] S801: Acquire the multimodal sensing signals output by the sensing and monitoring layer. The multimodal sensing signals include angle monitoring signals and grating monitoring signals.

[0075] S802: Determine the finger bone bending information based on the angle monitoring signal, and determine the stress information of each measuring point based on the grating monitoring signal.

[0076] S803: Based on the finger bone bending information and the stress information of each measuring point, control the output of the drive signal to the drive execution layer.

[0077] In some embodiments, step S803, controlling the drive signal output to the drive execution layer based on the finger bone bending information and the stress information of each measuring point, includes: determining whether the bionic hand is in contact with a foreign object based on the stress information of each measuring point, and when it is determined that the hand is in contact with a foreign object, judging the object characteristics based on the grating monitoring signal of each measuring point, and determining the target pressure sequence and target angle sequence based on the object characteristics; acquiring the first error information of each joint angle and the target angle sequence, and the second error information of each measuring point and the target pressure sequence in real time, and adjusting the drive current and heating time of each shape memory alloy spring using a fuzzy PID algorithm based on the first error information and the second error information.

[0078] It should be noted that the control method of the multimodal feedback bionic hand provided in this application embodiment is based on the same inventive concept as the multimodal feedback bionic hand provided in this application embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned multimodal feedback bionic hand, and the repeated parts will not be described again.

[0079] In some embodiments, an electronic device provided in this application includes a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the control method of the multimodal feedback bionic hand described above.

[0080] Specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include onboard memory for caching purposes. The processor may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0081] Memory can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0082] This application also provides a non-transitory computer storage medium storing a computer program thereon, which, when executed by a processor, implements the control method of the multimodal feedback bionic hand described above. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method as described in the embodiments of this application.

[0083] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0084] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A multimodal feedback bionic hand, characterized in that, include: The system consists of a skeletal structure layer, a drive execution layer, a bionic skin layer, a sensing and monitoring layer, and a processing module. The skeletal structure layer includes metacarpals and at least one phalanx, with each phalanx movably connected to the metacarpal. The drive execution layer includes at least one finger bone drive unit. Each finger bone drive unit includes a drive line and a shape memory alloy spring. The drive line is arranged along the extension direction of the finger bone. The shape memory alloy spring is connected to the drive line. The finger bone drive unit is used to stretch the drive line through the energized phase change contraction characteristics of the shape memory alloy spring to drive the flexion and extension of the corresponding finger bone. The bionic skin layer is used to cover the skeletal structure layer and the drive execution layer; The sensing and monitoring layer includes an angle sensing unit and a Bragg fiber grating sensing unit. The angle sensing unit is disposed at each joint of the phalanx, and the Bragg fiber grating sensing unit is disposed inside the bionic skin layer. The measuring points of the Bragg fiber grating sensing unit cover the metacarpals and each of the phalanges. The processing module is used to receive the multimodal sensing signal output by the sensing and monitoring layer, and control the driving signal output to the driving execution layer according to the multimodal sensing signal.

2. The multimodal feedback bionic hand as described in claim 1, characterized in that, The multimodal sensing signal includes an angle monitoring signal and a grating monitoring signal; the processing module is further configured to determine the finger bone bending information based on the angle monitoring signal, determine the stress information of each of the measuring points based on the grating monitoring signal, and control the driving signal output to the driving execution layer based on the finger bone bending information and the stress information of each of the measuring points.

3. The multimodal feedback bionic hand as described in claim 2, characterized in that, The processing module is further configured to determine whether the bionic hand is in contact with a foreign object based on the stress information of each measuring point, and when it is determined that the hand is in contact with a foreign object, to judge the object characteristics based on the grating monitoring signal of each measuring point, and to determine the target pressure sequence and the target angle sequence based on the object characteristics; to acquire in real time the first error information between each joint angle and the target angle sequence, and the second error information between the stress information of each measuring point and the target pressure sequence; and to adjust the driving current and heating time of each shape memory alloy spring using a fuzzy PID algorithm based on the first error information and the second error information.

4. The multimodal feedback bionic hand as described in claim 1, characterized in that, Each measurement point is provided with a set of Bragg fiber gratings. Each set of Bragg fiber gratings includes two Bragg fiber gratings with different center wavelengths. One Bragg fiber grating is directly placed in the bionic skin layer to sense the superposition of strain and temperature information, and the other Bragg fiber grating is placed in a strain isolation yarn tube to achieve strain isolation.

5. The multimodal feedback bionic hand as described in claim 4, characterized in that, The number of finger bones is five, and the Bragg fiber grating sensing unit includes multiple measuring optical fibers; Each of the finger bones is equipped with a corresponding measuring optical fiber, and the metacarpal bone region is equipped with four measuring optical fibers. Each optical fiber in the finger bone region is connected in series with three sets of Bragg fiber gratings, and each optical fiber in the palm region is connected in series with three sets of Bragg fiber gratings.

6. The multimodal feedback bionic hand as described in claim 1, characterized in that, The phalanges include proximal phalanges, middle phalanges and distal phalanges. The middle phalanges and the distal phalanges are connected by distal interphalangeal joints. The proximal phalanges and the middle phalanges are connected by proximal interphalangeal joints. The proximal phalanges and the metacarpals are connected by metacarpophalangeal joints. The bending angle of the metacarpophalangeal joint is 0°-90°, the swing angle of the metacarpophalangeal joint is -20°-20°, the bending angle of the proximal interphalangeal joint is 0°-100°, and the bending angle of the distal interphalangeal joint is 0°-80°.

7. The multimodal feedback bionic hand as described in claim 1, characterized in that, The finger bone driving unit includes two shape memory alloy springs, which are respectively arranged on the back and the pad of the finger bone. The shape memory alloy spring at the pad is used to drive the finger bone to bend towards the palm, and the shape memory alloy spring at the back is used to drive the finger bone to extend.

8. The multimodal feedback bionic hand as described in claim 1, characterized in that, The finger bone has a guide path inside or on the side, and the drive line is arranged along the guide path, wherein the guide path is a micro groove or a conduit.

9. The multimodal feedback bionic hand as described in claim 1, characterized in that, The drive execution layer also includes a temperature sensing element, which is used to monitor the operating temperature of the shape memory alloy in real time; The processing module is also used to reduce the amplitude of the drive signal or stop outputting the drive signal when the operating temperature of the shape memory alloy exceeds a set threshold.

10. A control method for a multimodal feedback bionic hand, characterized in that, The control method for the multimodal feedback bionic hand is applied to the multimodal feedback bionic hand as described in any one of claims 1 to 9; The control method for the multimodal feedback bionic hand includes: Acquire multimodal sensing signals output by the sensing and monitoring layer, wherein the multimodal sensing signals include angle monitoring signals and grating monitoring signals; The finger bone bending information is determined based on the angle monitoring signal, and the stress information of each measuring point is determined based on the grating monitoring signal. The drive signal output to the drive execution layer is controlled based on the phalanx bending information and the stress information of each measuring point.

Images