Glove system

Abstract

A glove system (100), comprising a glove body (110), and multiple first strain sensors (120) and multiple second strain sensors (130) arranged on the glove body (110). When a user wears the glove body (110), each first strain sensor (120) among the multiple first strain sensors (120) is located at an interphalangeal joint of the user, and is configured to measure the deformation of the interphalangeal joint in a single degree of freedom; and each second strain sensor (130) among the multiple second strain sensors (130) is located at a metacarpophalangeal joint, and is configured to measure the deformation of the metacarpophalangeal joint in two degrees of freedom, wherein the single degree of freedom and the two degrees of freedom each comprise a degree of freedom corresponding to finger flexion.

Description

A glove system

[0001] Cross-references

[0002] This specification claims priority to the international application with application number PCT / CN2023 / 124295 filed on October 12, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This specification relates to the field of sensor technology, and in particular to a glove system equipped with multiple sensors and a method for recognizing gestures based on the glove system. Background Art

[0004] With the gradual maturity of AR / VR technology and the rise of the metaverse concept, smart electronic devices are placing higher demands on human-computer interaction technology. Flexible angle sensors can be easily integrated into wearable devices such as smart clothing and smart gloves, enabling accurate recognition and reproduction of human movements. As one of the key underlying technologies of the metaverse, they have attracted extensive attention and research. To accurately recognize and reproduce hand movements, it is necessary to capture hand movements, especially finger movements, in multiple degrees of freedom.

[0005] Therefore, it is hoped to propose an electronic device that can improve the accuracy of hand motion acquisition while controlling costs.

[0006] Summary of the Invention

[0007] One of the embodiments of the present specification provides a glove system, comprising: a glove body and a plurality of first strain sensors and a plurality of second strain sensors arranged on the glove body, wherein, when a user wears the glove body, each of the plurality of first strain sensors is located at an interphalangeal joint of the user and is configured to measure the deformation of the interphalangeal joint in a single degree of freedom, and each of the plurality of second strain sensors is located at a metacarpophalangeal joint and is configured to measure the deformation of the metacarpophalangeal joint in two degrees of freedom, wherein the single degree of freedom and the two degrees of freedom both include a degree of freedom corresponding to finger bending.

[0008] In some embodiments, the plurality of first strain sensors and the plurality of second strain sensors together provide data reflecting deformation of the user's fingers in no less than 15 degrees of freedom.

[0009] In some embodiments, the glove system further includes two inertial sensors, wherein when the user wears the glove body, the two inertial sensors are respectively located on both sides of the user's wrist joint and are configured to jointly measure the movement of the wrist joint in three degrees of freedom.

[0010] In some embodiments, the glove system further includes a third strain sensor located at the user's wrist joint and an inertial sensor located outside the wrist joint.

[0011] In some embodiments, the first strain sensor comprises an inductive sensor, and the inductance of the inductive sensor changes with the deformation of the interphalangeal joint.

[0012] In some embodiments, the second strain sensor includes a plurality of capacitive structures, and the capacitances of the plurality of capacitive structures change respectively as the metacarpophalangeal joint changes in various degrees of freedom.

[0013] In some embodiments, the second strain sensor includes: a flexible substrate and a first capacitor structure and a second capacitor structure, wherein the first capacitor structure and the second capacitor structure each include a multilayer structure arranged on the same side surface in the thickness direction of the flexible substrate, and the layers of each multilayer structure are stacked along the thickness direction.

[0014] In some embodiments, the glove body includes fabric wrapped around each finger of the user and fabric located between adjacent fingers, wherein the elastic modulus of the fabric between adjacent fingers is smaller than the elastic modulus of the fabric on each finger.

[0015] In some embodiments, the fabric between adjacent fingers has at least one of a slit structure, an elastic fabric structure, or a pleated structure.

[0016] In some embodiments, the plurality of first strain sensors are arranged only at the interphalangeal joints of each finger of the user closest to the metacarpophalangeal joint.

[0017] In some embodiments, the glove system further includes a processor configured to: read data associated with the plurality of first strain sensors and the plurality of second strain sensors; and determine a gesture of the user based on the data.

[0018] In some embodiments, the processor is configured to: output a gesture representation corresponding to the user's gesture on a display interface; and output feedback information according to a distance between the gesture representation and a preset gesture.

[0019] In some embodiments, the processor is configured to: read a mapping relationship between a gesture and a voice data packet; and play voice information based on the user's gesture and the mapping relationship.

[0020] In some embodiments, the processor is configured to: read a mapping relationship between a gesture and an execution instruction; and execute the instruction based on the user's gesture and the mapping relationship.

[0021] One of the embodiments of the present specification provides a gesture reproduction method, comprising: obtaining sensor data of a user's hand based on a glove system, wherein the glove system includes a glove body and a plurality of first strain sensors and a plurality of second strain sensors arranged on the glove body, when the user wears the glove body, each of the plurality of first strain sensors is located at an interphalangeal joint of the user and is configured to measure the deformation of the interphalangeal joint in a single degree of freedom, each of the plurality of second strain sensors is located at a metacarpophalangeal joint and is configured to measure the deformation of the metacarpophalangeal joint in two degrees of freedom, the single degree of freedom and the two degrees of freedom both including a degree of freedom corresponding to finger bending; determining the user's gesture based on the sensor data; outputting a gesture representation corresponding to the gesture on a display interface; and outputting feedback information based on the distance between the gesture representation and a preset gesture.

[0022] In some embodiments, the plurality of first strain sensors and the plurality of second strain sensors together provide data reflecting deformation of the user's fingers in no less than 15 degrees of freedom.

[0023] One of the embodiments of this specification provides a voice playback method, comprising: acquiring sensor data of a user's hand based on a glove system, wherein the glove system includes a glove body and a plurality of first strain sensors and a plurality of second strain sensors arranged on the glove body, when the user wears the glove body, each of the plurality of first strain sensors is located at an interphalangeal joint of the user and is configured to measure the deformation of the interphalangeal joint in a single degree of freedom, each of the plurality of second strain sensors is located at a metacarpophalangeal joint and is configured to measure the deformation of the metacarpophalangeal joint in two degrees of freedom, the single degree of freedom and the two degrees of freedom both including a degree of freedom corresponding to finger bending; determining the user's gesture based on the sensor data; reading a mapping relationship between a gesture and a voice data packet; and playing voice information based on the user's gesture and the mapping relationship.

[0024] In some embodiments, the plurality of first strain sensors and the plurality of second strain sensors together provide data reflecting deformation of the user's fingers in no less than 15 degrees of freedom.

[0025] One of the embodiments of the present specification provides a gesture control method, comprising: obtaining sensor data of a user's hand based on a glove system, wherein the glove system comprises a glove body and a plurality of first strain sensors and a plurality of second strain sensors arranged on the glove body, and when the user wears the glove body, each of the plurality of first strain sensors is located at an interphalangeal joint of the user and is configured to measure the deformation of the interphalangeal joint in a single degree of freedom, and each of the plurality of second strain sensors is located at a metacarpophalangeal joint and is configured to measure the deformation of the metacarpophalangeal joint in two degrees of freedom, wherein the single degree of freedom and the two degrees of freedom both include a degree of freedom corresponding to finger bending; determining the user's gesture based on the sensor data; reading a mapping relationship between a gesture and an execution instruction; and executing an instruction based on the user's gesture and the mapping relationship.

[0026] In some embodiments, the plurality of first strain sensors and the plurality of second strain sensors together provide data reflecting deformation of the user's fingers in no less than 15 degrees of freedom. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] This specification will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, like numbers represent like structures, wherein:

[0028] FIG1 is a schematic diagram of a framework of an exemplary glove system according to some embodiments of the present specification;

[0029] FIG2A is a schematic diagram of joints of a user's hand according to some embodiments of this specification;

[0030] FIG2B is a schematic diagram of joint positions of a user's hand according to some embodiments of this specification;

[0031] FIG3 is an exemplary schematic diagram of a hand motion coordinate system according to some embodiments of this specification;

[0032] FIG4 is a schematic structural diagram of an exemplary glove system according to some embodiments of this specification;

[0033] FIG5 is a schematic diagram of a second strain sensor according to some embodiments of this specification;

[0034] FIG6 is a schematic diagram of a second strain sensor according to yet other embodiments of this specification;

[0035] FIG7A is a schematic diagram showing the distribution of four sensing structures on a flexible substrate according to some embodiments of this specification;

[0036] FIG7B is a schematic diagram of the cross-sectional structure along the Y-axis direction shown in FIG7A ;

[0037] FIG8A is a schematic diagram of a structure of a sensor without deformation according to some embodiments of this specification;

[0038] FIG8B is a schematic diagram of the deformation of the sensor shown in FIG8A after being stretched or compressed along the long axis;

[0039] FIG9 is a side view schematic diagram of the sensor shown in FIG8A after bending and deformation around an axis parallel to the short axis direction;

[0040] FIG10 is a schematic top view of the sensor shown in FIG8A after being bent and deformed around an axis parallel to the thickness direction;

[0041] FIG11 is a schematic cross-sectional view of a second strain sensor according to some embodiments of this specification;

[0042] FIG12 is a processing flow chart of a processor according to some embodiments of this specification;

[0043] FIG13 is a processing flow chart of a processor according to yet other embodiments of this specification;

[0044] FIG14 is a processing flow chart of a processor according to some embodiments of this specification;

[0045] FIG15 is a processing flow chart of a processor according to some embodiments of this specification;

[0046] FIG16 is a flowchart of a method for gesture replication according to some embodiments of this specification;

[0047] FIG17 is a flow chart of a method for voice playback according to some embodiments of this specification;

[0048] FIG18 is a flow chart of a gesture control method according to some embodiments of this specification. DETAILED DESCRIPTION

[0049] To more clearly illustrate the technical solutions of the embodiments of this application, the following is a brief introduction to the drawings required for describing the embodiments. Obviously, the drawings described below are merely examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without inventive effort. Unless otherwise apparent from the context or otherwise noted, the same reference numerals in the figures represent the same structure or operation.

[0050] It should be understood that the terms "system," "device," "unit," and / or "module" used herein are a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other terms can achieve the same purpose, the terms may be replaced by other expressions.

[0051] As used in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0052] Flowcharts are used in this application to illustrate the operations performed by the systems according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0053] FIG1 is a schematic diagram of an exemplary glove system according to some embodiments of this specification. In some embodiments, as shown in FIG1 , the glove system 100 may include a glove body 110, multiple strain sensors (including a first strain sensor 120 and a second strain sensor 130), and a processor 140. The multiple strain sensors may be distributed at different locations on the glove body 110.

[0054] The glove system 100 is a system for capturing a user's hand movements (including finger movements). The glove system 100 can recognize hand movements through one or more devices, such as motion capture gloves, sign language gloves, VR gloves, force feedback gloves, tactile feedback gloves, etc.

[0055] The glove body 110 refers to a wearable item for fitting with the hand. In some embodiments, in order to facilitate the glove body 110 to fit with the hand joints (for example, including the interphalangeal joints, metacarpophalangeal joints, wrist joints, etc.), the glove body 110 has a structure that adapts to the shape of the hand and extends along the hand joints. In some embodiments, the glove body 110 can be a flexible fabric that easily deforms (such as bending) along with the fingers when subjected to external force. In some embodiments, the glove body 110 can provide support for the strain sensor to facilitate the arrangement of the strain sensor. For example, the glove body 110 can include one or more layers of flexible structure (for example, flexible fabric). The strain sensor can be fixed to the surface of a layer of flexible structure by means of gluing, pressing or sewing.

[0056] Strain sensors are electronic devices that can convert deformation in corresponding areas of the glove body 110 into electrical signals. In some embodiments, strain sensors (e.g., the first strain sensor 120 and the second strain sensor 130) are placed in one or more areas of the glove body 110 corresponding to the joints of the user's hand. When the user wears the glove body 110 and moves their hand, the glove body 110 deforms due to external forces. The strain sensors can convert the deformation at the hand joints into electrical signals, which can reflect the direction and magnitude of the deformation in the corresponding area.

[0057] In some embodiments, the strain sensor includes a plurality of first strain sensors 120 and a plurality of second strain sensors 130 .

[0058] Each of the plurality of first strain sensors 120 can be configured to measure deformation in a single degree of freedom. In some embodiments, the first strain sensor 120 can be located at a location on the user's hand where deformation of the degree of freedom is relatively small, such as an interphalangeal joint, and can be configured to measure deformation in a single degree of freedom of the interphalangeal joint. In some embodiments, the first strain sensor 120 comprises a single degree of freedom sensor, such as a single-axis sensor (e.g., an inductive, resistive bending sensor, or a capacitive stretch sensor).

[0059] Each of the plurality of second strain sensors 130 can be configured to measure deformation in more than one degree of freedom. In some embodiments, the second strain sensor 130 can be located at a location on the user's hand where deformation has a greater degree of freedom, such as at the metacarpophalangeal joint, and can be configured to measure deformation of the metacarpophalangeal joint in two degrees of freedom. In some embodiments, the second strain sensor 130 comprises a multi-degree-of-freedom sensor, such as a multi-axis sensor (e.g., a capacitive multi-axis motion sensor).

[0060] For more details about the first strain sensor and the second strain sensor, please refer to the description related to Figure 4. For the description of the interphalangeal joint and metacarpophalangeal joint, please refer to the description related to Figures 2A and 2B.

[0061] Figure 2A is a schematic diagram of the joints of a user's hand according to some embodiments of this specification. Figure 2B is a schematic diagram of the joint positions of a user's hand according to some embodiments of this specification.

[0062] As shown in FIG2A , the metacarpophalangeal joints 211 , 212 , 213 , 214 , and 215 are the joints between the metacarpal capitulum and the proximal phalanx (the first phalanx closest to the metacarpal bone), and the interphalangeal joints 221 , 222 , 223 , 224 , 225 , 226 , 227 , 228 , and 229 are the joints between phalanges.

[0063] 2A and 2B , positions 0, 2, 5, 8, and 11 on the back of the hand (dorsum side) correspond one-to-one to the metacarpophalangeal joints 215, 214, 213, 212, and 211. Positions 1, 3, 6, 9, 12, 4, 7, 10, and 13 on the back of the hand correspond one-to-one to the interphalangeal joints 229, 227, 225, 223, 221, 228, 226, 224, and 222, respectively.

[0064] In some embodiments, first strain sensors 120 are disposed on the glove body 110 at positions corresponding to positions 1, 3, 6, 9, 12, 4, 7, 10, and 13 on the back of the hand. In some embodiments, first strain sensors 120 can be disposed on the front (palm side) or side of the hand of the glove body 110 at positions corresponding to interphalangeal joints 229, 227, 225, 223, 221, 228, 226, 224, and 222. In some embodiments, second strain sensors 130 are disposed on the glove body 110 at positions corresponding to positions 0, 2, 5, 8, and 11 on the back of the hand.

[0065] Each first strain sensor 120 can measure the deformation of one interphalangeal joint in a single degree of freedom among the interphalangeal joints 229, 227, 225, 223, 221, 228, 226, 224, and 222. Each second strain sensor 130 can measure the deformation of one metacarpophalangeal joint in two degrees of freedom among the metacarpophalangeal joints 215, 214, 213, 212, and 211.

[0066] The degrees of freedom described in this specification refer to the dimensions in which an object can move. To facilitate the explanation of the degrees of freedom of movement of the hand, a three-dimensional coordinate system is established for the movement of the hand, for example, the three-dimensional coordinate system shown in Figure 3. In this coordinate system, the origin is the metacarpophalangeal joint, the direction of finger extension is the Y axis, and the direction perpendicular to the hand plane is the Z axis. Among them, the hand plane can refer to the fitting plane of the palm part. For example, when the palm of the hand is placed on the table, the table or the plane passing through the hand and parallel to the table can be used as the hand plane. This three-dimensional coordinate system can be used to describe the degrees of freedom of hand movement. Specifically, in the three-dimensional coordinate system shown in Figure 3, the rotation of the hand around the X axis is defined as bending, the rotation of the hand around the Z axis is defined as swinging, and the rotation of the hand around the Y axis is defined as rotation. The interphalangeal joints (such as interphalangeal joints 229, 227, 225, 223, 221, 228, 226, 224, 222) can only bend and rotate. In other words, the deformation of the interphalangeal joints corresponds only to one degree of freedom, namely bending. The metacarpophalangeal joints (e.g., metacarpophalangeal joints 215, 214, 213, 212, 211) can bend, swing, and rotate. In other words, the deformation of the metacarpophalangeal joints corresponds to two degrees of freedom, namely bending and swinging. Thus, each first strain sensor 120 can measure the deformation of one interphalangeal joint among the interphalangeal joints 229, 227, 225, 223, 221, 228, 226, 224, 222 in the degree of freedom of bending. That is, when any interphalangeal joint performs bending motion, the first strain sensor 120 can measure the deformation of the glove body 110 at the corresponding interphalangeal joint. Each second strain sensor 130 can measure the deformation of one of the metacarpophalangeal joints 215, 214, 213, 212, and 211 in two degrees of freedom, namely, when any metacarpophalangeal joint performs bending or swinging motion, the second strain sensor 130 can measure the deformation of the glove body 110 at the corresponding metacarpophalangeal joint.

[0067] Continuing to refer to Figures 2A and 2B, a hand has a total of nine interphalangeal joints, of which interphalangeal joints 228, 226, 224, and 222 are distal interphalangeal joints relatively far from the palm, and interphalangeal joints 229, 227, 225, 223, and 221 are proximal interphalangeal joints relatively close to the palm. In some embodiments, the movement of the distal interphalangeal joints is manifested as following the movement of the previous joint (the proximal interphalangeal joint). Under normal circumstances, they do not bend independently, but bend together with the proximal interphalangeal joints. Therefore, the first strain sensor may not be separately arranged at the distal interphalangeal joints, but only at the proximal interphalangeal joints. In some embodiments, the deformation data at the distal interphalangeal joints can be generated by mapping the deformation data of the corresponding proximal interphalangeal joints. Thus, the glove system 100 collects deformation data of at least five proximal interphalangeal joints 229, 227, 225, 223, and 221 in a single degree of freedom through the multiple first strain sensors 120, and the glove system 100 collects deformation data of five metacarpophalangeal joints 215, 214, 213, 212, and 211 in two degrees of freedom through the multiple second strain sensors 130. In total, the multiple first strain sensors 120 and the multiple second strain sensors 130 jointly provide deformation data of at least 15 degrees of freedom on at least the user's fingers.

[0068] By using the deformation data at the proximal interphalangeal joint to deduce or map the deformation data at the distal interphalangeal joint, the first strain sensor at the distal interphalangeal joint can be omitted, thereby reducing the number of strain sensors set on the glove body, simplifying the structure of the glove system, and helping to control product costs while ensuring the accuracy of deformation data collection.

[0069] In some embodiments, the glove system 100 may further include a processor 140 configured to receive and process data collected by multiple strain sensors (including multiple first strain sensors 120 and multiple second strain sensors 130). For example, the processor 140 may obtain deformation data from the strain sensors and analyze and process it, for example, calculating the deformation magnitude and deformation angle of each finger joint using a mapping relationship (e.g., a mapping function or a trained machine learning model). In some embodiments, the processor 140 may directly process the data collected by the multiple strain sensors, or may receive data collected by the multiple strain sensors and transmit the data to an external device for data processing. In some embodiments, the processor 140 may perform corresponding functions based on the data processing results, such as gesture reproduction, voice broadcast, gesture control, etc. In some embodiments, the processor 140 may interact with an external processing device based on the data processing results. The external device may include a host computer such as a computer, mobile phone, or VR device. Interactions may include, for example, controlling the external device's power, volume adjustment, program execution (e.g., controlling the movement of a game character), and fitness feedback.

[0070] In some embodiments, the processor 140 may be a single server or a server group, and the server group may be centralized or distributed. In some embodiments, the processor 140 may be local or remote. In some embodiments, the data collected by the strain sensor is transmitted to the processor 140 via a wire. In some embodiments, the processor 140 may be implemented on a cloud platform. For example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud cloud, a multi-cloud, etc., or any combination thereof. In some embodiments, the processor 140 may include one or more processing chips, and the processing chip analyzes and processes the received data. In some embodiments, the processor 140 is an independent processing device. In some embodiments, the processor 140 may be an external device or a part of the glove body 110. For example, the processor 140 may be integrated into the external device or the glove body 110.

[0071] Typically, to capture multi-DOF finger motion, a gyroscope is placed at a non-joint location on the finger to measure the shape of the finger and infer the finger's bending. Alternatively, a single-DOF strain sensor (e.g., a first strain sensor) is placed at the finger joint, and a gyroscope is placed at a non-joint location on the finger. The measurement results of these two sensors are combined to calculate the finger's bending. However, some embodiments herein combine a dual-DOF strain sensor (e.g., a second strain sensor) with a single-DOF strain sensor to control product costs, reduce subsequent data processing costs, and ensure measurement accuracy.

[0072] In some embodiments, to more fully identify hand motion, the glove system 100 can also capture wrist motion. For example, the glove system 100 can measure wrist motion using multiple inertial sensors, or a combination of strain sensors and inertial sensors. For more information on capturing wrist motion, see other sections of this specification, such as Figure 4 and its related description.

[0073] FIG4 is a schematic structural diagram of an exemplary glove system according to some embodiments of the present specification.

[0074] As shown in FIG4 , in some embodiments, a glove system 400 includes a glove body 410, a plurality of first strain sensors (including a first strain sensor 421, a first strain sensor 422, a first strain sensor 423, a first strain sensor 424, and a first strain sensor 425), a plurality of second strain sensors (including a second strain sensor 431, a second strain sensor 432, a second strain sensor 433, a second strain sensor 434, and a second strain sensor 435), and a processor 440. The first strain sensors are located in any one or more regions of the glove body 410 corresponding to the user's interphalangeal joints (e.g., interphalangeal joints 229, 227, 225, 223, and 221). The second strain sensors are located in any one or more regions of the glove body 410 corresponding to the user's metacarpophalangeal joints (e.g., metacarpophalangeal joints 211, 212, 213, 214, and 215). The processor 440 is configured to receive and process data collected by the strain sensors.

[0075] In some embodiments, the first strain sensor is a uniaxial sensor that collects the deformation of the interphalangeal joint in a single degree of freedom (i.e., bending). In some embodiments, the first strain sensor includes various flexible bending sensors, such as inductive sensors, resistive sensors, capacitive sensors, optical fiber sensors, etc. As an example only, the first strain sensor includes an inductive sensor, the inductance of which can change with the deformation of the interphalangeal joint. Specifically, the inductive sensor deforms as the interphalangeal joint moves, and its inductance changes in response to the deformation of the inductive sensor. By collecting inductance-related parameters of the inductive sensor and analyzing their changes, the bending condition of the inductive sensor (such as bending angle, bending direction, etc.) can be calculated more accurately. For example, the electrical quantity of the inductance of the inductive sensor is proportional to the deformation of the inductive sensor. Therefore, the change in the electrical quantity of the inductance of the inductive sensor can characterize the deformation or bending degree of the interphalangeal joint.

[0076] Since the interphalangeal joint corresponds to only one bending degree of freedom, a single-axis sensor (such as an inductive sensor) is used to measure the movement of the interphalangeal joint, which has a simple structure and low product cost.

[0077] In some embodiments, referring to FIG2A and FIG4 , the first strain sensor is not separately arranged at the distal interphalangeal joint, and the first strain sensor is arranged only at the proximal interphalangeal joint. The deformation data at the distal interphalangeal joint can be generated by mapping the deformation data of the corresponding proximal interphalangeal joint. Referring to FIG2B and FIG4 , the first strain sensors are arranged at positions 1, 3, 6, 9 and 12 on the user's fingers. For example, the first strain sensor 425 is arranged at position 1 on the thumb, and position 1 corresponds to the proximal interphalangeal joint 229 on the thumb. For another example, the first strain sensor 424 is arranged at position 3 on the index finger, and position 3 corresponds to the proximal interphalangeal joint 227 on the index finger. For another example, the first strain sensor 423 is arranged at position 6 on the middle finger, and position 6 corresponds to the proximal interphalangeal joint 225 on the middle finger.

[0078] By arranging the first strain sensor only at the interphalangeal joint closest to the metacarpophalangeal joint of each finger of the user, the structure of the glove system 400 can be simplified, the product cost can be reduced, and the accuracy of deformation data collection can be ensured.

[0079] In some embodiments, the second strain sensor is a multi-axis sensor for collecting deformation of the metacarpophalangeal joint in two degrees of freedom (i.e., bending and swinging). In some embodiments, the second strain sensor includes various flexible bending sensors, such as inductive sensors, resistive sensors, capacitive sensors, optical fiber sensors, etc. As an example only, the second strain sensor includes multiple capacitor structures, and the capacitance of the multiple capacitor structures can change separately as the metacarpophalangeal joint changes in each degree of freedom. For the specific structure of the second strain sensor including multiple capacitor structures, please refer to Figures 5-8B and related descriptions.

[0080] 5 and 6 , in some embodiments, the second strain sensor 500 includes a flexible substrate 510, a first capacitive structure 520, and a second capacitive structure 530. In some embodiments, as shown in FIG5 , the first capacitive structure 520 and the second capacitive structure 530 are located on both sides of the flexible substrate 510 along the thickness direction, and the projection of the first capacitive structure 520 on the flexible substrate 510 at least partially overlaps with the projection of the second capacitive structure 530 on the flexible substrate 510. In some embodiments, as shown in FIG6 , the first capacitive structure 520 and the second capacitive structure 530 are located on the same side of the flexible substrate 510 along the thickness direction.

[0081] The flexible substrate 510 is flexible and easily deforms (e.g., bends) when subjected to external forces. In some embodiments, to facilitate conformation to human joints, the flexible substrate 510 may have a flat structure. In this case, the flexible substrate 510 has a thickness direction, which may be the Z-axis direction as shown in FIG5 .

[0082] The capacitor structure (including the first capacitor structure 520 and the second capacitor structure 530) is used to measure the structure of the bending deformation of the flexible substrate 510. Specifically, the capacitor structure has a corresponding area, and the electrical quantity of the capacitor structure (for example, capacitance) is proportional to its area (or a dimensional parameter affecting the area, such as the length, width of the capacitor structure or the bending angle of the second strain sensor, etc.), that is, the electrical quantity of the capacitor structure in different directions is proportional to the deformation of the second strain sensor in different degrees of freedom, thereby reflecting the deformation (such as bending angle, bending direction). It should be understood that the second strain sensor may include but is not limited to a capacitor structure, for example, it may also be a resistor structure or a capacitor-resistor composite structure. In some embodiments, the first capacitor structure 520 and the second capacitor structure 530 may each include a multilayer structure arranged on the same side surface of the thickness direction of the flexible substrate 510 (the Z-axis direction as shown in Figures 5 or 6), and the layers of each multilayer structure are stacked along the thickness direction. Wherein, the multilayer structure is a structure formed by stacking layer structures. The layers of each multilayer structure stacked along the thickness direction can be understood as follows: the layers of the multiple layered structures of the first capacitor structure 520 and the second capacitor structure 530 are arranged along the thickness direction of the flexible substrate 510 and stacked together. In some embodiments, the multilayer structure includes a second conductive layer, an intermediate layer, and a first conductive layer. In some embodiments, the multilayer structure includes a second conductive layer, a second intermediate layer, a third conductive layer, a first intermediate layer, and a first conductive layer. The specific layered structures included in the multilayer structure can be found below.

[0083] When different capacitive structures are arranged at different positions of the second strain sensor 500, these capacitive structures will produce different responses to the deformation of the flexible substrate 510 in different dimensions. When the flexible substrate 510 deforms in a certain dimension, the detection parameters generated by these capacitive structures have characteristics corresponding to the deformation of this dimension; and when the flexible substrate 510 deforms in another dimension, the detection parameters generated by these capacitive structures will have characteristics corresponding to it. It can be understood that the characteristics of the detection parameters generated by these capacitive structures will correspond to the deformation dimensions of the flexible substrate 510 one-to-one. Based on this, the deformation of the flexible substrate 510 can be identified based on the detection parameters of each capacitive structure. In some embodiments, the processor (e.g., processor 140) can determine the deformation of the flexible substrate 510 in at least two dimensions based on the capacitance-related parameters of each capacitive structure (e.g., the first capacitive structure 520, the second capacitive structure 530) using a specific algorithm. For example, the deformation of the flexible substrate 510 in at least two dimensions is determined using a machine learning model, a mapping relationship, or a functional relationship. The description of the deformation in at least two dimensions can be found below.

[0084] When the second strain sensor 500 (flexible substrate 510) bends and deforms, the physical shape of at least a portion of the capacitive structure (e.g., the first capacitive structure 520 and the second capacitive structure 530) located on the flexible substrate 510 changes, thereby causing the corresponding capacitance to change. For example, bending the second strain sensor 500 may cause the area of ​​the capacitive structure to change, thereby changing the capacitance of the capacitive structure. By collecting the capacitance of the capacitive structure or parameters related to capacitance and analyzing the changes in the capacitance of the capacitive structure or parameters related to capacitance, the bending condition of the second strain sensor 500 (e.g., bending angle, bending direction) can be more accurately sensed.

[0085] As shown in FIG5 , in some embodiments, a first capacitor structure 520 and a second capacitor structure 530 are respectively distributed on opposite sides of the flexible substrate 510 along the thickness direction. In some embodiments, the first capacitor structure 520 and the second capacitor structure 530 are symmetrically arranged along the flexible substrate 510 along the thickness direction. In this case, the projection of the first capacitor structure 520 on the flexible substrate 510 completely overlaps with the projection of the second capacitor structure 530 on the flexible substrate 510. When the first capacitor structure 520 and the second capacitor structure 530 are symmetrically arranged about the flexible substrate 510 along the thickness direction, the two capacitor structures have the same response to some external disturbances (for example, the overall stretching or compression of the second strain sensor along a certain direction). In this case, based on the capacitance-related parameters of the capacitor structures, a differential processing algorithm can be used to eliminate the influence of external disturbances on the second strain sensor 500, thereby improving the sensitivity of determining the deformation of the flexible substrate 510 in at least two degrees of freedom.

[0086] In some embodiments, the flexible substrate 510 includes a short axis and a long axis perpendicular to the thickness direction. The deformation in multiple degrees of freedom generated by the second strain sensor 500 (flexible substrate 510) shown in FIG5 includes at least bending deformation about an axis parallel to the short axis and tensile or compressive deformation along the long axis. The short axis can be the X-axis shown in FIG5 , and the long axis can be the Y-axis shown in FIG5 .

[0087] As shown in FIG5 , when the second strain sensor 500 is bent and deformed about an axis parallel to the minor axis (i.e., the X-axis), the first capacitor structure 520 and the second capacitor structure 530 undergo tensile deformation and compressive deformation, respectively. For example, the first capacitor structure 520 is stretched and elongated by the bending, while the second capacitor structure 530 is compressed by the bending. At this time, the capacitances of the first capacitor structure 520 and the second capacitor structure 530 undergo opposite changes. In this case, the difference in capacitance between the first capacitor structure 520 and the second capacitor structure 530 can reflect the bending direction and degree of the second strain sensor 500 about the axis parallel to the X-axis. Since the first capacitor structure 520 and the second capacitor structure 530 are stretched and compressed to substantially the same degree, the sum of their capacitances can remain substantially unchanged.

[0088] When the second strain sensor 500 is stretched or compressed along its longitudinal axis, the first capacitor structure 520 and the second capacitor structure 530 are stretched or compressed synchronously, and the capacitances of the first capacitor structure 520 and the second capacitor structure 530 change synchronously. In this case, the difference in capacitance between the first capacitor structure 520 and the second capacitor structure 530 remains substantially unchanged (or close to zero), and the sum of the two capacitances reflects the degree of stretching or compression of the second strain sensor 500 along its longitudinal axis.

[0089] In some embodiments, by comparing the capacitance characteristics of the first and second capacitive structures 520 and 530 in the two degrees of freedom, for example, the relationship between the sum (or difference) of their capacitances and the deformation of the flexible substrate 510, mutual interference between the deformations in the two degrees of freedom can be avoided, effectively distinguishing the deformations of the flexible substrate 510 in the two degrees of freedom. For example, when the second strain sensor 500 is stretched or compressed as a whole, the first and second capacitive structures 520 and 530 will stretch or compress synchronously. This synchronous stretching or compression of the first and second capacitive structures 520 and 530 is considered common-mode interference. In this case, differential processing of the signal (e.g., capacitance) can eliminate this common-mode interference, rendering the second strain sensor 500 insensitive to its own stretching or compression deformation and sensitive only to bending deformation about an axis parallel to the minor axis (i.e., the X-axis), enabling the second strain sensor 500 to accurately detect bending deformation in this degree of freedom. That is, the difference in capacitance between the first capacitor structure 520 and the second capacitor structure 530 primarily reflects the bending deformation of the flexible substrate 510 about an axis parallel to the minor axis (i.e., the X-axis). Similarly, the sum of the capacitances of the first capacitor structure 520 and the second capacitor structure 530 primarily reflects the tensile or compressive deformation of the flexible substrate 510 along the major axis. The detection principle for tensile or compressive deformation along the major axis is similar to that of the second strain sensor 500 having four capacitor structures. For detailed descriptions, please refer to Figures 8A to 10 and the related descriptions.

[0090] As shown in Figure 6, in some embodiments, the flexible substrate 510 includes a long axis direction perpendicular to the thickness direction, and the first capacitor structure 520 and the second capacitor structure 530 are distributed on the same side of the flexible substrate 510 in the thickness direction. The first capacitor structure 520 and the second capacitor structure 530 are arranged side by side and both extend along the long axis direction of the flexible substrate 510.

[0091] In some embodiments, the first capacitor structure 520 and the second capacitor structure 530 are symmetrically arranged along a mid-section parallel to a plane formed by the thickness direction and the long axis direction, and the mid-section is located on the flexible substrate 510. When the first capacitor structure 520 and the second capacitor structure 530 are symmetrically arranged about the flexible substrate 510 in the mid-section, the two capacitor structures have the same response to partial external interference (for example, the second strain sensor 500 is stretched or compressed as a whole along a certain direction). At this time, the processing circuit 16 can eliminate the influence of the external interference on the second strain sensor 500 through a differential processing algorithm based on the capacitance-related parameters of the capacitor structures (for example, the first capacitor structure 520 and the second capacitor structure 530), thereby improving the sensitivity of determining the deformation of the flexible substrate 510 in at least two degrees of freedom. For more information about the mid-section, please refer to Figures 7A and 7B and their related descriptions.

[0092] In some embodiments, the degree of freedom deformation generated by the second strain sensor 500 (flexible substrate 510 ) shown in FIG. 6 includes at least bending deformation around an axis parallel to the thickness direction and tensile or compressive deformation along the long axis direction.

[0093] As shown in FIG6 , when the second strain sensor 500 is bent about an axis parallel to the thickness direction (i.e., the Z-axis), the first capacitor structure 520 and the second capacitor structure 530 undergo tensile deformation and compressive deformation, respectively. For example, the first capacitor structure 520 is stretched by the bending, and the second capacitor structure 530 is compressed by the bending. At this time, the capacitances of the first capacitor structure 520 and the second capacitor structure 530 undergo opposite changes. In this case, the difference in capacitance between the first capacitor structure 520 and the second capacitor structure 530 can reflect the bending direction and degree of the second strain sensor 500 about the axis parallel to the Z-axis. Since the first capacitor structure 520 and the second capacitor structure 530 are stretched and compressed to substantially the same degree, the sum of their capacitances can remain substantially unchanged.

[0094] When the second strain sensor 500 shown in FIG6 is stretched or compressed along its long axis, the first capacitor structure 520 and the second capacitor structure 530 are stretched or compressed synchronously, and the capacitances of the first capacitor structure 520 and the second capacitor structure 530 change synchronously. In this case, the difference in capacitance between the first capacitor structure 520 and the second capacitor structure 530 remains substantially unchanged (or close to zero), and the sum of the two capacitances reflects the degree of stretching or compression of the second strain sensor 500 along its long axis.

[0095] In some embodiments, by comparing the capacitance characteristics of the first and second capacitive structures 520 and 530 in the two degrees of freedom, for example, the relationship between the sum (or difference) of their capacitances and the deformation of the flexible substrate 510, mutual interference between the deformations in the two degrees of freedom can be avoided, effectively distinguishing the deformations of the flexible substrate 510 in the two degrees of freedom. For example, when the second strain sensor 500 is stretched or compressed as a whole, the first and second capacitive structures 520 and 530 will stretch or compress synchronously. This synchronous stretching or compression of the first and second capacitive structures 520 and 530 is considered common-mode interference. In this case, differential processing of the signal (e.g., capacitance) can eliminate this common-mode interference, rendering the second strain sensor 500 insensitive to its own stretching or compression deformation and sensitive only to bending deformation along an axis parallel to the thickness direction (i.e., the Z-axis), enabling the second strain sensor 500 to accurately detect bending deformation in this degree of freedom. That is, the difference in capacitance between the first capacitor structure 520 and the second capacitor structure 530 primarily reflects the bending deformation of the flexible substrate 510 about an axis parallel to the thickness direction (i.e., the Z-axis). Similarly, the sum of the capacitances of the first capacitor structure 520 and the second capacitor structure 530 primarily reflects the tensile or compressive deformation of the flexible substrate 510 along the longitudinal axis. The detection principle for tensile or compressive deformation along the longitudinal axis is similar to that of the second strain sensor 500 having four capacitor structures. For detailed descriptions, please refer to Figures 8A to 10 and the related descriptions.

[0096] FIG7A is a schematic diagram showing the distribution of four capacitor structures on a flexible substrate according to some embodiments of this specification; FIG7B is a schematic diagram showing the cross-sectional structure along the Y-axis shown in FIG7A .

[0097] As shown in Figures 7A and 7B, in some embodiments, compared to Figure 6, the second strain sensor 500 further includes a third capacitor structure 540 and a fourth capacitor structure 550. The first capacitor structure 520 and the second capacitor structure 530 are arranged side by side on one side of the flexible substrate 510 along the thickness direction and both extend along the long axis direction; the third capacitor structure 540 and the fourth capacitor structure 550 are arranged side by side on the other side of the flexible substrate 510 along the thickness direction and both extend along the long axis direction.

[0098] In some embodiments, the processor reads capacitance-related parameters of the first capacitor structure 520 , the second capacitor structure 530 , the third capacitor structure 540 , and the fourth capacitor structure 550 , and determines deformation of the flexible substrate 510 in at least two degrees of freedom based on the parameters.

[0099] In some embodiments, the degrees of freedom deformation generated by the second strain sensor 500 (flexible substrate 510) shown in FIG7A include at least bending deformation about an axis parallel to the thickness direction, bending deformation about an axis parallel to the short axis direction, and tensile or compressive deformation along the long axis of the flexible substrate 510. For illustrative purposes, the capacitance-related parameters of various capacitive structures under different degrees of freedom deformation are described below with reference to FIG8A , FIG8B , FIG9 , and FIG10 .

[0100] In some embodiments, parameters related to capacitance include: C1, C2, C3, C4, Ctotal, dCx, and dCz; where Ctotal = C1 + C2 + C3 + C4, dCx (composite differential operation) = (C1 + C2) - (C3 + C4), and dCz (composite differential operation) = (C1 + C3) - (C2 + C4). C1, C2, C3, and C4 represent the capacitances of the first capacitor structure 520, the second capacitor structure 530, the third capacitor structure 540, and the fourth capacitor structure 550, respectively.

[0101] In some embodiments, the first capacitor structure 520 and the second capacitor structure 530 are symmetrically arranged about the first middle section S1, and the third capacitor structure 540 and the fourth capacitor structure 550 are symmetrically arranged about the first middle section S1. The first middle section S1 represents: a section of the flexible substrate 510 parallel to a plane formed by the thickness direction and the long axis direction; the first capacitor structure 520 and the third capacitor structure 540 are symmetrically arranged about the second middle section S2 of the flexible substrate 510, and the second capacitor structure 530 and the fourth capacitor structure 550 are symmetrically arranged about the second middle section S2. The second middle section S2 represents: a midplane parallel to a plane formed by the long axis direction and the short axis direction of the flexible substrate 510.

[0102] By providing four symmetrical capacitor structures on the second strain sensor 500 and performing a composite differential operation, common-mode interference can be effectively eliminated during the process of identifying sensor bending deformation. As an example, when the second strain sensor 500 is stretched or compressed as a whole, the first capacitor structure 520, the second capacitor structure 530, the third capacitor structure 540, and the fourth capacitor structure 550 will be stretched or compressed synchronously. The stretching or compression of these four capacitor structures is considered common-mode interference. The dCx and dCz obtained by providing four capacitor structures and performing composite differential processing on the signals remain unchanged, effectively eliminating this common-mode interference. This makes the flexible second strain sensor 500 insensitive to its own stretching or compression deformation and only sensitive to its own bending deformation, thereby increasing the accuracy of the second strain sensor 500.

[0103] FIG8A is a schematic diagram of a sensor without deformation according to some embodiments of the present specification; FIG8B is a schematic diagram of the deformation of the sensor shown in FIG8A after being stretched or compressed along the long axis.

[0104] In some embodiments, when the first capacitor structure 520 , the second capacitor structure 530 , the third capacitor structure 540 and the fourth capacitor structure 550 are all capacitor structures, the second strain sensor 500 can identify tensile or compressive deformation along the long axis of the flexible substrate 11 based on parameters related to capacitance.

[0105] In some embodiments, when the second strain sensor 500 shown in FIG8A is not deformed, the capacitance of the four capacitor structures can be obtained by the following formula (1). For illustrative purposes only, the four capacitor structures of the second strain sensor 500 are considered to be completely identical and symmetrical about the flexible substrate 510 (i.e., the first middle section S1 and the second middle section S2).

[0106] Wherein, ε0 represents the dielectric constant of vacuum; ε represents the relative dielectric constant of each of the four capacitor structures (i.e., the middle layer, see below); d represents the thickness of the dielectric layer of each of the four capacitor structures; A represents the area of ​​each of the four capacitor structures on the plane formed by the long axis direction and the short axis direction; L0 represents the initial length of each of the four capacitor structures in the long axis direction of the flexible substrate 510; w represents the width of each of the four capacitor structures in the short axis direction of the flexible substrate 510.

[0107] In some embodiments, as shown in FIG8B , after the second strain sensor 500 is stretched or compressed along the longitudinal direction (i.e., the Y-axis), the initial length L0 of each of the four capacitor structures is stretched or compressed to Lx. Considering that the thickness of each of the four capacitor structures changes very little, for ease of explanation, it is approximately assumed that the thickness of each of the four capacitor structures remains unchanged. Based on formula (1), the capacitance change of the four capacitor structures can be expressed as follows: formula (2).

[0108] At this time, Ctotal, dCx, and dCz are determined by the following formulas (3), (4), and (5), respectively. dCx=(C1+C2)-(C3+C4)=0 (4) dCz=(C1+C3)-(C2+C4)=0 (5)

[0109] When the second strain sensor 500 is stretched or compressed along the longitudinal axis, the four capacitor structures are stretched or compressed synchronously, and the capacitance of the four capacitor structures changes synchronously. In this case, the sum of the capacitances of the four capacitor structures can reflect the degree of stretching or compression of the second strain sensor 500 (flexible substrate 510) along the longitudinal axis. According to formulas (2) and (3), Ctotal is only related to the length Lx after the capacitor structure changes, and Lx can be used to represent the stretching or compressing deformation of the second strain sensor 500 along the longitudinal axis. Therefore, Ctotal can be used to reflect the stretching or compressing deformation of the second strain sensor 500 (flexible substrate 510) along the longitudinal axis.

[0110] According to formula (4), dCx reflects the difference between the sum of the capacitance C1 of the first capacitor structure 520 and the capacitance C2 of the second capacitor structure 530, and the sum of the capacitance C3 of the third capacitor structure 540 and the capacitance C4 of the fourth capacitor structure 550. Referring to the structural distribution of the four capacitor structures in FIG8B , it can be seen that the sum of the capacitance C1 of the first capacitor structure 520 and the capacitance C2 of the second capacitor structure 530, namely C1+C2, can reflect the bending deformation of the upper side of the flexible substrate 510 in the thickness direction; the sum of the capacitance C3 of the third capacitor structure 540 and the capacitance C4 of the fourth capacitor structure 550, namely C3+C4, can reflect the bending deformation of the lower side of the flexible substrate 510 in the thickness direction. Therefore, dCx is used to reflect the difference in bending deformation generated on the upper and lower sides (i.e., the upper and lower sides along the thickness direction) of the flexible substrate 510 when it is deformed. It is related to the bending deformation around an axis parallel to the short axis direction (i.e., the X-axis direction). Deformation caused by tension or compression along the long axis direction (i.e., the Y-axis direction) does not cause deformation difference between the upper and lower sides of the flexible substrate 510, so dCx is a constant 0.

[0111] Similarly, according to formula (5), dCz is used to reflect the difference in bending deformation generated on the two side surfaces along the short axis direction when the flexible substrate 510 is deformed. dCz is only related to the bending deformation around an axis parallel to the thickness direction (i.e., the Z-axis direction). Deformation caused by tension or compression along the long axis direction (i.e., the Y-axis direction) does not cause a difference in deformation between the two side surfaces of the flexible substrate 510 along the short axis direction, so dCz is also a constant of 0.

[0112] In some embodiments, the output parameter Ctotal of the second strain sensor 500 along the Y-axis varies linearly with the stretched or compressed length Lx, exhibiting extremely high linearity. Both dCz and dCx are zero, unaffected by the stretched or compressed length Lx. Therefore, the second strain sensor 500 can identify tensile or compressive deformation along the long axis of the flexible substrate 510 based on the capacitance-related parameter Ctotal.

[0113] FIG9 is a side view schematic diagram of the sensor shown in FIG8A after being bent and deformed around an axis parallel to the short axis direction.

[0114] In some embodiments, when the first capacitor structure 520, the second capacitor structure 530, the third capacitor structure 540 and the fourth capacitor structure 550 are all capacitor structures, the second strain sensor 500 can identify bending deformation along the flexible substrate 510 around an axis parallel to the short axis direction based on the parameter dCx related to capacitance.

[0115] As shown in FIG8A and FIG9, when the second strain sensor 500 is bent and deformed along an axis parallel to the short axis direction (i.e., the X-axis direction), for the convenience of explanation, the second strain sensor 500 after bending and deformation is approximately considered to be an arc, the first capacitor structure 520 and the second capacitor structure 530 are bent and elongated to L1, and the third capacitor structure 540 and the fourth capacitor structure 550 are bent and compressed to L3. Considering that the thickness of each of the four capacitor structures changes very little, for the convenience of explanation, it is approximately considered that the thickness of each of the four capacitor structures remains unchanged. According to formula (1), the capacitance C1 of the first capacitor structure 520 and the capacitance C2 of the second capacitor structure 530 change to the following formula (6), and the capacitance C3 of the third capacitor structure 540 and the capacitance C4 of the fourth capacitor structure 550 change to the following formula (7).

[0116] At this time, Ctotal, dCx, and dCz are determined by the following formulas (8), (9), and (10), respectively. dCz=(C1+C3)-(C2+C4)=0 (10)

[0117] In which, R0 represents the radius of the arc of the flexible substrate 510 when it is bent and deformed; R1 represents the radius of the arc of the first capacitor structure 520 and the second capacitor structure 530 when it is bent and deformed; R2 represents the radius of the arc of the third capacitor structure 540 and the fourth capacitor structure 550 when it is bent and deformed; t represents the thickness of the second strain sensor 500 as shown in Figure 8A; L1 represents the length of the arc of the first capacitor structure 520 and the second capacitor structure 530 when they are bent and deformed; L3 represents the length of the arc of the third capacitor structure 540 and the fourth capacitor structure 550 when they are bent and deformed.

[0118] When the second strain sensor 500 is bent and deformed along an axis parallel to the short axis, for example, the first capacitor structure 520 and the second capacitor structure 530 are bent and stretched, and the third capacitor structure 540 and the fourth capacitor structure 550 are bent and compressed. At this time, the sum of the capacitance C1 of the first capacitor structure 520 and the capacitance C2 of the second capacitor structure 530, that is, C1+C2, has an opposite change relationship with the sum of the capacitance C3 of the third capacitor structure 540 and the capacitance C4 of the fourth capacitor structure 550, that is, C3+C4. At this time, Ctotal obtained by adding the capacitances of the four capacitor structures can offset the change trends of C1+C2 and C3+C4. That is, Ctotal is the constant shown in formula (8) at this time, and Ctotal cannot be used to reflect the bending deformation of the second strain sensor 500 along the axis parallel to the short axis.

[0119] In some embodiments, the output parameter dCx of the second strain sensor 500 after bending about an axis parallel to the minor axis (i.e., the X-axis) varies linearly and proportionally with the bending angle β, exhibiting extremely high linearity. Both dCz and Ctotal are constants and are unaffected by the bending angle β. Therefore, the second strain sensor 500 can identify bending deformation about an axis parallel to the minor axis based on the capacitance-related parameter dCx.

[0120] It should be noted that, in addition to the case of the four capacitor structures described above, the second strain sensor 500 may have three or more capacitor structures distributed on both sides of the flexible substrate 510 along the thickness direction, and all extend along the long axis direction. As long as the deformations of different dimensions can be characterized by the capacitance-related parameters of these capacitor structures, deformations of multiple dimensions can be identified. As an example only, there are three capacitor structures on the second strain sensor 500, two of which are distributed on the upper side of the flexible substrate and the other is distributed on the lower side of the flexible substrate. At this time, the capacitance-related parameters of the three capacitor structures are also related to the bending deformation around the X-axis (or Z-axis). At this time, the bending deformation of the second strain sensor 500 around the X-axis (or Z-axis) can also be calculated by combining the capacitance-related parameters generated by the three sensor structures.

[0121] FIG10 is a schematic top view of the sensor shown in FIG8A after being bent and deformed around an axis parallel to the thickness direction.

[0122] In some embodiments, when the first capacitor structure 520, the second capacitor structure 530, the third capacitor structure 540 and the fourth capacitor structure 550 are all capacitor structures, the second strain sensor 500 can identify bending deformation along the flexible substrate 510 around an axis parallel to the thickness direction based on the parameter dCz related to capacitance.

[0123] As shown in FIG8A and FIG10, when the second strain sensor 500 is bent and deformed along an axis parallel to the thickness direction (i.e., the Z-axis direction), for the convenience of explanation, it is approximately considered that the second strain sensor 500 after bending and deformation can be a circular arc, the first capacitor structure 520 and the third capacitor structure 540 (not shown in FIG10) are bent and elongated to L4, and the second capacitor structure 530 and the fourth capacitor structure 550 (not shown in FIG10) are bent and compressed to L2. Considering that the thickness of each of the four capacitor structures changes very little, for the convenience of explanation, it is approximately considered that the thickness of each of the four capacitor structures remains unchanged. Then, according to formula (1), the capacitance C1 of the first capacitor structure 520 and the capacitance C3 of the third capacitor structure 540 are as follows (11), and the capacitance C2 of the second capacitor structure 530 and the capacitance C4 of the fourth capacitor structure 550 change to the following formula (12).

[0124] At this time, Ctotal, dCx, and dCz are determined by the following formulas (13), (14), and (15), respectively. dCx=(C1+C2)-(C3+C4)=0 (14)

[0125] In which, r0 represents the radius of the arc of the bending deformation of the flexible substrate 510; r1 represents the radius of the arc of the bending deformation of the first capacitor structure 520 and the third capacitor structure 540; r2 represents the radius of the arc of the bending deformation of the second capacitor structure 530 and the fourth capacitor structure 550; L2 represents the length of the arc of the bending deformation of the first capacitor structure 520 and the third capacitor structure 540; L4 represents the length of the arc of the bending deformation of the second capacitor structure 530 and the fourth capacitor structure 550.

[0126] In some embodiments, the output parameter dCz of the second strain sensor 500 after bending about an axis parallel to the thickness direction (i.e., the Z-axis) varies linearly with the bending angle α, exhibiting extremely high linearity. Both dCx and Ctotal are constants and are unaffected by the bending angle α. Therefore, the second strain sensor 500 can identify bending deformation about an axis parallel to the thickness direction based on the capacitance-related parameter dCz.

[0127] In some embodiments, when the first capacitor structure 520, the second capacitor structure 530, the third capacitor structure 540, and the fourth capacitor structure 550 are all resistor structures, the processor can determine the deformation of the flexible substrate 510 in at least two dimensions based on the resistance parameters related to the four capacitor structures. For example, the resistances of the four capacitor structures are: Substituting the areas of each capacitive structure after deformation of the second strain sensor 500 into the calculation and performing a differential operation, a result similar to a capacitance-related parameter can be obtained, enabling the processor to determine the bending deformation of the second strain sensor 500 about an axis parallel to the thickness direction, the bending deformation about an axis parallel to the short axis direction, and the tensile or compressive deformation along the long axis of the flexible substrate. Here, ρ represents the resistivity of each of the four capacitive structures (i.e., the intermediate layer, as described below).

[0128] In some embodiments, the processor can identify composite deformation of the second strain sensor 500, including bending deformation about an axis parallel to the thickness direction, bending deformation about an axis parallel to the short axis direction, and tensile or compressive deformation along the long axis of the flexible substrate 510. For example, the processor can simultaneously calculate changes in capacitance-related parameters Ctotal, dCx, and dCz, thereby simultaneously recovering components of deformation in different dimensions and restoring the actual bending deformation of the second strain sensor 500.

[0129] Therefore, the embodiments of this specification can measure the deformation of multiple degrees of freedom (for example, including bending around the X-axis, bending around the Z-axis, and stretching along the Y-axis) through a single sensor, thereby realizing the detection of motion information in multiple dimensions, which is conducive to simplifying the preparation process and miniaturization design while improving the detection accuracy.

[0130] FIG11 is a schematic cross-sectional view of a second strain sensor according to some embodiments of the present specification.

[0131] As shown in FIG11 , in some embodiments, each capacitor structure includes a second conductive layer, an intermediate layer, and a first conductive layer stacked sequentially away from the flexible substrate 510 along the thickness direction. For example, the first capacitor structure 520 includes a second conductive layer 522, intermediate layers 52-53, and a first conductive layer 521 stacked sequentially away from the flexible substrate 510; the second capacitor structure 530 includes a second conductive layer 532, intermediate layers 52-53, and a first conductive layer 531 stacked sequentially away from the flexible substrate 510; the third capacitor structure 540 includes a second conductive layer 542, intermediate layers 54-55, and a first conductive layer 541 stacked sequentially away from the flexible substrate 510; and the fourth capacitor structure 550 includes a second conductive layer 552, intermediate layers 54-55, and a first conductive layer 551 stacked sequentially away from the flexible substrate 510. In some embodiments, the intermediate layer is interposed between the first and second conductive layers.

[0132] In some embodiments, the first conductive layer and the second conductive layer of each capacitor structure include an elastic conductive material, which can make the first conductive layer and the second conductive layer conductive and can return to their original shape when the external force disappears. In some embodiments, the elastic conductive material includes but is not limited to conductive film, conductive ink, conductive polymer material, conductive gel, liquid metal, etc. In some embodiments, the first conductive layer and the second conductive layer include a conductive film, which is made by mixing conductive particles with a polymer material. In some embodiments, the polymer material includes but is not limited to silicone, rubber or resin, etc. In some embodiments, the first conductive layer and the second conductive layer include conductive ink, which is made by mixing conductive particles with an ink material, and the conductive ink can be printed to obtain a conductive pattern. In some embodiments, the first conductive layer and the second conductive layer include a conductive polymer material, conductive gel, liquid metal, etc. In some embodiments, the conductive polymer material includes but is not limited to polypyrrole, poly (3, 4-ethylenedioxythiophene) - polystyrene sulfonate (PEDOT:PSS, Poly (3, 4-ethylenedioxythiophene) polystyrene sulfonate), etc.

[0133] In some embodiments, the elastic conductive material includes an elastic material filled with conductive particles. The conductivity of the first and second conductive layers can be adjusted by adjusting the density of the conductive particles filled in the elastic material. In some embodiments, the elastic material includes silicone, rubber, resin, polydimethylsiloxane (PDMS), polyurethane, styrene-butadiene-styrene (SBS), etc. In some embodiments, the conductive particles include metal powder and carbon powder, carbon nanotubes, silver nanowires, carbon black, graphite powder, graphene, etc.

[0134] In some embodiments, the intermediate layer comprises an elastic insulating material, in which case each capacitor structure is a capacitor structure. For example, the elastic insulating material includes silicone, rubber, polydimethylsiloxane (PDMS), thermoplastic polyurethane (TPU), etc. In some embodiments, the intermediate layer comprises a high-resistance elastic material, and the resistivity of the intermediate layer is more than 1000 times greater than the resistivity of the first conductive layer and the second conductive layer. In this case, each capacitor structure is a capacitor-resistor composite capacitor structure.

[0135] In some embodiments, the thickness of the first conductive layer and the second conductive layer of each capacitor structure is in the range of 1 to 100 μm. In some embodiments, the width of the first conductive layer and the second conductive layer of each capacitor structure is in the range of 0.5 to 10 mm. In some embodiments, the spacing between the first conductive layer 521 of the first capacitor structure 520 and the first conductive layer 531 of the second capacitor structure 530 is in the range of 0.02 to 2 mm. In some embodiments, the spacing between the second conductive layer 522 of the first capacitor structure 520 and the second conductive layer 532 of the second capacitor structure 530 is in the range of 0.02 to 2 mm. In some embodiments, the spacing between the first conductive layer 541 of the third capacitor structure 540 and the first conductive layer 551 of the fourth capacitor structure 550 is in the range of 0.02 to 2 mm. In some embodiments, the spacing between the second conductive layer 542 of the third capacitor structure 540 and the second conductive layer 552 of the fourth capacitor structure 550 is in the range of 0.02 to 2 mm.

[0136] In some embodiments, the conductivity of the first conductive layer is greater than the conductivity of the intermediate layer.

[0137] Conductivity is a parameter used to describe the ease with which electric charge flows in a substance. Since the conductivity of the first conductive layer is greater than that of the second conductive layer, the first conductive layer has better conductivity. Therefore, the resistance of the first conductive layer is less than that of the second conductive layer. It should be noted that in this specification, the resistance of a component refers to the resistance between two surfaces of the component spaced apart along the thickness direction of the flexible substrate 510. The resistance of the first conductive layer refers to the resistance between two surfaces of the first conductive layer spaced apart along the thickness direction of the flexible substrate 510, and the resistance of the second conductive layer may refer to the resistance between two surfaces of the second conductive layer spaced apart along the thickness direction of the flexible substrate 510. In some embodiments, the conductivity of the first conductive layer may be more than 100 times the conductivity of the second conductive layer. In some embodiments, the conductivity of the first conductive layer may be greater than that of the second conductive layer by setting the density of the conductive particles filled in the elastic material of the first conductive layer to be greater than the density of the conductive particles filled in the elastic material of the second conductive layer.

[0138] When the capacitor structure (e.g., the first capacitor structure 520, the second capacitor structure 530, the third capacitor structure 540, and the fourth capacitor structure 550) is a capacitor-resistance composite capacitor structure or a single capacitor structure, the resistance between the two surfaces of the intermediate layer along the thickness direction of the flexible substrate 510 should not be too small to facilitate subsequent measurement of the resistance and analysis of the change in resistance. In some embodiments, the resistance between the two surfaces of the intermediate layer along the thickness direction of the flexible substrate 510 is greater than 0.8 MΩ. For example, when the capacitor structure (e.g., the first capacitor structure 520, the second capacitor structure 530, the third capacitor structure 540, and the fourth capacitor structure 550) is a capacitor-resistance composite capacitor structure, the resistance between the two surfaces of the intermediate layer along the thickness direction of the flexible substrate 510 can also be 0.8 MΩ to 550 GΩ. For another example, when the capacitor structure (e.g., the first capacitor structure 520, the second capacitor structure 530, the third capacitor structure 540, and the fourth capacitor structure 550) is a single capacitor structure, the resistance between the two surfaces of the intermediate layer spaced apart along the thickness direction of the flexible substrate 510 can be greater, for example, the resistance can be greater than 550 GΩ.

[0139] By setting the resistance between the two surfaces of the intermediate layer along the thickness direction of the flexible substrate 510 to be greater than 0.8 MΩ, the intermediate layer can be made conductive while having a sufficiently large resistance, thereby ensuring the measurement of the resistance of the intermediate layer and the analysis of the change in resistance, which can accurately reflect the bending condition of the second strain sensor 500.

[0140] In some embodiments, the relative dielectric constant of the intermediate layer is greater than 2. The relative dielectric constant is a physical parameter that characterizes the dielectric or polarization properties of a dielectric material. Its value is equal to the ratio of the capacitance of a capacitor of the same size made with the corresponding material as the dielectric medium to the capacitance of a capacitor made with a vacuum as the dielectric medium. The relative dielectric constant also characterizes the material's ability to store electricity, also known as relative permittivity. Materials with a relative dielectric constant greater than 2 are polar materials. In other words, an intermediate layer with a relative dielectric constant greater than 2 has a certain degree of electricity storage capacity. Therefore, the intermediate layer can be equivalent to a parallel combination of a resistor and a capacitor.

[0141] In some embodiments, the relative dielectric constant of the intermediate layer may be greater than or equal to 4. Preferably, the relative dielectric constant of the intermediate layer is greater than 5. In some embodiments, the relative dielectric constant of the intermediate layer is greater than 10.

[0142] Setting the relative dielectric constant of the intermediate layer to be greater than 2 allows the intermediate layer to function as both a resistor and a capacitor. During bending of the second strain sensor 500, both the capacitance and resistance of the intermediate layer change with the deformation of the flexible substrate 510. Therefore, parameters related to the capacitance and resistance of the intermediate layer can reflect the bending of the second strain sensor 500. In this embodiment, the simultaneous presence of capacitance and resistance to reflect the bending of the second strain sensor 500 significantly improves the sensitivity and accuracy of the second strain sensor 500.

[0143] In addition, by setting the resistance between the two surfaces of the intermediate layer along the thickness direction of the flexible substrate 510 to be greater than 0.8 MΩ, it can also be ensured that the intermediate layer can exhibit both better resistance performance and better capacitance performance, thereby ensuring the sensitivity and accuracy of the second strain sensor 500.

[0144] In some embodiments, the two capacitor structures on the same side surface of the flexible substrate 510 can share a first conductive layer, and the second conductive layers of the two capacitor structures are spaced apart along the short axis direction of the flexible substrate 510. In some embodiments, more conductive layers (such as a third conductive layer, etc.) can be provided in a stacked manner along the thickness direction of each inductive structure, which can effectively improve the sensitivity of the sensor 500 in detecting deformations in each degree of freedom. For example, each capacitor structure includes a second conductive layer, a second intermediate layer, a third conductive layer, a first intermediate layer, and a first conductive layer stacked in sequence away from the flexible substrate 510 along the thickness direction. Among them, the parameters of the third conductive layer, the first conductive layer, and the second conductive layer are consistent; the parameters of the second intermediate layer, the first intermediate layer, and the intermediate layer described above are consistent.

[0145] In some embodiments, the glove system 400 can capture wrist joint motion. In the three-dimensional coordinate system shown in FIG3 , wrist joint (e.g., wrist joint 230 shown in FIG2A ) motion is represented by three degrees of freedom: bending, swinging, and rotation. Bending and swinging of the wrist joint are controlled by the wrist joint, which can cause deformation of the wrist joint. Rotation of the wrist joint is controlled by the rotation of the user's forearm, causing the entire wrist joint to rotate.

[0146] In some embodiments, to capture wrist motion in three degrees of freedom, the glove system 400 also includes two inertial sensors. These sensors are located on the glove body 410, corresponding to regions on either side of the wrist. The processor 440 is configured to receive and process data captured by the inertial sensors. In some embodiments, the two inertial sensors are located on either side of the wrist in the direction of arm extension, namely, the front and back sides of the wrist. The front side of the wrist refers to the side facing the back of the hand, and the back side of the wrist refers to the side facing the forearm. For ease of understanding, the two sides of the wrist can be understood as regions of the hand or forearm that are close to the wrist in two directions but do not substantially deform with wrist deformation. For example, the glove system 400 includes a first inertial sensor 451 and a second inertial sensor 452, respectively, located on the glove body 410 at hand positions 14 and 15 shown in FIG2B . The two inertial sensors can respectively capture positional information on either side of the wrist, thereby cooperating to identify overall wrist motion in three degrees of freedom: rotation, flexion, and swing.

[0147] In some embodiments, in order to collect the movement of the wrist joint in three degrees of freedom, the glove system 400 may include an inertial sensor and a third strain sensor. The inertial sensor and the third strain sensor cooperate to collect the movement of the wrist joint. In some embodiments, the third strain sensor may include a multi-degree-of-freedom sensor, which can be used to collect the deformation of the wrist joint in multiple degrees of freedom. As an example only, the third strain sensor can be a sensor that is the same or similar to the second strain sensor and can collect the deformation of the wrist joint in two or more degrees of freedom. As shown in Figure 4, the inertial sensor and the third strain sensor 460 are arranged on the glove body 410, and the third strain sensor 460 is located at the wrist joint, that is, corresponding to the hand positions 14 and 15 shown in Figure 2B on the glove body 410. The inertial sensor is located in an area outside the wrist joint, such as the front or back side of the wrist joint described above. In some embodiments, the inertial sensor can be located on either side of the wrist joint in the direction of arm extension. For example, the inertial sensor is located on the glove body 410 at the position corresponding to the hand position 14 or 15 shown in FIG2B , that is, the inertial sensor is the first inertial sensor 451 or the second inertial sensor 452 shown in FIG4 . The processor 440 is configured to receive and process data collected by the inertial sensor and the third strain sensor.

[0148] By adopting two inertial sensors or combining an inertial sensor with a strain sensor, multiple degrees of freedom motion acquisition of the wrist joint can be achieved. Users can flexibly choose a combination of different sensor types to achieve multiple degrees of freedom motion acquisition according to different application scenarios.

[0149] Continuing with FIG4 , in some embodiments, the glove body 410 includes fabric wrapped around each finger and palm of the user, as well as fabric located between adjacent fingers. The fabric between adjacent fingers can connect the bases of adjacent fingers (near the metacarpophalangeal joints), which may cause the fabric at the bases of the fingers to pull against each other, thereby interfering with the data collected by the second strain sensors at adjacent metacarpophalangeal joints. To reduce this interference, a structure can be provided between the fabrics of adjacent fingers to reduce or prevent pulling between the fabrics on adjacent fingers, so that the elastic modulus of the fabric between adjacent fingers is smaller than the elastic modulus of the fabric on each finger.

[0150] By setting a structure between the fabrics of adjacent fingers to reduce or avoid pulling between the fabrics at the base of adjacent fingers when the fingers move, the elastic modulus of the fabric between adjacent fingers is made smaller than the elastic modulus of the fabric on each finger, thereby reducing the mutual interference between the second strain sensors at the base of different fingers, and ensuring that the data collected by each second strain sensor can accurately reflect the joint deformation at the corresponding position.

[0151] In some embodiments, the structure for reducing pull between the fabric on adjacent fingers includes at least one of a slit structure, an elastic fabric structure, or a pleated structure. That is, the fabric between adjacent fingers can have at least one of a slit structure, an elastic fabric structure, or a pleated structure. In some embodiments, as shown in Figure 4 , slit structure 470 refers to a slit provided in the glove fabric between the second strain sensors. The slit can have any shape, such as a diamond or rectangle. In some embodiments, the elastic fabric has greater elasticity than the fabric in other areas of the glove body. In some embodiments, the elastic fabric structure or pleated structure refers to elastic fabric provided between the second strain sensors, or fabric with pleats, to avoid damaging the overall aesthetics of the glove body. In some embodiments, to further reduce pull between adjacent fingers, the slit structure, elastic fabric structure, or pleated structure extends along the length of the fingers. In some embodiments, the pleated structure includes multiple pleats arranged along the length of the fingers. In some embodiments, the elastic fabric structure or the pleated structure may exist simultaneously with the slit structure 470 and fill the slit structure 470, thereby minimizing the pulling between the fabrics at the base of adjacent fingers while maintaining the integrity of the fabric of the glove body.

[0152] In some embodiments, the processor is integrated into the glove body. For example, the processor can be located in the corresponding palm part. For another example, as shown in Figure 4, the processor 440 can be located in the corresponding back of the hand. The processor can read data related to multiple first strain sensors, multiple second strain sensors, inertial sensors and third strain sensors, and the user's gesture can be determined based on the data. For example, a function or machine learning is used to construct the relationship between data and finger shape. Specifically, the original data set of sensor data and joint angles is obtained, and a function is constructed based on the data set, or a machine learning model is trained. This achieves the mapping between sensor data and joint angles (or the mapping between sensor data and gestures). After determining the user's gesture, it can be used in various application scenarios such as sign language learning applications (action reproduction), deaf-mute voice, gesture control methods, etc.

[0153] In some embodiments, after determining a user gesture, the processor may output a gesture representation corresponding to the user's gesture on a display interface and output feedback information based on the distance between the gesture representation and a preset gesture. The display interface is provided by an external device, such as a mobile phone screen or a computer screen. The feedback information may include, for example, the comparison results of the user's gesture representation and the preset gesture. The distance between the gesture representation and the preset gesture may reflect the degree of similarity between the user's gesture and the preset gesture. The closer the distance, the higher the similarity between the user's gesture and the preset gesture. In some embodiments, the distance may be obtained by first mapping the gesture representation to the preset gesture and then calculating the distance between the gesture representation and the corresponding point on the preset gesture. The feedback information may be used to inform the user whether their gesture is identical to the preset gesture or the degree of similarity between their gesture and the preset gesture. For example, the feedback information may be the similarity between the gesture representation and the preset gesture expressed as a percentage. The processing flow in which the processor outputs feedback information based on the user's gesture and the distance between the gesture and the preset gesture can be applied to a variety of application scenarios, such as the gesture game application scenario shown in FIG12 and the sign language learning application scenario shown in FIG13.

[0154] FIG12 is a processing flow chart of a processor according to some embodiments of the present specification.

[0155] As shown in FIG. 12 , in some embodiments, in a gesture game application scenario, the processing flow 12 of the processor includes the following steps.

[0156] 121. Play the sample animation.

[0157] In some embodiments, the processor controls the display interface to display or play an animation containing exemplary gestures. The exemplary gestures are used to guide the user to perform gestures. In some embodiments, the process of playing the exemplary animation can be triggered after the user puts on the glove body, or can be triggered by the user himself.

[0158] 122. Glove data calibration.

[0159] In some embodiments, the user wears the glove body and performs gestures following the exemplary gestures to calibrate the glove system data. The glove system data calibration includes calibrating the deviation between the position of the strain sensor on the glove body and the corresponding joint position caused by wearing. Optionally, the example animation in step 121 can include a gesture of a specific posture. When the user wears the glove body and imitates the gesture of the specific posture, the processor can calibrate the data in the standard database based on the data collected at this time, so that the calibrated data can more accurately reflect the user's finger posture. In some embodiments, step 122 is not necessary, or can be performed before step 121.

[0160] 123. Collect player gestures.

[0161] In some embodiments, when a player wears gloves and makes gestures, the strain sensors on the gloves corresponding to the joints of the user's hands generate strain data, thereby enabling the collection of the player's gestures.

[0162] 124. Recognize gestures based on sensor data.

[0163] In some embodiments, the processor can restore the user's gestures based on strain data from strain sensors on the glove corresponding to each joint of the user's hand. In some embodiments, the user can make multiple gestures in sequence, and the processor can continuously recognize the user's multiple gestures based on the sensor data and store the user's gestures in sequence.

[0164] 125. You will pass if you can reproduce the whole set of actions.

[0165] In some embodiments, the processor may compare the stored multiple gestures of the user with multiple preset gestures in sequence, determine the distance between the multiple gesture representations and the multiple preset gestures, and output feedback information. If the distance between the multiple gesture representations and the multiple preset gestures is less than a preset threshold, it means that the full set of actions are reproduced, and the output feedback information is passed and the next game process is triggered. The distance between the multiple gesture representations and the multiple preset gestures can reflect the average similarity between the multiple gestures of the user and the corresponding multiple preset gestures. The smaller the distance, the higher the average similarity between the multiple gestures and the one-to-one corresponding multiple preset gestures. The preset threshold may correspond to, for example, an average degree of fit between the multiple gesture representations and the multiple preset gestures of 70%.

[0166] 126. If the action is not reproduced, restart the game.

[0167] In some embodiments, the processor may sequentially compare the stored multiple gestures of the user with multiple preset gestures, determine the distance between the multiple gesture representations and the multiple preset gestures, and output feedback information. If the distance between the multiple gesture representations and the multiple preset gestures is greater than a preset threshold (for example, corresponding to a 70% consistency between the multiple gesture representations and the multiple preset gestures), it indicates that the entire set of actions has not been reproduced, and the output feedback information is a gesture error.

[0168] 127. Replay the player's gesture process and play special effect animation.

[0169] In some embodiments, the processor can control the display interface to sequentially display gesture representations corresponding to the user's gestures. In some embodiments, when the user completes the full set of action reproduction, the processor can replay the player's gesture process and play special effect animations.

[0170] FIG13 is a processing flow chart of a processor according to still other embodiments of the present specification.

[0171] As shown in FIG. 13 , in some embodiments, in a sign language learning application scenario, the processing flow 13 of the processor includes the following steps.

[0172] 131. Glove data calibration.

[0173] In some embodiments, glove system data calibration includes calibrating the deviation between the position of strain sensors on the glove body and the corresponding joint positions caused by wearing the glove. In some embodiments, the user wears the glove body and can perform gestures following the calibration process displayed on the display interface to calibrate the glove system data.

[0174] 132. Play sample sign language.

[0175] In some embodiments, the processor controls the display interface to display or play sample sign language. The sample sign language is used to demonstrate standard sign language gestures to the user, thereby guiding the user to learn sign language. In some embodiments, the process of playing the sample sign language can be automatically triggered after the glove data is calibrated, or it can be triggered by the user.

[0176] 133. Collect user gestures.

[0177] In some embodiments, when a user wears gloves and makes gestures, the strain sensors on the gloves corresponding to the joints of the user's hands can generate strain data, thereby enabling the collection of the player's gestures.

[0178] 134. Recognize gestures based on sensor data.

[0179] In some embodiments, the processor can restore the user's gestures based on the strain data of the strain sensors on the glove corresponding to each joint of the user's hand. In some embodiments, the processor can display the restored user gestures on the display interface in the form of gesture representations.

[0180] 135. When the action is repeated, the corresponding audio is played.

[0181] In some embodiments, the processor may compare the gesture representation with the example sign language, determine the distance between the gesture representation and the example sign language, and output feedback information. If the distance between the gesture representation and the example sign language is less than a preset threshold (e.g., corresponding to a 70% match between the gesture representation and the example sign language), the action is reproduced, and the feedback information output is to end the training process and play the corresponding audio (e.g., congratulations on passing the training).

[0182] 136. If the action is not reproduced, retrain.

[0183] In some embodiments, the processor may compare the gesture representation with the example sign language, determine the distance between the gesture representation and the example sign language, and output feedback information. If the distance between the gesture representation and the example sign language is greater than a preset threshold (e.g., corresponding to a 70% match between the gesture representation and the example sign language), the action is not replicated, and the feedback information output is to restart the training process.

[0184] 137. After a period of time, assess the learned movements.

[0185] In some embodiments, the processor may mark the preset gestures (i.e., sample sign language) reproduced by the user and redisplay them at a later time. For example, the processor may automatically re-execute the training process for the preset gestures reproduced or restored by the user based on the user's instructions or at a later time to achieve repeated assessment.

[0186] In some embodiments, after determining a user gesture, the processor can read the mapping relationship between the gesture and the voice data packet, and then play the voice information based on the user gesture and the mapping relationship. The mapping relationship between gestures and voice data packets can be pre-built and stored in a database, or provided by a third party. The processor plays the voice information based on the user gesture and the mapping relationship, which can be applied to various application scenarios, such as the application scenario of assisting the deaf and mute to speak as shown in Figure 14.

[0187] FIG14 is a flowchart illustrating processing of a processor according to some embodiments of the present specification.

[0188] As shown in FIG. 14 , in some embodiments, in an application scenario of assisting deaf-mute people in speaking, the processing flow 14 of the processor includes the following steps.

[0189] 141. Glove data calibration.

[0190] In some embodiments, glove system data calibration includes calibrating the deviation between the position of strain sensors on the glove body and the corresponding joint positions caused by wearing the glove. In some embodiments, the user wears the glove body and can perform gestures following the calibration process displayed on the display interface to calibrate the glove system data.

[0191] 142. Recognize gestures based on sensor data.

[0192] In some embodiments, when a user wears the gloves and makes a sign language gesture, the strain sensors on the gloves corresponding to the user's hand joints generate strain data. The processor can then reconstruct the user's gesture based on the strain data from the strain sensors on the gloves corresponding to the user's hand joints. In some embodiments, the user can make multiple gestures in succession to express a complete sentence. The processor can continuously recognize the user's multiple gestures based on the sensor data and store the gestures in sequence.

[0193] 143. The loudspeaker plays the sound corresponding to the sign language.

[0194] In some embodiments, the processor recognizes a user gesture and controls the speaker to play voice information corresponding to the user gesture. The speaker may be provided by an external device. In some embodiments, the processor may recognize a user gesture and determine the distance between the user gesture and a preset gesture. For example, if the distance between the user gesture and the preset gesture is less than a preset threshold (for example, the degree of consistency between the multiple gesture representations and the multiple preset gestures is 70%), the voice information is controlled to be played; if the distance between the user gesture and the multiple preset gestures is greater than a preset threshold (for example, the degree of consistency between the multiple gesture representations and the multiple preset gestures is 70%), the voice information is not played, and the user is prompted to re-enter the gesture.

[0195] FIG15 is a flowchart illustrating processing of a processor according to some embodiments of the present specification.

[0196] In some embodiments, after determining a user gesture, the processor can read a mapping between the gesture and an execution instruction, and then execute the instruction based on the user gesture and the mapping. The mapping between gestures and execution instructions can be pre-built and stored in a database, or provided by a third party. The processor executes instructions based on the user gesture and the mapping, which can be applied to a variety of application scenarios, such as the gesture control application scenario shown in Figure 15.

[0197] FIG15 is a flowchart illustrating processing of a processor according to some embodiments of the present specification.

[0198] As shown in FIG. 15 , in some embodiments, in an application scenario of gesture control, the processing flow 15 of the processor includes the following steps.

[0199] 151. Predefined operation gestures.

[0200] In some embodiments, the operation gesture can be predefined by the user, that is, the mapping relationship between the gesture and the execution instruction can be predefined by the user. In some embodiments, the user can pre-give the mapping relationship between the gesture and the corresponding execution instruction of the external device (such as a game console, an Internet of Things device). For example, the user can define a mapping relationship between a five-finger closing gesture and an external device executing a screenshot instruction. For example, the user can define a mapping relationship between a five-finger opening gesture and an external device executing a play instruction. For another example, the user can define a V-shaped gesture composed of the index finger and the middle finger and an external device executing a photo taking instruction to establish a mapping relationship.

[0201] 152. Glove data calibration.

[0202] In some embodiments, glove system data calibration includes calibrating the deviation between the position of strain sensors on the glove body and the corresponding joint positions caused by wearing the glove. In some embodiments, glove system data calibration can be performed periodically. The user wears the glove body and can perform gestures following the calibration process displayed on the display interface to calibrate the glove system data.

[0203] 153. Collect user gestures.

[0204] In some embodiments, when a user wears gloves and makes gestures, the strain sensors on the gloves corresponding to the joints of the user's hands can generate strain data, thereby enabling the collection of the player's gestures.

[0205] 154. Recognize gestures based on sensor data.

[0206] In some embodiments, the processor can restore the user's gestures based on the strain data of the strain sensors on the glove corresponding to each joint of the user's hand. In some embodiments, the processor can display the restored user gestures on the display interface in the form of gesture representations.

[0207] 155. Execute commands based on gestures.

[0208] In some embodiments, the processor recognizes user gestures and, based on a predefined mapping between user gestures and execution instructions, controls the external device to execute the instruction corresponding to the user gesture. For example, a user can input a five-finger pinch gesture to control the external device to execute a screenshot instruction. For example, a user can input a five-finger spread gesture to control the external device to execute a play instruction. For another example, a user can input a V-shaped gesture with the index and middle fingers to control the external device to execute a photo-taking instruction.

[0209] FIG16 is a flowchart of a method for gesture replication according to some embodiments of this specification.

[0210] As shown in FIG. 16 , in some embodiments, the gesture replication method 16 includes the following steps.

[0211] 161. Obtain sensor data from the user's hand based on the glove system.

[0212] In some embodiments, a glove system includes a glove body and a plurality of first strain sensors and a plurality of second strain sensors arranged on the glove body. When a user wears the glove body, each of the plurality of first strain sensors is located at an interphalangeal joint of the user and is configured to measure deformation of the interphalangeal joint in a single degree of freedom. Each of the plurality of second strain sensors is located at a metacarpophalangeal joint and is configured to measure deformation of the metacarpophalangeal joint in two degrees of freedom, where both the single degree of freedom and the two degrees of freedom include a degree of freedom corresponding to finger flexion.

[0213] In some embodiments, the first strain sensor comprises a single-degree-of-freedom sensor, such as a single-axis sensor (e.g., an inductive, resistive bending sensor, or a capacitive stretch sensor). In some embodiments, the second strain sensor comprises a multi-degree-of-freedom sensor, such as a multi-axis sensor (e.g., a capacitive multi-axis motion sensor). The deformation of the interphalangeal joint corresponds to the single degree of freedom of bending. The deformation of the metacarpophalangeal joint corresponds to the two degrees of freedom of bending and swinging. Thus, each first strain sensor can measure the deformation of one interphalangeal joint among the multiple interphalangeal joints of the hand in the bending degree of freedom. That is, when any interphalangeal joint undergoes bending motion, the first strain sensor can measure the deformation of the glove body at the corresponding interphalangeal joint. Each second strain sensor can measure the deformation of one metacarpophalangeal joint among the multiple metacarpophalangeal joints in the bending degree of freedom. That is, when any metacarpophalangeal joint undergoes bending or swinging motion, the second strain sensor can measure the deformation of the glove body at the corresponding metacarpophalangeal joint. In total, the multiple first strain sensors and the multiple second strain sensors jointly provide deformation data for at least 15 degrees of freedom on at least one finger of the user. For more information about the glove system, please refer to the relevant section above.

[0214] 162. Determine a user's gesture based on sensor data.

[0215] In some embodiments, when a user makes a gesture, the strain sensors on the glove corresponding to the user's hand joints can generate strain data. The processor can restore the user's gesture based on the strain data of the strain sensors on the glove corresponding to the user's hand joints.

[0216] 163. Output a gesture representation corresponding to the gesture on a display interface.

[0217] In some embodiments, after determining the user gesture, a gesture representation corresponding to the user's gesture can be output on the display interface. The gesture representation can be a hand posture represented or simulated by points, lines, surfaces, etc. or a combination thereof. For example, multiple points can be used on the display interface to represent multiple joints of the user's hand, and the lines between some of the points can be used to represent the bones or hand frames between the joints. For another example, a hand in a corresponding posture can be simulated based on the recognized gesture and displayed on the display interface. The display interface is provided by an external device, such as a mobile phone screen or a computer screen. In some embodiments, the user can make multiple gestures in sequence, and the processor can continuously recognize the user's multiple gestures based on the sensor data, and display the user's gestures in sequence.

[0218] 164. Output feedback information according to the distance between the gesture representation and the preset gesture.

[0219] In some embodiments, the processor can output feedback information based on the distance between the gesture representation and the preset gesture. The feedback information may include the comparison result of the user's gesture representation and the preset gesture, etc. The distance between the gesture representation and the preset gesture can reflect the similarity between the user's gesture and the preset gesture. The closer the distance, the higher the similarity between the user's gesture and the preset gesture. In some embodiments, the distance can be obtained by first mapping the gesture representation with the preset gesture, and further calculating the distance between the gesture representation and the corresponding point on the preset gesture. The feedback information can be used to remind the user whether his gesture is the same as the preset gesture or the similarity between his gesture and the preset gesture. For example, the feedback information can be the similarity between the gesture representation and the preset gesture expressed as a percentage. The processing flow of the processor outputting feedback information based on the user's gesture and the distance between it and the preset gesture can be applied to a variety of application scenarios, such as gesture game application scenarios or sign language learning application scenarios.

[0220] FIG17 is a flow chart of a method for voice playback according to some embodiments of this specification.

[0221] As shown in FIG. 17 , in some embodiments, the voice playing method 17 includes the following steps.

[0222] 171. Obtain sensor data from the user's hand based on the glove system.

[0223] In some embodiments, a glove system includes a glove body and a plurality of first strain sensors and a plurality of second strain sensors arranged on the glove body. When a user wears the glove body, each of the plurality of first strain sensors is located at an interphalangeal joint of the user and is configured to measure deformation of the interphalangeal joint in a single degree of freedom. Each of the plurality of second strain sensors is located at a metacarpophalangeal joint and is configured to measure deformation of the metacarpophalangeal joint in two degrees of freedom, where both the single degree of freedom and the two degrees of freedom include a degree of freedom corresponding to finger flexion.

[0224] In some embodiments, the first strain sensor comprises a single-degree-of-freedom sensor, such as a single-axis sensor (e.g., an inductive, resistive bending sensor, or a capacitive stretch sensor). In some embodiments, the second strain sensor comprises a multi-degree-of-freedom sensor, such as a multi-axis sensor (e.g., a capacitive multi-axis motion sensor). The deformation of the interphalangeal joint corresponds to the single degree of freedom of bending. The deformation of the metacarpophalangeal joint corresponds to the two degrees of freedom of bending and swinging. Thus, each first strain sensor can measure the deformation of one interphalangeal joint among the multiple interphalangeal joints of the hand in the bending degree of freedom. That is, when any interphalangeal joint undergoes bending motion, the first strain sensor can measure the deformation of the glove body at the corresponding interphalangeal joint. Each second strain sensor can measure the deformation of one metacarpophalangeal joint among the multiple metacarpophalangeal joints in the bending degree of freedom. That is, when any metacarpophalangeal joint undergoes bending or swinging motion, the second strain sensor can measure the deformation of the glove body at the corresponding metacarpophalangeal joint. In total, the multiple first strain sensors and the multiple second strain sensors jointly provide deformation data for at least 15 degrees of freedom on at least one finger of the user. For more information about the glove system, please refer to the relevant section above.

[0225] 172. Determine a user's gesture based on sensor data.

[0226] In some embodiments, when a user makes a gesture, the strain sensors on the glove corresponding to the user's hand joints can generate strain data. The processor can restore the user's gesture based on the strain data of the strain sensors on the glove corresponding to the user's hand joints.

[0227] 173. Read the mapping relationship between gestures and voice data packets.

[0228] In some embodiments, the mapping relationship between gestures and voice data packets can be predefined by the user. For example, the user can define a mapping relationship between a five-finger folded gesture and the voice "stop". For example, the user can define a mapping relationship between a five-finger spread gesture and the voice "hey". For another example, the user can define a mapping relationship between a V-shaped gesture composed of the index finger and the middle finger and the voice "take a photo". In some embodiments, the mapping relationship between gestures and voice data packets can also be established by a sign language database, which includes a mapping relationship between sign language gestures and expression vocabulary.

[0229] 174. Play voice information based on the user's gesture and the mapping relationship.

[0230] In some embodiments, the processor recognizes user gestures and controls an external device (such as a speaker) to play voice information based on the mapping relationship between the gestures and the voice data packets. For example, the user can input a gesture of five fingers together to control the external device to output the voice "stop". For example, the user can input a gesture of five fingers spread out to control the external device to output the voice "Hey". For another example, the user can input a V-shaped gesture of the index finger and middle finger to control the external device to output the voice "take a photo". For another example, the user can input a sign language gesture to control the external device to output the voice corresponding to the sign language.

[0231] For more information about the voice playback method, please refer to Figure 14 and its related description.

[0232] FIG18 is a flow chart of a gesture control method according to some embodiments of this specification.

[0233] As shown in FIG. 18 , in some embodiments, the gesture control method 18 includes the following steps.

[0234] 181. Obtain sensor data from the user's hand based on the glove system.

[0235] In some embodiments, the execution instruction glove system includes a glove body and a plurality of first strain sensors and a plurality of second strain sensors arranged on the execution instruction glove body. When the user wears the execution instruction glove body, each of the plurality of first strain sensors for executing instructions is located at an interphalangeal joint of the execution instruction user and is configured to measure the deformation of the interphalangeal joint for executing instructions in a single degree of freedom. Each of the plurality of second strain sensors for executing instructions is located at a metacarpophalangeal joint and is configured to measure the deformation of the metacarpophalangeal joint for executing instructions in two degrees of freedom. Both the single degree of freedom for executing instructions and the two degrees of freedom for executing instructions include the degree of freedom corresponding to finger bending.

[0236] In some embodiments, the first strain sensor comprises a single-degree-of-freedom sensor, such as a single-axis sensor (e.g., an inductive, resistive bending sensor, or a capacitive stretch sensor). In some embodiments, the second strain sensor comprises a multi-degree-of-freedom sensor, such as a multi-axis sensor (e.g., a capacitive multi-axis motion sensor). The interphalangeal joints correspond to the single degree of freedom of bending. The metacarpophalangeal joints correspond to the two degrees of freedom of bending and swinging. Thus, each first strain sensor can measure the deformation of one of the nine interphalangeal joints of the hand in the bending degree of freedom. That is, when any interphalangeal joint undergoes bending motion, the first strain sensor can measure the deformation of the glove body at the corresponding interphalangeal joint. Each second strain sensor can measure the deformation of one of the five metacarpophalangeal joints in the bending degree of freedom. That is, when any metacarpophalangeal joint undergoes bending or swinging motion, the second strain sensor can measure the deformation of the glove body at the corresponding metacarpophalangeal joint. In total, the multiple first strain sensors and the multiple second strain sensors collectively provide deformation data for at least 15 degrees of freedom on at least one user's finger. For more information about the glove system, please refer to the relevant section above.

[0237] 182. Determine a user's gesture based on sensor data.

[0238] In some embodiments, when a user wears gloves and makes gestures, strain sensors on the gloves corresponding to the user's hand joints generate strain data. A processor can reconstruct the user's gestures based on the strain data from the strain sensors on the gloves corresponding to the user's hand joints. In some embodiments, the processor can display the reconstructed user gestures on a display interface as gesture representations.

[0239] 183. Mapping relationship between reading gestures and executing instructions.

[0240] In some embodiments, the mapping relationship between gestures and execution instructions can be predefined by the user. In some embodiments, the user can pre-set the mapping relationship between gestures and corresponding execution instructions of external devices (such as game consoles). For example, the user can define a mapping relationship between a five-finger closing gesture and an external device executing a screenshot instruction. For example, the user can define a mapping relationship between a five-finger opening gesture and an external device executing a play instruction. For another example, the user can define a mapping relationship between a V-shaped gesture composed of the index finger and the middle finger and an external device executing a photo taking instruction. In some embodiments, the mapping relationship between gestures and execution instructions can also be defined by the manufacturer.

[0241] 184. Execute instructions based on the user's gesture and the mapping relationship.

[0242] In some embodiments, the processor recognizes user gestures and, based on a predefined mapping between user gestures and execution instructions, controls the external device to execute the instruction corresponding to the user gesture. For example, a user can input a five-finger pinch gesture to control the external device to execute a screenshot instruction. For example, a user can input a five-finger spread gesture to control the external device to execute a play instruction. For another example, a user can input a V-shaped gesture with the index and middle fingers to control the external device to execute a photo-taking instruction.

[0243] For more information about the gesture control method, please refer to Figure 15 and its related description.

[0244] While the basic concepts have been described above, it will be apparent to those skilled in the art that the detailed disclosure is merely illustrative and does not limit this specification. Although not explicitly stated herein, various modifications, improvements, and revisions to this specification may be made by those skilled in the art. Such modifications, improvements, and revisions are suggested in this specification and remain within the spirit and scope of the exemplary embodiments of this specification.

[0245] This specification also uses specific terms to describe the embodiments of this specification. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "one embodiment," "an embodiment," or "an alternative embodiment" two or more times in different locations in this specification do not necessarily refer to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of this specification may be appropriately combined.

[0246] In addition, unless expressly stated in the claims, the order of the processing elements and sequences, the use of alphanumeric characters, or the use of other names described in this specification are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some of the invention embodiments currently considered useful through various examples, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the spirit and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0247] Similarly, it should be noted that, in order to simplify the presentation of this specification and thus facilitate understanding of one or more embodiments of the invention, the foregoing descriptions of the embodiments of this specification sometimes combine multiple features into a single embodiment, figure, or description thereof. However, this disclosure method does not imply that the subject matter of this specification requires more features than those recited in the claims. In fact, an embodiment may have fewer features than all of the features of a single disclosed embodiment.

[0248] In some embodiments, numbers are used to describe the quantity of components and attributes. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise stated, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the description and claims are approximate values, which may change according to the required characteristics of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the settings of such numerical values ​​are as accurate as possible within the feasible range.

[0249] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, and documents, cited in this specification is hereby incorporated by reference in its entirety. This includes application history documents that are inconsistent with or conflict with the content of this specification, as well as documents (currently or subsequently attached to this specification) that limit the broadest scope of the claims of this specification. It should be noted that if the descriptions, definitions, and / or terminology used in the accompanying materials are inconsistent or conflicting with the content of this specification, the descriptions, definitions, and / or terminology used in this specification will control.

[0250] Finally, it should be understood that the embodiments described in this specification are intended only to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly described and illustrated in this specification.

Claims

1. A glove system, comprising: Glove body; as well as A plurality of first strain sensors and a plurality of second strain sensors are arranged on the glove body, wherein when a user wears the glove body, each of the plurality of first strain sensors is located at an interphalangeal joint of the user and is configured to measure the deformation of the interphalangeal joint in a single degree of freedom, and each of the plurality of second strain sensors is located at a metacarpophalangeal joint and is configured to measure the deformation of the metacarpophalangeal joint in two degrees of freedom, wherein the single degree of freedom and the two degrees of freedom both include a degree of freedom corresponding to finger bending.

2. The glove system according to claim 1, wherein the plurality of first strain sensors and the plurality of second strain sensors together provide data reflecting deformation of no less than 15 degrees of freedom on the user's fingers.

3. The glove system according to claim 1, further comprising two inertial sensors, wherein: When the user wears the glove body, the two inertial sensors are respectively located on both sides of the user's wrist joint and are configured to jointly measure the movement of the wrist joint in three degrees of freedom.

4. The glove system according to claim 1, further comprising a third strain sensor located at a wrist joint of the user and an inertial sensor located outside the wrist joint. 5 . The glove system according to claim 1 , wherein the first strain sensor comprises an inductive sensor, and an inductance of the inductive sensor changes with deformation of the interphalangeal joint. 6 . The glove system according to claim 1 , wherein the second strain sensor comprises a plurality of capacitive structures, and the capacitances of the plurality of capacitive structures change respectively as the metacarpophalangeal joint changes in each degree of freedom.

7. The glove system according to claim 6, wherein the second strain sensor comprises: Flexible substrate; The first capacitor structure and the second capacitor structure each include a multilayer structure arranged on the same side surface of the flexible substrate in the thickness direction, and each layer of the multilayer structure is stacked along the thickness direction.

8. The glove system according to claim 1, wherein the glove body comprises a cloth wrapped around each finger of the user and a cloth located between adjacent fingers, wherein: The elastic modulus of the cloth between the cloths of adjacent fingers is smaller than the elastic modulus of the cloth on each of the fingers.

9. The glove system according to claim 8, wherein the fabric between adjacent fingers has at least one of a slit structure, an elastic fabric structure or a pleated structure. 10 . The glove system according to claim 1 , wherein the plurality of first strain sensors are arranged only at interphalangeal joints of each finger of the user closest to the metacarpophalangeal joint.

11. The glove system of claim 1 , further comprising a processor configured to: reading data associated with the plurality of first strain sensors and the plurality of second strain sensors; and A gesture of the user is determined based on the data.

12. The glove system according to claim 11, wherein the processor is configured to: Outputting a gesture representation corresponding to the user's gesture on a display interface; and Output feedback information according to the distance between the gesture representation and the preset gesture.

13. The glove system according to claim 11, wherein the processor is configured to: Read the mapping relationship between gestures and voice data packets; and The voice information is played based on the user's gesture and the mapping relationship.

14. The glove system according to claim 11, wherein the processor is configured to: Read the mapping relationship between gestures and execution instructions; and An instruction is executed based on the user's gesture and the mapping relationship.

15. A gesture reproduction method, comprising: Acquiring sensor data of a user's hand based on a glove system, wherein the glove system includes a glove body and a plurality of first strain sensors and a plurality of second strain sensors arranged on the glove body, when the user wears the glove body, each of the plurality of first strain sensors is located at an interphalangeal joint of the user and is configured to measure the deformation of the interphalangeal joint in a single degree of freedom, each of the plurality of second strain sensors is located at a metacarpophalangeal joint and is configured to measure the deformation of the metacarpophalangeal joint in two degrees of freedom, wherein the single degree of freedom and the two degrees of freedom both include a degree of freedom corresponding to finger bending; determining a gesture of the user based on the sensor data; Outputting a gesture representation corresponding to the gesture on a display interface; and Output feedback information according to the distance between the gesture representation and the preset gesture. 16 . The gesture reproduction method according to claim 15 , wherein the plurality of first strain sensors and the plurality of second strain sensors jointly provide data reflecting deformation of no less than 15 degrees of freedom on the user's fingers.

17. A voice playing method, comprising: Acquiring sensor data of a user's hand based on a glove system, wherein the glove system includes a glove body and a plurality of first strain sensors and a plurality of second strain sensors arranged on the glove body, when the user wears the glove body, each of the plurality of first strain sensors is located at an interphalangeal joint of the user and is configured to measure the deformation of the interphalangeal joint in a single degree of freedom, each of the plurality of second strain sensors is located at a metacarpophalangeal joint and is configured to measure the deformation of the metacarpophalangeal joint in two degrees of freedom, wherein the single degree of freedom and the two degrees of freedom both include a degree of freedom corresponding to finger bending; determining a gesture of the user based on the sensor data; Read the mapping relationship between gestures and voice data packets; and The voice information is played based on the user's gesture and the mapping relationship.

18. According to the voice playing method of claim 17, the plurality of first strain sensors and the plurality of second strain sensors jointly provide data reflecting deformation of no less than 15 degrees of freedom on the user's fingers.

19. A gesture control method, comprising: Acquiring sensor data of a user's hand based on a glove system, wherein the glove system includes a glove body and a plurality of first strain sensors and a plurality of second strain sensors arranged on the glove body, when the user wears the glove body, each of the plurality of first strain sensors is located at an interphalangeal joint of the user and is configured to measure the deformation of the interphalangeal joint in a single degree of freedom, each of the plurality of second strain sensors is located at a metacarpophalangeal joint and is configured to measure the deformation of the metacarpophalangeal joint in two degrees of freedom, wherein the single degree of freedom and the two degrees of freedom both include a degree of freedom corresponding to finger bending; determining a gesture of the user based on the sensor data; Read the mapping relationship between gestures and execution instructions; and An instruction is executed based on the user's gesture and the mapping relationship. 20 . The gesture control method according to claim 19 , wherein the plurality of first strain sensors and the plurality of second strain sensors jointly provide data reflecting deformation of no less than 15 degrees of freedom on the user's fingers.

Images