Multimodal flexible glove, medical system, quantification method, and manufacturing method

Abstract

This invention provides a multimodal flexible glove, a medical system, a quantification method, and a manufacturing method, which can be applied to artificial intelligence, flexible electronics, and medical devices. The multimodal flexible glove may include: a flexible glove body; a flexible strain sensor array generating multiple sets of joint stress signals; a flexible pressure sensor array generating multiple sets of pressure intensity signals; and a data processing circuit that sends multiple sets of joint stress data and multiple sets of pressure intensity data to a host computer based on the multiple sets of joint stress signals and multiple sets of pressure intensity signals, respectively. This allows the host computer to input the multiple sets of joint stress data, the receiving time of the multiple sets of joint stress data, the setting position of the flexible strain sensor array on the flexible glove body, the multiple sets of pressure intensity data, the receiving time of the multiple sets of pressure intensity data, and the setting position of the flexible pressure sensor array on the flexible glove body into a trained neural network, outputting first action type information and a first quantification evaluation result for the medical action.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, flexible electronics, and medical device technology, specifically to a multimodal flexible glove, a medical system, a quantification method, and a manufacturing method. Background Technology

[0002] Traditional Chinese medicine techniques such as acupuncture and massage, as treasures of Chinese medicine, have a long history and remarkable clinical efficacy. Their therapeutic effects largely depend on the physician's skillful manipulation techniques. The precision of these techniques not only reflects the physician's experience level but is also a key factor affecting the consistency and repeatability of treatment. Therefore, objective and accurate quantitative evaluation of medical manipulation techniques has significant clinical and educational value. Summary of the Invention

[0003] In view of this, the present invention provides a multimodal flexible glove, a medical system, a quantification method, and a manufacturing method.

[0004] According to a first aspect of the present invention, a multimodal flexible glove is provided, comprising: a flexible glove body having a first outer surface corresponding to the back of the hand and a second outer surface corresponding to the palm; a flexible strain sensor array disposed at multiple joint positions on the first outer surface, for generating multiple sets of joint stress signals at multiple times based on the degree of joint flexion of the target object when the target object wears the multimodal flexible glove and performs a medical action; wherein the medical action includes acupuncture, the multiple joint positions are the interphalangeal joint and metacarpophalangeal joint of the thumb, the interphalangeal joint and metacarpophalangeal joint of the index finger, and the interphalangeal joint and metacarpophalangeal joint of the middle finger, and the multiple sets of joint stress signals correspond to the amplitude of the lifting and thrusting techniques in acupuncture; and a flexible pressure sensor array disposed at at least one position on the palm or fingertip of the second outer surface, for generating multiple sets of joint stress signals based on the degree of joint flexion of the target object when the target object wears the multimodal flexible glove and performs a medical action. The pressure is applied to generate multiple sets of pressure intensity signals at multiple times; at least one location is the fingertip of the thumb and index finger, and the pressure intensity signals correspond to the pressure distribution during needle handle twisting and acupoint pressing; a data processing circuit, at least partially located on the back of the hand on the first outer surface, is electrically connected to a flexible strain sensor array and a flexible pressure sensor array, and is used to send multiple sets of joint stress data and multiple sets of pressure intensity data to the host computer based on multiple sets of joint stress signals and multiple sets of pressure intensity signals, so that the host computer inputs the multiple sets of joint stress data, the receiving time of the multiple sets of joint stress data, the setting position of the flexible strain sensor array on the flexible glove body, the multiple sets of pressure intensity data, the receiving time of the multiple sets of pressure intensity data, and the setting position of the flexible pressure sensor array on the flexible glove body into a trained neural network, and outputs the first action type information and the first quantitative evaluation result of the medical action.

[0005] According to a second aspect of the present invention, a medical system is provided, comprising the aforementioned multimodal flexible glove and host computer.

[0006] According to a third aspect of the present invention, a quantification method is provided, comprising: receiving multiple sets of joint stress data and multiple sets of pressure intensity data at multiple times from the aforementioned multimodal flexible glove; inputting the multiple sets of joint stress data, the receiving times of the multiple sets of joint stress data, the setting position of the flexible strain sensor array on the flexible glove body, the multiple sets of pressure intensity data, the receiving times of the multiple sets of pressure intensity data, and the setting position of the flexible pressure sensor array on the flexible glove body into a trained neural network, and outputting first action type information of the medical action and a first quantification evaluation result.

[0007] According to a fourth aspect of the present invention, a method for manufacturing the above-mentioned multimodal flexible glove is provided, comprising: obtaining a flexible glove body; preparing a flexible strain sensor array and disposing the flexible strain sensor array on the flexible glove body; preparing a flexible pressure sensor array and disposing the flexible pressure sensor array on the flexible glove body; disposing a data processing circuit on the flexible glove body and electrically connecting the data processing circuit to the flexible strain sensor array and the flexible pressure sensor array.

[0008] According to an embodiment of the present invention, a multimodal flexible glove is provided. During medical procedures performed by a target subject wearing the multimodal flexible glove, a flexible strain sensor array on the glove body can generate multiple sets of joint stress signals at multiple times based on the joint flexion degree of the target subject. Simultaneously, a flexible pressure sensor can generate multiple sets of pressure intensity signals at multiple times based on the pressure applied to at least one of the fingertips and palms of the target subject. A data processing circuit can send multiple sets of joint stress data and multiple sets of pressure intensity data to a host computer based on the multiple sets of joint stress signals and multiple sets of pressure intensity signals, respectively.

[0009] In this way, the host computer can input multiple sets of joint stress data, the receiving time of multiple sets of joint stress data, the setting position of the flexible strain sensor array on the flexible glove body, multiple sets of pressure data, the receiving time of multiple sets of pressure data, and the setting position of the flexible pressure sensor array on the flexible glove body into the trained neural network, and output the first action type information and the first quantitative evaluation result of the medical action. Thus, when the target object is performing a medical action, the specific type of medical action performed by the target object can be accurately determined, and the standardization of that type of medical action can be accurately quantified.

[0010] Furthermore, in this embodiment of the invention, the flexible glove body, flexible strain sensor, and flexible pressure sensor are lightweight, breathable, and have a high degree of fit, which can reduce the interference of wearing gloves on the doctor's hand movements, ensure natural feel and flexibility in real acupuncture and massage operation scenarios, and improve the clinical reliability of the collected joint stress data and pressure data.

[0011] Based on this, a flexible strain sensor is disposed at a joint position on the first outer surface of the flexible glove body, a flexible pressure sensor is disposed at at least one position on the fingertip or palm of the second outer surface of the flexible glove body, and a data processing circuit is at least partially disposed on the back of the hand on the first outer surface of the flexible glove body. This arrangement reduces the impact of at least one of the flexible strain sensor, flexible pressure sensor, or data processing circuit on the target object's medical movements. This allows for multi-dimensional real-time acquisition of key manipulation parameters (i.e., the aforementioned joint stress data and pressure data) while the target object performs the medical movements naturally, providing a reliable data foundation for the digital evaluation, intelligent teaching, and standardized inheritance of traditional Chinese medicine acupuncture and massage techniques. Attached Figure Description

[0012] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, which will be explained in conjunction with the drawings.

[0013] Figure 1 A schematic diagram of a medical system according to an embodiment of the present invention is shown.

[0014] Figure 2 A schematic diagram of a multimodal flexible glove according to an embodiment of the present invention is shown.

[0015] Figure 3 A connection diagram of a data processing circuit according to an embodiment of the present invention is shown.

[0016] Figure 4 A schematic diagram of a data processing circuit according to an embodiment of the present invention is shown. Figure 1 .

[0017] Figure 5 A schematic diagram of a data processing circuit according to an embodiment of the present invention is shown. Figure 2 .

[0018] Figure 6 A schematic diagram of a medical system according to another embodiment of the present invention is shown.

[0019] Figure 7 A schematic diagram of a neural network according to an embodiment of the present invention is shown.

[0020] Figure 8A schematic diagram of a method for quantifying medical actions according to an embodiment of the present invention is shown.

[0021] Figure 9 A schematic diagram of a method for manufacturing a multimodal flexible glove according to an embodiment of the present invention is shown.

[0022] Figure 10 The test curves for the cyclic stability test of the flexible strain sensor according to an embodiment of the present invention are shown.

[0023] Figure 11 Test curves for the response / recovery time of a flexible strain sensor according to an embodiment of the present invention are shown.

[0024] Figure 12 The test curve of the strain-resistance response curve of the flexible strain sensor according to an embodiment of the present invention is shown.

[0025] Figure 13 The test curves for the load-unload hysteresis test of a flexible strain sensor according to an embodiment of the present invention are shown.

[0026] Figure 14 The test curve of a stepped strain cycle test of a flexible strain sensor according to an embodiment of the present invention is shown.

[0027] Figure 15 The test curves for the continuous gradual strain response test of the flexible strain sensor according to an embodiment of the present invention are shown.

[0028] Figure 16 A schematic diagram of a polytetrafluoroethylene mold according to an embodiment of the present invention is shown.

[0029] Figure 17 A schematic diagram of a polytetrafluoroethylene mold according to another embodiment of the present invention is shown.

[0030] Figure 18 A schematic diagram of a sensor array of a flexible pressure sensor according to an embodiment of the present invention is shown.

[0031] Figure 19 The test curves for the cyclic stability test of the flexible pressure sensor according to an embodiment of the present invention are shown.

[0032] Figure 20 Test curves for the response / recovery time of a flexible pressure sensor according to an embodiment of the present invention are shown.

[0033] Figure 21 The test curve of the stepped pressure response test of the flexible pressure sensor according to an embodiment of the present invention is shown. Detailed Implementation

[0034] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0035] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0036] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0037] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0038] In the embodiments of this invention, the collection, updating, analysis, processing, use, transmission, provision, and storage of data (e.g., including but not limited to user personal information) comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures have been taken to prevent unauthorized access to user personal information data and to maintain user personal information security and network security.

[0039] In the embodiments of the present invention, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0040] It should be noted that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. The terms "installed," "connected," and "linked" should be interpreted broadly, for example, they can be fixed connections, detachable connections, or integral connections; they can be mechanical connections or electrical connections; they can be direct connections or indirect connections through an intermediate medium; they can be internal connections between two elements. The terms "parallel," "perpendicular," and "equal" include the described situation and situations similar to the described situation, the range of which is within an acceptable deviation range, wherein the acceptable deviation range is determined by those skilled in the art taking into account the measurement under discussion and the error associated with the measurement of a particular quantity (i.e., the limitations of the measurement system). For example, "parallel" includes absolute parallelism and approximate parallelism, where an acceptable deviation range for approximate parallelism can be, for example, within 5°; "perpendicular" includes absolute perpendicularity and approximate perpendicularity, where an acceptable deviation range for approximate perpendicularity can also be, for example, within 5°. "Equal" includes absolute equality and approximate equality, where an acceptable deviation range for approximate equality can be, for example, a difference between the two equal items being less than or equal to 5% of either one. Those skilled in the art will understand the specific meaning of the above terms in this application based on the specific circumstances.

[0041] Traditional Chinese medicine, including acupuncture and massage, boasts a long history and remarkable clinical efficacy. The therapeutic effect largely depends on the physician's skillful manipulation techniques. The core of acupuncture lies in the precise control of needling techniques such as "lifting and inserting" and "twisting," while massage encompasses various techniques including "pressing, pressing, tapping, kneading, and pinching." The intensity, frequency, amplitude, and dynamic combination of these techniques collectively determine the generation of the "deqi" sensation and the effectiveness of the treatment. The precision of the manipulation not only reflects the physician's experience level but is also a key factor affecting the consistency and repeatability of treatment. Therefore, objective and quantitative evaluation of medical techniques (such as acupuncture, massage, or cardiopulmonary resuscitation) has significant clinical and educational value.

[0042] Furthermore, the transmission and teaching of traditional Chinese medicine acupuncture and massage primarily rely on the traditional apprenticeship system and oral transmission model, where learners master operational techniques through observation, imitation, and personal perception. While this experience-based teaching method retains the humanistic characteristics of traditional medicine, its evaluation system mainly depends on the instructor's subjective judgment and the patient's feedback, lacking unified quantitative standards. This results in inconsistent teaching quality, difficulty in accurately recording and standardizing high-level techniques, and learners' inability to obtain real-time, objective feedback on mechanical parameters. The inability to effectively quantify key parameters such as force, frequency, and angle during the manipulation process is a major bottleneck in the objectification and standardization of TCM external treatment operations.

[0043] In some proposed solutions, researchers have attempted to use motion capture systems or data gloves as sensing devices to digitally record and analyze traditional Chinese medicine acupuncture and massage techniques. However, these sensing devices, which rely on rigid sensors, still suffer from significant technical compatibility issues when applied to acupuncture and massage scenarios. The main problems are as follows: First, in terms of sensing capabilities, these devices lack effective means to perceive the minute displacements during acupuncture, such as the "lifting and thrusting" motion, and the subtle changes in contact force caused by the "twisting" operation. This weakens or even eliminates some crucial operational details at the data level. Second, regarding multi-scale information acquisition, limited by the inherent design of sensor range and performance parameters, these devices struggle to simultaneously detect minute force changes and large-scale deformations. Information at different scales is processed in segments, making it difficult to simultaneously meet the combined requirements of micro-Newton-level pressure sensitivity and large-scale strain monitoring.

[0044] In practical use, the aforementioned sensing devices also exhibit shortcomings in human-machine adaptability. These devices often employ rigid structures or wired connections, which to some extent restrict natural hand movements, potentially altering the user's force application and operating habits, thus affecting the accuracy of the collected data. At the data processing level, these solutions largely remain at the stage of single-modal feature analysis or simple fusion, lacking effective mechanisms for temporal alignment of multimodal signals, cross-modal correlation modeling, and high-level semantic expression, making it difficult to achieve stable identification and interpretable quantification of complex manipulation patterns.

[0045] Therefore, the aforementioned sensing devices have limitations in terms of sensing dimensions, detection performance, structural fit, and algorithm intelligence, making it difficult to accurately and comprehensively reflect the multidimensional mechanical characteristics of traditional Chinese medicine acupuncture and massage techniques. This limits their application potential in clinical teaching, operational evaluation, and quantitative efficacy research. Therefore, there is a need for an intelligent flexible glove system and supporting algorithms that can simultaneously acquire multidimensional mechanical information and perform high-precision dynamic quantification without interfering with natural operation. This system can provide reliable data support for standardized teaching, objective evaluation, and efficacy mechanism research of acupuncture and massage techniques, promoting the intelligent and scientific development of traditional Chinese medicine external treatment techniques.

[0046] In view of the above problems, the present invention provides a medical system that at least partially solves the above problems.

[0047] Figure 1 A schematic diagram of a medical system according to an embodiment of the present invention is shown.

[0048] like Figure 1 As shown, the medical system of this embodiment includes a multimodal flexible glove 100 and a host computer 200. The host computer 200 can be connected to the multimodal flexible glove 100 via wired and / or wireless communication links.

[0049] In this embodiment of the invention, the target object can wear the multimodal flexible glove 100 to perform medical actions. For example, the target object can refer to a user using the multimodal flexible glove 100. Medical actions may include acupuncture, massage, or cardiopulmonary resuscitation. During the process of the target object wearing the multimodal flexible glove 100 to perform medical actions, the multimodal flexible glove 100 can accurately measure the large-scale deformation, continuous displacement, and frequency of joints in medical actions such as acupuncture "lifting and inserting" and massage "swinging". Furthermore, it can detect the minute contact forces generated by medical actions such as acupuncture "twisting" and acupoint "pressing" with high sensitivity and wide dynamic range. Subsequently, the multimodal flexible glove 100 can send the monitored multi-source mechanical signals (such as joint stress signals and pressure signals of the target object's hand) to the host computer 200 so that the host computer 200 can quantify the multi-source mechanical signals. Thus, this medical system can monitor and quantify multi-source mechanical signals of medical movements in real time and synchronously, while ensuring almost no interference with the natural medical movements of the target subject. This enables high-fidelity acquisition of multimodal data, thereby achieving objective feedback and digital transmission of traditional Chinese medicine techniques. The multimodal flexible glove of this invention will be further described below with reference to the accompanying drawings.

[0050] Figure 2 A schematic diagram of a multimodal flexible glove according to an embodiment of the present invention is shown.

[0051] like Figure 2As shown, the multimodal flexible glove of this embodiment includes a flexible glove body 110, a flexible strain sensor array 120, a flexible pressure sensor array 130, and a data processing circuit 140. The flexible glove body 110 is exemplified by the glove worn by the right hand of the target object.

[0052] In some embodiments, the flexible glove body 110 is based on a flexible, breathable elastic textile material to ensure the glove fits and is comfortable on the hand, thereby enabling the multimodal flexible glove to combine flexibility and wearing comfort.

[0053] Furthermore, the flexible glove body 110 may have a first outer surface corresponding to the back of the hand. Figure 2 The glove surface shown on the left) and the second outer surface corresponding to the palm ( Figure 2 (Glove surface shown on the right). Specifically, when the target object is wearing the multimodal flexible glove, the back of the target object's hand can contact the first inner surface of the flexible glove body 110, and the first outer surface and the first inner surface can be two opposing surfaces on the flexible glove body 110. The palm of the target object can contact the second inner surface of the flexible glove body 110, and the second outer surface and the second inner surface can be two opposing surfaces on the flexible glove body 110.

[0054] Multiple locations can be defined on the flexible glove body 110 based on the structure of the hand. For example, these multiple locations may include joint locations on the first outer surface, fingertips on the second outer surface, and palm locations on the second outer surface.

[0055] The flexible strain sensor can be disposed at a joint location on the first outer surface. For example, this joint location may include a metacarpophalangeal joint, an interphalangeal joint, and a wrist joint. Specifically, the metacarpophalangeal joint location may be the joint location where the back of the hand and fingers connect on the flexible glove body 110. The interphalangeal joint location may be the joint location between adjacent phalanges of each finger on the flexible glove body 110. The wrist joint location may refer to the location at the wrist joint where the back of the hand connects to the arm on the flexible glove body 110. Further, the flexible strain sensor can be one or multiple. When there is one flexible strain sensor, it can be located at one of the metacarpophalangeal joint, interphalangeal joint, or wrist joint locations. When there are multiple flexible strain sensors, they can be distributed across multiple metacarpophalangeal joint locations, multiple interphalangeal joint locations, and wrist joint locations to form a flexible strain sensor array 120.

[0056] The flexible strain sensor array 120 can generate multiple sets of joint stress signals at multiple times based on the degree of joint flexion of the target object when wearing a multimodal flexible glove and performing medical actions. Each set of joint stress signals includes multiple flexible strain signals collected simultaneously by multiple flexible strain sensors. For example, when the flexible strain sensor array 120 is disposed on the first outer surface of the flexible glove body 110, it can monitor large-scale deformations such as flexion and extension of the fingers and wrists during medical actions, thereby generating corresponding joint stress signals based on the amplitude of acupuncture "lifting and thrusting" techniques and the angle of massage "swinging" techniques. Furthermore, the frequency of the medical actions can also be monitored based on the frequency of the generated joint stress signals.

[0057] A flexible pressure sensor can be disposed at at least one location on the second outer surface, either the fingertip or the palm. The fingertip location can include key pressure points such as the fingertips, finger pads (especially the thumb pad), etc., on the flexible glove body 110. The palm location can include key pressure points such as the heel of the palm on the flexible glove body 110. For example, there can be one or more flexible pressure sensors. When there is only one flexible pressure sensor, it can be located at either the fingertip or palm location. When there are multiple flexible pressure sensors, they can be located at multiple fingertip and palm locations to form a flexible pressure sensor array 130. However, the embodiments of the present invention are not limited to this. In some embodiments, the flexible pressure sensors disposed at each location can also be in array form. Thus, in this embodiment, multiple flexible pressure sensor arrays 130 can each be located at multiple fingertip and palm locations, thereby forming a large flexible pressure sensor array 130.

[0058] The flexible pressure sensor array 130 can generate multiple sets of pressure intensity signals at multiple times based on the pressure at at least one location when the target object is wearing a multimodal flexible glove and performing medical actions. Each set of pressure intensity signals includes multiple pressure intensity signals collected simultaneously by multiple flexible pressure sensors. Thus, this flexible pressure sensor can monitor the minute contact forces and pressure distribution generated during needle handle "twisting" and acupoint "pressing" when the target object performs medical actions, achieving precise capture of the micro-dynamic characteristics of acupuncture techniques.

[0059] The data processing circuit 140 is at least partially disposed on the back of the hand of the first outer surface. For example, two modules of the data processing circuit 140 can be integrated onto a first circuit board and a second circuit board, respectively. The first and second circuit boards can be electrically connected via a flexible circuit board 150. For example, a module of the first circuit board can be electrically connected to a flexible strain sensor array 120, and a module of the second circuit board can be electrically connected to a flexible pressure sensor array 130. Furthermore, the first circuit board is positioned closer to the flexible strain sensor array 120 than the second circuit board to acquire high-quality joint stress signals. Thus, the data processing circuit 140 can receive multiple sets of joint stress signals from the flexible strain sensor array 120 and multiple sets of pressure intensity signals from the flexible pressure sensor array 130, and encapsulate the joint stress signals and pressure intensity signals respectively to obtain joint stress data and pressure intensity data. Subsequently, the data processing circuit 140 can send the joint stress data and pressure intensity data to a host computer.

[0060] The host computer can obtain the first movement type information and the first quantitative assessment result of the medical movement based on the joint stress data and the pressure data.

[0061] For example, upon receiving the aforementioned multiple sets of joint stress data, the host computer can determine the reception time of the multiple sets of joint stress data and determine the location of the flexible strain sensor corresponding to each joint stress data in each set of joint stress data (e.g., multiple joint positions on the aforementioned first outer surface). It should be understood that different flexible stress sensors can transmit joint stress data via different data transmission channels (e.g., lines). Based on this, joint stress data received via different data transmission channels can correspond to different location positions. On this basis, the location positions can be pre-associated and stored with the data transmission channels, so that the host computer can determine the location corresponding to the joint stress data transmitted by that data transmission channel based on the data transmission channel. The same applies to the pressure data, and will not be elaborated further. Furthermore, upon receiving the aforementioned multiple sets of pressure data, the host computer can determine the reception time of the multiple sets of pressure data and determine the location of the flexible pressure sensor corresponding to each pressure data in each set of pressure data (e.g., at least one location on the aforementioned palm or fingertip position on the aforementioned second outer surface).

[0062] Subsequently, the host computer can input multiple sets of joint stress data, the receiving time of multiple sets of joint stress data, the setting position of the flexible strain sensor array 120 on the flexible glove body 110, multiple sets of pressure data, the receiving time of multiple sets of pressure data, and the setting position of the flexible pressure sensor array 130 on the flexible glove body 110 into a trained neural network (e.g., a multimodal TCM manipulation quantitative evaluation model) to obtain the first action type information and the first quantitative evaluation result of the medical action. This first action type information characterizes the specific action type (e.g., the specific manipulation technique) of the medical action determined based on the joint stress data and pressure data. For example, for acupuncture, this first action type information may include information on action types such as "lifting and thrusting"; for massage, this first action type information may include information on action types such as "swinging". The first quantitative evaluation result can be used to assess the completion status of the medical action performed by the target object. For example, the first quantitative evaluation result can characterize the similarity between the medical action performed by the target object, determined based on the joint stress data and pressure data, and the standard medical action, so that the target object can adjust the medical action.

[0063] Based on this, in this embodiment of the invention, during the medical action performed by the target object wearing the aforementioned multimodal flexible glove, the flexible strain sensor array 120 on the flexible glove body 110 can generate multiple sets of joint stress signals at multiple times according to the joint bending degree of the target object. Simultaneously, the flexible pressure sensor can generate multiple sets of pressure intensity signals at multiple times according to the pressure at least one of the fingertips and palms of the target object. The data processing circuit 140 can send multiple sets of joint stress data and multiple sets of pressure intensity data to the host computer based on the multiple sets of joint stress signals and multiple sets of pressure intensity signals, respectively. Thus, the host computer can input the multiple sets of joint stress data, the receiving time of the multiple sets of joint stress data, the setting position of the flexible strain sensor array 120 on the flexible glove body 110, the multiple sets of pressure intensity data, the receiving time of the multiple sets of pressure intensity data, and the setting position of the flexible pressure sensor array 130 on the flexible glove body 110 into a trained neural network, and output the first action type information and the first quantitative evaluation result of the medical action. It should be understood that when a medical action (such as acupuncture) is performed on a target, the flexible strain sensor can generate strain corresponding to the pressure applied to the needle handle by the target object via the flexible pressure sensor. For example, for flexible strain sensors and flexible pressure sensors located close to each other (e.g., on the same finger) (e.g., a flexible pressure sensor on the thumb pad and a flexible strain sensor on the thumb interphalangeal joint), the joint stress signal and the pressure intensity signal acquired at the same time have a coupling relationship under the same dimension (e.g., a mechanical coupling relationship). Specifically, for example, when the pressure applied by the target object using the thumb increases, the degree of deformation of the thumb interphalangeal joint also increases; this is the aforementioned mechanical coupling relationship. Based on this, by using a trained neural network to process the joint stress signal and the pressure intensity signal that have this coupling relationship, the characteristics of the acupuncture action used to simultaneously generate the joint stress signal and the pressure intensity signal can be determined based on this coupling relationship. Thus, even if the displacement of "lifting and inserting" and the contact force of "twisting" are relatively small, the accurate type of acupuncture movement and the quantitative evaluation result can still be determined based on the aforementioned pressure intensity signal, joint stress signal, corresponding setting position, and receiving time. In this way, while the target subject is performing medical movements, the specific type of medical movement performed can be accurately determined, and the standardization of that type of medical movement can be accurately quantified.

[0064] Based on this, the multimodal flexible glove of this invention transforms subjective experience-based "feelings" and "insights" into quantifiable physical parameters (such as joint stress and pressure), thereby enabling the objective recording and analysis of medical actions such as acupuncture and massage techniques. It fundamentally solves the core problem of inconsistent teaching quality and distorted skill transmission caused by vague subjective standards, providing a scientific basis for manual technique teaching, assessment, and the transmission of renowned doctors' experience.

[0065] Furthermore, in this embodiment of the invention, the flexible glove body 110, the flexible strain sensor, and the flexible pressure sensor are lightweight, breathable, and have a high degree of fit, which can reduce the interference of wearing gloves on the doctor's hand activities, ensure a natural feel and flexibility in real acupuncture and massage operation scenarios, and improve the clinical reliability of the collected data.

[0066] Based on this, a flexible strain sensor is disposed at the joint position on the first outer surface of the flexible glove body 110, a flexible pressure sensor is disposed at at least one position on the fingertip or palm of the second outer surface of the flexible glove body 110, and a data processing circuit 140 is at least partially disposed on the back of the hand on the first outer surface of the flexible glove body 110. This arrangement reduces the impact of at least one of the flexible strain sensor, flexible pressure sensor, or data processing circuit 140 on the target object's medical movements. This allows for multi-dimensional real-time acquisition of key manipulation parameters while the target object performs the medical movements naturally, providing a reliable data foundation for the digital evaluation, intelligent teaching, and standardized inheritance of traditional Chinese medicine acupuncture and massage techniques.

[0067] In this embodiment of the invention, the flexible strain sensor can be manufactured based on a modified metal (e.g., liquid metal, specifically gallium-based liquid alloy, etc.). For example, the flexible strain sensor can be based on liquid metal encapsulated in an Ecoflex (i.e., ultra-soft liquid silicone) elastomer, exhibiting high tensile strength (>100%) and a stable sensitivity coefficient.

[0068] For example, a flexible strain sensor may include a first flexible insulating film (e.g., the material may include an ultra-soft liquid silicone elastomer), a flexible sensing layer, and a second flexible insulating film (e.g., the material may include an ultra-soft liquid silicone elastomer) stacked sequentially. The flexible sensing layer is formed based on a first dispersion mixture of a metal (e.g., the aforementioned liquid metal) and a thickener (e.g., silica). In some embodiments, the flexible sensing layer may also include an active conductive material, such as at least one of carbon black, graphene (e.g., graphene nanosheets), or carboxylated multi-walled carbon nanotubes.

[0069] In this embodiment of the invention, the flexible pressure sensor may include a first flexible electrode layer, a pressure-sensitive layer, and a second flexible electrode layer stacked sequentially. Each flexible pressure sensor's pressure-sensitive layer is configured to have a biomimetic dome structure (e.g., a spherical shell shape), and the pressure-sensitive layer is formed based on a second dispersion mixture of the aforementioned active conductive material.

[0070] Furthermore, both the first and second flexible electrode layers may include active conductive materials. However, it should be understood that the conductive active materials in the flexible strain sensor, flexible pressure sensor, and the first and second flexible electrode layers are not limited to the aforementioned carbon black, graphene, and carboxylated multi-walled carbon nanotubes. Alternatively, other materials with high conductivity and nanostructures may be used, including but not limited to one or more combinations of unfunctionalized multi-walled or single-walled carbon nanotubes, silver nanowires, conductive polymers, and metal nanoparticles (such as gold and silver nanoparticles). The conductive polymers include poly(3,4-ethylenedioxythiophene):polystyrene sulfonate (PEDOT:PSS), etc.

[0071] In some embodiments, the flexible pressure sensor may be based on microstructured carbon materials (such as carbon black, graphene, and carboxylated multi-walled carbon nanotubes mentioned above), and its pressure-sensitive layer is a multi-walled carbon nanotube / TPU (i.e., thermoplastic polyurethane) composite material with a biomimetic dome structure, exhibiting high sensitivity (>4.54 kPa). -1 ) and low detection limit (<100 Pa).

[0072] Based on this, by synergistically integrating wide-range strain sensing and high-precision tactile sensing into the multimodal flexible glove of this invention, the glove can simultaneously capture the motion and mechanical information of complex techniques such as "lifting and thrusting," "twisting," "pressing," and "kneading," overcoming the limitations of incomplete information from traditional single-sensor dimensions. Thus, the high-dimensional, multimodal data (such as joint stress data and pressure intensity data) provided by this multimodal flexible glove lays a data foundation for in-depth research into the biomechanical mechanisms of composite techniques and their therapeutic effects. Furthermore, the flexible strain sensor and flexible pressure sensor of this invention are lightweight, breathable, and highly conforming, minimizing interference with the target subject's hand movements while wearing the multimodal flexible glove, improving the natural feel and flexibility in real acupuncture and massage scenarios, and enhancing the clinical reliability of the collected data.

[0073] Figure 3 A connection diagram of a data processing circuit according to an embodiment of the present invention is shown. Figure 4 A schematic diagram of a data processing circuit according to an embodiment of the present invention is shown. Figure 1 . Figure 5 A schematic diagram of a data processing circuit according to an embodiment of the present invention is shown. Figure 2 The illustration is shown below. Figure 1 and indication Figure 2 The two opposite surfaces of the data processing circuit can be shown separately.

[0074] like Figure 3As shown, the data processing circuit may include a data processing module and a control module. The data processing module may be disposed on a first circuit board located on the back of the hand. The control module may be disposed on a second circuit board and electrically connected to the data processing module via a flexible circuit board. In some embodiments, the second circuit board containing the control module may be disposed on the back of the wrist of the flexible glove body, but may also be disposed elsewhere, which will not be elaborated here. The flexible circuit board may be a flexible printed circuit board.

[0075] In terms of physical architecture, the first circuit board can serve as a data acquisition sub-board, and the second circuit board can serve as a main control board. Together, the main control board and the data acquisition sub-board constitute the data acquisition terminal. Based on this design, the two are reliably connected through flexible printed circuit board cables, thereby enabling the separate structure of the first and second circuit boards to adapt to the distributed high-frequency acquisition requirements under complex conditions.

[0076] like Figure 3 As shown, the data processing module may include a multiplexer, operational amplifier, and analog-to-digital converter, etc. Furthermore, a constant current source is integrated on the data acquisition sub-board, which can be used to power the devices on the operational amplifier and analog-to-digital converter, etc., on the data acquisition sub-board. The control module may include a controller and a communication unit, etc. In addition, the control module may also include a 3.7V polymer lithium battery and a power management unit, etc. The power management unit may include a low-dropout linear regulator. Thus, the voltage of the 3.7V polymer lithium battery can be converted into a stable 3.3V power supply voltage through the low-dropout linear regulator, providing low-noise power support for the first circuit board.

[0077] like Figure 4 As shown, the multiple input terminals of the multiplexer are electrically connected to the output terminals of multiple flexible strain sensors via the strain sensor interface 1411 of the data acquisition subboard 141. Thus, under the control of the control signal received via the control terminal of the multiplexer, the multiplexer sequentially outputs the joint stress signals of the multiple flexible strain sensors via its output terminal.

[0078] The input of the operational amplifier is electrically connected to the output of the multiplexer. In this way, the operational amplifier can amplify the analog joint stress signal from each flexible strain sensor to obtain an amplified joint stress signal, thus enabling the amplification of joint stress signals from multiple flexible strain sensors. It should be noted that the amplified joint stress signal output by the operational amplifier is still in analog form.

[0079] The analog-to-digital converter (ADC) can be, for example, a 16-bit high-precision ADC, such as the ADS1115. The input of the ADC can be electrically connected to the output of an operational amplifier. In this way, the ADC can convert amplified joint stress signals in analog form into digital joint stress signals, thereby enabling analog-to-digital conversion of joint stress signals from multiple flexible strain sensors.

[0080] Thus, based on the aforementioned devices, the data acquisition sub-board 141 can be used for precise conditioning and acquisition of weak joint stress signals. It can also multiplex, amplify, condition, and digitize the joint stress signals from multiple (e.g., 6) flexible strain sensors, and then transmit them to the main control board 142 via the flexible circuit board 150 for further processing.

[0081] The controller 1421 can be a microcontroller implemented using a single-chip microcomputer, such as an STM32. The first input terminal of the controller 1421 can be electrically connected to the output terminal of the analog-to-digital converter via the wiring of the aforementioned flexible circuit board 150. For example, this wiring can be based on the Inter-Integrated Circuit (I2C) bus protocol. 2 C) Signal transmission is performed, and this line can be a wiring or cable, etc. In this way, the controller 1421 can receive multiple digital joint stress signals corresponding to multiple flexible strain sensors from the analog-to-digital converter.

[0082] The second input terminal of the controller 1421 can be directly connected to the output terminal of the flexible pressure sensor via the pressure sensor interface 1422 of the main control board 142. In this way, the controller 1421 can convert the analog pressure signal received from the flexible pressure sensor into a digital pressure signal. For example, during the data acquisition phase, the controller 1421 can utilize its built-in analog-to-digital converter combined with direct memory access technology to achieve high-speed synchronous acquisition and analog-to-digital conversion of pressure signals from a multi-channel (e.g., 27-channel) flexible pressure sensor, thereby obtaining a digital joint stress signal.

[0083] Based on this, the controller 1421 can perform frame encapsulation on the digital pressure signal and the digital joint stress signal according to a predetermined communication protocol to obtain pressure data and joint stress data. For example, the predetermined communication protocol can be a custom binary communication protocol, etc.

[0084] like Figure 5As shown, the communication unit 1423 can be, for example, a wireless communication unit, such as the ESP8266. This communication unit 1423 can be electrically connected to the first output terminal of the controller and can also communicate with a host computer. Thus, after the data acquisition sub-board 141 transmits the aforementioned pressure data and joint stress data to the main control board 142 via the flexible circuit board 150 and completes packaging, the communication unit 1423 can receive the pressure data and joint stress data from the controller via a high-speed serial interface connected to the controller, and compress the floating-point data and long integer data of each of the pressure data and joint stress data respectively, and then send the compressed pressure data and compressed joint stress data to the host computer. For example, the communication unit 1423 can compress the floating-point data and long integer data of the pressure data and the floating-point data and long integer data of the joint stress data based on a predetermined lossy compression algorithm to reduce the occupancy of the communication bandwidth. Subsequently, the communication unit 1423 can utilize a low-latency point-to-point wireless communication protocol (i.e., the ESP-NOW protocol) to package the compressed pressure data and compressed joint stress data into ESP-NOW wireless data packets and transmit them to the host computer via a high-frequency communication mechanism. Furthermore, the controller's second output can be electrically connected to the multiplexer's control terminal to provide the aforementioned control signals to the multiplexer's control terminal based on a predetermined program, thereby controlling the multiplexer to sequentially provide joint stress signals from different input terminals to the output terminal.

[0085] Thus, the aforementioned data processing circuit, acting as a data acquisition terminal, can complete the data processing and wireless transmission stages. Subsequently, the host computer, acting as a remote data receiver, can perform the data reception stage.

[0086] In the data receiving stage, the host computer may include a data receiving unit, for example, built on a microcontroller. Based on this, the host computer can continuously monitor and receive the aforementioned ESP-NOW wireless data packets. Furthermore, the host computer has a built-in strict data packet integrity verification and packet loss detection mechanism, capable of efficiently decompressing the received ESP-NOW wireless data packets to obtain compressed pressure data and compressed joint stress data. It should be understood that, in this embodiment of the invention, the aforementioned data processing circuit can transmit compressed pressure data and compressed joint stress data at multiple time points. Thus, the host computer can obtain a time-series data stream based on the pressure data and joint stress data, thereby achieving high-frequency acquisition and low-latency transmission of heterogeneous data from multiple sensors. The heterogeneous data from multiple sensors refers to joint stress data corresponding to multiple flexible strain sensors and pressure data corresponding to multiple flexible pressure sensors.

[0087] Furthermore, in one embodiment of the present invention, reference continues to be made to... Figure 4The multimodal flexible glove also includes multiple inertial sensors 1412. These inertial sensors 1412 can be respectively disposed on a first circuit board and a second circuit board. Each of the multiple inertial sensors 1412 can collect multiple sets of inertial signals corresponding to medical actions at multiple moments when the target object is wearing the multimodal flexible glove and performing medical actions. Each set of inertial signals includes inertial signals collected by multiple inertial sensors 1412 at the same moment. For example, the inertial signals can be digital signals.

[0088] The controller 1421 is electrically connected to multiple inertial sensors 1412. In this way, multiple sets of inertial signals can be frame-encapsulated according to a predetermined communication protocol to obtain multiple sets of inertial data. For example, the controller can communicate via the same bus (e.g., I / O). 2 The C-bus is electrically connected to multiple inertial sensors 1412. The address selection pins of these multiple inertial sensors 1412 (e.g., AD0 hardware pins) can be configured to connect to ground and power supply (e.g., 3.3V power supply), respectively, so that the controller can communicate via the same I-bus. 2 The C-bus is electrically connected to the aforementioned multiple inertial sensors 1412, and can control the signals transmitted via the same I-bus based on the level of the address selection pin of each inertial sensor 1412 (e.g., a low level "0" for ground and a high level "1" for power supply). 2 The inertial signals transmitted via the C-bus are differentiated by address, thereby enabling synchronous and accurate measurement of the inertial signals. In this way, the controller 1421 can acquire multi-source heterogeneous sensor data, including multi-channel joint stress data, multi-channel pressing force data, and multi-channel inertial data, realizing the acquisition of multi-modal sensor data.

[0089] The communication unit can also compress floating-point and long integer data of multiple sets of inertial data using lossy compression algorithms, as described above, and send the compressed inertial data to the host computer. For example, the compressed inertial data can be sent via Bluetooth. In this way, the host computer can obtain the second action type information and the second quantitative evaluation result of the medical action based on the joint stress data, pressure data, and inertial data.

[0090] For example, a host computer can input multiple sets of joint stress data, the receiving time of these joint stress data, the location of the flexible strain sensor array on the flexible glove body, multiple sets of pressure data, the receiving time of these pressure data, the location of the flexible pressure sensor array on the flexible glove body, multiple sets of inertial data, and the location of multiple inertial sensors on the flexible glove body (e.g., the positions of the first and second circuit boards mentioned above) into a trained neural network, and output second action type information and a second quantitative evaluation result for the medical action. This second action type information characterizes the specific action type (e.g., the specific technique of the medical action) determined based on the joint stress data, pressure data, and inertial data. The second quantitative evaluation result can be used to assess the completion status of the medical action performed by the target object. For example, the second quantitative evaluation result can characterize the similarity between the medical action performed by the target object, determined based on the joint stress data, pressure data, and inertial data, and a standard medical action, so that the target object can adjust the medical action.

[0091] However, it should be understood that the flexible strain sensor, flexible pressure sensor, and inertial sensor in this embodiment of the invention are not limited to the locations described above. Alternatively, depending on the specific identification task, the flexible strain sensor, flexible pressure sensor, and inertial sensor can also be placed on other relevant parts of the body, such as the elbow, shoulder, or back. The number and arrangement of the flexible strain sensor, flexible pressure sensor, and inertial sensor can also be adjusted accordingly, as long as they can effectively capture the relevant signals of medical movements.

[0092] In some embodiments, when the medical action is acupuncture, the flexible strain sensor can be mainly deployed at the interphalangeal and metacarpophalangeal joints of the thumb, index finger, and middle finger of the flexible glove body. The flexible pressure sensor can be deployed at the fingertips of the thumb and index finger.

[0093] In this way, the data processing circuit can simultaneously collect joint stress data and pressure data of the target object during acupuncture and transmit them to the host computer. Subsequently, the host computer can use a trained neural network to identify the specific techniques of the medical movements performed on the target object based on the joint stress data and pressure data, and output the corresponding technique category. At the same time, the system will quantitatively evaluate the acupuncture techniques of the operator, intuitively reflecting the degree of similarity between the current medical movement (which may be a specific operation technique under a certain type of medical movement) and the standard medical movement (which may be a specific operation technique under a certain type of medical movement).

[0094] In some embodiments, when the medical action is identified as a massage technique, flexible strain sensors can be primarily deployed at the interphalangeal and metacarpophalangeal joints of the thumb, index, and middle fingers of the flexible glove body. Flexible pressure sensors can be deployed at the fingertips of the thumb and index finger. Thus, the data processing circuit can simultaneously collect joint stress data and pressure data of the target object during the massage process and transmit them to a host computer. Subsequently, the host computer can, based on a trained neural network, identify the specific technique of the medical action of the target object according to the joint stress data and pressure data, and output its corresponding technique category. Furthermore, in addition to the similarity between the current medical action and a standard medical action, at least one of the first or second quantitative evaluation results may also include a specific evaluation of the quality of the medical action, such as quantitative feedback information generated regarding the uniformity of force distribution, the correctness of the movement trajectory, and the stability of the rhythm.

[0095] In some embodiments, when the medical action is a cardiopulmonary resuscitation (CPR) technique, flexible strain sensors can be primarily deployed at the interphalangeal and metacarpophalangeal joints of the thumb, index, and middle fingers of the flexible glove body. Flexible pressure sensors can be deployed at the fingertips of the thumb and index finger. Thus, the data processing circuit can simultaneously collect joint stress data and compression pressure data of the target object during CPR and transmit them to a host computer. Subsequently, the host computer can, based on a trained neural network, identify the specific technique of the medical action performed on the target object according to the joint stress data and compression pressure data, and output its corresponding technique category. Furthermore, in addition to the similarity between the current medical action and a standard medical action, at least one of the first or second quantitative evaluation results may also include a specific evaluation of the quality of the medical action, such as quantitative feedback information generated regarding the uniformity of force distribution, the correctness of the movement trajectory, and the stability of the rhythm.

[0096] Figure 6 A schematic diagram of a medical system according to another embodiment of the present invention is shown.

[0097] like Figure 6As shown, the data processing circuit, after processing the joint stress signal from the flexible strain sensor, the pressure signal from the flexible pressure sensor, and the inertial signal from the inertial sensor into joint stress data, pressure signal, and inertial data respectively, can send these data to the host computer. Upon receiving the joint stress data, pressure signal, and inertial data, the host computer can perform preprocessing on them. For example, the preprocessing process may include data cleaning (such as handling missing and duplicate data), data transformation (such as data standardization and label encoding), and data augmentation (such as noise addition, masking, normalization, and data offsetting) to improve data quality and consistency. During this process, the joint stress data can be associated with the corresponding flexible strain sensor's location (e.g., the joint location on the flexible glove body). Furthermore, the pressure signal can be associated with the corresponding flexible pressure sensor's location (e.g., the fingertip and palm positions on the flexible glove body). Subsequently, the preprocessed joint stress data, pressure data, and inertial data can be input into a trained neural network (such as a deep neural network) to achieve automatic recognition and quantitative evaluation of different medical movements.

[0098] In some embodiments of the present invention, the following description uses the process of neural network processing of joint stress data and pressure data as an example. It should be understood that the embodiments of the present invention are not limited thereto.

[0099] In this embodiment, the joint locations include metacarpophalangeal joints, interphalangeal joints, and wrist joints. The multimodal flexible glove includes multiple flexible strain sensors and multiple flexible pressure sensors. The multiple flexible strain sensors are respectively disposed at multiple metacarpophalangeal joints, multiple interphalangeal joints, and the wrist joint. The multiple flexible pressure sensors are disposed at multiple fingertips and palm pads.

[0100] The host computer can receive multiple sets of joint stress data and multiple sets of pressure data at consecutive time points from the data processing circuit of the multimodal flexible gloves when the target object is wearing them and performing medical actions. It should be understood that the host computer can determine the reception time of each set of joint stress data and each set of pressure data upon receiving them. Each set of joint stress data includes multiple joint stress data points from multiple flexible strain sensors, and each set of pressure data includes multiple pressure data points from multiple flexible pressure sensors.

[0101] The host computer can input multiple sets of joint stress data and the corresponding receiving times, as well as multiple sets of pressure intensity data and the corresponding receiving times, into a trained neural network. In this way, the neural network can process the multiple sets of joint stress data and the corresponding receiving times, as well as the multiple sets of pressure intensity data and the corresponding receiving times, to obtain the first action type information and the first quantitative evaluation result.

[0102] Figure 7 A schematic diagram of a neural network according to an embodiment of the present invention is shown.

[0103] like Figure 7 As shown, a neural network may include an input layer, a two-stream neural network, a weighted bidirectional cross-modal attention module, a fusion layer, and an output layer.

[0104] The input layer can receive input data related to various sensors in the multimodal flexible glove, such as the aforementioned multiple sets of joint stress data, the receiving time of multiple sets of joint stress data, the setting position of the flexible strain sensor array on the flexible glove body, multiple sets of pressure data, the receiving time of multiple sets of pressure data, and the setting position of the flexible pressure sensor array on the flexible glove body.

[0105] Two-stream neural networks include graph convolutional neural networks and temporal convolutional neural networks, which extract features from input data in parallel.

[0106] For example, graph convolutional neural networks can adaptively construct a data graph structure based on the K-nearest neighbor algorithm, establishing spatial correlation features between data points in each group of sensor data. For instance, a graph convolutional neural network can extract multiple first spatial correlation features corresponding to multiple groups of joint stress data based on the correlation between multiple joint stress data points in each group and the placement position of the flexible strain sensor array on the flexible glove body. Furthermore, a graph convolutional neural network can extract multiple second spatial correlation features corresponding to multiple groups of pressure intensity data based on the correlation between multiple pressure intensity data points in each group and the placement position of the flexible pressure sensor array on the flexible glove body.

[0107] In some embodiments, the graph convolutional neural network outputs a first spatial correlation feature based on the correlation between the placement positions of flexible strain sensors corresponding to multiple joint stress data points in each group of joint stress data, and based on the correlation between the values ​​of the multiple joint stress data points in each group of joint stress data. Furthermore, the graph convolutional neural network outputs a second spatial correlation feature based on the correlation between the placement positions of flexible pressure sensors corresponding to multiple pressure data points in each group of pressure data, and based on the correlation between the values ​​of the multiple pressure data points in each group of pressure data.

[0108] For example, a temporal convolutional neural network (TCNN) may include multiple residual connection modules for modeling the temporal dynamics of the data. For instance, a TCNN can extract a first temporal variation feature based on the dynamic characteristics of the evolution of multiple sets of joint stress data over time, using multiple sets of joint stress data and their corresponding reception times. Furthermore, a TCNN can extract a second temporal variation feature based on the dynamic characteristics of the evolution of multiple sets of pressing force data over time, using multiple sets of pressing force intensity data and their corresponding reception times.

[0109] For example, a temporal convolutional neural network can extract first spatial features based on the values ​​of each joint stress data point in each set of joint stress data and the placement locations of flexible strain sensors corresponding to multiple joint stress data points in each set of joint stress data. It can also extract first temporal features based on the values, placement locations, and reception times of each joint stress data point in multiple sets of joint stress data. Subsequently, first temporal variation features can be determined based on the first spatial features and the first temporal features.

[0110] In some embodiments, the temporal convolutional neural network can extract a second spatial feature based on the value of each pressure data point in each set of pressure data and the placement position of the flexible strain sensor corresponding to multiple pressure data points in each set of pressure data. It can also extract a second temporal feature based on the value of each pressure data point in multiple sets of pressure data, the placement position, and the receiving time. Subsequently, a second temporal variation feature can be determined based on the second spatial feature and the second temporal feature.

[0111] It should be noted that graph convolutional neural networks and temporal convolutional neural networks can perform dimensionality reduction after receiving data such as joint stress data and pressure intensity data. Then, the aforementioned processing is performed on the dimensionality-reduced joint stress features and pressure intensity features to obtain spatial correlation features and temporal variation features, which will not be elaborated here.

[0112] Subsequently, the features extracted by the two-stream neural network are input into a weighted bidirectional cross-modal attention module for fusion and interaction. The weighted bidirectional cross-modal attention module may include two parts: a bidirectional attention mechanism and a modality-weighted fusion mechanism.

[0113] In this embodiment of the invention, the weighted cross-modal attention module can adaptively weight the first spatial correlation feature and the first temporal variation feature based on a modal weighting fusion mechanism to obtain the first spatial correlation weighted feature and the first temporal variation weighted feature. Furthermore, the weighted cross-modal attention module can adaptively weight the second spatial correlation feature and the second temporal variation feature based on a modal weighting fusion mechanism to obtain the second spatial correlation weighted feature and the second temporal variation weighted feature.

[0114] For example, the modality weighted fusion mechanism adaptively assigns weights to features from the temporal convolutional branch and the graph convolutional branch, dynamically adjusting the contribution of spatial correlation features and temporal variation features according to the characteristics of the input data, so as to enhance the expression of key features and suppress redundant information.

[0115] Specifically, the weighted cross-modal attention module can adaptively assign weights to the first spatial association feature and the first temporal change feature based on the modal weighted fusion mechanism, according to their respective types (e.g., spatial type and temporal type), to obtain the first spatial association weighted feature and the first temporal change weighted feature.

[0116] The weighted cross-modal attention module can adaptively assign weights to the second spatial correlation feature and the second temporal change feature based on the modal weighted fusion mechanism, according to their respective types (e.g., spatial type and temporal type), to obtain the second spatial correlation weighted feature and the second temporal change weighted feature.

[0117] Subsequently, the weighted cross-modal attention module can fuse the first spatial correlation weighted features and the first spatial correlation features to obtain the first spatial fused features, and can fuse the first temporal change weighted features and the first temporal change features to obtain the first temporal fused features.

[0118] Similarly, the weighted cross-modal attention module can fuse the second spatial correlation weighted features and the second spatial correlation features to obtain the second spatial fusion features, and can fuse the second temporal variation weighted features and the second temporal variation features to obtain the second temporal fusion features.

[0119] In some embodiments, the first spatial correlation features corresponding to multiple time points can be weighted by a spatiotemporal graph convolutional network to obtain first spatial correlation weighted features, and the first temporal variation weighted features can be weighted by a temporal convolutional network to obtain first temporal variation weighted features. In some embodiments, the second spatial correlation features corresponding to multiple time points can be weighted by a spatiotemporal graph convolutional network to obtain second spatial correlation weighted features, and the second temporal variation weighted features can be weighted by a temporal convolutional network to obtain second temporal variation weighted features.

[0120] Furthermore, the weighted bidirectional cross-modal attention module in the neural network can calculate the first bidirectional attention feature based on the first spatial fusion feature and the first temporal fusion feature, and can calculate the second bidirectional attention feature based on the second spatial fusion feature and the second temporal fusion feature.

[0121] For example, bidirectional attention mechanisms include two multi-head attention substructures, etc.

[0122] In one embodiment, one multi-head attention substructure uses a first spatial correlation feature as the query vector and a first temporal variation feature as the key and value; another multi-head attention substructure uses the first temporal variation feature as the query vector and the first spatial correlation feature as the key and value. Thus, the weighted bidirectional attention module can output the first bidirectional attention feature, enabling cross-modal bidirectional information interaction for joint stress data.

[0123] In another embodiment, one multi-head attention substructure uses a second spatial correlation feature as the query vector and a second temporal variation feature as the key and value; another multi-head attention substructure uses a second temporal variation feature as the query vector and a second spatial correlation feature as the key and value. Thus, the weighted bidirectional attention module can output a second bidirectional attention feature, enabling cross-modal bidirectional information interaction for pressure intensity data. This allows for the establishment of highly fine-grained feature associations, achieving alignment and complementarity between temporal and spatial features.

[0124] The fusion layer of the neural network fuses the first spatial fusion feature, the first temporal fusion feature, the first bidirectional attention feature, the second spatial fusion feature, the second temporal fusion feature, and the second bidirectional attention feature to obtain the attention fusion feature.

[0125] Subsequently, the output layer of the neural network can be processed based on the attention fusion features to output the first action type information and the first quantitative evaluation result of the medical action.

[0126] For example, the output layer may include a normalization layer and a classification network.

[0127] The normalization layer can perform layer-level normalization on the attention fusion features to obtain normalized features. In this way, through the aforementioned weighted bidirectional cross-modal attention module and normalization layer, normalized features with better stability and discriminative ability can be obtained.

[0128] Subsequently, the classification network can process the normalized features to obtain the first action type information and the first quantitative assessment result of the medical action.

[0129] Based on this, when the aforementioned neural network outputs the first action type information and the first quantitative evaluation result, these can be displayed on the host computer's visualization interface. Of course, the second action type information and the second quantitative evaluation result can also be displayed. In this way, the aforementioned medical system can present the action type information and quantitative evaluation results of traditional Chinese medicine manipulation techniques in the form of real-time waveforms, numerical indicators, and visual charts, providing the target user with immediate and intuitive operational feedback, helping them clearly identify the differences between their own operation and the standard technique. This transforms the learning process of medical movements from "passive guessing" to "active correction," significantly shortening the mastery period and thus assisting the target user in efficiently establishing correct technique memory and operational habits.

[0130] In some embodiments, the aforementioned neural network can be trained using training samples to train an initial preset neural network. For example, the recipient (e.g., a manual therapy instructor) can wear the aforementioned multimodal flexible gloves and repeatedly perform the same medical action at different levels, thereby collecting multiple sets of multimodal data corresponding to the medical action at different levels. Each set of multimodal data may include joint stress signals and pressure intensity signals. Subsequently, each set of multimodal data can be labeled based on the specific type of the medical action, and can also be labeled based on the degree of difference between each set of multimodal data and the standard multimodal data corresponding to the standard medical action, thereby obtaining training samples. For example, the specific types of medical actions may include 8 types of traditional Chinese medicine acupuncture techniques, 12 types of massage techniques, and cardiopulmonary resuscitation techniques, etc. Furthermore, in some embodiments, to improve the model's generalization ability and robustness, data augmentation strategies are introduced during the training phase. By processing the original data with noise, masking, normalization, and data offsetting, diverse training samples are generated, thereby improving the neural network's adaptability to complex environments and individual differences.

[0131] Based on the above, this embodiment of the invention, through the neural network of multimodal data fusion and feature modeling, can perform spatiotemporal alignment, feature extraction, and deep fusion of collected joint stress data, pressure intensity data, and inertial data. This allows for accurate quantification of key features of acupuncture techniques based on the mechanical coupling relationship between joint stress data and pressure intensity data corresponding to joint stress signals collected simultaneously by flexible strain sensors and flexible pressure sensors located in close proximity. Furthermore, by mapping joint stress data and pressure intensity data into objective quantitative evaluation results that can be used for real-time teaching feedback and standardized assessment, reliance on subjective experience is reduced, promoting the scientific, quantitative, and intelligent evaluation of medical actions such as acupuncture and massage techniques.

[0132] Figure 8 A schematic diagram of a method for quantifying medical actions according to an embodiment of the present invention is shown.

[0133] like Figure 8 As shown, the quantization method in this embodiment includes operations S810 to S820.

[0134] During operation of S810, multiple sets of joint stress data and multiple sets of pressure data at multiple times are received from the multimodal flexible glove.

[0135] During operation of S820, multiple sets of joint stress data, the receiving time of multiple sets of joint stress data, the setting position of the flexible strain sensor array on the flexible glove body, multiple sets of pressure data, the receiving time of multiple sets of pressure data, and the setting position of the flexible pressure sensor array on the flexible glove body are input into the trained neural network, and the first action type information and the first quantitative evaluation result of the medical action are output.

[0136] It should be understood that the quantization method in the embodiments of the present invention is not limited thereto; for details, please refer to the foregoing description, which will not be repeated here.

[0137] The multimodal gloves, medical systems, and corresponding quantification methods of the present invention have been described above. The manufacturing process of the multimodal flexible gloves of the present invention will be described below.

[0138] Figure 9 A schematic diagram of a method for manufacturing a multimodal flexible glove according to an embodiment of the present invention is shown.

[0139] like Figure 9 As shown, the manufacturing method of this embodiment includes operations S910 to S940.

[0140] Using the S910, the flexible glove body is obtained.

[0141] In operation S920, a flexible strain sensor array is fabricated and placed on the flexible glove body.

[0142] In operation S930, a flexible pressure sensor array is fabricated and placed on the flexible glove body.

[0143] In operation of S940, the data processing circuit is located on the flexible glove body and electrically connected to the flexible strain sensor array and the flexible pressure sensor array.

[0144] In this embodiment of the invention, a nylon-spandex blend fabric with good elasticity and breathability can be used as the body of the flexible glove, and the sensor layout can be based on the biomechanical structure of the human hand. For example, a flexible strain sensor array can be positioned at each finger joint to monitor bending movements and capture joint stress signals, while a flexible pressure sensor array can be concentrated at the fingertips to capture pressure signals.

[0145] For fixation and encapsulation, the flexible strain sensor array is bonded to the flexible glove body using adhesive. Its wires are routed along the natural creases of the fingers and non-primarily active areas on the back of the hand to reduce stress fatigue during gripping. The lead-out portions of the wires can be encapsulated with a flexible polyurethane film and secured to the back of the hand using 3M tape. The flexible pressure sensor is also secured to the fingertips of the glove using adhesive.

[0146] In this embodiment of the invention, fabricating a flexible strain sensor may include: preparing a first flexible insulating film; adding a thickener (e.g., silica powder) to a liquid metal to obtain a first dispersion mixture, the first dispersion mixture including a dispersion sensing ink; coating the dispersion sensing ink onto the surface of the first flexible insulating film to form a flexible sensing layer; and forming a second flexible insulating film on the flexible sensing layer to form a flexible strain sensor.

[0147] For example, part A (which may include vinyl-terminated polydimethylsiloxane and a small amount of platinum catalyst, etc.) and part B (which may include vinyl-terminated polydimethylsiloxane, hydrogen-containing silicone oil crosslinking agent and inhibitor, etc.) of a curable liquid silicone rubber (e.g., Ecoflex 00-30, etc.) are mixed in a 1:1 mass ratio. The mixture is stirred evenly using a magnetic rotor, and air bubbles are removed from the curable liquid silicone rubber solution using a vacuum pump. Subsequently, the curable liquid silicone rubber solution is poured onto a polytetrafluoroethylene mold, allowing it to flow naturally until it covers the entire mold. After remaining still for a period of time, it is placed in an 80 °C electric thermostatic drying oven for 30 min to dry and cure, thus obtaining a cured liquid silicone rubber film. This cured liquid silicone rubber film can be used as a first flexible insulating film.

[0148] Subsequently, silica powder (e.g., silica powder with a particle size of 40 micrometers, accounting for 5% of the total mass of the liquid metal) is added as a filler material to liquid metal (e.g., gallium indium alloy) and stirred (e.g., for 12 hours) to obtain a paste-like particulate dispersion sensing ink. This paste-like particulate dispersion sensing ink can be used as the aforementioned first dispersion mixture.

[0149] A polyethylene terephthalate (PET) mask of the desired shape can be laser-cut. The PET mask is then placed on a first flexible insulating film. A first dispersion mixture is placed within the PET mask and coated using a mask doctor blade to form a pattern for the flexible sensing layer. Subsequently, the PET mask can be removed, and specified fine wires are used as the sensing lines for the flexible sensing layer and led out. Finally, a curable liquid silicone rubber solution is spin-coated onto the pattern of the flexible sensing layer and dried in an 80 °C oven to form a second flexible insulating film, thus producing a flexible strain sensor.

[0150] It should be understood that the polymer matrix in the first flexible insulating film and the first dispersion mixture of the embodiments of the present invention is not limited to the above-mentioned curable liquid silicone rubber and thermoplastic polyurethane. Other polymers with good film-forming properties, flexibility and biocompatibility can be used, such as polyvinyl alcohol, silica gel or agarose.

[0151] Figure 10 The test curves for the cyclic stability test of the flexible strain sensor according to an embodiment of the present invention are shown.

[0152] like Figure 10 As shown, the flexible strain sensor of this embodiment maintains a stable resistance change rate of approximately 8% throughout 1000 repeated stretch-release cycles, without significant attenuation, drift, or fluctuation. This demonstrates that the flexible strain sensor of this embodiment possesses excellent linearity and stability during long-term cyclic use, ensuring high consistency and reliability in signal acquisition.

[0153] Figure 11 Test curves for the response / recovery time of a flexible strain sensor according to an embodiment of the present invention are shown.

[0154] like Figure 11 As shown, the flexible strain sensor of this embodiment exhibits a rapid increase in resistance change rate from 0 strain to 1% strain, with a response time of 136 ms; and a rapid decrease in resistance change rate from 1% strain back to 0 strain, with a recovery time of 172 ms. Therefore, the flexible strain sensor of this embodiment can respond quickly to medical procedures under different deformations.

[0155] Figure 12 The test curve of the strain-resistance response curve of the flexible strain sensor according to an embodiment of the present invention is shown.

[0156] like Figure 12As shown, for the flexible strain sensor of this embodiment, as the strain increases from 0 to 100%, the resistance change rate continuously increases from 0 to approximately 37%, and the curve shows a monotonically increasing trend, maintaining good responsiveness over a wide strain range. Therefore, the flexible strain sensor of this embodiment possesses good tensile strength and can operate stably within an ultra-wide strain range of 0~100%, meeting the needs of large-deformation medical procedures.

[0157] Figure 13 The test curves for the load-unload hysteresis test of a flexible strain sensor according to an embodiment of the present invention are shown.

[0158] like Figure 13 As shown, the two curves represent the loading (i.e., stretching, corresponding to the upward arrow) and unloading (releasing, corresponding to the downward arrow) processes, respectively. The two curves almost completely overlap, and the hysteresis loop is extremely small. Therefore, it can be seen that the flexible strain sensor of this embodiment possesses extremely low hysteresis, with highly consistent resistance responses during the loading and unloading processes, high signal acquisition accuracy, and no significant hysteresis error.

[0159] Figure 14 The test curve of a stepped strain cycle test of a flexible strain sensor according to an embodiment of the present invention is shown.

[0160] like Figure 14 As shown, for the flexible strain sensor of this embodiment, each strain level (e.g., strain levels of 10%, 30%, 50%, 70%, and 90%) corresponds to a stable rate of change in resistance. The responses under the same strain in two cycles are highly consistent, with no significant drift. Therefore, it can be seen that the flexible strain sensor of this embodiment possesses accurate step response and excellent repeatability to strains of different amplitudes, can accurately identify deformations of different magnitudes, and has strong anti-interference capabilities.

[0161] Figure 15 The test curves for the continuous gradual strain response test of the flexible strain sensor according to an embodiment of the present invention are shown.

[0162] like Figure 15 As shown, the voltage change rate perfectly follows the gradual change of strain (e.g., strain levels can be 0%, 5%, 10%, 15%, 20%, 25%, and 30%), exhibiting a perfect "trapezoidal" response. Furthermore, each strain level corresponds to a stable resistance plateau, with no overshoot or hysteresis during the rise and fall processes. Therefore, the flexible strain sensor of this invention possesses excellent dynamic tracking performance and signal fidelity, accurately capturing continuously changing deformations, and is suitable for dynamic monitoring scenarios such as medical procedures.

[0163] It should be noted that, in Figures 10-15In this context, the resistance change rate can be a relative resistance change rate, or it can be determined based on the ratio of the resistance change value of the flexible strain sensor under strain conditions to the resistance of the flexible strain sensor under no strain conditions, which will not be elaborated here.

[0164] In this embodiment of the invention, the fabrication of a flexible pressure sensor may include: adding an active conductive material to an organic solvent to obtain a second dispersion mixture, wherein the active conductive material includes at least two of carbon black, graphene, or carbon nanotubes; adding thermoplastic polyurethane to the second dispersion mixture to form a mixed solution; casting the mixed solution into a mold with a ball array on its surface and thermosetting it to form a pressure-sensitive layer with a biomimetic dome structure; depositing the pressure-sensitive layer on a first flexible electrode layer, and depositing a second flexible electrode layer on the pressure-sensitive layer to form a flexible pressure sensor.

[0165] For example, a microstructure-enhanced pressure-sensitive layer can be prepared. Active conductive materials such as carbon black, graphene, and carbon nanotubes are dispersed in an organic solvent such as N,N-dimethylformamide at a predetermined mass ratio to form a stable second dispersion mixture.

[0166] Figure 16 A schematic diagram of a polytetrafluoroethylene mold according to an embodiment of the present invention is shown. Figure 17 A schematic diagram of a polytetrafluoroethylene mold according to another embodiment of the present invention is shown.

[0167] In this embodiment of the invention, thermoplastic polyurethane powder can be added, and the mixture is mechanically stirred to form a homogeneous carbon / thermoplastic polyurethane mixed solution. Then, this mixed solution can be poured into a container such as... Figure 16 or Figure 17 The pressure-sensitive layer is obtained by thermosetting and demolding a polytetrafluoroethylene mold with a microsphere array on its surface. It should be understood that... Figure 16 In the polytetrafluoroethylene mold 1610 shown, the microsphere array may include microspheres of the same size. Figure 17 In the polytetrafluoroethylene mold 1710 shown, the microsphere array may include microspheres of different sizes, which will not be elaborated here. It should be noted that the pressure-sensitive layer here has the aforementioned biomimetic dome structure and redundant edges.

[0168] In this embodiment of the invention, circuit design software can be used to draw the upper and lower electrode patterns of the flexible pressure sensor. The required flexible upper and lower electrodes are then fabricated on a polyimide / copper / gold composite substrate using laser cutting. The flexible lower electrode can serve as the first flexible electrode layer, and the flexible upper electrode can serve as the second flexible electrode layer. Subsequently, device integration and packaging are performed. The redundant edges of the prepared pressure-sensitive layer are cut to obtain the biomimetic dome-shaped pressure-sensitive layer. Then, an insulating tape of a predetermined thickness (such as 3M VHB) can be used.TM Using insulating tape (of the model name), following a multi-layer stacking scheme, the first flexible electrode layer, the pressure-sensitive layer, and the second flexible electrode layer are precisely aligned and bonded to form a sandwich structure, thereby achieving integrated device design. This encapsulation method ensures unobstructed sensing position and maintains the initial cavity. It should be understood that there can be multiple pressure-sensitive layers, and the cavity is the gap between multiple pressure-sensitive layers. A flexible pressure sensor array on a single fingertip can be constructed based on these multiple pressure-sensitive layers. In this way, the resulting flexible pressure sensor array has good flexibility and can conform well to the finger.

[0169] Figure 18 A schematic diagram of a sensor array of a flexible pressure sensor according to an embodiment of the present invention is shown.

[0170] exist Figure 18 The diagram illustrates a flexible pressure sensor array and flexible leads that should be placed on each fingertip or palm. These flexible leads can be used for signal extraction. In this way, the flexible pressure sensor enables distributed sensing of multi-point pressure / tactile sensation, adapted for sensing the pressure intensity signal of medical actions.

[0171] Figure 19 The test curves for the cyclic stability test of the flexible pressure sensor according to an embodiment of the present invention are shown.

[0172] like Figure 19 As shown, the flexible pressure sensor of this embodiment of the invention maintains a stable current change rate during 8000 repeated loading-unloading pressure cycles, without significant attenuation or drift. Furthermore, in Figure 19 The illustration shows the periodicity of the pressure signal during the cycle, with highly consistent waveforms. This demonstrates that the flexible pressure sensor of this embodiment possesses excellent long-term cyclic stability and fatigue resistance, meeting the requirements for long-term use.

[0173] Figure 20 The test curves for the response / recovery time of a flexible pressure sensor according to an embodiment of the present invention are shown. The vertical axis can represent the rate of change of current.

[0174] like Figure 20 As shown, the flexible pressure sensor of this embodiment exhibits a rapid increase in current change rate during loading, with a response time of only 69ms; and a rapid decrease in current change rate during unloading, with a recovery time of only 26ms. Therefore, the flexible pressure sensor of this embodiment possesses excellent response and recovery speeds, capable of capturing minute and dynamic pressure signals in real time without significant lag.

[0175] Figure 21 The test curve of the stepped pressure response test of the flexible pressure sensor according to an embodiment of the present invention is shown.

[0176] like Figure 21 As shown, when a stepped pressure of 0N, 1N, 2N, 3N, 4N, and 5N is sequentially applied to the flexible pressure sensor of this embodiment, each pressure level corresponds to a stable rate of change of current. The higher the pressure, the higher the rate of change of current, exhibiting a monotonically increasing trend without significant overshoot or drift. Therefore, the sensor demonstrates accurate stepped response and excellent linearity to pressures of varying magnitudes, accurately identifying different pressure levels, and is suitable for quantitative pressure detection in medical procedures.

[0177] It should be noted that the aforementioned rate of change of current can be a relative rate of change of current, which can be determined based on the ratio of the current change value of the flexible pressure sensor under pressure to the reference current value of the flexible pressure sensor when it is not under pressure. This will not be elaborated here.

[0178] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0179] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A multimodal flexible glove, characterized in that, include: The flexible glove body has a first outer surface corresponding to the back of the hand and a second outer surface corresponding to the palm; A flexible strain sensor array, disposed at multiple joint positions on the first outer surface, is used to generate multiple sets of joint stress signals at multiple times based on the degree of joint bending of the target object when the target object is wearing a multimodal flexible glove and performing medical actions. The medical actions include acupuncture, and the multiple joint positions are the interphalangeal and metacarpophalangeal joints of the thumb, the interphalangeal and metacarpophalangeal joints of the index finger, and the interphalangeal and metacarpophalangeal joints of the middle finger. The multiple sets of joint stress signals correspond to the amplitude of the lifting and thrusting techniques in acupuncture. A flexible pressure sensor array is disposed at at least one location on the palm or fingertip of the second outer surface, for generating multiple sets of pressure intensity signals at multiple times based on the pressure at at least one location when the target object is wearing a multimodal flexible glove and performing medical actions; the at least one location is the fingertip of the thumb and index finger, and the pressure intensity signals correspond to the pressure distribution during needle handle twisting and acupoint pressing. A data processing circuit, at least partially disposed on the back of the hand on the first outer surface, is electrically connected to a flexible strain sensor array and a flexible pressure sensor array. It is used to send multiple sets of joint stress data and multiple sets of pressure intensity data to a host computer based on multiple sets of joint stress signals and multiple sets of pressure intensity signals, respectively. This allows the host computer to input the multiple sets of joint stress data, the time of receipt of the multiple sets of joint stress data, the placement position of the flexible strain sensor array on the flexible glove body, the multiple sets of pressure intensity data, the time of receipt of the multiple sets of pressure intensity data, and the placement position of the flexible pressure sensor array on the flexible glove body into a trained neural network, and output the first action type information and the first quantitative evaluation result of the medical action.

2. The multimodal flexible glove according to claim 1, characterized in that, The flexible strain sensors in the flexible strain sensor array each include a first flexible insulating film, a flexible sensing layer and a second flexible insulating film stacked sequentially, wherein the flexible sensing layer is formed of a first dispersion mixture including liquid metal and a thickener; The flexible pressure sensors in the flexible pressure sensor array each include: a first flexible electrode layer, a pressure-sensitive layer, and a second flexible electrode layer stacked sequentially. The pressure-sensitive layer has a biomimetic dome structure and is formed from a second dispersion mixture including an active conductive material and an organic solvent.

3. The multimodal flexible glove according to claim 1 or 2, characterized in that, The data processing circuit includes: A data processing module, comprising a first circuit board located on the back of the hand, and including: A multiplexer has multiple input terminals that are electrically connected to multiple flexible strain sensors in a flexible strain sensor array, and is used to sequentially output the joint stress signals of multiple flexible strain sensors through the output terminal of the multiplexer under the control of the control signal received through the control terminal of the multiplexer. An operational amplifier, electrically connected to the output of a multiplexer, is used to amplify the joint stress signal from each flexible strain sensor to obtain an amplified joint stress signal in analog form. An analog-to-digital converter, electrically connected to the output of an operational amplifier, is used to convert amplified joint stress signals in analog form into digital joint stress signals. A control module, disposed on the second circuit board, is electrically connected to the data processing module via a flexible circuit board, and includes: The controller is directly electrically connected to the flexible pressure sensor array and electrically connected to the analog-to-digital converter via the lines of the flexible circuit board. It is used to convert the analog pressure signal received from the flexible pressure sensor array into a digital pressure signal, receive the digital joint stress signal from the analog-to-digital converter, and frame encapsulate the digital pressure signal and the digital joint stress signal according to a predetermined communication protocol to obtain pressure data and joint stress data. The communication unit is electrically connected to the controller and is used to compress the floating-point data and long integer data of the pressing force data and joint stress data respectively, and send the compressed pressing force data and compressed joint stress data to the host computer.

4. The multimodal flexible glove according to claim 3, characterized in that, The second circuit board is located on the back of the wrist. The multimodal flexible glove also includes multiple inertial sensors, which are respectively disposed on the first circuit board and the second circuit board. They are used to collect multiple sets of inertial signals corresponding to the medical actions at multiple times when the target object wears the multimodal flexible glove and performs medical actions. Each set of inertial signals includes inertial signals collected by multiple inertial sensors at the same time. The controller is electrically connected to multiple inertial sensors and is also used to frame-encapsulate multiple sets of inertial signals according to a predetermined communication protocol to obtain multiple sets of inertial data. The communication unit is also used to compress floating-point data and long integer data of multiple sets of inertial data, and send the compressed inertial data to the host computer. The host computer is also used to input multiple sets of joint stress data, the receiving time of multiple sets of joint stress data, the setting position of the flexible strain sensor array on the flexible glove body, multiple sets of pressure data, the receiving time of multiple sets of pressure data, the setting position of the flexible pressure sensor array on the flexible glove body, multiple sets of inertial data, and the setting position of multiple inertial sensors on the flexible glove body into a trained neural network, and output the second action type information and the second quantitative evaluation result of the medical action.

5. A medical system, characterized in that, The medical system includes the multimodal flexible glove and host computer as described in any one of claims 1 to 4.

6. The medical system according to claim 5, characterized in that, Joint positions include metacarpophalangeal joints, interphalangeal joints, and wrist joints; The multimodal flexible glove includes a flexible strain sensor array based on multiple flexible strain sensors and a flexible pressure sensor array based on multiple flexible pressure sensors. The multiple flexible strain sensors are respectively located at multiple metacarpophalangeal joints, multiple interphalangeal joints and wrist joints, and the multiple flexible pressure sensors are respectively located at multiple fingertips and palms. The host computer is used for: When the target object wears a multimodal flexible glove to perform medical actions, multiple sets of joint stress data and multiple sets of pressure data at multiple times are received from the data processing circuit of the multimodal flexible glove. Each set of joint stress data includes multiple joint stress data from multiple flexible strain sensors, and each set of pressure data includes multiple pressure data from multiple flexible pressure sensors. Multiple sets of joint stress data, the reception time of multiple sets of joint stress data, the placement position of the flexible strain sensor array on the flexible glove body, multiple sets of pressure data, the reception time of multiple sets of pressure data, and the placement position of the flexible pressure sensor array on the flexible glove body are input into a trained neural network to achieve the following: The graph convolutional neural network in the neural network extracts multiple first spatial correlation features corresponding to multiple sets of joint stress data based on multiple sets of joint stress data and the setting position of the flexible strain sensor array on the flexible glove body. It also extracts multiple second spatial correlation features corresponding to multiple sets of pressing pressure data based on multiple sets of pressing pressure data and the setting position of the flexible pressure sensor array on the flexible glove body. The temporal convolutional neural network in the neural network extracts the first temporal variation feature based on the dynamic characteristics of the evolution of the joint stress data over time, based on multiple sets of joint stress data and the corresponding receiving time. It also extracts the second temporal variation feature based on the dynamic characteristics of the evolution of the pressing force data over time, based on multiple sets of pressing force data and the corresponding receiving time. The weighted cross-modal attention module in the neural network is based on a modal weighted fusion mechanism. It adaptively weights the first spatial correlation feature and the first temporal variation feature to obtain the first spatial correlation weighted feature and the first temporal variation weighted feature, respectively. It then fuses the first spatial correlation weighted feature and the first spatial correlation feature to obtain the first spatial fusion feature, and fuses the first temporal variation weighted feature and the first temporal variation feature to obtain the first temporal fusion feature. Furthermore, it adaptively weights the second spatial correlation feature and the second temporal variation feature to obtain the second spatial correlation weighted feature and the second temporal variation weighted feature, respectively. It then fuses the second spatial correlation weighted feature and the second spatial correlation feature to obtain the second spatial fusion feature, and finally fuses the second temporal variation weighted feature and the second temporal variation feature to obtain the second temporal fusion feature. The weighted bidirectional cross-modal attention module in the neural network is based on the bidirectional attention mechanism. It calculates the first bidirectional attention feature based on the first spatial fusion feature and the first temporal fusion feature, and calculates the second bidirectional attention feature based on the second spatial fusion feature and the second temporal fusion feature. The fusion layer of the neural network fuses the first spatial fusion feature, the first temporal fusion feature, the first bidirectional attention feature, the second spatial fusion feature, the second temporal fusion feature, and the second bidirectional attention feature to obtain the attention fusion feature; The output layer of the neural network processes the attention fusion features and outputs the first action type information and the first quantitative evaluation result of the medical action.

7. A quantification method, characterized in that, The quantization method includes: Receive multiple sets of joint stress data and multiple sets of pressure data at multiple times from the multimodal flexible glove as described in any of claims 1 to 4; Multiple sets of joint stress data, the receiving time of multiple sets of joint stress data, the setting position of the flexible strain sensor array on the flexible glove body, multiple sets of pressure data, the receiving time of multiple sets of pressure data, and the setting position of the flexible pressure sensor array on the flexible glove body are input into a trained neural network, which outputs the first action type information and the first quantitative evaluation result of the medical action.

8. A method for manufacturing a multimodal flexible glove according to any one of claims 1 to 4, characterized in that, The manufacturing method includes: Obtain the flexible glove body; A flexible strain sensor array was fabricated and then placed on the body of a flexible glove. A flexible pressure sensor array was fabricated and then placed on the body of a flexible glove. The data processing circuit is located on the flexible glove body and electrically connected to the flexible strain sensor array and the flexible pressure sensor array.

9. The manufacturing method according to claim 8, characterized in that, Fabrication of flexible strain sensors in a flexible strain sensor array includes: Preparation of the first flexible insulating film; A thickener is added to liquid metal to obtain a first dispersion mixture, the first dispersion mixture including a dispersion sensing ink; The dispersion sensing ink is coated on the surface of the first flexible insulating film to form a flexible sensing layer; A second flexible insulating film is formed on the flexible sensing layer to form a flexible strain sensor.

10. The manufacturing method according to claim 8 or 9, characterized in that, Fabricating a flexible pressure sensor in a flexible pressure sensor array includes: An active conductive material is added to an organic solvent to obtain a second dispersion mixture; Thermoplastic polyurethane is added to the second dispersion mixture to form a mixed solution; The mixed solution is poured into a mold with a ball array on the surface and thermosetting to form a pressure-sensitive layer with a biomimetic dome structure; A pressure-sensitive layer is disposed on a first flexible electrode layer, and a second flexible electrode layer is disposed on the pressure-sensitive layer to form a flexible pressure sensor.

Images