A neural network driven multicore fiber grating array hand motion capture method and device

Abstract

The application discloses a neural network driven multi-core fiber grating array hand motion capture method and device. The method comprises the following steps: arranging a multi-core fiber grating array on a flexible glove; synchronously collecting a wavelength sequence and an angle true value in a training stage (TS1-TS3), and training a hand joint angle prediction neural network model after data preprocessing and division; collecting and processing wavelength signals in real time in an application stage (AS1-AS4), and inputting the model to solve and output joint angles. The device comprises a sensing unit, a communication unit, a solving unit, a calibration unit and a training unit. The application utilizes the powerful nonlinear fitting capability of the neural network and the multi-core fiber sensing advantage, effectively solves the problems of the traditional scheme, such as sensitivity to arrangement, linear mapping limitation and weak anti-interference capability, and realizes high-precision, high-robustness and flexible arrangement hand motion capture, which is suitable for medical treatment, robots and virtual reality fields.

Description

Technical Field

[0001] This invention relates to the field of fiber optic sensing technology and hand motion capture, specifically to a neural network-driven method and device for capturing hand motion using a multi-core fiber optic grating array. Background Technology

[0002] In recent years, with the rapid development of natural human-computer interaction technology, it has shown broad application prospects in fields such as minimally invasive medicine, robotic teleoperation, and virtual reality. Traditional interactive devices such as keyboards and joysticks have limited flexibility and lack immersion, making it difficult to meet the refined interaction needs in complex scenarios. As the most dexterous part of the human body, the real-time perception and capture of the hand's posture has become a key technology for improving the naturalness of interaction. Existing hand motion capture technologies mainly include inertial sensing, visual imaging, strain sensing, and mechanical exoskeleton control, but all have significant limitations: inertial sensors are susceptible to electromagnetic interference, resulting in drift and insufficient positioning accuracy; visual solutions are greatly affected by ambient light and occlusion, and are costly and have complex algorithms; mechanical exoskeletons have high control weight, poor wearing comfort, and inflexible movements; resistive strain sensing faces challenges such as limited sensor layout and signal stability, making it difficult to achieve high-precision and robust hand motion capture.

[0003] Fiber optic strain sensors are sensing devices based on optical mechanisms. Using fiber optic gratings as the sensing element, they accurately measure the strain state of the object being measured by sensing the characteristic changes in light signals caused by external strain during propagation. These sensors possess outstanding advantages such as high accuracy, high spatial resolution, resistance to electromagnetic interference, corrosion resistance, and compact structure, making them suitable for long-term, high-precision monitoring of minute strains. Multi-core optical fiber refers to optical fiber formed by integral drawing or bonding two or more optical fibers. Multi-core grating optical fiber refers to optical fiber with gratings inscribed on multi-core fibers, or optical fiber formed by first inscribing gratings on a single fiber and then bonding two or more grating fibers together.

[0004] Existing technology discloses a data glove and interaction system based on fiber Bragg gratings. The data glove consists of a bending sensor deployed at the finger joints of the glove body. The bending sensor includes a fiber Bragg grating, shape memory alloy wire, and a metal capillary. This method calculates the bending angle as a linear function, with the bending angle linearly mapped to the center wavelength. However, due to the deployment and finger movement mechanism, finger bending and strain have a nonlinear mapping. This method does not consider the nonlinearity of the fiber Bragg grating center wavelength data calculation. Furthermore, deploying too few gratings on a single fiber of a single finger makes it difficult to simultaneously calculate the opening and closing and bending of the finger joint.

[0005] Existing technology also discloses a hand motion capture device and system based on multi-core fiber grating array sensing. This technical solution captures hand movements through a multi-core fiber grating array. This algorithm has high requirements for the deployment of flexible sensors and for the linearity of sensor data. Although it can simultaneously sense the bending and opening / closing of joints using multi-core fiber gratings, it still struggles to address the nonlinearity between sensor data and joint angles. Furthermore, different finger movements can interfere with each other, cross-influencing skin strain and interfering with the calculation of joint angles using the multi-core fiber grating array wavelength, thus reducing the accuracy and stability of hand motion capture.

[0006] A common problem with the above-disclosed technologies is that, due to limitations in the calculation and deployment methods, the nonlinear calculation of joint angles from sensor data is not considered. Existing technology 1, influenced by sensor deployment, has high requirements for sensor placement, requiring the grating to be positioned at the joint's bending location; existing technology 2 does not achieve a nonlinear mapping between curvature and joint angles. For complex hand motion capture, sensor data is affected by deployment, different hands, and other external interferences, making it difficult to achieve high-precision and high-stability mapping of sensor data to joint angles using the above-disclosed technologies.

[0007] In order to achieve gesture dynamic capture more conveniently, flexibly and accurately, and in order to effectively promote the development and application of human-computer interaction technology, there is an urgent need for a method and device for motion capture of flexible sensing gloves based on neural networks that can achieve free sensor deployment, high precision, good adaptability and strong robustness. Summary of the Invention

[0008] The present invention aims to provide a neural network-driven multi-core fiber optic grating array hand motion capture method and device to solve the problems of the existing technology in handling the complex nonlinear mapping between sensing signals and joint angles, as well as the problems of sensitivity to sensor deployment and weak anti-interference ability, thereby achieving high-precision, high-robust and flexible hand motion capture.

[0009] To achieve the above objectives, the present invention provides a neural network-driven method for capturing hand motion using a multi-core fiber grating array, characterized by including the deployment of the multi-core fiber grating array in a flexible glove, model training, and model application.

[0010] The deployment of the multi-core fiber grating array in the flexible glove refers to the multi-core fiber grating array being deployed along a preset trajectory in at least a portion of the deformation-to-be-captured area of ​​the flexible glove.

[0011] The model training includes the following steps: TS1: Synchronously acquire wavelength time series data of the multi-core fiber grating array and true value data of hand joint angles provided by the calibration unit; TS2: Preprocess and partition the data, wherein the preprocessing includes at least one of outlier removal, temperature compensation and normalization, and the data partitioning includes at least partitioning the data into a training set; TS3: Construct and train a neural network model for predicting hand joint angles, the model including at least one of a temporal feature extraction unit and an angle regression unit.

[0012] The application of the model includes the following steps: AS1: Real-time acquisition of wavelength signals from multi-core fiber optic grating arrays; AS2: The data is preprocessed, and the preprocessing includes at least one of outlier removal, temperature compensation, and normalization; AS3: Input the preprocessed multi-core fiber grating array wavelength signal into the hand joint angle prediction neural network model; AS4: Outputs the calculated hand joint angle data.

[0013] Furthermore, the multi-core fiber Bragg grating array is formed by longitudinally bonding multiple fiber Bragg grating arrays or by integrally fabricating multiple fiber cores.

[0014] Furthermore, the calibration unit is used to measure and record the true value data of the angles of each joint of the hand during synchronous acquisition.

[0015] Furthermore, the temperature compensation is achieved by using a reference grating to eliminate the combined effects of temperature through subtraction, resulting in temperature-compensated arbitrary wavelength data. satisfy: ,in This represents the wavelength data before temperature compensation. The wavelength of the reference grating at the initial temperature. The wavelength of the reference grating under temperature changes.

[0016] Furthermore, the hand joint angles include three-dimensional angles of multiple joints, expressed as Euler angles or quaternions, which are absolute angles relative to the palm or relative angles between adjacent joints.

[0017] This invention provides a neural network-driven multi-core fiber grating array hand motion capture device, comprising: a sensing unit, a communication unit, a calculation unit, a calibration unit, and a training unit.

[0018] The sensing unit includes a flexible sensing glove with a multi-core fiber Bragg grating array; the communication unit includes a fiber Bragg grating demodulator for reading the wavelengths of the multi-core fiber Bragg grating array and converting the grating wavelength signals into digital signals for transmission; the calculation unit has a built-in neural network model for predicting hand joint angles for real-time output of joint angles; the calibration unit and the training unit are used to assist in building the neural network model for predicting hand joint angles; the calibration unit includes, but is not limited to, a high-precision optical motion capture system or a mechanical calibration device or an inertial motion capture system or a shape sensing system; the training unit is used for training the neural network model.

[0019] The sensing unit, communication unit, calibration unit, and training unit are used for model training.

[0020] The sensing unit, communication unit, and calculation unit are used for the application of the model.

[0021] This invention provides a neural network-driven method and device for capturing hand motion using a multi-core fiber Bragg grating array. Addressing the limitations of existing traditional interactive devices and fiber optic data gloves, such as linear mapping constraints, stringent deployment requirements, and weak anti-interference capabilities, this invention proposes an innovative hardware-software collaborative solution, achieving the following beneficial effects:

[0022] (1) Breaking through the bottleneck of traditional linear mapping, extremely high-precision hand motion capture is achieved. Addressing the shortcomings of existing technologies that simply assume a linear mapping between the bending angle and the center wavelength of the fiber optic grating, which fails to handle the complex movement mechanisms of the fingers, this invention creatively introduces a neural network model for predicting hand joint angles. By constructing a neural network model containing temporal feature extraction units and angle regression units, the system can learn and fit the complex nonlinear mapping relationship between the wavelength of the multi-core fiber optic grating and multiple hand joints (three-dimensional Euler angles or quaternions). This end-to-end data-driven approach completely transcends the accuracy ceiling of traditional physical or mechanistic modeling, significantly improving the accuracy of refined motion capture.

[0023] (2) The invention removes the stringent restrictions on sensor placement, greatly improving the system's adaptability and mass production feasibility. Existing technologies require the fiber optic grating to be precisely placed at specific locations where the finger joints are bent, placing extremely high demands on the physical alignment accuracy of the sensors. This invention places the fiber optic grating along a preset trajectory in at least a portion of the deformation area to be captured on the flexible glove, and combines this with the powerful feature extraction and nonlinear fault tolerance capabilities of neural networks to achieve "free sensor placement." This means that the system no longer relies excessively on the absolute coordinate accuracy of the fiber optic grating on the glove, which not only perfectly adapts to individual differences in hand size among different users, but also significantly reduces the difficulty and manufacturing cost of fitting and calibrating the flexible sensing gloves in industrial mass production.

[0024] (3) Effectively overcomes the interference of multi-finger cross-strain, significantly enhancing the robustness and stability of the system. Addressing the problem that "different finger movements can cross-influence skin strain, thus interfering with the calculation," this invention utilizes a multi-core fiber grating array to acquire rich, high-dimensional strain information and deeply mines the dynamic spatiotemporal dependencies in the wavelength time series through a recurrent neural network structure. This model achieves decoupling of complex cross-interference at the algorithmic level, ensuring that even when users perform complex gestures such as clenching their fists or multi-finger coordinated movements, it can still output highly stable, drift-free continuous joint angle data.

[0025] (4) Integrating the inherent advantages of optical fiber with data preprocessing mechanisms, it possesses excellent environmental anti-interference capabilities. Addressing the inherent limitations of inertial sensors, such as susceptibility to electromagnetic interference, and the limitations of visual solutions due to obstruction and lighting, this invention fully leverages the physical advantages of fiber optic gratings—their resistance to strong electromagnetic interference and corrosion—enabling them to operate stably in extreme environments such as strong magnetic fields and obstructed views. Furthermore, this invention introduces specific preprocessing steps before model calculation, particularly utilizing a reference grating for difference operations, effectively eliminating wavelength drift caused by environmental temperature fluctuations, further ensuring the purity of the sensing signal and the absolute reliability of the dynamic calculation results.

[0026] (5) The system architecture is highly optimized, making it lightweight, comfortable to wear, and supporting low-latency real-time interaction. Compared to the bulkiness of mechanical exoskeletons and the unstable signal of traditional resistive sensors, this invention seamlessly integrates thin multi-core optical fibers into a flexible sensing glove, achieving a near-zero-burden wearing experience. In addition, this invention adopts a system architecture that separates "training" and "application," using independent calibration and training units to complete high-computing-power model optimization, and deploying lightweight models in integrated solution units. While ensuring low latency and high response speed, it greatly enhances the user's immersion, perfectly meeting the refined human-computer interaction needs in complex scenarios such as minimally invasive medical care, robot teleoperation, and virtual reality. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the overall architecture of a neural network-driven multi-core fiber optic grating array for hand motion capture, provided in an embodiment of the present invention.

[0028] Figure 2 This is a schematic diagram of a flexible sensing glove and its multi-core fiber optic grating layout provided in an embodiment of the present invention.

[0029] Figure 3 This is a schematic diagram of a neural network model structure for predicting hand joint angles provided in an embodiment of the present invention.

[0030] Figure 4 This is a schematic diagram of a neural network-driven multi-core fiber optic grating array hand motion capture device according to an embodiment of the present invention. Detailed Implementation

[0031] The present invention will now be further described with reference to the accompanying drawings.

[0032] Please see Figure 1 This illustration shows a schematic diagram of the overall architecture of a neural network-driven multi-core fiber Bragg grating array for hand motion capture according to an embodiment of the present invention. The implementation scheme of the entire process is described as follows:

[0033] First, a multi-core fiber grating array is deployed on the flexible sensing glove to accurately sense the physical deformation caused by hand movements and convert it into wavelength signals.

[0034] During the model training phase: TS1: Synchronously acquire wavelength time series data generated by the multi-core fiber grating array, and true value data of hand joint angles synchronously measured by the calibration unit; TS2: Preprocessing and data partitioning, performing quality optimization on the collected raw data, including at least one of outlier removal, temperature compensation and normalization, and partitioning the data into test, validation and training sets; TS3: Model Building and Training. Based on the processed data, build and train a neural network model for predicting hand joint angles. The model structure includes at least one of a temporal feature extraction unit for capturing dynamic patterns and an angle regression unit for nonlinear feature mapping.

[0035] During the model application phase: AS1: Real-time acquisition of wavelength signals generated by a multi-core fiber optic grating array during hand movements in a flexible sensing glove; AS2: Performs preprocessing operations on the acquired wavelength signals in the same manner as during the training phase to improve data quality; AS3: Input the preprocessed wavelength signal into the hand joint angle prediction neural network model that has been deployed in the solution unit; AS4: The model performs forward calculations and outputs the calculated multi-joint angle data of the hand in real time, achieving accurate hand motion capture.

[0036] Figure 2This is a schematic diagram of a flexible sensing glove and its multi-core fiber Bragg grating array layout in a specific embodiment of the present invention, including a flexible glove 1, a multi-core fiber Bragg grating array 2, and a fiber Bragg grating demodulator 3. The flexible glove 1 is the basic carrier of the entire flexible sensing glove, designed for direct wear on the hand; the multi-core fiber Bragg grating array 2 is composed of three fiber Bragg grating arrays longitudinally bonded together, extending along the direction of the fingers and distributed in a wavy pattern on the back of the flexible glove; short horizontal lines on the multi-core fiber Bragg grating array 2 are marked as grating nodes, and gratings are etched on the fiber Bragg grating arrays near the grating nodes; the fiber Bragg grating demodulator 3 is located near the wrist position on the back of the glove, where the multi-core fiber Bragg grating array converges; the fiber Bragg grating array is directly connected to multiple channels of the fiber Bragg grating demodulator 3; preferably, a beam splitter is used, connecting multiple fiber Bragg grating arrays to one channel of the fiber Bragg grating demodulator 3. The sensing unit is the flexible sensing glove with the multi-core fiber Bragg grating array; the communication unit is the fiber Bragg grating demodulator, which converts the center wavelength optical information of the fiber Bragg grating into digital wavelength information.

[0037] In one specific embodiment of the present invention, the hand joint angles are represented in the form of three-dimensional Euler angles or quaternions. Each hand has 15 sets of mapped hand joint angles, and each finger has 3 sets of mapped joint angles: the wrist-metacarpal joint, metacarpophalangeal joint, and interphalangeal joint of the thumb; and the absolute angles of the metacarpophalangeal joint, proximal interphalangeal joint, and distal interphalangeal joint of the index, middle, ring, and little fingers relative to the palm, or their relative angles relative to the previous joint (i.e., the joint closer to the palm, with the root joint set as the palm). The process of simultaneously acquiring two data streams includes: executing a preset calibration action sequence, which includes at least one of finger extension, fist clenching, individual finger bending, and multi-finger coordinated movements.

[0038] In a specific embodiment of the present invention, the synchronous acquisition in step TS1 is achieved through timestamp-based data interpolation technology, and the calibration unit is an optical motion capture system comprising multiple high-speed infrared cameras. Since the acquisition frame rate of the wavelength time series data of the multi-core fiber grating array typically differs from the acquisition frame rate of the calibration unit for acquiring the true value data of the hand joint angle, the two original data streams cannot be naturally aligned on the time axis. Therefore, utilizing the timestamp information carried by each of the two data streams during synchronous acquisition, a preset interpolation algorithm is used to resample and fit the two sets of data with different acquisition frame rates to the time axis, ensuring a strict one-to-one correspondence between the wavelength data and the true value data of the joint angle at the same time node. This provides a high-quality sample set with high time synchronization accuracy for the subsequent training of the neural network model.

[0039] Step TS2 includes: preprocessing and partitioning the data, wherein the preprocessing includes at least one of outlier removal, temperature compensation and normalization.

[0040] In a specific embodiment of the present invention, outlier removal in step TS2 refers to identifying and removing outlier records that contain missing values ​​or significantly deviate from physiological motion logic and normal physical distribution. Addressing potential quality defects in the original dataset during acquisition, effectively removing these abnormal data affected by hardware jitter or sudden environmental noise can prevent invalid samples from negatively interfering with subsequent neural network training, ensuring the accuracy of the model's learned features.

[0041] In a specific embodiment of the present invention, the temperature compensation described in step TS2 employs a reference grating to implement a temperature compensation mechanism, thereby eliminating the center wavelength shift caused by ambient temperature fluctuations and separating the wavelength signal affected only by hand movement strain. This compensation process uses a reference fiber grating, which is only affected by temperature and not by strain, as a benchmark, and the temperature-compensated arbitrary wavelength data... Satisfying function: , in This represents the wavelength data before temperature compensation. The wavelength of the reference grating at the initial temperature. The wavelength of the reference grating under temperature changes.

[0042] In a specific embodiment of the present invention, the normalization process described in step TS2 is to scale the center wavelengths of different gratings to a uniform numerical scale in order to eliminate the differences between different sensing gratings and demodulation channels and prevent certain high-amplitude features from occupying too much weight during model training.

[0043] In a specific embodiment of the present invention, the data partitioning described in step TS2 involves scientifically dividing the preprocessed high-quality dataset according to a preset strategy. By employing a timestamp-based sliding window method or random sampling, the wavelength sequence and its corresponding angle ground truth values ​​are divided into independent training, validation, and test sets. The training set is used for iterative updates of the weight parameters within the neural network model; the validation set is used for real-time monitoring of the training process and hyperparameter tuning; and the test set is used to simulate real-world application scenarios after training to conduct a final independent evaluation of the model's capture accuracy, real-time performance, and robustness.

[0044] Step TS3 includes: constructing and training a neural network model for predicting hand joint angles, the model including a temporal feature extraction unit and an angle regression unit.

[0045] Preferably, the temporal feature extraction unit employs a long short-term memory (LSTM) network structure. This unit receives a wavelength data window consisting of t consecutive time steps. Internally, the LSM network selectively memorizes and transmits sequence information through the synergistic action of forget gates, input gates, and output gates, thereby effectively capturing long-range dependencies and dynamic patterns in the hand movement wavelength signal and encoding the information of the entire time window into a fixed-length one-dimensional feature vector.

[0046] The angle regression unit is preferably composed of multiple fully connected layers cascaded sequentially. This unit receives the feature vector output from the temporal feature extraction unit as input. The first fully connected layer performs nonlinear transformation and fusion of the features; subsequent fully connected layers progressively abstract and refine the features; finally, the output layer maps the high-level features to specific numerical values, corresponding to the predicted angle values ​​for the 15 joints of the hand.

[0047] Figure 3 This is a schematic diagram of a neural network model structure for predicting hand joint angles according to an embodiment of the present invention. Figure 3-1 The image shows a temporal feature extraction unit. Figure 3-2 The image shows the angle regression unit.

[0048] Figure 3-1 In the long short-term memory network cells shown, the input signal X t This refers to the wavelength data at the current moment; H t-1 This refers to the hidden state of the previous moment, i.e., short-term memory; C t-1 The previous cell state is long-term memory; the forgetting gate determines the previous cell state C through the sigmoid layer. t-1 The input gate determines which information needs to be discarded; the sigmoid layer decides which new information needs to be updated, and works with the tanh layer to generate new candidate memories, which are then added to the cell state; the output gate is based on the updated cell state C. t By combining Sigmoid and tanh, the amount of information output to the hidden state H at the current time step is determined. t Output signal C t For the updated long-term memory to be passed on to the next moment; H t This is the feature vector at the current time step, which will be passed to the next time step and also enter as the output of the current layer. Figure 3-2 In the angle regression unit shown.

[0049] Figure 3-2 The diagram shows a fully connected neural network, where each circle represents a neuron. The temporal features H extracted from the Long Short-Term Memory network are shown. t Mapped to joint angle Hout t The input layer is the output vector H from the Long Short-Term Memory network.t The hidden layer can have 0, 1, or more layers; the output layer outputs the joint angle Hout. t .

[0050] It should be understood that Figure 3 The model structure shown is merely a typical embodiment of the present invention and not the only limitation. Those skilled in the art can flexibly adjust the model according to actual computational constraints or capture accuracy requirements: for example, the feature extraction unit is not limited to a long short-term memory network structure, but can also employ gated recurrent units, temporal convolutional structures, or other neural network components suitable for processing sequential data; in some simple application scenarios with lower requirements for temporal features, the feature extraction unit can be omitted, and the regression unit can directly perform the calculations. Furthermore, the depth (number of layers) and width (number of neurons per layer) of the regression unit can be parameterized according to the complexity of the joint mapping to achieve the optimal balance between performance and efficiency.

[0051] Figure 4 This is a schematic diagram of a neural network-driven multi-core fiber Bragg grating array hand motion capture device according to an embodiment of the present invention. White arrows connect the model training section, and gray arrows connect the model application section. The sensing unit includes a flexible sensing glove with a multi-core fiber Bragg grating array; the communication unit includes a fiber Bragg grating demodulator for reading the wavelengths of the multi-core fiber Bragg grating array, converting the grating wavelength signals into digital signals, and transmitting them; the calculation unit has a built-in neural network model for predicting hand joint angles, used to output joint angles in real time; a calibration unit and a training unit are used to assist in constructing the neural network model for predicting hand joint angles; the calibration unit includes, but is not limited to, a high-precision optical motion capture system, a mechanical calibration device, an inertial motion capture system, or a shape sensing system; the training unit is used for training the neural network model.

[0052] During model training, the sensing unit and communication unit provide wavelength time series data, while the calibration unit synchronously provides true joint angle data. After preprocessing and data partitioning, a neural network model for predicting hand joint angles is obtained in the training unit.

[0053] In the model application, the sensing unit and the communication unit acquire wavelength time series data in real time, and after preprocessing, input it into the calculation unit to output the joint angle; the calculation unit has a built-in neural network model for predicting hand joint angles output by the training unit.

Claims

1. A neural network-driven method for capturing hand motion using a multi-core fiber Bragg grating array, characterized in that, This includes the application of multi-core fiber Bragg grating arrays in flexible gloves, model training, and model application. The deployment of the multi-core fiber grating array in the flexible glove refers to the multi-core fiber grating array being deployed along a preset trajectory in at least a portion of the deformation area to be captured in the flexible glove. The model training includes the following steps: TS1: Synchronously acquire wavelength time series data of the multi-core fiber grating array and true value data of hand joint angles provided by the calibration unit; TS2: Preprocess and partition the data, wherein the preprocessing includes at least one of outlier removal, temperature compensation and normalization, and the data partitioning includes at least partitioning the data into a training set; TS3: Construct and train a neural network model for predicting hand joint angles, the model including at least one of a temporal feature extraction unit and an angle regression unit; The application of the model includes the following steps: AS1: Real-time acquisition of wavelength signals from multi-core fiber optic grating arrays; AS2: The data is preprocessed, and the preprocessing includes at least one of outlier removal, temperature compensation, and normalization; AS3: Input the preprocessed multi-core fiber grating array wavelength signal into the hand joint angle prediction neural network model; AS4: Outputs the calculated hand joint angle data.

2. The method according to claim 1, characterized in that, The multi-core fiber grating array is formed by longitudinally bonding multiple fiber grating arrays or by integrating multiple fiber cores into a single fabrication.

3. The method according to claim 1, characterized in that, The calibration unit is used to measure and record the true value data of the angles of each joint of the hand during synchronous acquisition.

4. The method according to claim 1, characterized in that, The temperature compensation is achieved by using a reference grating, eliminating the combined effects of temperature through subtraction, resulting in temperature-compensated data for any wavelength. satisfy: ,in This represents the wavelength data before temperature compensation. The wavelength of the reference grating at the initial temperature. The wavelength of the reference grating under temperature changes.

5. The method according to claim 1, characterized in that, The temporal feature extraction unit adopts a network structure with time series feature capture function, and the angle regression unit adopts a neural network structure with nonlinear mapping function.

6. The method according to claim 1, characterized in that, The hand joint angles include three-dimensional angles of multiple joints, expressed as Euler angles or quaternions, and are absolute angles relative to the palm or relative angles between adjacent joints.

7. An apparatus for implementing the method as described in any one of claims 1–6, characterized in that, include: Sensing unit, communication unit, calculation unit, calibration unit, and training unit; The sensing unit includes a flexible sensing glove with a multi-core fiber grating array. The communication unit includes a fiber optic grating demodulator for reading the wavelengths of a multi-core fiber optic grating array and converting the grating wavelength signals into digital signals for transmission; the calculation unit has a built-in neural network model for predicting hand joint angles for real-time output of joint angles; the calibration unit and training unit are used to assist in building the neural network model for predicting hand joint angles; the calibration unit includes, but is not limited to, a high-precision optical motion capture system, a mechanical calibration device, an inertial motion capture system, or a shape sensing system; the training unit is used for training the neural network model. The sensing unit, communication unit, calibration unit, and training unit are used for model training as described in claim 1; The sensing unit, communication unit, and calculation unit are used in the model application described in claim 1.

Images