Forearm rotation detection sensor system and method based on chiral multimode fiber speckle

Abstract

The application provides a kind of based on chiral multimode optical fiber speckle forearm rotation detection sensor system and method, belong to optical fiber sensing and human-computer interaction technical field.The application solves the technical problem that traditional optical fiber sensor is difficult to distinguish forearm rotation movement direction movement.The technical scheme is: using the chiral sensitive characteristics of multimode optical fiber speckle to the twist direction, the optical fiber is fixed on the forearm of human body, through collecting speckle pattern under different rotation angles, combined with convolutional neural network for deep learning training, realize the direction and angle synchronous identification of forearm pronation and supination movement.The beneficial effects of the application are: the identification of forearm pronation and supination of each 10 angle categories in the range of 0° to 90° is realized, the accuracy is more than 99%, with high sensitivity, strong direction recognition ability, low cost, light weight and flexibility, etc., can be widely used in virtual reality human-computer interaction, robot teleoperation and medical rehabilitation monitoring fields.

Description

Technical Field

[0001] This invention relates to the fields of fiber optic sensing and human-computer interaction technology, specifically to a forearm rotation detection sensor system and method based on chiral multimode fiber speckle. Background Technology

[0002] With the deep integration of the metaverse and physical systems, human-computer interaction is undergoing a paradigm shift from discrete command triggering to continuous, high-fidelity motion mapping. In cutting-edge applications such as immersive virtual reality and robotic teleoperation, interactive systems not only need to accurately perceive the amplitude of human joint movements to ensure quantitative precision in operations, but also need to reproduce the user's motion intentions in real time and with high accuracy. Especially in tasks involving fine manipulation, such as adjusting knobs on a robotic arm or complex assembly, the system must possess a keen ability to perceive the directionality of movements to avoid control accidents caused by misjudgment of direction.

[0003] Currently, wearable motion sensing solutions mainly include inertial measurement units (IMUs) based on microelectromechanical systems (MEMS) and vision-based motion capture systems. IMUs suffer from inherent cumulative integral drift, limiting long-term measurement stability, and there is a constraint between sampling rate and attitude calculation accuracy, posing challenges when capturing high-frequency dynamic movements. While vision-based motion capture systems offer higher accuracy, they are highly susceptible to limb occlusion or changes in lighting conditions in complex environments, making it difficult to maintain robustness for monitoring fine hand movements under multi-degree-of-freedom occlusion.

[0004] Fiber optic sensors, with their resistance to electromagnetic interference, lightweight flexibility, and good biocompatibility, are considered a potential ideal solution for next-generation wearable interactive devices. In recent years, the introduction of novel stretchable luminescent materials has further improved the adaptability of fiber optic sensors to human skin surfaces. Recent research has proposed a wearable joint monitoring system based on plastic optical fiber (POF), which achieves high-precision measurement of complex joint angles by utilizing changes in light intensity caused by joint bending.

[0005] However, current mainstream fiber optic sensing technologies generally rely on wavelength or intensity modulation principles. This modulation is essentially scalar sensing; while it can accurately measure the magnitude of deformation, it loses the directionality of motion, i.e., it loses vector information. This makes it difficult for sensors to distinguish between actions with opposite directions but similar amplitudes, facing the physical bottleneck of "directional ambiguity." To overcome this limitation, researchers have introduced chirality into the design and fabrication of sensors to detect the directionality of sensing. For example, spiral conical long-term fiber gratings or tilted arc-shaped gratings are used to break the geometric symmetry of the grating structure, thereby achieving the identification of the torsion direction. Other studies have developed high-sensitivity vector torsion sensors based on helical side-hole fibers. Although these special fiber optic solutions solve the directionality problem in terms of physical mechanism, their fabrication usually relies on complex laser processing or special wire drawing processes, resulting in high fabrication costs. Furthermore, the sensors require special packaging, reducing the robustness of the devices and limiting their large-scale application in low-cost wearable devices.

[0006] In recent years, multimode fiber (MMF) speckle imaging technology has provided a novel solution for the fabrication of low-cost fiber optic sensors. By combining with artificial intelligence to learn and analyze interference patterns, it enables the detection of physical parameters such as curvature and temperature. MMF speckle imaging is based on the fact that external parameters induce microscopic geometric deformations within the fiber, causing changes in the fiber's refractive index, resulting in fiber speckles possessing "fingerprint characteristics" of external physical parameters. By analyzing and learning these fingerprint characteristics, the detection of external physical parameters can be achieved. However, to date, there is no research on the application of MMF speckle technology in human motion monitoring, especially for the detection of directional rotational motion. Summary of the Invention

[0007] This invention aims to solve the technical problem that traditional fiber optic sensors have difficulty distinguishing rotational motions with opposite directions but the same amplitude. It provides a forearm rotation detection sensor based on chiral multimode fiber speckle, which can realize synchronous and high-precision perception of the amplitude and direction of forearm rotational motion, and provide a reliable vectorized perception solution for fields such as immersive virtual reality, robot teleoperation and medical rehabilitation monitoring.

[0008] This invention is achieved through the following measures: a forearm rotation detection sensor based on chiral multimode fiber speckle, comprising a coherent light source, a multimode fiber, an image acquisition device, and a data processing unit. The coherent light source is used to generate coherent laser light, preferably a helium-neon laser with a wavelength of 632.8 nm. One end of the multimode fiber is coupled to the laser light output from the coherent light source through a microscope objective, preferably using a step-index multimode fiber with a core diameter of 62.5 μm and a cladding diameter of 125 μm. The middle section of the multimode fiber, serving as the sensing area (approximately 0.5 m in length), is fixed to the human forearm. Specifically, the proximal end is fixed to the posterior aspect of the upper arm, approximately 10 cm above the lateral epicondyle of the humerus, as a reference fixing end for torsion; the middle section is fixed to the midline of the dorsal aspect of the forearm, approximately 5 cm below the olecranon of the ulna; and the distal end is fixed near the radial styloid process on the dorsal aspect of the wrist. It is naturally laid along the dorsal aspect of the arm using medical tape, with an appropriate curvature reserved at the elbow joint to accommodate a 90° flexion posture. The image acquisition device is a CCD camera used to acquire speckle patterns from the output end face of a multimode fiber, with a preferred resolution of 1368×918 pixels. The data processing unit includes a multi-task classification model based on a convolutional neural network, used for feature extraction and classification of the acquired speckle patterns, simultaneously outputting the amplitude of the forearm rotation angle and the direction of movement. This invention is the first to use the chiral characteristics of speckle in ordinary multimode fiber for human rotational motion direction perception, eliminating the need for special fiber fabrication and employing multi-task learning to achieve vectorized synchronous output, thus solving the problem of directional ambiguity in traditional solutions.

[0009] The multi-task classification model is built on the ResNet-50 architecture, including a shared backbone feature extraction network and two independent classification heads. The backbone network adopts a 50-layer residual architecture, utilizing a bottleneck structure containing multiple convolutions, batch normalization layers, and the ReLU activation function to enhance the network's ability to express complex texture features in speckle images and improve training stability. The angle amplitude classification head outputs 10 angle categories, corresponding to 0° to 90°, with a stride of 10°; the motion direction classification head outputs 3 categories: pronation, supination, and neutral. The acquired speckle images are uniformly adjusted to a 224×224 pixel RGB three-channel format for neural network input. The two classification heads share backbone features but have independent parameters, and can synchronously output angle and direction in a single inference, unlike existing fiber optic sensing schemes that can only output scalar information.

[0010] In practical use, the subject maintains a natural standing posture with the upper arm adducted close to the torso, the elbow joint flexed at 90° and placed in an anatomically neutral position. A multimode optical fiber is fixed to the forearm as described above. Laser output from a coherent light source is coupled into the multimode optical fiber through a microscope objective. When the subject rotates their forearm, the forearm and wrist segments of the fiber undergo clockwise or counterclockwise torsional deformation, altering the refractive index distribution within the fiber and inducing energy coupling and phase shift between different modes, resulting in a chiral-sensitive evolution of the speckle texture at the output end. A CCD camera acquires speckle images in real time and transmits them to a data processing unit. After preprocessing, the images are input into a multi-task classification model. The model simultaneously outputs the angular amplitude and direction of motion. Through combined mapping, accurate identification of 19 signed angles covering -90° to 90° is achieved, completing the acquisition of vectorized perception information of forearm rotation.

[0011] The present invention has the following advantages over the prior art:

[0012] 1. High directional discrimination capability: By utilizing the intrinsic chirality of multimode fiber speckle to the direction of torsion, the "directional ambiguity" bottleneck of traditional fiber optic sensors has been successfully solved, enabling precise differentiation of rotational motions with opposite directions but the same amplitude. This is the first time that the chiral characteristics of ordinary multimode fiber speckle have been used to realize the perception of human rotation direction without the need for special optical fibers or complex packaging.

[0013] 2. High recognition accuracy: The recognition accuracy for 10 angle categories of forearm pronation and supination within the range of 0° to 90° is over 99%, achieving synchronous high-precision perception of the amplitude and direction of forearm rotation.

[0014] 3. Simple fabrication and low cost: It uses ordinary step-index multimode fiber, without the need for complex laser processing or special drawing process, and without special packaging, which greatly reduces the fabrication cost and difficulty.

[0015] 4. Lightweight, flexible, and comfortable to wear: The fiber optic sensor is lightweight and flexible. It is naturally laid on the skin surface with medical tape, making it highly comfortable to wear and suitable for long-term use.

[0016] 5. Strong anti-interference capability: Fiber optic sensors have the advantages of being resistant to electromagnetic interference and unaffected by lighting conditions and obstructions, and can maintain stable detection performance even in complex environments.

[0017] 6. Broad application prospects: It can be widely used in fields such as virtual reality human-computer interaction, robot teleoperation, medical rehabilitation monitoring, and motion posture analysis, providing a reliable vectorized perception solution for the next generation of immersive human-computer interaction. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 Schematic diagram of the experimental setup for verifying the sensitivity of multimode fiber speckle torsion direction;

[0020] Figure 2 This is a quantitative ZNCC analysis diagram showing the variation of optical fiber speckle pattern with torsion angle, where... Figure 2 (a) shows the curve of speckle correlation coefficient as a function of angle. Figure 2 (b) is a chiral feature verification diagram;

[0021] Figure 3 This is a schematic diagram of the forearm rotation detection experimental system;

[0022] Figure 4 An angle classification confusion matrix that includes directional information for forearm movement.

[0023] Explanation of reference numerals in the attached figures:

[0024] 1. Helium-neon laser; 2. Microscope objective; 3. Fiber optic pigtail; 4. Fiber optic clamp; 5. Multimode fiber; 6. Fiber optic rotator; 7. Collimating lens; 8. CCD camera; 9. Human forearm. Detailed Implementation

[0025] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Example 1

[0027] Multimode fiber torsion direction sensitivity verification experimental system, such as Figure 1As shown, the system includes 1-a helium-neon laser; 2-microscope objective; 3-fiber pigtail; 4-fiber clamp; 5-multimode fiber; 6-fiber rotator; 7-collimating lens; 8-CCD camera. The experiment was conducted in a laboratory environment with a constant temperature of 27±0.5°C and stable background illumination. The helium-neon laser 1 outputs a coherent laser with a wavelength of 632.8 nm. This laser beam is coupled into a step-index multimode fiber 5 (core diameter 62.5 μm, cladding diameter 125 μm, length 2.0 m) through the microscope objective 2 and the fiber pigtail 3. The middle section of the fiber was selected as the torsion test area, and the effective sensing length was set to 0.5 m. One end of the test section was fixed to the fiber clamp 4, and the other end was fixed to the high-precision fiber rotator 6. Both ends of the test section were kept horizontally fixed and naturally straight. A clockwise rotation of the other end relative to the fixed end of the fiber clamp 4 is defined as positive torsion, and a counterclockwise rotation is defined as negative torsion. An alternating cyclic acquisition strategy was employed, with each round of experiments including complete measurements in both clockwise and counterclockwise directions. Before each round of measurements, the optical fiber was initialized to a naturally straight and untwisted state, and 100 speckle images were continuously acquired in this initial state as reference images for that round of experiments. Subsequently, the fiber rotator 6 was adjusted to apply twisting to the test section of the fiber in both clockwise (CW) and counterclockwise (CCW) directions, with a twist range from 0° to 90° and an acquisition step size set at 10°. At each angle node, the speckle light field at the fiber output end was collimated by the collimating lens 7 and imaged onto the CCD camera 8 (resolution 1368×918 pixels) for image acquisition. 100 speckle images were acquired for each angle, and then the acquired images were transmitted to a computer via a USB 3.0 interface for image processing and recognition.

[0028] Test results are as follows Figure 2 As shown in (a), the ZNCC coefficient decreases significantly in the early stages of rotation and rapidly enters the decorrelation saturation region within a small angular range, indicating that the output speckle is highly sensitive to small torsional disturbances. Figure 2 As shown in (b), in repeated tests, the ZNCC index of speckle patterns with the same rotation direction and angle remained at an extremely high level of around 0.95, verifying the excellent physical stability and signal reproduction capability of the sensing system. However, compared with speckle patterns with the same rotation angle but opposite directions, even if the amplitude of the external torsion angle is exactly the same, the correlation coefficient between the output speckle textures is significantly reduced to around 0.55-0.60. This confirms from a physical mechanism perspective that the clockwise and counterclockwise stress fields induce fundamentally different mode coupling paths inside the optical fiber, forming chiral characteristics with unique directions.

[0029] Example 2

[0030] Based on the mechanism verified in Embodiment 1 above, the present invention constructs as follows: Figure 3The system shown is a forearm rotation detection sensor system based on a novel chiral multimode fiber speckle pattern. The system consists of a helium-neon laser 1, a microscope objective 2, an optical fiber pigtail 3, a multimode fiber 5, a collimating lens 7, and a CCD camera 8, connected sequentially. The helium-neon laser 1 uses a 632.8 nm wavelength laser source to provide a highly coherent light field. The microscope objective 2 is positioned at the output end of the light source and, together with the optical fiber pigtail 3, couples the laser light into the input end of the multimode fiber 5. The multimode fiber 5, as the core sensing element, is a step-index fiber with a core diameter of 62.5 μm and a cladding diameter of 125 μm, with a total length of approximately 2.0 meters. A section of approximately 0.5 meters is selected as the effective sensing area and fixed to the human forearm 9. The speckle light field emitted from the output end of the multimode fiber 5 is collimated by the collimating lens 7 and projected onto the photosensitive surface of the CCD camera 8. To capture the chiral characteristics of the human forearm 9 during rotation, the multimode fiber 5 is arranged across the joint on the human forearm 9. Specifically, the proximal end of the multimode fiber 5 is fixed to the posterior aspect of the upper arm, approximately 10 cm above the lateral epicondyle of the humerus, as a reference fixation end; the middle section is fixed to the midline of the dorsal aspect of the forearm, approximately 5 cm below the olecranon of the ulna; and the distal end is fixed near the radial styloid process on the dorsal aspect of the wrist. The fiber is laid naturally along the dorsal aspect of the arm, with an appropriate curvature reserved at the elbow joint to accommodate a 90-degree flexion posture. Based on the above system, the detection procedure is as follows: First, data acquisition is performed. The subject stands naturally with the upper arm close to the torso, the elbow flexed at 90 degrees and placed in a neutral position. Pronation (0-90 degrees) and supination (0--90 degrees) of the forearm 9 are performed, with a step length set to 10 degrees. After stabilization at each angle node, the CCD camera 8 continuously acquires 100 frames of speckle images. Subsequently, the acquired raw images are uniformly adjusted to 224×224 pixel RGB format and input into the data processing unit. The data processing unit deploys a multi-task convolutional neural network based on the ResNet-50 architecture. This network contains two independent output heads, outputting 10 angular amplitude categories and 3 motion direction categories, respectively. By mining high-dimensional features in the speckle pattern, it achieves synchronous demodulation of the forearm motion state. This invention improves the standard ResNet-50 for multi-task speckle chirality recognition: a shared fully connected fusion module is introduced at the backbone network output, compressing the 2048-dimensional features to 512 and 256 dimensions respectively, before branching into two parameter-independent classification heads; the angular amplitude classification head outputs 10 angular categories from 0° to 90°, and the motion direction classification head outputs 3 direction categories: pronation, supination, and neutral position; the two heads are combined and mapped to finally output 19 signed angles from -90° to 90°. These improvements enable the model to complete the decoupled recognition of rotation amplitude and direction in a single inference, overcoming the defect of directly using standard single-task classification networks that cannot synchronously output vector information. This is the core innovation of this invention at the algorithm level.

[0031] The actual performance test results of the system are as follows: Figure 4 As shown, the model converges quickly on both angle and orientation classification tasks. Figure 4 The confusion matrix shown further demonstrates that, across the entire range from -90 degrees to 90 degrees, the present invention achieves an accuracy rate exceeding 99% for recognizing 10 angle categories each for pronation and supination of the human forearm. This indicates that the present invention, utilizing the chiral sensitivity of multimode optical fiber, effectively solves the directional ambiguity problem inherent in traditional sensors, achieving high-precision, vectorized motion capture.

[0032] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A forearm rotation detection sensor system based on chiral multimode fiber speckle, characterized in that, The system includes a helium-neon laser (1), a microscope objective (2), an optical fiber pigtail (3), a multimode optical fiber (5), a collimating lens (7), a CCD camera (8), and a data processing unit. The helium-neon laser (1) is used to generate a highly coherent laser beam. The microscope objective (2) is located at the output end of the helium-neon laser (1) and, together with the optical fiber pigtail (3), couples the laser beam into the input end of the multimode optical fiber (5). The middle section of the multimode optical fiber (5) serves as the effective sensing area and is configured to be fixed across the joint to the forearm (9) of the human body to be tested, for use in sensing the forearm (9). When rotational motion occurs, torsional deformation is generated to change the internal refractive index distribution and output a speckle light field with chiral characteristics; the collimating lens (7) is set at the output end of the multimode fiber (5) to collimate the emitted speckle light field; the photosensitive surface of the CCD camera (8) faces the collimating lens (7) to continuously acquire the collimated speckle image; the data processing unit is communicatively connected to the CCD camera (8) to extract features and classify the acquired speckle image, and synchronously demodulate and output the angle amplitude and direction of motion of the human forearm (9).

2. The forearm rotation detection sensor system based on chiral multimode fiber speckle according to claim 1, characterized in that, The multimode fiber (5) is a step-index multimode fiber with a core diameter of 62.5 μm and a cladding diameter of 125 μm.

3. The forearm rotation detection sensor system based on chiral multimode fiber speckle according to claim 2, characterized in that, The multimode fiber (5) is fixed across the joint on the human forearm (9) as follows: the proximal end of the multimode fiber (5) is fixed to the posterior side of the upper arm about 10 cm above the lateral epicondyle of the humerus as a reference fixing end for torsion, the middle section is fixed to the midline of the dorsal side of the forearm about 5 cm below the olecranon of the ulna, and the distal end is fixed near the radial styloid process on the dorsal side of the wrist. The multimode fiber (5) is reserved at the elbow joint to accommodate a 90° flexion posture.

4. The forearm rotation detection sensor system based on chiral multimode fiber speckle according to claim 3, characterized in that, The data processing unit is equipped with a multi-task classification model based on a deep residual network architecture. This model includes a shared backbone feature extraction network, an angle amplitude classification head, and a motion direction classification head. The angle amplitude classification head outputs 10 angle categories, corresponding to 0° to 90°. The motion direction classification head outputs three categories: pronation, supination, and neutral. Through the combined mapping of the angle amplitude classification head and the motion direction classification head, the signed angle recognition result in the range of -90° to 90° is finally output.

5. The forearm rotation detection sensor system based on chiral multimode fiber speckle according to claim 4, characterized in that, The output of the shared backbone feature extraction network is introduced with a shared fully connected fusion module; the speckle image acquired by the CCD camera (8) is uniformly adjusted to RGB format of 224×224 pixels and then input into the multi-task classification model.

6. The forearm rotation detection sensor system based on chiral multimode fiber speckle according to claim 5, characterized in that, A fiber optic clamp (4) is provided next to the fiber optic clamp to fix one end of the fiber optic test section.

7. The forearm rotation detection sensor system based on chiral multimode fiber speckle according to claim 6, characterized in that, A fiber optic rotator (6) is provided next to the fiber optic clamp to fix the other end of the fiber optic test section.

8. A method for detecting the human forearm based on the forearm rotation detection sensor system according to any one of claims 1 to 7, characterized in that, Includes the following steps: S1. Fix the effective sensing area of ​​the multimode fiber (5) across the joint to the subject's forearm (9); the subject stands naturally with the upper arm close to the torso, the elbow flexed at 90 degrees and placed in a neutral position. S2. Turn on the helium-neon laser (1), and the laser beam is coupled into the multimode fiber (5) through the microscope objective (2) and the fiber optic pigtail (3) in sequence. S3. The subject performs forearm rotation. The CCD camera (8) acquires speckle images in real time at different angle nodes. The multimode fiber (5) outputs the images and collimates them through the collimating lens (7). S4. The data processing unit receives the speckle image, inputs it into the built-in multi-task classification model, and synchronously drives the angle amplitude classification head and the motion direction classification head to perform feature extraction and classification calculation through the shared backbone network. S5. Synchronously output the angle amplitude and motion direction to complete the vectorized motion capture of the rotation state of the human forearm (9).

Images