Wearable optical sensor glove, system, gesture recognition method and article recognition method

Abstract

The invention discloses a wearable optical sensor glove, a system, a gesture recognition method and an article recognition method. Comprising a customized flexible glove body, a flexible circuit board, a photoelectric detector, a flexible force-induced luminescence optical fiber, a multipath gating device and a single-chip microcomputer control module. Wherein the customized flexible glove body is made of a flexible material; the flexible circuit boards are arranged on finger parts and a hand back part of the glove; the photoelectric detector is mounted in the finger region and is electrically connected with the multi-path gating device; the flexible force-induced luminescence optical fiber is fixed on the photoelectric detector in each finger region; the multi-channel gating device and the single-chip microcomputer control module are also integrated on the flexible circuit board on the back of the hand and are electrically communicated with each other. The system has good anti-interference performance, and can realize the functions of high-speed dynamic detection, article identification and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of human-computer interaction gesture recognition, and more specifically, relates to a wearable optical sensor glove, system, gesture recognition method and item recognition method. Background Art

[0002] Human-computer interaction technology refers to the technology of communication between humans and computers or intelligent machines through language or actions, and is an important part of artificial intelligence and the Internet of Things. In recent years, with the continuous progress of artificial intelligence technology, the field of human-computer interaction has achieved rapid development. Among them, gesture recognition, as one of the most effective and widely adopted human-computer interaction technologies, can be classified as vision-based technology or sensor-based technology. Due to the maturity of camera imaging technology and imaging processing algorithms, the former has rapidly developed into the mainstream in the field of human-computer interaction. Although its accuracy is easily affected by ambient light. Its application prospects are limited by natural defects. In contrast, sensor-based technology has stable data acquisition and simple signal processing. Electronic sensors can also be easily integrated with signal processing electronic components for data analysis. However, devices based on electronic equipment are vulnerable to electromagnetic interference and chemical corrosion.

[0003] On the other hand, optical sensors have received great attention in wearable strain sensing and gesture recognition due to their versatility and electromagnetic interference resistance. Given the significant advantages of optical wearable sensors in human-computer interaction, there is an increasing demand for portable and wearable systems that do not require external laser sources and spectrometers. Therefore, self-powered wearable sensors that can directly convert mechanical stimuli into sensing signals have become a promising solution. It is worth noting that mechanoluminescent (ML) materials have received great attention because they can directly utilize the piezoelectric effect to convert external stress signals directly into optical signals for sensing. These self-powered optical wearable sensors have the advantages of portability and low power consumption, making them particularly suitable for human-computer interaction applications.

[0004] The invention patent with the publication number CN118732851A in the prior art proposes an intelligent glove and a human-computer interaction device with gesture capture and tactile feedback functions. The method includes: a glove body, a sensing module, and a feedback driving module. The sensing module is arranged on the back of the glove body; the sensing module includes a first encapsulation layer, a second encapsulation layer, and a liquid metal electrode; the liquid metal electrode is arranged in a multi-loop bent shape between the first encapsulation layer and the second encapsulation layer; the feedback driving module is arranged on the palm of the glove body; the feedback driving module includes a first substrate, a second substrate, and a piezoelectric composite layer; the piezoelectric composite layer is between the first substrate and the second substrate, and there are flexible electrodes between the piezoelectric composite layer and the first substrate and between the second substrate respectively. The two flexible electrodes are respectively connected to an alternating voltage. This solution realizes gesture capture and force feedback through a flexible resistive sensing + tactile feedback module, but it has deficiencies in anti-interference, high-speed dynamic detection, machine learning algorithm support, etc., and more stays in the mode of "bending resistance change → uploading to the host computer → driving piezoelectric vibration". Summary of the Invention

[0005] In order to overcome the deficiencies of the gesture recognition solution in the prior art in terms of anti-interference, high-speed dynamic detection, machine learning algorithm support, etc., the present invention provides a wearable optical sensor glove, a system, a gesture recognition method, and an item recognition method.

[0006] The primary object of the present invention is to solve the above technical problems, and the technical solution of the present invention is as follows:

[0007] The first aspect of the present invention provides a wearable optical sensor glove, including: a customized flexible glove body, a flexible circuit board, a photodetector, a flexible force-induced luminescent optical fiber, a multiplexer, and a single-chip microcomputer control module; wherein, the customized flexible glove body is made of a flexible material; the flexible circuit board is integrated in the finger and back regions of the customized flexible glove body; the photodetector is integrated on the flexible circuit board in the finger region and is electrically connected to the multiplexer; the flexible force-induced luminescent optical fiber is fixed on the photodetectors in each finger region; the multiplexer is integrated on the flexible circuit board in the back region of the hand and is electrically connected to the photodetector and the single-chip microcomputer control module; the single-chip microcomputer control module is integrated on the flexible circuit board in the back region of the hand and is electrically connected to the multiplexer.

[0008] Further, the luminescent material of the flexible force-induced luminescent optical fiber is zinc sulfide doped with copper or manganese elements and is embedded in an elastic polymer matrix to generate visible light in the green or orange wavelength band when being stretched by an external force.

[0009] Further, the base material of the flexible circuit board is polyimide, and routing areas of several sensing channels are arranged at the five fingers respectively.

[0010] Further, the photodetector is a VEML6035 detector; the multiplexer is a PCA9548A multiplexer.

[0011] Further, the single-chip microcomputer control module is an Arduino single-chip microcomputer control module. By presetting the acquisition rate and data format, combined with pull-up, pull-down resistors and bypass capacitors, it can obtain the light intensity data of five finger channels in real time.

[0012] Further, the Arduino single-chip microcomputer control module also includes a wireless communication module, which sends the data to the host computer through the wireless communication module.

[0013] Further, the flexible force-induced luminescent optical fiber is fixed on the photodetector by a polymer adhesive.

[0014] The second aspect of the present invention provides a wearable optical sensor glove system, which uses a wearable optical sensor glove and a host computer, and specifically includes:

[0015] Information acquisition module: It includes a customized flexible glove body, a photodetector and a flexible force-induced luminescent optical fiber, which is used to convert mechanical strain into optical signals when the flexible force-induced luminescent optical fiber on the finger bends or stretches, and the original gesture data is captured by the photodetector;

[0016] Information transfer module: It includes a flexible circuit board, a multiplexer and a single-chip microcomputer control module, which is used to summarize the multi-channel signals from n information acquisition modules through the multiplexer and uniformly input them into the single-chip microcomputer control module. The single-chip microcomputer control module packs the received multi-channel electrical signals and sends them to the information processing module through the wireless communication module;

[0017] Information processing module: It is used to run machine learning algorithms, perform feature analysis, classification recognition and visual display on the received multi-channel electrical signals from the information transfer module, and output corresponding action recognition results or grasping judgment information.

[0018] The third aspect of the present invention provides a gesture recognition method, which uses a wearable optical sensor glove for gesture recognition, and includes the following steps:

[0019] When the user's finger bends or stretches, the flexible force-induced luminescent optical fiber is subjected to mechanical strain and generates corresponding optical signals;

[0020] The photodetector is used to detect the optical signal and output corresponding electrical signals to the multiplexer;

[0021] The multiplexer uniformly inputs the electrical signals from multiple photodetectors into the single-chip microcomputer control module;

[0022] The single-chip microcomputer control module packs the received multi-channel electrical signals and sends them to the host computer through the wireless communication module;

[0023] The host computer preprocesses the received multi-channel electrical signals and inputs them into a pre-trained neural network for gesture recognition, outputs the recognition results and visualizes them on the monitoring terminal.

[0024] Further, the neural network is a feedforward neural network, and the training method of the neural network includes the following steps:

[0025] Collect various gesture data and balance various gesture data samples by the truncation method;

[0026] Perform standardization processing on the various gesture data samples to obtain a data set that conforms to the supervised learning format;

[0027] Divide the data set into a training set and a test set by the holdout method;

[0028] Use the training set to iteratively train the model in combination with the cross-entropy loss function and L2 regularization; when the loss function converges to a preset threshold, use the test set to evaluate the recognition accuracy and generalization performance of the model to obtain the finally trained model.

[0029] The fourth aspect of the present invention provides an object recognition method based on a grasping action, which uses a wearable optical sensor glove for object recognition, including the following steps:

[0030] Sample multi-channel optical signals generated by flexible force-induced luminescent optical fibers, and record the change amplitude of the optical signals and their time series;

[0031] Input the time series and the optical signal change amplitude data into the neural network model, classify the gesture switching speed according to the change amplitude and gesture switching speed of the sensing signal during the grasping process, and determine the bending mode and posture of the user's gesture;

[0032] According to the bending mode and posture of the gesture, judge the shape and size of the target object, and output the object recognition result.

[0033] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0034] By combining self-powered optical sensing with neural network algorithms, the present invention can effectively combat electromagnetic interference and achieve comprehensive recognition of multiple gestures, speeds, and object grasping without the need for an additional laser light source, improving the flexibility and accuracy of the human-computer interaction system. Relying on the multi-channel real-time monitoring and the efficient classification ability of the machine learning model, this glove demonstrates good adaptability and scalability in the fields of wearable devices and human-computer interaction, and can be applied to various occasions such as medical rehabilitation, robot collaboration, VR / AR scenarios, etc., significantly expanding the application boundaries of gesture recognition technology and enhancing the actual usage experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] To make the objectives and technical solutions of the present invention clearer, the present invention provides the following drawings and descriptions:

[0036] Figure 1 Schematic diagram of the glove structure provided by an embodiment of the present invention;

[0037] Figure 2 Schematic diagram of the glove system structure provided by an embodiment of the present invention;

[0038] Figure 3 Circuit diagram for implementing the signal acquisition function provided by an embodiment of the present invention;

[0039] Figure 4 Corresponding effect diagram of the gesture sensor provided by an embodiment of the present invention;

[0040] Figure 5 Corresponding relationship diagram between the training scale and accuracy of the gesture recognition function provided by an embodiment of the present invention;

[0041] Figure 6 Accuracy result diagram for recognizing six gestures provided by an embodiment of the present invention;

[0042] Figure 7 Online detection test result diagram provided by an embodiment of the present invention;

[0043] Figure 8 Sensor signal intensity diagram for three different gesture movement speeds provided by an embodiment of the present invention;

[0044] Figure 9 Corresponding relationship diagram between the training scale and accuracy of the gesture movement speed recognition function provided by an embodiment of the present invention;

[0045] Figure 10 Accuracy result diagram for recognizing three different gesture movement speeds provided by an embodiment of the present invention;

[0046] Figure 11 Item recognition experiment result diagram provided by an embodiment of the present invention;

[0047] Figure 12 This is the experimental result diagram of object size recognition provided by the embodiment of the present invention.

[0048] Among them, 1-customized flexible glove body; 2-flexible circuit board; 3-photodetector; 4-flexible force-induced luminescent optical fiber; 5-multiplexer; 6-single-chip microcomputer control module. Specific embodiments

[0049] In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0050] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0051] Embodiment 1:

[0052] The present invention provides a wearable optical sensor glove, as Figure 1 shown in the schematic structural diagram of a wearable optical sensor glove, which specifically includes: a customized flexible glove body 1, a flexible circuit board 2, a photodetector 3, a flexible force-induced luminescent optical fiber 4, a multiplexer 5, and a single-chip microcomputer control module 6; among them, the customized flexible glove body 1 is made of flexible material, adapted to the human hand and available for five fingers to wear; the flexible circuit board 2 is integrated in the finger and back-of-hand areas of the customized flexible glove body 1 for laying out traces and mounting relevant electronic components; the photodetector 3 is integrated on the flexible circuit board 2 in the finger area and is electrically connected to the multiplexer 5 for receiving the optical signal generated by the force-induced luminescent optical fiber under mechanical strain and outputting a corresponding electrical signal; the flexible force-induced luminescent optical fiber 4 is fixed on the photodetector 3 in each finger area and generates fluorescence when subjected to external force stretching and is captured by the photodetector 3; the multiplexer 5 is integrated on the flexible circuit board 2 in the back-of-hand area and is electrically connected to the photodetector 3 and the single-chip microcomputer control module 6 for uniformly transmitting the electrical signals of multiple sensing channels under the I 2 C interface; the single-chip microcomputer control module 6 is integrated on the flexible circuit board 2 in the back-of-hand area and is electrically connected to the multiplexer 5 for data acquisition, storage and preliminary processing of the acquired electrical signals.

[0053] More specifically, the luminescent material of the flexible force-induced luminescent optical fiber 4 is zinc sulfide (ZnS) doped with copper or manganese elements and is embedded in an elastic polymer matrix to generate visible light in the green or orange wavelength band when subjected to external force stretching.

[0054] More specifically, the substrate of the flexible circuit board 2 is a polyimide (PI) polymer film material that can be bent to 360°, and the routing areas of several sensing channels are arranged at five fingers respectively.

[0055] More specifically, the photodetector 3 is a surface-mounted VEML6035 detector to adapt to the emission peak of the force-induced luminescence optical fiber 4; the multiplexer 5 is a PCA9548A multiplexer.

[0056] More specifically, the single-chip microcomputer control module 6 is an Arduino single-chip microcomputer control module, which acquires the light intensity data of five finger channels in real time in standard I 2 C timing by presetting the acquisition rate and data format, and combining pull-up, pull-down resistors and bypass capacitors.

[0057] More specifically, the Arduino single-chip microcomputer control module also includes a wireless communication module (Bluetooth module), and sends the data to the host computer through the wireless communication module (Bluetooth module). The Arduino control module used in this embodiment adopts a standard I 2 C interface circuit to complete data transceiver and signal processing tasks. The Arduino single-chip microcomputer control module has programmability, and can collect data rate, accuracy, data display format, store relevant data, and other various independent modules. What is used in this embodiment is a driving Bluetooth wireless communication module.

[0058] More specifically, the flexible force-induced luminescence optical fiber 4 is fixed on the photodetector 3 by a polymer adhesive (such as electric melting glue).

[0059] As Figure 3 shown is the circuit diagram corresponding to the signal acquisition function of the present invention. In this design, 5 surface-mounted VEML6035 detectors are used, and their specific ports and functions are as follows: the VCC port is connected to VDD to provide positive power voltage for the device; the GND port is used as the ground wire to ensure the integrity of the current loop. The SDA and SCL ports respectively correspond to I 2Two data buses in the C communication protocol are used for data transmission. In addition, the INT port is used to output an interrupt signal. In addition, bypass capacitors are designed in each VEML6035-related circuit to stabilize the power supply voltage, while other resistors act as pull-up resistors to achieve complete circuit functions. Additionally, in order to transmit 5 groups of independent data simultaneously, the design uses a PCA9548A multiplexer to facilitate the real-time transmission of the data collected by 5 groups of VEML6035 to the Arduino controller. In addition, bypass capacitors, pull-up and pull-down resistors are also designed in the PCA9548A-related circuit to achieve complete circuit functions. Additionally, this design specifically adds 5 rows of ports, thereby increasing the robustness of the system. Even if one row of ports is damaged, there is no need to replace the entire circuit board. Only another row of ports can be used. The ports are shown in the lower part of the circuit diagram.

[0060] In this embodiment, 5 strain sensors based on flexible force-induced luminescent polymer optical fibers are integrated onto a flexible printed circuit (FPC) using an I 2 C multiplexer to form a glove-type circuit system with 5 independent sensing channels. Each optical fiber contains zinc sulfide powder doped with copper elements, and when subjected to mechanical strain, it triggers a local piezoelectric effect through lattice deformation to release photon signals (luminescence can be achieved with elements such as copper and manganese, and is not limited to green or orange wavelengths). Since this mechanism does not require an external light source, the system only relies on a micro battery to provide low-power power supply for the FPC, and can directly convert strain into visible fluorescence signals when the user bends their fingers. After the photodetector (PD) captures the signal, it is input into the Arduino control module through the multiplexed I 2 C and then wirelessly transmitted to the host through a Bluetooth module. The feedforward neural network (FNN) algorithm at the host side normalizes the data and then can perform gesture recognition or other pattern analysis.

[0061] In this embodiment, the sensors embedded in the glove correspond to five fingers and are labeled as Sensor 1 to Sensor 5 in the illustration of Figure 4 (a). During testing, first verify whether there is interference in the five channels. The results show that each channel is independent; then the user wears the glove and makes movements according to the Figure 4 predefined gestures shown (such as open, partially bent, large bend of the index finger, etc.). The stress luminescence intensities of the five sensing channels corresponding to each gesture are plotted in Figure 4 (a)-(f). By comparing the stress luminescence intensity distributions of each finger, different gesture luminescence patterns can be clearly distinguished. For example, for Gesture 1 of an open hand, where all five fingers remain in the initial unbent position, the stress luminescence intensities of all sensors remain at zero. When the fingers are bent to different angles to form various gestures, different stress luminescence patterns are observed, such asFigure 4 (b)-(f). For example, in gesture 2, sensors 2 and 3 are bent to a large angle, while the other three sensors remain in their initial state. Therefore, the stress luminescence intensities of sensors 2 and 3 are around 0.6 and 1, respectively, while the remaining three values ​​remain zero. In gesture 3, sensors 1 to 3 are bent, resulting in non-zero values ​​for these three sensors. In gesture 4, the large bend of the index finger results in the highest stress luminescence signal in sensor 4. In gesture 6, both fingers remain in their initial state, resulting in zero values ​​for sensors 3 and 4. Different stress luminescence intensity patterns are observed in different gestures, which is consistent with the strain test results. To facilitate subsequent machine learning training, these multi-channel signals will be normalized and passed into the FNN model, thereby achieving accurate recognition and real-time monitoring of complex multi-gestures.

[0062] Embodiment 2:

[0063] This embodiment provides a wearable optical sensor glove system, such as Figure 2 The figure shows a schematic diagram of the structure of a wearable optical sensor glove system. The system adopts the wearable optical sensor glove and a host computer described in Example 1, including:

[0064] Information collection module: including a customized flexible glove body 1, a photodetector 3 and a flexible mechanoluminescent optical fiber 4, which is used to convert mechanical strain into an optical signal when the finger is bent or stretched, and the original gesture data is captured by the photodetector 3;

[0065] Information transfer module: includes a flexible circuit board 2, a multiplexer 5 and a single-chip control module 6, which is used to aggregate the multi-channel signals from n information acquisition modules through the multiplexer 5 and uniformly input them into the single-chip control module 6. The single-chip control module 6 packages the received multi-channel electrical signals and sends them to the information processing module via the wireless communication module. In this embodiment, the value of n is 5;

[0066] Information processing module: used to run machine learning algorithms, perform feature analysis, classification recognition and visualization on the multi-channel electrical signals received from the information transfer module, and output corresponding action recognition results or grasping judgment information.

[0067] The research on signal capture and analysis of multi-joint coordinated motion of the hand using a wearable optical sensor glove provided by the present invention includes but is not limited to:

[0068] Gesture recognition function: single finger flexion and extension, combination of multiple knuckles (such as close together, open, cross, etc.) and dynamic feature extraction of typical gestures (such as digital expression gestures 1-5, gaming gestures rock-paper-scissors, general gestures, etc.);

[0069] Action speed analysis function: It can sensitively detect the gesture conversion speed and quantitatively record the time parameters of finger movements from start to completion.

[0070] Item grasping and recognition function: Such as some common item features: curved objects (such as bananas, apples), cylindrical objects (such as bottles with a diameter of 2.5 - 5 cm), flat objects (such as books), regular cubes (such as packaging boxes), etc.

[0071] Object size determination function: Automatically estimate the object size according to the grasping posture, and achieve the classification recognition of the cylinder size with a grasping span (detection range 2 - 5 cm).

[0072] Embodiment 3:

[0073] This embodiment provides a gesture recognition method. The method uses a wearable optical sensor glove described in Embodiment 1 for gesture recognition, including the following steps:

[0074] When the user's finger bends or extends, the flexible force - induced luminescent optical fiber 4 is mechanically strained and generates corresponding optical signals.

[0075] Use the photodetector 3 to detect the optical signals and output corresponding electrical signals to the multiplexer 5.

[0076] The multiplexer 5 uniformly inputs the electrical signals from multiple photodetectors 3 into the single - chip microcomputer control module 6.

[0077] The single - chip microcomputer control module 6 packs the received multi - channel electrical signals and sends them to the host computer through the wireless communication module.

[0078] The host computer pre - processes the received multi - channel electrical signals and inputs them into a pre - trained neural network for gesture recognition, outputs the recognition results, and visualizes them on the monitoring terminal.

[0079] More specifically, the neural network is a feed - forward neural network, and the training method of the neural network includes the following steps:

[0080] Collect various gesture data and use the truncation method to balance the gesture data samples of various types.

[0081] Perform standardization processing on the gesture data samples of various types to obtain a data set that conforms to the format of supervised learning.

[0082] Use the hold - out method to divide the data set into a training set and a test set.

[0083] Use the training set to iteratively train the model in combination with the cross - entropy loss function and L2 regularization; when the loss function converges to a preset threshold, use the test set to evaluate the recognition accuracy and generalization performance of the model to obtain the finally trained model.

[0084] To implement a truly intelligent human-machine interaction system, the host computer uses a feedforward neural network (FNN) machine learning algorithm to train the designed sensing gloves. The FNN algorithm is written in the Python language and includes an input layer, a configurable hidden layer, and an output layer. Its simple and effective architecture offers several advantages, including flexible configuration, fast training speed, and low resource consumption, making it very suitable for classification tasks. To achieve the network's classification ability, we use Softmax as the activation function for the output layer. During the training process, cross-entropy is used as the loss function to evaluate the effectiveness of the model. The equation for binary cross-entropy is defined as follows:

[0085]

[0086] where y is the actual label (in binary classification, it equals 0 or 1), is the predicted probability that the model assigns to the sample belonging to the "positive class (y = 1)", and the value range is usually [0,1]. The cross-entropy loss will give a large penalty when the predicted probability is inconsistent with the true label. By minimizing this loss, the model can learn more accurate probability estimates.

[0087] For multi-class tasks, the cross-entropy equation is written as:

[0088]

[0089] where i is the class index, C is the total number of classes, y i is the one-hot vector of the true label, is the predicted probability for the corresponding class. Intuitively, when the predicted probability of the model for the true class is larger ( close to 1), this loss will be smaller; if the predicted probability of the model for the true class is very low ( far less than 1), then the logarithmic value will be a large negative number, resulting in a significant increase in the loss, thus guiding the model to continuously correct the parameters during training to improve the predicted probability for the correct class.

[0090] To train and test the FNN, the collected data is randomly divided into 70% for model training and 30% for testing. After normalizing the initial input data and processing it using the Leaky Relu activation function, the accuracy of the model can be easily increased to over 90%. The accuracies of gesture recognition and grasping actions reach 98% and 94% respectively. To further improve the performance of the model, L2 regularization is defined as a penalty term in the loss function to reduce overfitting and improve the accuracy. In this way, the accuracies of gesture recognition and grasping recognition are further increased to 99% and 95% respectively.

[0091] Subsequently, the experimental results are input into the model for training. The relationship between the recognition accuracy and the training scale is evaluated, as Figure 5 shown. When the training scale reaches approximately 50%, the accuracy rate reaches 99%. After 100% of the training iterations, the accuracy rate is further increased to a high level of 99.2%.

[0092] Furthermore, the six predefined gestures shown in Figure 4 are tested using the present invention, and the trained model is applied to demonstrate a human-computer interaction application program. Volunteers are required to present the six predefined gestures, which are then recognized by the trained model. The experimental results are as shown in Figure 6 where the vertical axis and the horizontal axis respectively correspond to the actual and predicted gestures. According to the results presented in the figure, the system has successfully achieved a perfect gesture recognition accuracy rate of 100%. However, before using the L2 regularization optimization algorithm, minor errors were observed in gestures 2 and 5, where approximately 3% of the cases were misclassified as gesture 1 and gesture 6 respectively. As shown in Figure 7 the effectiveness of the present invention is further demonstrated through online detection, where the experimental gestures and the corresponding predicted gestures are presented in the left and right columns respectively.

[0093] Another key application provided by the present invention is motion speed detection, which poses a major challenge to traditional wearable technologies. For example, the transition from gesture 1 to gesture 2 is tested at different speeds. As shown in Figure 8 when the gesture makes 10 circles in 30 seconds (speed 1), 10 seconds (speed 2), and 4 seconds (speed 3) respectively, the sensor signal changes significantly. In the state of speed 1, due to the slow gesture movement, all sensor signals remain below 0.2. When the gesture speed increases to speed 2, the signal peak exceeds 0.6, which is approximately 3 times that at speed 1. Further enhancement of the gesture speed will bring higher signal intensity. Therefore, the sensing glove can effectively recognize the gesture actions of human-computer interaction. By completing the same gesture transition actions at different speeds, the change in the amplitude of the sensor signal can be clearly observed, thereby achieving the detection of the gesture motion speed.

[0094] Training a machine learning model for the detection of gesture motion speed, as shown in Figure 9 when the training scale increases from 10% to approximately 100%, the accuracy rate of the model increases from 97% to 99%, indicating that a larger training data set improves the recognition accuracy. As shown in Figure 10As shown, when applied to gesture action recognition, the model achieved 100% perfect accuracy at gesture speeds 1 and 2. For gesture speed 3, only 3% of the actions were misclassified as gesture speed 2. Since gesture speed 3 is approximately 0.4 seconds per revolution, which is too fast for normal gesture movements, it induced small errors in recognition. Nevertheless, the system performed exceptionally well in normal and slow gesture movements, providing a reliable and effective method for gesture and action recognition.

[0095] Example 4:

[0096] This embodiment provides an object recognition method based on grasping actions. The method uses a wearable optical sensor glove described in Example 1 for object recognition, including the following steps:

[0097] Sample multi-channel optical signals generated by the flexible force-induced luminescent optical fiber 4, and record the change amplitude of the optical signals and their time series;

[0098] Input the time series and the optical signal change amplitude data into a neural network model, classify the gesture switching speed according to the change amplitude and gesture switching speed of the sensing signal during the grasping process, and determine the bending mode and posture of the user's gesture;

[0099] Based on the bending mode and posture of the gesture, judge the shape and size of the target object, and output the object recognition result.

[0100] Another application of the glove is grasping action recognition, which can be used to recognize different objects. In this work, the sensing glove was applied to object detection. As Figure 11 (a) shows, the left and right columns are the detected signals and the corresponding grasping actions respectively. The glove successfully recognized five different objects: a banana, a bottle, a book, an apple, and a rectangular box. The obvious differences in the detected signals of the fiber optic sensors made it possible to effectively recognize different objects. For example, although both the banana and the bottle have a similar cylindrical shape, the curved front part of the banana is in sharp contrast to the completely straight structure of the bottle. Therefore, the signal intensities of sensors 3 and 4 for the banana are much higher than those for the bottle. In the case of the apple and the box, high signal intensities appeared in the apple. Especially during the grasping action of the box, since the thumb does not need to bend, the signal of sensor 5 is zero. Therefore, the signal pattern is determined by the shape of the grasped object. As Figure 11 (b) shows, the recognition accuracy increases with the increase in the number of training samples. When the training scale is about 100%, the recognition accuracy reaches 95%. Subsequently, the trained model was verified by testing its ability to recognize 5 objects. The recognition rates of all five objects exceeded 90%. As Figure 11 (c) shows, two of the objects were successfully recognized with an accuracy of 100%.

[0101] Grasping action recognition can also be applied to determine the size of an object. For example, the test results of cylinders with diameters of 5 cm, 3 cm, and 2.5 cm are as Figure 12 (a - b) shown. It can be observed that during the grasping action, the smaller cylinder results in a higher signal intensity. This is because a larger bending angle is required to grasp a smaller object. As Figure 12 (c) shown, after machine learning training, the detection accuracy of the system reaches about 95%, and it can accurately identify three objects of different sizes. The recognition accuracy is as Figure 12 (d) shown.

[0102] By combining self - powered optical sensing and neural network algorithms, the present invention can effectively combat electromagnetic interference and, without the need for an additional laser light source, achieve comprehensive recognition of multiple gestures, speeds, and object grasping, improving the flexibility and accuracy of the human - machine interaction system. Relying on the multi - channel real - time monitoring and the efficient classification ability of the machine learning model, this glove demonstrates good adaptability and scalability in the fields of wearable devices and human - machine interaction, and can be applied to various occasions such as medical rehabilitation, robot collaboration, VR / AR scenarios, etc., significantly expanding the application boundary of gesture recognition technology and enhancing the actual usage experience.

[0103] Obviously, the above - mentioned embodiments of the present invention are only examples for clearly illustrating the present invention, rather than limitations on the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation manners here. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the claims of the present invention.

Claims

1. A wearable optical sensor glove, characterized in that: include: A customized flexible glove body (1), a flexible circuit board (2), a photoelectric detector (3), a flexible mesoluminescent optical fiber (4), a multiplexer (5), and a single-chip control module (6); wherein the customized flexible glove body (1) is made of a flexible material; the flexible circuit board (2) is integrated in the finger and back of hand regions of the customized flexible glove body (1); the photoelectric detector (3) is integrated on the flexible circuit board (2) in the finger region and is electrically connected to the multiplexer (5); the flexible mesoluminescent optical fiber (4) is fixed on the photoelectric detectors (3) in each finger region; the multiplexer (5) is integrated on the flexible circuit board (2) in the back of hand region and is electrically connected to the photoelectric detector (3) and the single-chip control module (6); the single-chip control module (6) is integrated on the flexible circuit board (2) in the back of hand region and is electrically connected to the multiplexer (5).

2. A wearable optical sensor glove according to claim 1, characterized in that: The luminescent material of the flexible mechanoluminescent optical fiber (4) is zinc sulfide doped with copper or manganese and embedded in an elastic polymer matrix so as to generate visible light in the green or orange band when stretched by external force.

3. A wearable optical sensor glove according to claim 1, characterized in that: The base material of the flexible circuit board (2) is polyimide, and wiring areas of a plurality of sensing channels are arranged at the five fingers respectively.

4. The wearable optical sensor glove according to claim 1, characterized in that: The photoelectric detector (3) is a VEML6035 detector; the multiplexer (5) is a PCA9548A multiplexer.

5. The wearable optical sensor glove according to claim 1, characterized in that: The single-chip control module (6) is an Arduino single-chip control module, which obtains light intensity data of five finger channels in real time by presetting the acquisition rate and data format and combining pull-up and pull-down resistors and bypass capacitors.

6. A wearable optical sensor glove according to claim 5, characterized in that: The Arduino single-chip microcomputer control module also includes a wireless communication module, through which data is sent to a host computer.

7. A wearable optical sensor glove system, using a wearable optical sensor glove according to any one of claims 1 to 7, characterized in that: include: The information collection module comprises a customized flexible glove body (1), a photoelectric detector (3) and a flexible mechanoluminescent optical fiber (4), and is used to convert mechanical strain into an optical signal when the finger is bent or stretched, and the photoelectric detector (3) captures the original gesture data; The information transfer module comprises a flexible circuit board (2), a multiplexer (5) and a single-chip control module (6), and is used to aggregate the multi-channel signals from the n information acquisition modules through the multiplexer (5) and uniformly input the signals into the single-chip control module (6); the single-chip control module (6) packages the received multi-channel electrical signals and sends them to the information processing module via the wireless communication module; Information processing module: used to run machine learning algorithms, perform feature analysis, classification recognition and visualization on the multi-channel electrical signals received from the information transfer module, and output corresponding action recognition results or grasping judgment information.

8. A gesture recognition method, using a wearable optical sensor glove according to any one of claims 1 to 7, characterized in that: The steps include: When the user bends or stretches the finger, the flexible mechanoluminescent optical fiber (4) is subjected to mechanical strain and generates a corresponding light signal; Utilizing the photodetector (3) to detect the optical signal and outputting a corresponding electrical signal to a multiplexer (5); The multiplexer (5) uniformly inputs the electrical signals from the plurality of photodetectors (3) into the single chip control module (6); The single-chip control module (6) packages the received multi-channel electrical signals and sends them to the host computer via the wireless communication module; The host computer pre-processes the received multi-channel electrical signals and inputs them into a pre-trained neural network for gesture recognition, outputs the recognition results and displays them visually on the monitoring terminal.

9. A gesture recognition method according to claim 8, characterized in that: The neural network is a feedforward neural network, and the training method of the neural network comprises the following steps: Collect various gesture data and use the truncation method to balance the gesture data samples; Standardizing the various gesture data samples to obtain a data set that conforms to a supervised learning format; Using a holdout method to divide the data set into a training set and a test set; The model is iteratively trained using the training set combined with the cross entropy loss function and L2 regularization; when the loss function converges to the preset threshold, the recognition accuracy and generalization performance of the model are evaluated using the test set to obtain the final trained model.

10. A method for object recognition based on grasping action, using a wearable optical sensor glove according to any one of claims 1 to 7, characterized in that: The steps include: Sampling the multi-channel optical signal generated by the flexible mechanoluminescent optical fiber (4), and recording the variation amplitude of the optical signal and its time series; The time series and the light signal change amplitude data are input into a neural network model, and the gesture switching speed is classified according to the change amplitude of the sensor signal during the grasping process and the gesture switching speed, so as to determine the bending mode and posture of the user's gesture; According to the bending pattern and posture of the gesture, the shape and size of the target object are judged and the object recognition result is output.

Images