Massage manipulation measuring device and recognition method based on visual tactile sensor

Abstract

The invention relates to the technical field of tactile sensing and traditional Chinese medicine intelligent diagnosis, in particular to a massage manipulation measuring device based on a visual tactile sensor and an identification method.The device synchronously collects six-dimensional force / torque data and force application position information in the massage process through the visual tactile sensor, and the six-dimensional force / torque data and the force application position information are obtained after data preprocessing and feature extraction; and through combination of deep learning and a random forest algorithm, precise identification of a force application position and a massage manipulation is realized. Experimental verification shows that the force estimation error of the device is about 0.27 N, the torque estimation error is 4.3 N mm, the force application position recognition accuracy rate reaches 97%, the recognition accuracy rate of seven traditional Chinese medicine massage manipulations is 97.3%, the problems that the traditional Chinese medicine massage manipulations are difficult to quantify and low in recognition precision are solved, objective standardization of the manipulations is achieved, and the method is suitable for popularization and application. And technical support is provided for traditional Chinese medicine massage standardized teaching, clinical application and intelligent rehabilitation equipment development.

Description

Technical Field

[0001] This invention relates to the fields of tactile sensing and intelligent diagnosis in traditional Chinese medicine, specifically to a massage technique measurement device and recognition method based on a visual tactile sensor. Background Technology

[0002] Traditional Chinese massage (also known as Tuina), as an important part of traditional medicine, has significant therapeutic effects in improving neuromuscular function, relieving chronic pain, and enhancing the body's immune function, and is widely used in rehabilitation therapy and health care. A complete TCM massage procedure usually consists of a combination of various basic techniques such as kneading, pressing, rolling, vibrating, pushing, striking, and rubbing. However, traditional massage techniques mainly rely on the experience passed down by practitioners. Different doctors vary significantly in terms of the strength, frequency, and application of force, resulting in inconsistent treatment effects for the same ailment. This seriously restricts the standardized inheritance and modern promotion of TCM massage.

[0003] The core of solving the above problems lies in quantifying and accurately identifying massage techniques. While researchers have developed various measurement schemes in existing technologies—such as the single-axis force sensor designed by Chen X et al., which can only detect vertical pressure and the sequence of operations; and the triaxial force sensor developed by Liu G et al., which can identify the magnitude and direction of spatial force, but its measurement range is limited to 0-2N, failing to cover the actual force applied in traditional Chinese medicine massage; and the flexible six-axis force / torque sensor proposed by Liu L et al., which can distinguish some techniques, ignores the importance of the application location—when the same technique is applied to different acupoints, the difference in the application location (such as the thumb tip, fingertip, or joint) leads to different contact areas, resulting in drastically different biomechanical stimulation effects. Relying solely on multidimensional mechanical parameters is insufficient to comprehensively characterize the features of massage techniques. Furthermore, existing recognition algorithms largely rely on convolutional neural networks, requiring a large number of samples, resulting in insufficient generalization ability in small-sample clinical scenarios and difficulty in achieving real-time recognition.

[0004] In recent years, visual tactile sensors (such as GelSight and TacTip) have provided a new direction for the quantification of massage techniques due to their advantages of high spatial resolution and simultaneous acquisition of multiple information. These sensors can simultaneously acquire information such as contact force and contact area by capturing images of the deformation of the elastic contact layer. However, existing technologies have not been optimized for traditional Chinese medicine massage scenarios, and cannot achieve simultaneous and accurate measurement of six-dimensional force / torque and the position of force application. Furthermore, they lack dedicated technique recognition algorithms, making it difficult to meet the needs of clinical applications. Summary of the Invention

[0005] This invention addresses the technical problems of limited information, low recognition accuracy, and narrow applicability in existing massage technique measurement technologies by providing a massage technique measurement device and recognition method based on a visual tactile sensor.

[0006] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: A massage technique measurement device based on a visual tactile sensor, comprising a visual tactile sensor body, which consists of an elastic skin, a transparent acrylic support plate, an RGB three-color LED lighting unit, a black resin shell, a miniature camera, and a base; The elastic skin is sequentially layered with a transparent gel layer, a black circular marker layer, a white backlight layer, and a black protective layer. The black resin shell has a slot inside for fixing the RGB three-color LED lighting unit, the transparent acrylic support plate and the miniature camera. The miniature camera is mounted on the base and is used to capture images of the deformation of the black circular markers on the elastic skin after it is compressed; The data acquisition module is connected to a miniature camera and a calibrated six-dimensional force sensor to simultaneously acquire tactile image data and six-dimensional force / torque true value data.

[0007] On the other hand, the present invention also provides a method for recognizing massage techniques using a visual-touch sensor-based measurement device, the method comprising the following steps: S1: Sensor calibration and dataset construction: The calibrated ATINano17 six-dimensional force sensor was fixed to the base. The elastic skin was pressed with different massage techniques, and 32,000 sets of tactile images and corresponding six-dimensional force / torque data were collected simultaneously. The data were divided into training set and test set in a 6:4 ratio. Five hundred images each of five force application positions of the thumb—finger tip, fingertip, spiral surface, edge, and joint—were collected and divided into training, validation, and test sets in a 6:3:1 ratio. S2: Force application location recognition model training: Set the batch size to 16 and the number of training rounds to 50 for the YOLOv8 classification model, adjust the size of the force application location image uniformly, input it into the model for training, and obtain the force application location recognition model. S3: Force / Torque Estimation Model Training: A modified VGG16 network is used, retaining 13 convolutional layers and 3 fully connected layers, and the network output layer is replaced with 6 neurons to correspond to the force and torque on the X, Y, and Z axes; The SGD optimization algorithm, StepLR learning rate scheduling and MSE loss function are used to fine-tune the model based on ImageNet pre-training. After 100 training rounds, the force / torque estimation model is obtained. S4: Massage Technique Feature Extraction: Using the feature extraction module, extract time-domain features, frequency-domain features, and synthetic force / torque features from the force / torque time-series data output by the force / torque estimation model, and construct feature vectors; S5: Massage Technique Recognition: Input the feature vector into a random forest classifier, set the number of decision trees to 200 and the maximum depth to 10, and train the model to recognize pressing, rolling, rubbing, kneading, pushing, striking and vibrating techniques.

[0008] The beneficial effects of this invention are:

[0009] This invention can simultaneously collect six-dimensional force / torque and force application location information through a visual and tactile sensor. The force measurement range is extended to 0-12N, covering the actual operating force of traditional Chinese massage (traditional sensors are mostly 0-2N). This solves the problem of single information collection in the existing technology and can comprehensively characterize the features of massage techniques. The accuracy rate of force application location recognition reaches 97%, the accuracy rate of seven massage techniques recognition reaches 97.3%, the force estimation error is only 0.27N, and the torque estimation error is 4.3Nmm, which is far superior to existing similar technologies and can accurately distinguish the differences between different techniques. Employing YOLOv8 and the random forest algorithm, the model is highly lightweight and has a fast inference speed. It does not require a large number of samples and is suitable for real-time clinical identification. The device has a compact structure, and the elastic skin can adapt to the concave and convex shapes of different force application sites, closely matching actual massage operation scenarios. It can achieve objective quantification and standardization of traditional Chinese massage techniques, providing accurate data references for massage teaching (such as the standard of technique strength and force application location), providing objective indicators for clinical efficacy evaluation, and helping traditional Chinese massage to integrate into the modern evidence-based medical system. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the overall structure of the measuring device in this invention; Figure 2 This is a schematic diagram of the layered structure of the elastic skin in this invention; Figure 3 This is a flowchart of the method for identifying traditional Chinese massage techniques in this invention; Figure 4 This is a timing characteristic diagram of the force / torque of the pressing technique in this invention; Figure 5 This is a time-series characteristic diagram of the force / torque of the massage technique in this invention; Figure 6 This is a feature diagram illustrating the accuracy of force prediction and actual measurement in this invention; Figure 7 This is a feature diagram illustrating the accuracy of torque prediction versus actual measurement in this invention. Figure 8 This is a diagram showing the different force application positions in this invention; Figure 9 These are characteristic diagrams of different massage techniques in this invention; Figure 10 This invention illustrates the positional distribution of markers on elastic skin under different stress conditions.

[0011] The attached diagram lists the components represented by each number as follows: 1. Elastic skin; 2. Transparent acrylic support plate; 3. RGB three-color LED lighting unit; 4. Black resin shell; 5. Miniature camera; 6. Base; 7. Transparent gel layer; 8. Black circular marker layer; 9. White backlight layer; 10. Black protective layer. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0014] In the description of this application, it should also be noted that, unless otherwise expressly specified and limited, the terms "set up," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this technology based on the specific circumstances.

[0015] In the description of this application, spatial relation terms such as “below,” “under,” “below,” “below,” “above,” “over,” etc., are used herein to describe the relationship between an element or feature shown in the figures and other elements or features. It should be understood that, in addition to the orientation shown in the figures, spatial relation terms also include different orientations of the device in use and operation. For example, if the device in the figures is flipped, an element or feature described as “below,” “under,” or “below” will be oriented “over” the other element or feature. Therefore, the exemplary terms “below” and “under” can include both upper and lower orientations. Furthermore, the device may also include other orientations (e.g., rotated 90 degrees or other orientations), and the spatial descriptive terms used herein are interpreted accordingly.

[0016] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0017] See Figures 1 to 10 A massage technique measurement device based on a visual tactile sensor includes a visual tactile sensor body, a data acquisition module, a data preprocessing module, a feature extraction module, and a technique recognition module.

[0018] The main body of the visual-tactile sensor serves as the core of the device, undertaking the function of collecting mechanical and positional information during the massage process. It consists of elastic skin 1, transparent acrylic support plate 2, RGB three-color LED lighting unit 3, black resin shell 4, miniature camera 5, and base 6. The elastic skin 1 is sequentially layered with a transparent gel layer 7, a black circular marker layer 8, a white backlight layer 9, and a black protective layer 10. The black resin shell 4 has a slot inside for fixing the RGB three-color LED lighting unit 3, the transparent acrylic support plate 2 and the miniature camera 5. The miniature camera 5 is mounted on the base 6 and is used to capture images of the deformation of the black circular markers on the elastic skin after it is compressed. The data acquisition module is connected to the miniature camera 5 and the calibrated six-dimensional force sensor respectively, and synchronously acquires tactile image data and six-dimensional force / torque true value data.

[0019] The transparent gel layer 7 of the elastic skin 1 is made of C2H6OSi, with a temperature resistance range of -30℃ to 200℃. The black circular marker layer 8 is made by iron powder spraying, with the marker size being 2mm×2mm and evenly distributed on the surface of the transparent gel layer 7. The white backlight layer 9 is made by diamond polishing powder spraying to enhance the imaging contrast between the black markers and the background.

[0020] The RGB tri-color LED lighting unit 3 is installed on the side of the black resin shell 4. The light is mixed by the transparent gel layer 7 and then evenly projected onto the inside of the elastic skin 1 to avoid external light interfering with the imaging quality of the miniature camera 5. The black resin shell 4 is made of light-shielding material to further isolate the influence of external ambient light on imaging.

[0021] The data acquisition module consists of a synchronization control unit, an image acquisition card, and a signal acquisition card. The synchronization control unit coordinates the acquisition timing of the miniature camera 5 and the ATINano17 six-dimensional force sensor to ensure the synchronization of tactile images and six-dimensional force / torque data. The image acquisition card receives image data output by the miniature camera 5, and the signal acquisition card acquires mechanical data from the six-dimensional force sensor. Both transmit the data to subsequent modules for processing.

[0022] The data preprocessing module implements image preprocessing based on the Python OpenCV library. It converts color images into black and white images (white background, black markers) using a threshold segmentation algorithm to remove environmental noise and light and shadow interference. It uses the Min-Max normalization method to process the six-dimensional force / torque data, unifying the dimensions of different dimensions to [-1,1] to eliminate the impact of scale differences on subsequent model training.

[0023] The feature extraction module is based on the Python SciPy and NumPy libraries for feature extraction. For six-dimensional force / torque time-series data, the mean, maximum, minimum, standard deviation, skewness, and kurtosis are calculated through time-domain analysis. The time-domain data is transformed to the frequency domain using Fast Fourier Transform (FFT) to extract the dominant frequency (the frequency with the most concentrated energy) and spectral entropy (reflecting the degree of disorder in the frequency distribution). The average resultant force, maximum resultant force, resultant force fluctuation (standard deviation of the resultant force), average resultant torque, maximum resultant torque, and torque fluctuation are calculated through vector synthesis. Simultaneously, the correlation between force and torque is calculated using the Pearson correlation coefficient. Finally, a 15-dimensional feature vector is constructed to comprehensively characterize the massage technique.

[0024] The technique recognition module includes a force application location recognition submodule and a massage technique recognition submodule. The force application location recognition submodule uses a pre-trained YOLOv8 classification model, which is lightweight, fast inference speed, and adaptable to real-time clinical recognition needs. It can recognize five commonly used force application locations in massage: thumb tip, fingertip, spiral surface, oblique point, and joint. The massage technique recognition submodule uses a random forest classifier, which is highly adaptable to small sample data, requires no complex feature engineering, improves generalization ability by integrating 200 decision trees, limits the maximum depth to 10 to avoid overfitting, and can recognize seven basic massage techniques: pressing, rolling, rubbing, kneading, pushing, striking, and vibration.

[0025] Furthermore, this invention also provides a method for recognizing massage techniques using a vision-based tactile sensor, specifically including the following steps: S1: Sensor Calibration and Dataset Construction Sensor calibration: Fix the calibrated ATINano17 six-dimensional force sensor to the base 6, and calibrate the sensor using a standard force source to ensure its measurement accuracy, which will serve as the true value benchmark for subsequent force / torque estimation models. Mechanical data construction: Three TCM massage therapists with more than 5 years of clinical experience were invited to simulate clinical massage operations on elastic skin 1 using seven techniques: pressing, rolling, rubbing, kneading, pushing, striking, and vibration. Simultaneously, 32,000 sets of tactile images and corresponding six-dimensional force / torque data (covering the measurement ranges of Fx: [-4,4]N, Fy: [-4,4]N, Fz: [0,-14]N, Tx: [-200,200]Nmm, Ty: [-200,200]Nmm, Tz: [-200,200]Nmm) were collected and divided into a training set (19,200 sets) and a test set (12,800 sets) in a 6:4 ratio. Example location data construction: Collect 500 images each of five force application locations of the thumb: fingertip, finger tip, threaded surface, edge, and joint. Divide them into a training set (1500 images), a validation set (750 images), and a test set (250 images) in a 6:3:1 ratio.

[0026] S2: Training of the Force Application Location Recognition Model Data preprocessing: The image size of the force application location was uniformly adjusted to 224×224 pixels (to adapt to the input requirements of YOLOv8), and data augmentation techniques such as random flipping and brightness adjustment were used to expand the training set to avoid model overfitting; Model parameter settings: Batch size is set to 16 (balancing training efficiency and memory usage), number of training epochs is 50, Adam optimization algorithm is used (initial learning rate of 0.001, dynamically decaying), and cross-entropy loss function is used (suitable for classification tasks). Model training and validation: The preprocessed training set was input into the YOLOv8 model. After each round of training, the model performance (accuracy and recall) was evaluated using the validation set. After training, the model was tested using the test set. The final force position recognition accuracy reached 97%.

[0027] S3: Force / Torque Estimation Model Training Network construction and modification: Based on the pre-trained VGG16 network, 13 convolutional layers (each convolutional layer is followed by a ReLU activation function to enhance nonlinear fitting ability) and 3 fully connected layers are retained. The 1000-dimensional output layer of the original classification task is replaced with a 6-dimensional output layer (corresponding to force and torque on the X, Y, and Z axes) to adapt to the requirements of the regression task. Training parameter settings: SGD optimization algorithm was used (learning rate 0.01, momentum 0.9, weight decay 1e). -3 To prevent excessive weights), StepLR learning rate scheduling (step size 10, γ=0.1, learning rate decays by a factor of 10 every 10 rounds), MSE loss function (measures the squared difference between the predicted value and the true value, suitable for regression tasks). Model Training and Convergence Validation: Tactile images and corresponding six-dimensional force / torque data were input into a modified VGG16 network. Fine-tuning was performed based on ImageNet pre-trained weights. After 100 training epochs, the model converged in approximately 20 epochs. At convergence, the RMSE (Root Mean Square Error) was 0.12, the R² (Coefficient of Determination, reflecting model goodness of fit) was 0.87, the force estimation error was approximately 0.27 N, and the torque estimation error was 4.3 Nmm. The loss function used was the mean squared error (MSE), defined as: ; In the formula, n is the number of samples. Let be the true value of the i-th sample. This is the predicted value for the i-th sample. Training and inference are performed using a GeForce RTX™ 4090 GPU and an Intel® Core™ i9 14900KF CPU. Measurements from an ATI Nano17 sensor are used as ground-based data. The dataset consists of 32,000 RGB images and their corresponding forces / torques, divided into training and testing sets in a 6:4 ratio. To prevent noise interference, the color images are preprocessed with a white background and black markers. In the training and testing datasets, the x-axis force measured by this invention... F x and y-axis force F y The actual range is [-4 N, 4 N], z-axis force F z It is [0, -14 N], x-axis torque T x y-axis torque T y and z-axis torque T z It is [-200 Nmm, 200 Nmm]. After 100 training epochs, the model converges after 20 epochs. RMSE It is 0.12. It is 0.87, of which: ; ; In the formula, n is the number of samples. Let be the true value of the i-th sample. Let be the predicted value for the i-th sample. This is the average of the true values ​​of the sample.

[0028] This invention employs different massage techniques to press on a designed visual-tactile sensor. Simultaneously, it records the model's predicted values ​​and the actual values ​​from the ATI Nano 17 sensor. By comparing the numerical difference between the two values, the accuracy of the force and torque estimation can be determined. The mean absolute error (MAE) of force / torque is calculated from 300 sets of continuous data. The calculation method is as follows: ; In the formula, n is the number of samples. Let be the true value of the i-th sample. Let be the predicted value for the i-th sample. This is the average of the true values ​​of the sample.

[0029] Experimental results are as follows Figure 6 and Figure 7 As shown, the force estimation accuracy is approximately 0.27 N, and the torque estimation accuracy is approximately 4.3 N / mm. This indicates that the force and torque outputs of this sensor have a good correlation with the true values ​​of the Nano 17 sensor. This demonstrates that the method has the capability to accurately acquire force and torque in the field of massage.

[0030] S4: Massage technique feature extraction data acquisition: Input the tactile images in the test set into the force / torque estimation model to obtain the corresponding six-dimensional force / torque time series data; at the same time, determine the force application position information through the force application position recognition model; Feature Calculation: For six-dimensional force / torque time series data, the feature extraction module is used to calculate time domain features (mean, maximum, minimum, standard deviation, skewness, kurtosis), frequency domain features (dominant frequency, spectral entropy), and composite force / torque features (average resultant force, maximum resultant force, resultant force fluctuation, average resultant torque, maximum resultant torque, torque fluctuation, force-torque correlation), and construct a vector containing 15 features; Feature visualization and key feature identification: The feature vectors of different methods were visualized using radar charts. Comparative analysis revealed that the standard deviation of the Tz-axis torque, the mean of the composite torque, and the standard deviation of the Ty-axis torque contributed the most to the differentiation of methods. For example, the normalized standard deviation of the Tz-axis torque of the pushing method was 0.08-0.15 (medium level), while the normalized standard deviation of the Tz-axis torque of the kneading method was 0.25-0.40 (high level), which can be used as core distinguishing features.

[0031] S5: Massage Technique Recognition Model training: Input the feature vectors and corresponding method labels (7 classes) into the random forest classifier, set the number of decision trees to 200 (to improve the ensemble effect), the maximum depth to 10 (to avoid overfitting), and use 5-fold cross-validation to optimize the model parameters; First, 500 images were collected from each of five different pressure application points, representing commonly used massage areas: the thumb tip, fingertip, spiral surface, side of the thumb, and joint. Figure 8 As shown. Next, a pre-trained YOLOv8 classification model is selected. Then, the dataset is divided into training, validation, and test sets in a 6:3:1 ratio.

[0032] Subsequently, training parameters were set: batch size of 16, number of training epochs of 50, and image size uniformly set to 224×224 pixels. The YOLOv8 model was then used for training, achieving an accuracy of 97%. Experimental results are shown in Table 1, demonstrating that this method can effectively distinguish the location of applied force.

[0033] Table 1. Confusion matrix for force application location classification

[0034] To classify different massage techniques, this invention first collected force and torque data over time for seven different types of techniques: pressing, rolling, rubbing, kneading, pushing, striking, and vibration. Then, it extracted independent features for each axis, including the average, maximum, minimum, standard deviation, skewness, and kurtosis of time-domain features, and the dominant frequency and spectral entropy of frequency-domain features. It also extracted composite force and composite torque features, such as average composite force, maximum composite force, composite force fluctuation, average composite torque, maximum composite torque, torque fluctuation, and force-torque correlation.

[0035] These features can effectively distinguish different massage techniques. This invention uses radar charts to display the distribution of these features under different massage techniques. Each feature is assigned to a different angle, and the value of each feature determines the distance from the corresponding point to the center. The larger the value, the farther the point is from the center. By comparing the radar charts of different techniques and observing the differences in the shape and area of ​​the graphs, the characteristics and differences of different techniques can be seen. Figure 9 As shown in the table. Next, the training set and test set were divided, with the test set accounting for 30%. Due to the small sample size and the need to meet the requirements of fast inference, CNN requires a large number of samples to ensure accuracy. Therefore, a random forest classifier was used to train the training set, with 200 decision trees and a maximum depth of 10. By sorting the feature weights, it was found that the standard deviation of torque on the Tz axis, the composite mean torque, and the standard deviation of torque on the Ty axis are the most critical features for distinguishing massage techniques. For example, the normalized range of the standard deviation of torque on the Tz axis for the pushing technique is 0.08-0.15, which is moderate among all techniques, while the range for the kneading technique is 0.25-0.40, which is relatively large. These features can effectively distinguish different massage techniques. The classification results are shown in Table 2, with an accuracy of 97.3%, verifying the effectiveness of the massage technique classification algorithm.

[0036] Table 2. Confusion Matrix for Massage Technique Classification

[0037] like Figure 10 As shown, when subjected to vertical pressure, the marked points exhibit a radial pattern due to the vertical compression of the gel; when subjected to a horizontal tangential force, the marked points shift along the direction of the force; and the vertical torque manifests as a radial pattern along the direction of rotation. Figure 4 and Figure 5 As shown, the pressing method, for example, involves dominant and relatively large normal force, slight fluctuations in tangential force, and relatively small torque. This technique typically involves pressing vertically on the acupoint with the tip or pad of the thumb, gradually increasing the force, pausing briefly after reaching a certain pressure, and then slowly reducing the force to stimulate the acupoint and unblock the meridians. The pushing method, on the other hand, involves dominant tangential force, relatively small normal force, and relatively small torque. The tip or pad of the thumb is used to push on the acupoint along a specific direction or meridian pathway, gradually increasing the force to allow it to penetrate deep into the tissues. After reaching a certain range, the force is slowly reduced and the user returns to the starting point.

[0038] Model testing and performance evaluation: The model performance was evaluated using the feature vectors of the test set, and the recognition results were analyzed by confusion matrix. The recognition accuracy of the seven massage techniques reached 97.3%, with the recognition accuracy of pushing and pressing techniques reaching 100%, and the confusion rate of rubbing and rolling techniques being only 2%, which meets the clinical recognition needs.

[0039] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the above embodiments are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A massage technique measuring device based on a visual-tactile sensor, characterized in that, include: The main body of the visual tactile sensor consists of an elastic skin (1), a transparent acrylic support plate (2), an RGB three-color LED lighting unit (3), a black resin shell (4), a miniature camera (5), and a base (6); The elastic skin (1) is sequentially layered with a transparent gel layer (7), a black circular marker layer (8), a white backlight layer (9), and a black protective layer (10). The black resin shell (4) has slots inside for fixing the RGB three-color LED lighting unit (3), the transparent acrylic support plate (2) and the miniature camera (5); The miniature camera (5) is mounted above the base (6) and is used to capture images of the deformation of the black circular markers after the elastic skin is compressed; The data acquisition module is connected to the miniature camera (5) and the calibrated six-dimensional force sensor respectively, and synchronously acquires tactile image data and six-dimensional force / torque true value data.

2. The massage technique measuring device based on a visual-tactile sensor according to claim 1, characterized in that, The transparent gel layer (7) of the elastic skin (1) is made of C2H6OSi, with a temperature resistance range of -30℃ to 200℃. The white backlight layer (9) is made by spraying diamond polishing powder, and the black circular marking dot layer (8) is made by spraying iron powder and is evenly distributed on the surface of the transparent gel layer (7); and / or The size of the marker point is 2mm × 2mm.

3. The massage technique measuring device based on a visual-tactile sensor according to claim 1, characterized in that, The RGB three-color LED lighting unit (3) is installed on the side of the black resin shell (4). The light is mixed by the transparent gel layer (7) and then evenly projected onto the inside of the elastic skin (1). The black resin shell (4) is made of light-shielding material.

4. The massage technique measuring device based on a visual-tactile sensor according to any one of claims 1-3, characterized in that, It also includes a data acquisition module, a data preprocessing module, a feature extraction module, and a technique recognition module; The data preprocessing module is used to perform background normalization and noise removal processing on the tactile images acquired by the miniature camera (5), convert the color image into a black and white image with a white background and black markers, and at the same time unify the dimensions of the six-dimensional force / torque data to eliminate the scale differences of data in different dimensions.

5. The massage technique measuring device based on a visual-tactile sensor according to claim 4, characterized in that, The feature extraction module is used to extract the time-domain features, frequency-domain features, and composite force / torque features of the six-dimensional force / torque data; The time-domain features include mean, maximum, minimum, standard deviation, skewness, and kurtosis; The frequency domain features include the dominant frequency and the spectral entropy; The combined force / torque characteristics include average resultant force, maximum resultant force, resultant force fluctuation, average resultant torque, maximum resultant torque, torque fluctuation, and force-torque correlation.

6. The massage technique measuring device based on a visual-tactile sensor according to claim 5, characterized in that, The technique recognition module includes a force application location recognition submodule and a massage technique recognition submodule; The force application location recognition submodule uses a pre-trained YOLOv8 classification model; The massage technique recognition submodule uses a random forest classifier.

7. A method for recognizing massage techniques based on a visual-tactile sensor, using the massage technique measuring device based on a visual-tactile sensor as described in claim 6, characterized in that... The method includes the following steps: S1: Sensor calibration and dataset construction: The calibrated ATINano17 six-dimensional force sensor was fixed on the base (6), and the elastic skin was pressed by different massage techniques (1). 32,000 sets of tactile images and corresponding six-dimensional force / torque data were collected simultaneously and divided into training set and test set in a 6:4 ratio. Five hundred images each of five force application positions of the thumb—finger tip, fingertip, spiral surface, edge, and joint—were collected and divided into training, validation, and test sets in a 6:3:1 ratio. S2: Force application location recognition model training: Set the batch size to 16 and the number of training rounds to 50 for the YOLOv8 classification model, adjust the size of the force application location image uniformly, input it into the model for training, and obtain the force application location recognition model. S3: Force / Torque Estimation Model Training: A modified VGG16 network is used, retaining 13 convolutional layers and 3 fully connected layers, and the network output layer is replaced with 6 neurons to correspond to the force and torque on the X, Y, and Z axes; The SGD optimization algorithm, StepLR learning rate scheduling and MSE loss function are used to fine-tune the model based on ImageNet pre-training. After 100 training rounds, the force / torque estimation model is obtained. S4: Massage Technique Feature Extraction: Using the feature extraction module, extract time-domain features, frequency-domain features, and synthetic force / torque features from the force / torque time-series data output by the force / torque estimation model, and construct feature vectors; S5: Massage Technique Recognition: Input the feature vector into a random forest classifier, set the number of decision trees to 200 and the maximum depth to 10, and train the model to recognize pressing, rolling, rubbing, kneading, pushing, striking and vibrating techniques.

8. The identification method according to claim 7, characterized in that, In S4, the feature vectors of different massage techniques are visualized and analyzed using radar charts to determine the standard deviation of torque on the Tz axis, the mean of composite torque, and the standard deviation of torque on the Ty axis as key features to distinguish different massage techniques. The normalized standard deviation of the Tz-axis torque for the pushing method ranges from 0.08 to 0.15, while that for the kneading method ranges from 0.25 to 0.

40.

9. The identification method according to claim 7, characterized in that, In S1, the measurement range of the ATI Nano17 six-dimensional force sensor is as follows: X-axis force Fx and Y-axis force Fy are both [-4 N, 4 N], Z-axis force Fz is [0, -14 N], and X-axis torque Tx, Y-axis torque Ty, and Z-axis torque Tz are all [-200 Nmm, 200 Nmm].

Images