Massage method measuring device and recognition method based on visual tactile sensor

Abstract

The present application relates to the technical field of tactile sensing and traditional Chinese medicine intelligent diagnosis, and in particular to a massage method measuring device and recognition method based on a visual tactile sensor, which synchronously collects six-dimensional force / torque data and force application position information during the massage process through the visual tactile sensor, and after data preprocessing and feature extraction, the force application position and massage method are accurately recognized by combining deep learning and random forest algorithm. Experimental verification shows that the force estimation error of the device is about 0.27 N, the torque estimation error is 4.3 Nmm, the force application position recognition accuracy is 97%, and the recognition accuracy of seven traditional Chinese massage methods is 97.3%, solving the problems of traditional Chinese massage method quantization difficulty and low recognition accuracy, and realizing the standardization of massage methods, providing technical support for the standardized teaching, clinical application and intelligent rehabilitation equipment development of traditional Chinese massage.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of tactile sensing and intelligent diagnosis of traditional Chinese medicine, and in particular to a massage method measuring device and recognition method based on a visual tactile sensor. BACKGROUND

[0002] Traditional Chinese massage (also known as tuina) is an important part of traditional medicine and has significant therapeutic effects in improving neuromuscular function, relieving chronic pain, and enhancing immune function, and is widely used in rehabilitation and health care. A complete traditional Chinese massage procedure is usually composed of a combination of various basic techniques such as rubbing, pressing, rolling, vibrating, pushing, striking, and massaging. However, traditional massage techniques mainly rely on the experience of practitioners, and there are significant differences in the force, frequency, and position of force application among different practitioners, resulting in inconsistent treatment effects for the same condition, which seriously hinders the standardization and modernization of traditional Chinese massage.

[0003] Quantitative and accurate recognition of massage techniques is the key to solving the above problems. In existing related technologies, researchers have developed various measurement schemes: for example, the single-axis force sensor designed by Chen X et al. can only detect vertical pressure and operation sequence; the three-axis force sensor developed by Liu G et al. can identify the magnitude and direction of spatial force, but the measurement range is limited to 0-2N, which cannot cover the actual operating force of traditional Chinese massage; the flexible six-axis force / torque sensor proposed by Liu L et al. can distinguish between some techniques, but it ignores the importance of force application position - when the same technique is applied to different acupoints, the difference in force application position (such as the tip of the thumb, the end of the finger, and the joint) will result in different contact areas and thus different biomechanical stimulation effects, making it difficult to fully characterize the massage technique characteristics relying solely on multi-dimensional force parameters. In addition, existing recognition algorithms mostly rely on convolutional neural networks, which require a large number of samples to support and have insufficient generalization ability in small-sample clinical scenarios, and are difficult to implement real-time recognition.

[0004] In recent years, visual tactile sensors (such as GelSight and TacTip) have provided a new direction for massage technique quantification due to their high spatial resolution and multi-information synchronous acquisition advantages. Such sensors can capture deformation images of the elastic contact layer to simultaneously acquire information such as contact force and contact area, but existing technologies have not been optimized for traditional Chinese massage scenarios, making it difficult to achieve simultaneous and accurate measurement of six-dimensional force / torque and force application position, and lacking specialized technique recognition algorithms, making it difficult to meet the needs of clinical applications. SUMMARY

[0005] The present application provides a massage technique measuring device and recognition method based on a visual tactile sensor to address the technical problems of single measurement information, low recognition accuracy, and narrow application range in existing technologies.

[0006] The technical solution of the present application to solve the above technical problems is as follows:

[0007] The massage method measuring device based on a visual tactile sensor comprises a visual tactile sensor body composed of an elastic skin, a transparent acrylic support plate, an RGB three-color LED lighting unit, a black resin shell, a miniature camera and a base;

[0008] The elastic skin is sequentially stacked with a transparent gel layer, a black circular marker point layer, a white backlight layer and a black protective layer.

[0009] The black resin shell is internally provided with a clamping groove for fixing the RGB three-color LED lighting unit, the transparent acrylic support plate and the miniature camera.

[0010] The miniature camera is installed above the base and is used to capture the deformation image of the black circular marker point after the elastic skin is pressed.

[0011] The data acquisition module is connected with the miniature camera and the calibrated six-dimensional force sensor respectively, and synchronously acquires tactile image data and six-dimensional force / torque true value data.

[0012] In another aspect, the present application also provides an identification method of the massage method measuring device based on the visual tactile sensor, which comprises the following steps:

[0013] S1: sensor calibration and data set construction: fix the calibrated ATINano17 six-dimensional force sensor on the base, press the elastic skin by different massage methods, synchronously acquire 32,000 groups of tactile images and corresponding six-dimensional force / torque data, and divide them into a training set and a test set in a 6:4 ratio;

[0014] Acquire 500 images of each of the five force application positions of the thumb tip, the finger end, the thread surface, the partial fin and the joint, and divide them into a training set, a validation set and a test set in a 6:3:1 ratio;

[0015] S2: force application position recognition model training: set the batch size of the YOLOv8 classification model to 16 and the training round number to 50, uniformly adjust the size of the force application position images, input the model for training, and obtain the force application position recognition model;

[0016] S3: force / torque estimation model training: use the modified VGG16 network, retain 13 convolutional layers and 3 fully connected layers, replace the network output layer with 6 neurons to correspond to the forces and torque of the X, Y and Z axes;

[0017] Use the SGD optimization algorithm, StepLR learning rate scheduling and MSE loss function, fine-tune the model on the basis of ImageNet pre-training, and obtain the force / torque estimation model after 100 rounds of training.

[0018] S4: Massage technique feature extraction: using the feature extraction module to extract the time domain features, frequency domain features and combined force / torque features from the force / torque time series data output by the force / torque estimation model, and construct a feature vector;

[0019] S5: Massage technique recognition: input the feature vector into the random forest classifier, set the number of decision trees to 200 and the maximum depth to 10, and realize the recognition of massage, rolling, rubbing, kneading, pushing, hitting and vibrating after training the model.

[0020] The beneficial effects of the present application are:

[0021] The present application can synchronously collect six-dimensional force / torque and force application position information through a visual tactile sensor, and the force measurement range is expanded to 0-12N, covering the actual operation force of traditional Chinese massage (most traditional sensors are 0-2N), solving the problem of single information collection in the prior art, and fully representing the massage technique characteristics;

[0022] The force application position recognition accuracy is 97%, the seven massage technique recognition accuracy is 97.3%, the force estimation error is only 0.27N, and the torque estimation error is 4.3Nmm, which is much better than the existing similar technology, and can accurately distinguish the differences between different techniques;

[0023] The YOLOv8 and random forest algorithm are adopted, the model has high lightweight degree and fast inference speed, does not need a large number of sample supports, is suitable for real-time identification in clinical practice, the device structure is compact, the elastic skin can adapt to the concave-convex morphology of different force application parts, and the actual massage operation scene is fitted, so that the objective quantification and standardization of traditional Chinese massage techniques are realized, precise data references (such as technique force, force application position standard) for massage teaching are provided, objective indexes for clinical efficacy evaluation are provided, and the traditional Chinese massage is integrated into the modern evidence-based medical system. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 It is a schematic diagram of the overall structure of the measuring device in the present application;

[0025] Figure 2 It is a schematic diagram of the layered structure of the elastic skin in the present application;

[0026] Figure 3 It is a flowchart of the traditional Chinese massage technique recognition method in the present application;

[0027] Figure 4 It is a force / torque time series feature map of the pressing technique in the present application;

[0028] Figure 5 It is a force / torque time series feature map of the pushing technique in the present application;

[0029] Figure 6The force prediction and real measurement accuracy feature map in the present application;

[0030] Figure 7 The torque prediction and real measurement accuracy feature map in the present application;

[0031] Figure 8 The different force application position map in the present application;

[0032] Figure 9 The different massage method feature map in the present application;

[0033] Figure 10 The position distribution of the mark on the elastic skin under different force conditions in the present application.

[0034] In the drawings, the components represented by each reference numeral are listed as follows:

[0035] 1, elastic skin; 2, transparent acrylic support plate; 3, RGB three-color LED lighting unit; 4, black resin shell; 5, micro camera; 6, base; 7, transparent gel layer; 8, black circular mark point layer; 9, white backlight layer; 10, black protective layer. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0037] In the description of the present application, the terms "first" and "second" are used only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0038] In the description of the present application, it should be further pointed out that, unless otherwise specifically specified and limited, the terms "set", "mount", "connected", "connected" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present technology can be understood according to the specific circumstances.

[0039] In the description of the application, spatially relative terms, such as "under", "below", "lower", "on", "above", "upper", and the like, can be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientations depicted in the figures. For example, if a device in the figures is inverted, elements described as "below" or "under" other elements or features would then be oriented "above" the other elements or features. Thus, the exemplary term "below" can encompass both an orientation of above and below. The device can also be oriented in the other directions (such as rotated 90 degrees or along other orientations) and the included spatial descriptions are to be interpreted accordingly.

[0040] In the description of the application, the word "exemplary" is used to mean "serving as an example, instance, or illustration." Any implementation described as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations. The following description is presented to enable any person skilled in the art to make and use the application. Details are set forth in the following description for purpose of explanation. It should be appreciated that one of ordinary skill in the art will readily recognize that the application can be practiced without the use of these specific details. In other instances, well-known structures and processes have not been described in detail in order to avoid obscuring the description of the application. Thus, the present application is not intended to be limited by the embodiments shown, but is to be accorded with the widest scope consistent with the principles and features disclosed.

[0041] Referring to Figures 1 to 10 The massage method measuring device based on a visual tactile sensor comprises a visual tactile sensor body, a data acquisition module, a data preprocessing module, a feature extraction module, and a method identification module.

[0042] The visual tactile sensor body serves as the core of the device and is responsible for collecting mechanical and position information during the massage process, and is composed of an elastic skin 1, a transparent acrylic support plate 2, an RGB three-color LED lighting unit 3, a black resin shell 4, a miniature camera 5, and a base 6.

[0043] The elastic skin 1 has a transparent gel layer 7, a black circular marker point layer 8, a white backlight layer 9, and a black protective layer 10 stacked in sequence.

[0044] The black resin shell 4 is internally provided with a clamping groove for fixing the RGB three-color LED lighting unit 3, the transparent acrylic support plate 2, and the miniature camera 5.

[0045] The miniature camera 5 is installed above the base 6 and is used to capture the deformation image of the black circular marker point after the elastic skin is pressed.

[0046] The data acquisition module is connected to the miniature camera 5 and the calibrated six-dimensional force sensor respectively, and synchronously acquires tactile image data and six-dimensional force / torque true value data.

[0047] The transparent gel layer 7 of the elastic skin 1 is made of C2H6OSi, with a temperature resistance range of -30℃ to 200℃. The black circular marker layer 8 is made by iron powder spraying, with the marker size being 2mm×2mm and evenly distributed on the surface of the transparent gel layer 7. The white backlight layer 9 is made by diamond polishing powder spraying to enhance the imaging contrast between the black markers and the background.

[0048] The RGB tri-color LED lighting unit 3 is installed on the side of the black resin shell 4. The light is mixed by the transparent gel layer 7 and then evenly projected onto the inside of the elastic skin 1 to avoid external light interfering with the imaging quality of the miniature camera 5. The black resin shell 4 is made of light-shielding material to further isolate the influence of external ambient light on imaging.

[0049] The data acquisition module consists of a synchronization control unit, an image acquisition card, and a signal acquisition card. The synchronization control unit coordinates the acquisition timing of the miniature camera 5 and the ATINano17 six-dimensional force sensor to ensure the synchronization of tactile images and six-dimensional force / torque data. The image acquisition card receives image data output by the miniature camera 5, and the signal acquisition card acquires mechanical data from the six-dimensional force sensor. Both transmit the data to subsequent modules for processing.

[0050] The data preprocessing module implements image preprocessing based on the Python OpenCV library. It converts color images into black and white images (white background, black markers) using a threshold segmentation algorithm to remove environmental noise and light and shadow interference. It uses the Min-Max normalization method to process the six-dimensional force / torque data, unifying the dimensions of different dimensions to [-1,1] to eliminate the impact of scale differences on subsequent model training.

[0051] The feature extraction module is based on the Python SciPy and NumPy libraries for feature extraction. For six-dimensional force / torque time-series data, the mean, maximum, minimum, standard deviation, skewness, and kurtosis are calculated through time-domain analysis. The time-domain data is transformed to the frequency domain using Fast Fourier Transform (FFT) to extract the dominant frequency (the frequency with the most concentrated energy) and spectral entropy (reflecting the degree of disorder in the frequency distribution). The average resultant force, maximum resultant force, resultant force fluctuation (standard deviation of the resultant force), average resultant torque, maximum resultant torque, and torque fluctuation are calculated through vector synthesis. Simultaneously, the correlation between force and torque is calculated using the Pearson correlation coefficient. Finally, a 15-dimensional feature vector is constructed to comprehensively characterize the massage technique.

[0052] The massage method recognition module comprises a force application position recognition submodule and a massage method recognition submodule; the force application position recognition submodule adopts a pre-trained YOLOv8 classification model, which is high in lightweight degree and fast in inference speed, is suitable for real-time recognition requirements in clinical practice, and can recognize five commonly used force application positions in massage, namely, the tip of the thumb, the end of the finger, the thread surface, the partial finger and the joint; the massage method recognition submodule adopts a random forest classifier, which is strong in adaptability to small sample data, does not require complex feature engineering, enhances the generalization ability by integrating 200 decision trees, limits the maximum depth to 10 to avoid overfitting, and can recognize seven basic massage methods, namely, pressing, rolling, rubbing, kneading, pushing, hitting and vibrating.

[0053] In addition, the application also provides a massage method recognition method of a visual-based tactile sensor, which specifically comprises the following steps:

[0054] S1: sensor calibration and data set construction

[0055] Sensor calibration: fix the calibrated ATINano17 six-dimensional force sensor on the base 6, calibrate the sensor through a standard force source, ensure the measurement accuracy, and use it as a true value benchmark for the subsequent force / torque estimation model;

[0056] Mechanical data construction: invite three Chinese medicine massagers with more than five years of clinical experience to simulate clinical massage operations on the elastic skin 1 by using seven methods, namely, pressing, rolling, rubbing, kneading, pushing, hitting and vibrating, and synchronously collect 32,000 groups of tactile images and corresponding six-dimensional force / torque data (covering the measurement range of Fx: [-4, 4] N, Fy: [-4, 4] N, Fz: [0, -14] N, Tx: [-200, 200] Nmm, Ty: [-200, 200] Nmm and Tz: [-200, 200] Nmm) in a proportion of 6:4 to divide them into a training set (19200 groups) and a test set (12800 groups);

[0057] Example position data construction: collect 500 images of five force application positions, namely, the tip of the thumb, the end of the finger, the thread surface, the partial finger and the joint, and divide them into a training set (1500 images), a validation set (750 images) and a test set (250 images) in a proportion of 6:3:1.

[0058] S2: force application position recognition model training

[0059] Data preprocessing: uniformly adjust the size of the force application position images to 224x224 pixels (adapt to the input requirements of YOLOv8), use data enhancement methods such as random flipping and brightness adjustment to expand the training set, and avoid model overfitting;

[0060] Model parameter settings: Set the batch size to 16 (balance training efficiency and memory usage), the number of training rounds to 50, use the Adam optimization algorithm (learning rate initial value 0.001, dynamic decay), and the cross-entropy loss function (suitable for classification tasks);

[0061] Model training and verification: Input the preprocessed training set into the YOLOv8 model, evaluate the model performance (accuracy, recall rate) after each training round using the validation set, and test the final force position recognition accuracy rate reaches 97% using the test set.

[0062] S3: Force / torque estimation model training

[0063] Network construction and modification: Based on the pre-trained VGG16 network, retain 13 convolutional layers (each convolutional layer is followed by a ReLU activation function to enhance non-linear fitting ability) and 3 fully connected layers, replace the original 1000-dimensional output layer of the classification task with a 6-dimensional output layer (corresponding to X, Y, Z axis force and torque), and adapt to the regression task requirements;

[0064] Training parameter settings: Use the SGD optimization algorithm (learning rate 0.01, momentum 0.9, weight decay 1e -3 , to prevent excessive weight), StepLR learning rate scheduling (step 10, γ = 0.1, learning rate decay by 10 times every 10 rounds), and MSE loss function (measure the squared difference between predicted and true values, suitable for regression tasks);

[0065] Model training and convergence verification: Input the haptic image and corresponding six-dimensional force / torque data into the modified VGG16 network, fine-tune based on the ImageNet pre-trained weights, and after 100 rounds of training, the model converges in about 20 rounds, with an RMSE (root mean square error) of 0.12 and an R² (determination coefficient, reflecting the goodness of fit of the model) of 0.87. The force estimation error of the test set is about 0.27N, and the torque estimation error is 4.3Nmm. The loss function used is mean squared error (MSE), defined as:

[0066] ;

[0067] where n is the number of samples, is the true value of the i-th sample, is the predicted value of the i-th sample. The training and inference are performed using a GeForce RTX™ 4090 GPU and an Intel® Core™ i9 14900KF CPU. The measurements of the ATI Nano 17 sensor are used as ground truth. The dataset consists of 32000 RGB images and corresponding force / torque, which is divided into training and test sets in a ratio of 6:4. In order to prevent the interference of noise, the color image is pre-processed to be white background and black marker point. The true range of the measured x-axis force F x and y-axis force F y is [-4 N, 4 N], the z-axis force F z is [0, -14 N], the x-axis torque T x , y-axis torque T y and z-axis torque T z is [-200 Nmm, 200 Nmm]. After 100 rounds of training, the model converges after 20 epochs, RMSE is 0.12, is 0.87, where:

[0068] ;

[0069] ;

[0070] where n is the number of samples, is the true value of the i-th sample, is the predicted value of the i-th sample, is the average value of the sample true value.

[0071] The present application adopts different massage methods to press on the designed visual-tactile sensor. At the same time, the model prediction value and the ATI Nano 17 sensor true value are recorded. By comparing the numerical difference between the two values, the accuracy of force and torque estimation can be judged. The mean absolute error (MAE) of force / torque is calculated from 300 consecutive data sets, and the calculation method is:

[0072] ;

[0073] where n is the number of samples, is the true value of the i-th sample, is the predicted value of the i-th sample, is the average value of the sample true value.

[0074] The experimental results are as followsFigure 6 and Figure 7 As shown in FIG. 6, the force estimation accuracy is about 0.27 N, and the torque estimation accuracy is about 4.3 Nmm. It is shown that the force and torque outputs of the sensor have good correlation with the true values of the Nano 17 sensor. It is proved that the method has the ability to accurately obtain force and torque in the massage field.

[0075] S4: Massage method feature extraction data acquisition: input the tactile image in the test set into the force / torque estimation model to obtain the corresponding six-dimensional force / torque time series data; at the same time, determine the force application position information through the force application position recognition model;

[0076] Feature calculation: for the six-dimensional force / torque time series data, calculate the time domain features (mean, maximum, minimum, standard deviation, skewness, kurtosis), frequency domain features (main frequency, spectral entropy), and combined force / torque features (average combined force, maximum combined force, combined force fluctuation, average combined torque, maximum combined torque, torque fluctuation, force-torque correlation) using the feature extraction module to construct a vector containing 15 features;

[0077] Feature visualization and key feature determination: visualize the feature vectors of different methods through radar charts. Comparative analysis shows that the Tz-axis torque standard deviation, combined torque mean, and Ty-axis torque standard deviation contribute the most to method differentiation - for example, the Tz-axis torque standard deviation of the pushing method is normalized to 0.08-0.15 (medium level), and the Tz-axis torque standard deviation of the kneading method is normalized to 0.25-0.40 (high level), which can be used as core distinguishing features.

[0078] S5: Massage method recognition

[0079] Model training: input the feature vector and corresponding method label (7 categories) into the random forest classifier, set the number of decision trees to 200 (to improve the integration effect), the maximum depth to 10 (to avoid overfitting), and use 5-fold cross-validation to optimize the model parameters;

[0080] First, 500 images of 5 different force application positions were collected, which are commonly used in massage, including the thumb tip, finger end, thread surface, partial finger, and joint, as shown in FIG. 7. Figure 8 Then, a pre-trained YOLOv8 classification model was selected. Then, the dataset was divided into training set, validation set and test set according to the ratio of 6:3:1.

[0081] Subsequently, the training parameters were set as follows: batch size 16, training rounds 50, and image size 224x224 pixels. Then, the YOLOv8 model was called for training, and the accuracy was 97%. The experimental results are shown in Table 1, which shows that this method can effectively distinguish the force application position.

[0082] Table 1 Confusion matrix of force position classification

[0083]

[0084] To realize the classification of different massage methods, first of all, the present application collects force and torque data of 7 different types of methods, including pressing, rolling, rubbing, kneading, pushing, hitting and vibrating, over time. Then, independent features of each axis are extracted, including mean, maximum, minimum, standard deviation, skewness, kurtosis of time domain features, and main frequency, spectral entropy of frequency domain features. The features of combined force and combined torque are extracted, such as average combined force, maximum combined force, combined force fluctuation, average combined torque, maximum combined torque, torque fluctuation, and force-torque correlation.

[0085] These features can effectively distinguish different massage methods. The present application displays the distribution of these features under different massage methods through radar chart. Each feature is assigned to a different angle, and the value of each feature determines the distance of the corresponding point to the center. The larger the value, the farther the point from the center. By comparing the radar charts of different methods and observing the differences in shape and area of the graph, the characteristics and differences of different methods in features can be seen, as shown in Figure 9 Next, the training set and test set are divided, and the test set accounts for 30%. Because the number of samples is small and needs to meet the demand of fast reasoning, a large number of samples are needed for CNN to ensure accuracy, so a random forest classifier is used to train the training set, with 200 decision trees and a maximum depth of 10. Through feature weight ordering, it is found that the Tz axis torque standard deviation, combined torque mean and Ty axis torque standard deviation are the most critical features to distinguish methods. For example, the Tz axis torque standard deviation of pushing method after normalization ranges from 0.08 to 0.15, which is in the middle among all methods, while the range of kneading method is 0.25-0.40, with a larger value, which can effectively distinguish different massage methods through features. The classification results are shown in Table 2, with an accuracy of 97.3%, which verifies the effectiveness of the massage method classification algorithm.

[0086] Table 2 Confusion matrix of massage method classification

[0087]

[0088] As shown in Figure 10 When subjected to vertical pressure, the marker points exhibit a radial pattern due to the vertical compression of the gel; when subjected to a tangential force in the horizontal direction, the marker points shift along the direction of the force; the vertical torque exhibits a radial pattern along the direction of rotation; as shown in Figure 4 and Figure 5As shown, for example, the normal force of the pressing method is dominant and large, the tangential force fluctuates slightly, and the torque is small. This method usually uses the tip or the palm of the thumb to press vertically on the acupoint, gradually increasing the force, stopping for a few moments after reaching a certain degree of force, and then slowly reducing the force to achieve the effect of stimulating the acupoint and dredging the meridians; the tangential force of the pushing method is dominant, the normal force is small, and the torque is small. The tip or the palm of the thumb is used to push and press on the acupoint along a certain direction or meridian running route, gradually increasing the force, allowing the force to penetrate deep into the tissue, and then slowly reducing the force and returning to the starting point after reaching a certain range.

[0089] Model testing and performance evaluation: The feature vectors of the test set are used to evaluate the performance of the model, and the recognition results are analyzed through the confusion matrix. The recognition accuracy of the seven massage methods is 97.3%, among which the recognition accuracy of the pushing method and the pressing method is 100%, and the confusion rate of the rubbing method and the rolling method is only 2%, which meets the clinical recognition requirements.

[0090] Although embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-described embodiments are merely exemplary embodiments or examples, and the scope of the present disclosure is not limited by these embodiments or examples, but is only limited by the granted claims and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by equivalent elements. In addition, the steps can be performed in an order different from that described in the present disclosure. Further, various elements in the embodiments or examples can be combined in various ways. It is important that many of the elements described herein can be replaced by equivalent elements that appear after the present disclosure as technology evolves.

Claims

1. A method for recognizing a massage technique based on a visual tactile sensor, characterized by, The method comprises the following steps: S1: sensor calibration and data set construction: fix the calibrated ATINano17 six-axis force sensor on the base (6), press the elastic skin (1) by different massage methods, and synchronously collect 32000 groups of tactile images and corresponding six-axis force / torque data, and divide them into training set and test set in the ratio of 6:4; Collect 500 images of each of the five force application positions of the thumb tip, finger end, thread surface, partial fin, and joint, and divide them into training set, validation set, and test set in the ratio of 6:3:1; S2: force application position recognition model training: set the batch size of the YOLOv8 classification model to 16, the training round to 50, uniformly adjust the size of the force application position image, input the model for training, and obtain the force application position recognition model; S3: force / torque estimation model training: use the modified VGG16 network, retain 13 convolutional layers and 3 fully connected layers, replace the network output layer with 6 neurons to correspond to the forces of X, Y, and Z axes and torque; Use the SGD optimization algorithm, StepLR learning rate scheduling, and MSE loss function to fine-tune the model based on ImageNet pre-training, and obtain the force / torque estimation model after 100 rounds of training; S4: massage method feature extraction: use the feature extraction module to extract the time domain features, frequency domain features, and combined force / torque features from the force / torque time series data output by the force / torque estimation model, and construct a feature vector; S5: massage method recognition: input the feature vector into the random forest classifier, set the number of decision trees to 200 and the maximum depth to 10, train the model, and realize recognition of massage, rolling, rubbing, kneading, pushing, and striking; The method uses a massage method measurement device based on a visual tactile sensor for recognition, and the measurement device comprises: A visual tactile sensor body composed of an elastic skin (1), a transparent acrylic support plate (2), an RGB three-color LED lighting unit (3), a black resin shell (4), a miniature camera (5), and a base (6); The elastic skin (1) has a transparent gel layer (7), a black circular marker point layer (8), a white backlight layer (9), and a black protective layer (10) stacked in order; The black resin shell (4) is internally provided with a card slot for fixing the RGB three-color LED lighting unit (3), the transparent acrylic support plate (2), and the miniature camera (5); The miniature camera (5) is installed above the base (6) and is used to capture the deformation image of the black circular marker point after the elastic skin is pressed; A data acquisition module connected with the miniature camera (5) and the calibrated six-axis force sensor respectively, which synchronously collects tactile image data and six-axis force / torque true value data; It also includes a data acquisition module, a data preprocessing module, a feature extraction module, and a method recognition module; The data preprocessing module is used for background normalization and noise removal processing of the tactile image collected by the miniature camera (5), converts the color image into a black and white image with white background and black marker points, and simultaneously performs dimension unification on the six-dimensional force / torque data to eliminate the scale difference of different dimensions of data. The feature extraction module is used for extracting time domain features, frequency domain features and resultant force / torque features of the six-dimensional force / torque data. The time domain features include mean value, maximum value, minimum value, standard deviation, skewness and kurtosis. The frequency domain features include main frequency and frequency spectrum entropy. The resultant force / torque features include average resultant force, maximum resultant force, resultant force fluctuation, average resultant torque, maximum resultant torque, torque fluctuation and force-torque correlation. The hand method recognition module includes a force application position recognition sub-module and a massage method recognition sub-module. The force application position recognition sub-module adopts a pre-trained YOLOv8 classification model. The massage method recognition sub-module adopts a random forest classifier.

2. The recognition method according to claim 1, wherein in S4, the feature vectors of different massage methods are visually analyzed by radar charts to determine that the Tz-axis torque standard deviation, the resultant torque mean value and the Ty-axis torque standard deviation are key features for distinguishing different massage methods; wherein the normalized Tz-axis torque standard deviation of the pushing method ranges from 0.08 to 0.15, and the normalized Tz-axis torque standard deviation of the kneading method ranges from 0.25 to 0.

40.

3. The recognition method according to claim 1, wherein in S1, the measurement range of the ATI Nano17 six-dimensional force sensor is: the X-axis force Fx and the Y-axis force Fy are both [-4 N, 4 N], the Z-axis force Fz is [0, -14 N], and the X-axis torque Tx, the Y-axis torque Ty and the Z-axis torque Tz are all [-200 Nmm, 200 Nmm].

4. The recognition method according to claim 1, wherein the transparent gel layer (7) of the elastic skin (1) is made of C2H6OSi, has a temperature resistance range of -30°C to 200°C, the white backlight layer (9) is made of diamond polishing powder sprayed, and the black circular marker point layer (8) is made of iron powder sprayed and uniformly distributed on the surface of the transparent gel layer (7); and / or the size of the marker point is 2mm x 2mm.

5. The recognition method according to claim 1, wherein the RGB three-color LED lighting unit (3) is installed on the side edge of the black resin shell (4), the light is mixed through the transparent gel layer (7) and then uniformly projected into the inside of the elastic skin (1), and the black resin shell (4) is made of light shielding material. ​ ​ ​ ​ ​ ​

Images