A massage teaching method and system based on multi-modal data collaboration

Abstract

The present application relates to a kind of based on multimodal data cooperation's massage teaching method and system, belong to massage teaching technical field, wherein, the method gathers the pressure distribution data and kinematics data in the process of student massage, to obtain pressure parameter and including palm overall motion parameter and fingertip local motion parameter Motion parameter, through the action correlation model of pre-training, the action coordination palm overall motion parameter and the action coordination fingertip local motion parameter are screened out, it is fused with pressure parameter, obtains including palm and the action synchronization rate of fingertip and the force speed synchronization rate of pressure and speed Dual-mode fusion feature.According to dual-mode fusion feature, pressure parameter and action coordination palm overall motion parameter and action coordination fingertip local motion parameter obtain current massage technique type, to call matched standard massage parameter to calculate the difference degree generation current massage score and individualized massage teaching guidance suggestion.

Description

Technical Field

[0001] This invention relates to the field of massage teaching technology, and in particular to a massage teaching method and system based on multimodal data collaboration. Background Technology

[0002] The current teaching of massage techniques has the following two major problems:

[0003] (i) The training mainly relies on oral instruction from instructors, lacking quantitative standards for key parameters of massage techniques. Trainees find it difficult to accurately perceive and replicate standard techniques. At the same time, instructors can only evaluate trainees' operations through visual observation. The evaluation is highly subjective and has large individual differences, making it impossible to form an objective and unified assessment basis, resulting in uneven training effects among trainees.

[0004] (ii) The few existing auxiliary teaching devices mostly use a single type of sensor, which cannot fully capture the multi-dimensional characteristics of massage techniques, make it difficult to analyze the defects in students' operation from multiple dimensions, and cannot provide students with accurate error correction guidance, resulting in low teaching efficiency.

[0005] To address the aforementioned challenges in teaching massage techniques, there is an urgent need for a teaching technology that can collaboratively collect multimodal parameters of massage techniques, objectively quantify and evaluate them, and provide precise guidance. Summary of the Invention

[0006] The technical problem to be solved by this invention is: This invention provides a massage teaching method and system based on multimodal data collaboration, which realizes the collaborative acquisition of multimodal parameters of massage techniques, objectively and quantitatively evaluates and accurately guides students' operations, and improves teaching efficiency.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0008] In a first aspect, the present invention provides a massage teaching method based on multimodal data collaboration, comprising:

[0009] The system collects pressure distribution data and kinematic data during the massage process in real time. The pressure distribution data and kinematic data are preprocessed to obtain preprocessed pressure distribution data and preprocessed kinematic data, respectively. Pressure features are extracted from the preprocessed pressure distribution data to obtain pressure parameters. Simultaneously, motion features are extracted from the preprocessed kinematic data to obtain motion parameters. The kinematic data includes overall palm motion data and fingertip local motion data. The pressure parameters include overall palm pressure parameters and fingertip local pressure parameters. The motion parameters include overall palm motion parameters and fingertip local motion parameters.

[0010] Obtain the current acquisition timestamp, and slice the overall hand motion parameters and the local fingertip motion parameters according to the current acquisition timestamp to obtain the overall hand motion parameters and local fingertip motion parameters under all time slices. Input the overall hand motion parameters and local fingertip motion parameters under each time slice into the pre-trained motion association model to calculate the motion association strength to obtain the corresponding motion association strength under all time slices. Filter out the coordinated overall hand motion parameters and coordinated local fingertip motion parameters according to the corresponding motion association strength under all time slices.

[0011] The overall motion parameters of the hand and the local motion parameters of the fingertips are fused with the pressure parameters to obtain dual-mode fusion features. The dual-mode fusion features include the motion synchronization rate of the hand and fingertips and the force-velocity synchronization rate of pressure and speed.

[0012] The dual-mode fusion features, pressure parameters, overall hand motion parameters, and fingertip motion parameters are used as input to a pre-trained massage technique recognition model to identify the massage technique. This process yields the current massage technique type. Based on this type, matching standard massage parameters are retrieved from a pre-built standard technique library. The difference between the standard massage parameters and the current massage parameters is calculated. A current massage score is generated based on the difference score, and personalized massage teaching guidance suggestions are generated based on the current massage score.

[0013] The beneficial effects of this invention are as follows: It simultaneously and in real-time acquires dual-modal pressure distribution data and kinematic data, overcoming the limitations of traditional single-modal acquisition. Simultaneously, it extracts corresponding pressure and motion features from the pressure distribution and kinematic data. The obtained pressure parameters include overall palm pressure parameters and fingertip local pressure parameters, while the motion parameters include overall palm motion parameters and fingertip local motion parameters. By decomposing the pressure and motion parameters from both the overall palm and fingertip dimensions, the accuracy and comprehensiveness of the obtained pressure and motion parameters are improved. Furthermore, the overall palm motion parameters and fingertip local motion parameters are time-sliced, and the overall palm motion parameters and fingertip local motion parameters from each time slice are input one by one into a pre-trained motion association model to calculate the motion association strength. This improves the accuracy of the calculated motion association strength, thereby ensuring the accuracy of the selected coordinated overall palm motion parameters and coordinated fingertip local motion parameters. When recognizing massage techniques, this method not only considers the overall hand motion parameters and fingertip local motion parameters, but also the dual-mode fusion features obtained by fusing these parameters, such as the synchronization rate of hand and fingertip movements and the force-velocity synchronization rate of pressure and speed. This overcomes the one-sidedness of focusing only on a single mode and failing to reflect the synergistic relationship between the two modes, thus improving the accuracy of massage technique recognition. This, in turn, ensures the accuracy of the generated current massage score, enabling objective quantitative evaluation and precise guidance for students' operations, thereby improving teaching efficiency.

[0014] Optionally, the preprocessing includes noise reduction processing. The pressure distribution data and the kinematic data are preprocessed separately to obtain preprocessed pressure distribution data and preprocessed kinematic data. Pressure features are extracted from the preprocessed pressure distribution data to obtain pressure parameters. Simultaneously, motion features are extracted from the preprocessed kinematic data to obtain motion parameters, including:

[0015] The pressure distribution data is denoised by median filtering and Gaussian smoothing respectively to obtain denoised pressure distribution data. Based on the preset hand contact area division rules, the denoised pressure distribution data is divided into palm area pressure distribution data and fingertip local pressure distribution data.

[0016] The pressure distribution data of the palm region and the local pressure distribution data of the fingertips were converted into two-dimensional pressure heat maps using bilinear interpolation, resulting in a two-dimensional pressure heat map of the entire palm and a two-dimensional pressure distribution map of the local fingertips. Then, voxel reconstruction technology was used to reconstruct the two-dimensional pressure heat maps of the entire palm and the local pressure distribution maps of the local fingertips into three-dimensional pressure heat maps, resulting in a three-dimensional pressure heat map of the entire palm and a three-dimensional pressure heat map of the local fingertips. Pressure features were extracted from the three-dimensional pressure heat maps of the entire palm and the local pressure heat maps of the local fingertips to obtain the overall pressure parameters of the palm and the local pressure parameters of the fingertips.

[0017] The overall hand motion data is denoised using Kalman filtering to obtain denoised overall hand motion data. Simultaneously, the local fingertip motion data is denoised using particle filtering to obtain denoised local fingertip motion data. Motion features are extracted from both the denoised overall hand motion data and the denoised local fingertip motion data to obtain the corresponding overall hand motion parameters and local fingertip motion parameters.

[0018] As described above, median filtering and Gaussian smoothing were used to denoise the pressure distribution data, effectively removing environmental interference during the acquisition process and improving the quality of the pressure distribution data. The denoised pressure distribution data was then divided into palm region pressure distribution data and fingertip local pressure distribution data, and pressure features were extracted using the corresponding 3D pressure heatmap, accurately capturing the pressure parameters of the entire palm and the fingertips. For the kinematic data, Kalman filtering, adapted for smooth motion, was used to denoise the overall palm motion data, improving the smoothness of the overall palm motion trajectory. Particle filtering, adapted for fine motion, was used to denoise the fingertip local motion data, effectively preserving minute details and improving the quality of the denoised fingertip local motion data, thus ensuring the accuracy of the obtained overall palm motion parameters and fingertip local motion parameters.

[0019] Optionally, the overall hand motion parameters and the local fingertip motion parameters are time-sliced ​​according to the current acquisition timestamp to obtain the overall hand motion parameters and local fingertip motion parameters for all time slices. The overall hand motion parameters and local fingertip motion parameters for each time slice are then input one by one into a pre-trained action association model to calculate the action association strength, resulting in the action association strength for all time slices, including:

[0020] Using the motion cycle of the overall hand movement corresponding to the preprocessed overall hand movement data as the time reference, the overall hand movement is divided into overall stages to obtain the overall stage time sequence. Based on the fingertip local movement parameters, the fingertip local movement corresponding to the preprocessed fingertip local movement data is divided into local stages to obtain the local stage time sequence. The local stage time sequence is associated and mapped with the overall stage time sequence to construct the stage association mapping rule between the overall hand movement and the fingertip local movement.

[0021] Based on the stage association mapping rule and the current acquisition timestamp, the overall palm motion parameters and the local fingertip motion parameters are sliced ​​according to a preset time slice to obtain the overall palm motion parameters and local fingertip motion parameters under all time slices.

[0022] A three-dimensional coordinate system for the hand is established with the center coordinate of the back of the hand in the overall hand motion parameters paired in each time slice as the origin coordinate. Based on the three-dimensional coordinate system, the fingertip coordinates in the corresponding fingertip local motion parameters are converted into relative coordinates to achieve spatial calibration of the overall hand motion parameters and fingertip local motion parameters paired in each time slice, so as to obtain the spatially calibrated overall hand motion parameters and fingertip local motion parameters paired in all time slices.

[0023] The overall hand motion parameters and local fingertip motion parameters paired under all time slices after spatial calibration are input one by one into the pre-trained motion association model to calculate the motion association strength, thus obtaining the corresponding motion association strength under all time slices.

[0024] As described above, when performing time-slicing of the overall palm motion parameters and the local fingertip motion parameters, a stage-related mapping rule constructed from the overall palm motion and the local fingertip motion is introduced. This avoids stage mismatches caused by simply slicing based on the current acquisition timestamp, ensuring the rationality of the pairing of overall palm motion parameters and local fingertip motion parameters in each time slice. By establishing a three-dimensional coordinate system for the hand, spatial calibration of the paired overall palm motion parameters and local fingertip motion parameters in each time slice can be achieved, effectively eliminating the influence of differences in hand size among different trainees on the motion parameters and improving the accuracy and objectivity of the calculated motion correlation strength.

[0025] Optionally, the step of inputting the overall hand motion parameters and fingertip local motion parameters paired under all time slices after spatial alignment into the pre-trained motion association model to calculate the motion association strength, and obtaining the corresponding motion association strength under all time slices, includes:

[0026] The overall hand motion parameters and fingertip local motion parameters paired under all time slices after spatial calibration are input one by one into the pre-trained motion association model. The constraint deviation of the overall hand motion parameters and fingertip local motion parameters paired under each time slice is calculated by the four types of preset constraints built into the motion association model, so as to obtain the four types of constraint deviations corresponding to all time slices. The four types of preset constraints include displacement constraints, velocity constraints, acceleration constraints and posture constraints. The four types of constraint deviations include displacement constraint deviation, velocity constraint deviation, acceleration constraint deviation and posture constraint deviation.

[0027] The displacement constraint condition is as follows:

[0028] ;

[0029] in, Indicates displacement constraint deviation. This represents the angle between the tangent directions of the overall hand motion trajectory in the paired overall hand motion parameters at time slice i. This represents the angle between the tangent directions of the fingertip local motion trajectory in the paired fingertip local motion parameters under time slice i. Indicates the first threshold;

[0030] The speed constraint condition is:

[0031] ;

[0032] ;

[0033] in, This represents the overall hand velocity in the overall hand motion parameters paired at time slice i. This represents the local velocity of the fingertip in the paired local motion parameters for time slice i. Indicates speed constraint deviation. Indicates the first coefficient. Indicates the second coefficient;

[0034] The acceleration constraint condition is as follows:

[0035] ;

[0036] in, Indicates acceleration constraint deviation. This represents the peak value of the overall hand acceleration in the overall hand motion parameters paired at time slice i. This represents the peak value of the local fingertip acceleration in the paired local fingertip motion parameters at time slice i. Indicates the second threshold;

[0037] The attitude constraint is as follows:

[0038] ;

[0039] ;

[0040] in, This represents the rate of change of the overall hand posture angle in the overall hand motion parameters paired at time slice i. This represents the rate of change of the local fingertip posture angle among the paired local fingertip motion parameters at time slice i. This represents the third threshold. This represents the fourth threshold. Indicates attitude constraint deviation;

[0041] Substitute the four types of constraint deviations corresponding to all time slices into the weighting formula to calculate the motion association strength, obtaining the motion association strength corresponding to all time slices. The weighting formula is as follows:

[0042] ;

[0043] in, Indicates the action correlation strength under time slice i. Indicates the first weight. Indicates the second weight. Indicates the third weight. This indicates the fourth weight.

[0044] As described above, the constraint deviations between the overall hand motion parameters and the local fingertip motion parameters in each time slice under each constraint condition are calculated using the four types of preset constraints built into the motion association model. That is, the motion association strength between the overall hand motion and the local fingertip motion is calculated from four aspects: displacement constraint, velocity constraint, acceleration constraint, and posture constraint. This solves the problem that a single dimension cannot fully reflect motion coordination. Furthermore, by quantifying various constraint deviations through explicit mathematical formulas, subjective judgment is replaced, making the calculation of motion association strength more objective and scientific.

[0045] Optionally, the process of fusing the overall hand motion parameters and the local fingertip motion parameters with the pressure parameters to obtain dual-mode fusion features includes:

[0046] Calculate the time difference between the overall hand motion parameters and the local fingertip motion parameters in the same time slice at the preset key nodes to obtain the time difference for each time slice. Determine whether the time difference for each time slice is lower than the first threshold. If so, the time slice with the time difference lower than the first threshold is taken as the candidate time slice.

[0047] Calculate the ratio of the overall pressure change rate in the overall palm pressure parameter to the local pressure change rate in the local fingertip pressure parameter under each candidate time slice to obtain the ratio corresponding to each candidate time slice. Determine whether the ratio corresponding to each candidate time slice is within the first preset interval. If not, filter out the candidate time slices whose ratio is not within the first preset interval to obtain the effective time slice.

[0048] The overall hand motion parameters and local fingertip motion parameters under the effective time slice are fused with the pressure parameters to obtain dual-mode fusion features.

[0049] As described above, by first filtering candidate time slices through the time difference of preset key nodes, and then filtering effective time slices through the ratio of the overall pressure change rate to the local pressure change rate, motion parameters with insufficient motion coordination can be accurately eliminated. This not only reduces the amount of data and improves the efficiency of fusion processing, but also allows the focus to be on motion parameters under effective time slices during fusion processing, thereby improving the reliability of dual-mode fusion features.

[0050] Optionally, the process of fusing the overall hand motion parameters and the local fingertip motion parameters under the effective time slice with the pressure parameters to obtain dual-mode fusion features includes:

[0051] Based on the overall palm motion parameters and the local fingertip motion parameters under the effective time slice, a preliminary technique type is identified to obtain a preliminary technique type. Based on the preliminary technique type, a matching standard pressure distribution entropy is called from a pre-built standard technique library. The overall palm pressure distribution entropy in the overall palm pressure parameters and the local fingertip pressure distribution entropy in the local fingertip pressure parameters are integrated to obtain the current pressure distribution entropy.

[0052] The current pressure distribution entropy, the standard pressure distribution entropy, and the effective time slice are input into the first synchronization rate formula to calculate the synchronization rate of hand and fingertip movements. The first synchronization rate formula is as follows:

[0053] ;

[0054] ;

[0055] in, This indicates the synchronization rate between the movements of the palm and fingertips. This represents the pressure distribution entropy correction coefficient. This represents the current pressure distribution entropy. Represents the standard pressure distribution entropy. Indicates the number of valid time slices. Indicates the total number of time slices;

[0056] The average pressure value of the entire palm from the overall palm pressure parameters and the average pressure value of the fingertip from the local fingertip pressure parameters are integrated to obtain the current average pressure value sequence. Based on the preliminary technique type, a matching standard speed sequence is retrieved from a pre-built standard technique library. The standard speed sequence and the current average pressure value sequence are input into a second synchronization rate formula for calculation to obtain the force-velocity synchronization rate of pressure and speed. The second synchronization rate formula is:

[0057] ;

[0058] in, The force-velocity synchronization rate, representing pressure and velocity. This represents the Pearson correlation coefficient between the current average pressure series and the standard velocity series. This represents the current average pressure value sequence. Represents a standard velocity sequence. This represents the dynamic time-warped distance between the current average pressure value sequence and the standard velocity sequence. This represents the largest dynamic time warp distance in the standard technique library.

[0059] As described above, preliminary technique type identification is performed based on the overall hand motion parameters and the local fingertip motion parameters within the effective time slice. This allows for the retrieval of matching standard pressure distribution entropy and standard velocity sequences based on the preliminary technique type, ensuring the rationality of the baseline for calculating the synchronization rate of hand and fingertip movements, as well as the force-velocity synchronization rate of pressure and velocity. By integrating the overall hand pressure distribution entropy and the local fingertip pressure distribution entropy for calculating the synchronization rate of hand and fingertip movements, the synchronization rate reflects not only temporal coordination but also the rationality of pressure distribution, improving the accuracy of the synchronization rate calculation. In calculating the force-velocity synchronization rate of pressure and velocity, the Pearson correlation coefficient, representing linear correlation, and the dynamic time warping distance, representing temporal similarity, are used to comprehensively reflect the coordination quality of pressure and velocity, improving the scientific rigor of the force-velocity synchronization rate calculation.

[0060] Optionally, the step of inputting the dual-mode fusion features, the pressure parameters, the overall hand motion parameters coordinated by the movement, and the local fingertip motion parameters coordinated by the movement as the current massage parameters into the pre-trained massage technique recognition model to perform massage technique recognition, thereby obtaining the current massage technique type, and calling the matching standard massage parameters from the pre-built standard technique library according to the current massage technique type, and calculating the difference between the standard massage parameters and the current massage parameters includes:

[0061] The current massage parameters are normalized to the [0,1] interval using Min-Max normalization to achieve standardization of the current massage parameters and obtain the standardized current massage parameters.

[0062] The random forest algorithm is used to select N core parameters from the standardized current massage parameters, and the principal component analysis algorithm is used to reduce the dimensionality of the N core parameters to obtain the N core parameters after dimensionality reduction.

[0063] The N core parameters after dimensionality reduction are input into the pre-trained massage technique recognition model to identify the massage technique and obtain the current massage technique type. The pre-trained massage technique recognition model is built and trained based on an LSTM network.

[0064] Based on the current massage technique type, N standard massage parameters that match the N core parameters after dimensionality reduction are called from the pre-built standard technique library, and the difference between the N standard massage parameters and the N core parameters after dimensionality reduction is calculated.

[0065] As described above, Min-Max normalization is used to standardize the current massage parameters. Then, the core parameters are selected through the random forest algorithm and dimensionality is reduced by principal component analysis. This reduces the interference of redundant parameters on the massage technique recognition model and lowers the computational complexity of the model. The massage technique recognition model based on LSTM network can effectively capture the temporal dependency of the core parameters during massage and improve the accuracy of massage technique recognition.

[0066] Optionally, the difference between the calculated N standard massage parameters and the N core parameters after dimensionality reduction includes:

[0067] Obtain the parameter type for each core parameter. When the parameter type is numerical, use the difference formula to calculate the difference between the core parameter and the corresponding standard massage parameter.

[0068] When the parameter type is directional, the difference between the core parameter and the corresponding standard massage parameter is calculated using the vector angle cosine theorem.

[0069] When the parameter type is time series, determine whether the time series is a periodic uniform time series. If yes, use the coefficient of variation method to calculate the difference between the core parameter and the corresponding standard massage parameter. If no, use the waveform similarity algorithm to calculate the difference between the core parameter and the corresponding standard massage parameter.

[0070] When the parameter type is stable, the difference between the core parameter and the corresponding standard massage parameter is calculated using the standard deviation formula.

[0071] As described above, the difference is calculated using a type-adaptive method for different types of core parameters to ensure the scientific validity and accuracy of the calculated difference. Furthermore, the categorized calculation facilitates the subsequent identification of core parameters with differences, providing support for personalized guidance.

[0072] Secondly, the present invention provides a massage teaching system based on multimodal data collaboration, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the massage teaching method based on multimodal data collaboration described in the first aspect.

[0073] The technical effects of the massage teaching system based on multimodal data collaboration provided in the second aspect are the same as those of the massage teaching method based on multimodal data collaboration provided in the first aspect. Attached Figure Description

[0074] Figure 1 A flowchart illustrating a massage teaching method based on multimodal data collaboration provided in this embodiment;

[0075] Figure 2 This is a schematic diagram of the overall process of a massage teaching method based on multimodal data collaboration provided in this embodiment;

[0076] Figure 3 This is a schematic diagram of the structure of a massage teaching system based on multimodal data collaboration provided in this embodiment.

[0077] Explanation of reference numerals in the attached figures

[0078] 1. A massage teaching system based on multimodal data collaboration;

[0079] 2. Processor;

[0080] 3. Memory. Detailed Implementation

[0081] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0082] Example 1

[0083] Please refer to Figures 1 to 2 This invention provides a massage teaching method based on multimodal data collaboration, comprising the following steps:

[0084] S1. Real-time acquisition of pressure distribution data and kinematic data during the student's massage process; preprocessing of the pressure distribution data and kinematic data to obtain preprocessed pressure distribution data and preprocessed kinematic data; extraction of pressure features from the preprocessed pressure distribution data to obtain pressure parameters; and extraction of motion features from the preprocessed kinematic data to obtain motion parameters. The kinematic data includes overall palm motion data and fingertip local motion data; the pressure parameters include overall palm pressure parameters and fingertip local pressure parameters; and the motion parameters include overall palm motion parameters and fingertip local motion parameters.

[0085] In this embodiment, the trainee wears a smart data glove during the massage. This smart data glove integrates a flexible thin-film pressure sensor array and multiple inertial measurement units (IMUs). The flexible thin-film pressure sensor array is arranged in a biomimetic arc shape: "3 points at the fingertips + 6×8 array on the thenar eminence + 6×8 array on the hypothenar eminence," conforming to the physiological curve of the palm. The pressure detection range of a single sensor is 0-50N, with an accuracy of ±0.1N, a sampling rate ≥100Hz, and a response time ≤8ms. The IMUs adopt a "center of the palm + fingertips" layout, with one 6-axis IMU located at the center of the back of the hand and at the tips of the thumb and index finger. The sampling frequency of the central IMU is ≥100Hz, with an acceleration measurement error ≤±0.05g and an angular velocity measurement error ≤±0.1° / s. The sampling frequency of the peripheral IMUs is ≥200Hz.

[0086] like Figure 2As shown, the smart data glove can actually collect and transmit pressure distribution data and kinematic data of the trainee during the massage process. The kinematic data includes overall palm movement data and fingertip local movement data. The collected pressure distribution data and kinematic data are preprocessed to obtain preprocessed pressure distribution data and preprocessed kinematic data, respectively. Preprocessing includes, but is not limited to, noise reduction and filtering. Pressure features are extracted from the preprocessed pressure distribution data to obtain pressure parameters, and motion features are extracted from the preprocessed kinematic data to obtain motion parameters. The pressure parameters include overall palm pressure parameters and fingertip local pressure parameters, and the motion parameters include overall palm movement parameters and fingertip local movement parameters.

[0087] At this point, the preprocessing in step S1 includes noise reduction processing. The pressure distribution data and the kinematic data are preprocessed separately to obtain preprocessed pressure distribution data and preprocessed kinematic data. Pressure features are extracted from the preprocessed pressure distribution data to obtain pressure parameters. Simultaneously, motion features are extracted from the preprocessed kinematic data to obtain motion parameters, including:

[0088] S11. The pressure distribution data is denoised by median filtering and Gaussian smoothing respectively to obtain denoised pressure distribution data. Based on the preset hand contact area division rule, the denoised pressure distribution data is divided into palm area pressure distribution data and fingertip local pressure distribution data.

[0089] S12. The pressure distribution data of the palm region and the local pressure distribution data of the fingertips are converted into two-dimensional pressure heat maps by bilinear interpolation, respectively, to obtain a two-dimensional pressure heat map of the whole palm and a two-dimensional pressure heat map of the local fingertips. Then, the voxel reconstruction technology is used to reconstruct the two-dimensional pressure heat map of the whole palm and the two-dimensional pressure distribution map of the local fingertips into three-dimensional pressure heat maps, respectively, to obtain a three-dimensional pressure heat map of the whole palm and a three-dimensional pressure heat map of the local fingertips. Pressure features are extracted from the three-dimensional pressure heat map of the whole palm and the three-dimensional pressure heat map of the local fingertips to obtain the overall pressure parameters of the palm and the local pressure parameters of the fingertips.

[0090] S13. The overall hand motion data is denoised using Kalman filtering to obtain denoised overall hand motion data. Simultaneously, the local fingertip motion data is denoised using particle filtering to obtain denoised local fingertip motion data. Motion features are extracted from the denoised overall hand motion data and the denoised local fingertip motion data to obtain the corresponding overall hand motion parameters and local fingertip motion parameters.

[0091] In this embodiment, as Figure 2As shown, a hybrid algorithm is used to perform secondary denoising on the pressure distribution data. Median filtering and Gaussian smoothing are applied to denoise the pressure distribution data, resulting in denoised pressure distribution data. Based on a preset hand contact area segmentation rule, the denoised pressure distribution data is divided into palm region pressure distribution data and fingertip local pressure distribution data. Specifically, the preset hand contact area segmentation rule is to divide the data into regions of interest (RIOs) based on the palm and fingertip contact areas. Bilinear interpolation is used to convert the palm region pressure distribution data and fingertip local pressure distribution data into two-dimensional pressure heatmaps. Voxel reconstruction technology is then used to reconstruct the overall palm two-dimensional pressure heatmap and the fingertip local two-dimensional pressure heatmap into overall palm three-dimensional pressure heatmap and fingertip local three-dimensional pressure heatmap, respectively. Pressure features are extracted from both the overall palm three-dimensional pressure heatmap and the fingertip local three-dimensional pressure heatmap to obtain overall palm pressure parameters and fingertip local pressure parameters. The overall palm pressure parameters include, but are not limited to: peak value of overall palm pressure, average overall palm pressure, entropy of overall palm pressure distribution, rise time of overall palm pressure, fall time of overall palm pressure, area of ​​overall palm pressure, and gradient of overall palm pressure. The local fingertip pressure parameters include, but are not limited to: peak value of local fingertip pressure, average local fingertip pressure, entropy of local fingertip pressure distribution, rise time of local fingertip pressure, fall time of local fingertip pressure, area of ​​local fingertip pressure, and gradient of local fingertip pressure.

[0092] Different algorithms were used to denoise different kinematic data. Kalman filtering was used to denoise the overall palm motion data, while particle filtering was used to denoise the fingertip local motion data, resulting in denoised overall palm motion data and fingertip local motion data. Motion feature extraction was then performed on both denoised data to obtain the corresponding overall palm motion parameters and fingertip local motion parameters. The overall palm motion parameters include, but are not limited to: the trajectory of the overall palm motion, the curvature of the trajectory, the peak velocity, the peak angular velocity, the amplitude of the attitude angle change, the motion period, the peak acceleration, the trajectory length, and the motion uniformity. The fingertip local motion parameters include, but are not limited to: the trajectory of the fingertip local motion, the curvature of the trajectory, the peak velocity, the peak angular velocity, the amplitude of the attitude angle change, the motion period, the peak acceleration, the trajectory length, and the motion uniformity. The trajectory, velocity, and acceleration parameters were calculated using quaternion integration.

[0093] S2. Obtain the current acquisition timestamp, and perform time slices on the overall hand motion parameters and the local fingertip motion parameters according to the current acquisition timestamp to obtain the overall hand motion parameters and local fingertip motion parameters under all time slices. Input the overall hand motion parameters and local fingertip motion parameters under each time slice into the pre-trained motion association model to calculate the motion association strength, and obtain the corresponding motion association strength under all time slices. Filter out the coordinated overall hand motion parameters and coordinated local fingertip motion parameters according to the corresponding motion association strength under all time slices.

[0094] At this point, in step S2, the overall hand motion parameters and the local fingertip motion parameters are sliced ​​according to the current acquisition timestamp to obtain the overall hand motion parameters and local fingertip motion parameters for all time slices. The overall hand motion parameters and local fingertip motion parameters for each time slice are then input into the pre-trained motion association model to calculate the motion association strength, resulting in the motion association strength for all time slices, including:

[0095] S21. Using the motion cycle of the overall palm motion corresponding to the preprocessed overall palm motion data as the time reference, the overall palm motion is divided into overall stages to obtain the overall stage time sequence. Based on the fingertip local motion parameters, the fingertip local motion corresponding to the preprocessed fingertip local motion data is divided into local stages to obtain the local stage time sequence. The local stage time sequence is associated and mapped with the overall stage time sequence to construct the stage association mapping rule between the overall palm motion and the fingertip local motion.

[0096] In this embodiment, as Figure 2 As shown, using the motion cycle of the preprocessed overall hand motion data as the time base, the overall hand motion is divided into stages, such as the pressure application stage, the maintenance stage, and the decompression stage, resulting in an overall stage time sequence. Based on the fingertip local motion parameters, the preprocessed fingertip local motion data is divided into local stages, such as the inflection points of the motion trajectory and the abrupt changes in velocity, resulting in the contact sub-stage, the maintenance sub-stage, and the ion withdrawal stage, resulting in a local stage time sequence. The local stage time sequences are then correlated and mapped with the overall stage time sequences, such as the pressure application stage corresponding to the contact sub-stage, the maintenance stage corresponding to the maintenance sub-stage, and the decompression stage corresponding to the ion withdrawal stage, to construct a stage correlation mapping rule between the overall hand motion and the fingertip local motion.

[0097] S22. According to the stage association mapping rule and the current collection timestamp, the overall palm motion parameters and the local fingertip motion parameters are time-sliced ​​according to a preset time slice to obtain the overall palm motion parameters and local fingertip motion parameters under all time slices.

[0098] S23. Establish a three-dimensional coordinate system for the hand with the center coordinate of the back of the hand in the overall hand motion parameters paired in each time slice as the origin coordinate. Based on the three-dimensional coordinate system of the hand, convert the fingertip coordinates in the corresponding fingertip local motion parameters into relative coordinates to achieve spatial calibration of the overall hand motion parameters and fingertip local motion parameters paired in each time slice, and obtain the spatially calibrated overall hand motion parameters and fingertip local motion parameters paired in all time slices.

[0099] S24. Input the overall hand motion parameters and fingertip local motion parameters of all time slices after spatial calibration into the pre-trained motion association model to calculate the motion association strength, and obtain the corresponding motion association strength for all time slices.

[0100] In this embodiment, as Figure 2 As shown, based on the stage association mapping rules constructed in step S21 and the current acquisition timestamp, the overall palm motion parameters and fingertip local motion parameters are time-sliced ​​according to preset time slices to obtain the overall palm motion parameters and fingertip local motion parameters under all time slices. That is, time synchronization of the overall palm motion parameters and fingertip local motion parameters is achieved. A three-dimensional coordinate system for the hand is established with the center coordinate of the back of the hand in the paired overall palm motion parameters under each time slice as the origin coordinate. Based on the three-dimensional coordinate system of the hand, the fingertip coordinates in the corresponding fingertip local motion parameters are converted into relative coordinates to achieve spatial calibration of the paired overall palm motion parameters and fingertip local motion parameters under each time slice, obtaining the spatially calibrated overall palm motion parameters and fingertip local motion parameters under all time slices. That is, spatial calibration of the overall palm motion parameters and fingertip local motion parameters is achieved. The spatially calibrated overall palm motion parameters and fingertip local motion parameters under all time slices are then input one by one into the pre-trained action association model to calculate the action association strength, obtaining the corresponding action association strength under all time slices.

[0101] At this point, step S24 includes:

[0102] S241. Input the overall hand motion parameters and fingertip local motion parameters paired under all time slices after spatial calibration into the pre-trained motion association model one by one. Calculate the constraint deviation of the overall hand motion parameters and fingertip local motion parameters paired under each time slice under each type of constraint condition using the four types of preset constraints built into the motion association model, so as to obtain the four types of constraint deviations corresponding to all time slices. The four types of preset constraints include displacement constraint conditions, velocity constraint conditions, acceleration constraint conditions, and posture constraint conditions. The four types of constraint deviations include displacement constraint deviation, velocity constraint deviation, acceleration constraint deviation, and posture constraint deviation.

[0103] The displacement constraint condition is as follows:

[0104] ;

[0105] in, Indicates displacement constraint deviation. This represents the angle between the tangent directions of the overall hand motion trajectory in the paired overall hand motion parameters at time slice i. This represents the angle between the tangent directions of the fingertip local motion trajectory in the paired fingertip local motion parameters under time slice i. Indicates the first threshold;

[0106] The speed constraint condition is:

[0107] ;

[0108] ;

[0109] in, This represents the overall hand velocity in the overall hand motion parameters paired at time slice i. This represents the local velocity of the fingertip in the paired local motion parameters for time slice i. Indicates speed constraint deviation. Indicates the first coefficient. Indicates the second coefficient;

[0110] The acceleration constraint condition is as follows:

[0111] ;

[0112] in, Indicates acceleration constraint deviation. This represents the peak value of the overall hand acceleration in the overall hand motion parameters paired at time slice i. This represents the peak value of the local fingertip acceleration in the paired local fingertip motion parameters at time slice i. Indicates the second threshold;

[0113] The attitude constraint is as follows:

[0114] ;

[0115] ;

[0116] in, This represents the rate of change of the overall hand posture angle in the overall hand motion parameters paired at time slice i. This represents the rate of change of the local fingertip posture angle among the paired local fingertip motion parameters at time slice i. This represents the third threshold. This represents the fourth threshold. Indicates attitude constraint deviation;

[0117] S242. Substitute the four types of constraint deviations corresponding to all time slices into the weighting formula to calculate the motion association strength, obtaining the motion association strength corresponding to all time slices. The weighting formula is as follows:

[0118] ;

[0119] in, Indicates the action correlation strength under time slice i. Indicates the first weight. Indicates the second weight. Indicates the third weight. This indicates the fourth weight.

[0120] In this embodiment, as Figure 2As shown, the overall hand motion parameters and fingertip local motion parameters paired under all time slices after spatial calibration are input one by one into the pre-trained motion association model. The motion association model has four types of preset constraints: displacement constraints, velocity constraints, acceleration constraints, and posture constraints. The constraint deviations of the overall hand motion parameters and fingertip local motion parameters paired under each type of constraint are calculated for each time slice to obtain the four types of constraint deviations for all time slices: displacement constraint deviation, velocity constraint deviation, acceleration constraint deviation, and posture constraint deviation. The four types of constraint deviations for all time slices are then substituted into the weighted formula to calculate the motion association strength for all time slices. The sum of the first, second, third, and fourth weights in the weighted formula is 1. Each weight is obtained by training the motion association model using standard operation data from multiple sets of expert massage procedures. Furthermore, when subsequently filtering out the overall hand motion parameters and coordinated fingertip local motion parameters based on the motion correlation strength across all time slices, the process involves comparing the motion correlation strength for each time slice with a correlation threshold. Time slices with motion correlation strengths below the threshold are filtered out, meaning that time slices with motion correlation strengths above the threshold are considered to have coordinated overall hand motion parameters and coordinated fingertip local motion parameters. The correlation threshold is set at 0.8, but can be adjusted based on specific circumstances.

[0121] S3. The overall motion parameters of the hand and the local motion parameters of the fingertips are fused with the pressure parameters to obtain dual-mode fusion features. The dual-mode fusion features include the motion synchronization rate of the hand and fingertips and the force-velocity synchronization rate of pressure and speed.

[0122] At this point, the process of fusing the overall hand motion parameters and the local fingertip motion parameters with the pressure parameters in step S3 to obtain the dual-mode fusion features includes:

[0123] S31. Calculate the time difference between the overall motion parameters of the palm and the local motion parameters of the fingertips in the same time slice at the preset key nodes, obtain the time difference corresponding to each time slice, and determine whether the time difference corresponding to each time slice is lower than the first threshold. If so, the time slice corresponding to the time difference lower than the first threshold is taken as the candidate time slice.

[0124] S32. Calculate the ratio of the overall pressure change rate in the overall palm pressure parameter to the local pressure change rate in the local fingertip pressure parameter under each candidate time slice, obtain the ratio corresponding to each candidate time slice, determine whether the ratio corresponding to each candidate time slice is within the first preset interval, if not, filter out the candidate time slices whose ratio is not within the first preset interval to obtain the effective time slices.

[0125] S33. The overall hand motion parameters and the local fingertip motion parameters under the effective time slice are fused with the pressure parameters to obtain dual-mode fusion features.

[0126] In this embodiment, as Figure 2 As shown, the time difference between the overall hand motion parameters and the local fingertip motion parameters in the same time slice is calculated at preset key nodes. These preset key nodes include: the pressure initiation time, pressure peak time, and pressure release initiation time of the overall hand motion; and the contact time, pressure peak time, and withdrawal time of the local fingertip motion. This yields the time difference for each time slice. Time slices with time differences lower than a first threshold are designated as candidate time slices, thus further synchronizing the time. The ratio of the overall pressure change rate in the overall hand motion parameters to the local pressure change rate in the local fingertip pressure parameters is calculated for each candidate time slice. Candidate time slices with ratios not falling within the first preset interval are filtered out, achieving intensity synchronization. The first preset interval is [0.9, 1.1], and this can be adjusted based on actual conditions. The overall hand motion parameters and local fingertip motion parameters in all effective time slices are fused with the pressure parameters to obtain dual-mode fusion features.

[0127] At this point, step S33 includes:

[0128] S331. Based on the overall palm motion parameters and the local fingertip motion parameters of the coordinated movement under the effective time slice, a preliminary manual technique type is identified to obtain a preliminary manual technique type. Based on the preliminary manual technique type, a matching standard pressure distribution entropy is called from the pre-built standard manual technique library, and the overall palm pressure distribution entropy in the overall palm pressure parameters and the local fingertip pressure distribution entropy in the local fingertip pressure parameters are integrated to obtain the current pressure distribution entropy.

[0129] S332. Input the current pressure distribution entropy, the standard pressure distribution entropy, and the effective time slice into the first synchronization rate formula for calculation to obtain the synchronization rate of the hand and fingertips. The first synchronization rate formula is:

[0130] ;

[0131] ;

[0132] in, This indicates the synchronization rate between the movements of the palm and fingertips. This represents the pressure distribution entropy correction coefficient. This represents the current pressure distribution entropy. Represents the standard pressure distribution entropy. Indicates the number of valid time slices. Indicates the total number of time slices;

[0133] S333. Integrate the overall palm pressure parameter with the fingertip pressure parameter to obtain the current average pressure value sequence. Based on the preliminary technique type, retrieve the matching standard speed sequence from the pre-built standard technique library. Input the standard speed sequence and the current average pressure value sequence into the second synchronization rate formula for calculation to obtain the force-speed synchronization rate of pressure and speed. The second synchronization rate formula is:

[0134] ;

[0135] in, The force-velocity synchronization rate, representing pressure and velocity. This represents the Pearson correlation coefficient between the current average pressure series and the standard velocity series. This represents the current average pressure value sequence. Represents a standard velocity sequence. This represents the dynamic time-warped distance between the current average pressure value sequence and the standard velocity sequence. This represents the largest dynamic time warp distance in the standard technique library.

[0136] In this embodiment, preliminary technique type identification is performed based on the overall palm motion parameters and the local fingertip motion parameters under the effective time slice. This is done using a pre-trained lightweight motion classification model. Based on the preliminary technique type, a matching standard pressure distribution entropy is retrieved from a pre-built standard technique library. This library stores standard massage parameters for different technique types from experts. The overall palm pressure distribution entropy and the local fingertip pressure distribution entropy are integrated to obtain the current pressure distribution entropy. The current pressure distribution entropy, the standard pressure distribution entropy, and the effective time slice are input into a first synchronization rate formula to calculate the synchronization rate between the palm and fingertips. The overall average palm pressure value and the local average fingertip pressure value are integrated to obtain the current average pressure value sequence. Similarly, based on the preliminary technique type, a matching standard speed sequence is retrieved from the pre-built standard technique library. The standard speed sequence and the current average pressure value are input into a second synchronization rate formula to calculate the pressure and appropriate force-speed synchronization rate.

[0137] S4. The dual-mode fusion features, the pressure parameters, the overall hand motion parameters of the coordinated movement, and the local fingertip motion parameters of the coordinated movement are used as the current massage parameters to input into the pre-trained massage technique recognition model to perform massage technique recognition, thereby obtaining the current massage technique type. Based on the current massage technique type, the matching standard massage parameters are called from the pre-built standard technique library. The difference between the standard massage parameters and the current massage parameters is calculated. Based on the difference, the current massage score is generated, and personalized massage teaching guidance suggestions are generated based on the current massage score.

[0138] In this embodiment, as Figure 2 As shown, the dual-mode fusion features, pressure parameters, overall hand motion parameters, and fingertip local motion parameters are used as inputs to a pre-trained massage technique recognition model to identify massage techniques. This further refines the massage technique identification to determine the current massage technique type. Based on the current massage technique type, matching standard massage parameters are retrieved from a pre-built standard technique library. The difference between the standard massage parameters and the current massage parameters is calculated, and a current massage score is generated based on this difference to further generate personalized massage teaching guidance suggestions.

[0139] At this point, step S4 involves inputting the dual-mode fusion features, the pressure parameters, the overall hand motion parameters coordinated by the movement, and the local fingertip motion parameters coordinated by the movement as the current massage parameters into the pre-trained massage technique recognition model to identify the massage technique, thereby obtaining the current massage technique type. Based on the current massage technique type, matching standard massage parameters are retrieved from the pre-built standard technique library, and the difference between the standard massage parameters and the current massage parameters is calculated, including:

[0140] S41. The current massage parameters are normalized to the [0,1] interval using Min-Max normalization to achieve the standardization of the current massage parameters and obtain the standardized current massage parameters.

[0141] S42. Use the random forest algorithm to select N core parameters from the standardized current massage parameters, and use the principal component analysis algorithm to reduce the dimensionality of the N core parameters to obtain the N core parameters after dimensionality reduction.

[0142] S43. Input the N core parameters after dimensionality reduction into the pre-trained massage technique recognition model to recognize the massage technique and obtain the current massage technique type. The pre-trained massage technique recognition model is built and trained based on LSTM network.

[0143] S44. Based on the current massage technique type, retrieve N standard massage parameters from the pre-built standard technique library that match the N core parameters after dimensionality reduction, and calculate the degree of difference between the N standard massage parameters and the N core parameters after dimensionality reduction.

[0144] In this embodiment, the current massage parameters are standardized, and a random forest algorithm is used to select N core parameters from the standardized parameters. N is 10, which can be adjusted according to the actual situation. Principal component analysis is then used to reduce the dimensionality of the N core parameters, resulting in N dimensionality-reduced core parameters. These N dimensionality-reduced core parameters are then input into a massage technique recognition model built and trained based on an LSTM network to identify the current massage technique type. Therefore, based on the current massage technique type, N standard massage parameters matching the N dimensionality-reduced core parameters are retrieved from a pre-built standard technique library, and the difference between each standard massage parameter and its corresponding dimensionality-reduced core parameter is calculated.

[0145] At this point, the calculation of the difference between the N standard massage parameters and the N core parameters after dimensionality reduction in step S44 includes:

[0146] S441. Obtain the parameter type of each core parameter. When the parameter type is numerical, use the difference formula to calculate the difference between the core parameter and the corresponding standard massage parameter.

[0147] S442. When the parameter type is directional, the difference between the core parameter and the corresponding standard massage parameter is calculated using the vector angle cosine theorem.

[0148] S443. When the parameter type is time series, determine whether the time series is a periodic uniform time series. If so, use the coefficient of variation method to calculate the difference between the core parameter and the corresponding standard massage parameter. If not, use the waveform similarity algorithm to calculate the difference between the core parameter and the corresponding standard massage parameter.

[0149] S444. When the parameter type is stable, the difference between the core parameter and the corresponding standard massage parameter is calculated using the standard deviation formula.

[0150] In this embodiment, when calculating the difference between the standard massage parameters and the N core parameters after dimensionality reduction, different difference calculation methods are selected according to the parameter type of the core parameters. When the parameter type of the core parameters is numerical, the difference formula is used to calculate the difference between the core parameters and the corresponding standard massage parameters. This difference formula can be either an absolute value difference formula or a relative value difference formula. The absolute value difference formula is as follows:

[0151] Difference = |Core Parameters - Standard Massage Parameters|;

[0152] The formula for the relative value difference is:

[0153] Difference = |Core Parameters - Standard Massage Parameters| / Standard Massage Parameters;

[0154] When the parameter type is directional, the difference between the core parameter and the corresponding standard massage parameter is calculated using the vector angle cosine theorem. The vector angle cosine theorem in this case is:

[0155] ;

[0156] in, This represents the angle between the force direction vector of the core parameter and the force direction vector of the standard massage parameter;

[0157] When the parameter type is time-series, if the time-series type specifically refers to periodic uniformity, then the coefficient of variation method is used to calculate the difference between the core parameters and the corresponding standard massage parameters. The coefficient of variation method is as follows:

[0158] ;

[0159] If the temporal sequence is not periodically uniform, a waveform similarity algorithm is used to calculate the difference between the core parameters and the corresponding standard massage parameters. The waveform similarity algorithm is as follows:

[0160] ;

[0161] If the parameter type is stable, the difference between the core parameter and the corresponding standard massage parameter is calculated using the standard deviation formula, where the standard deviation formula is:

[0162] ;

[0163] After calculating N differences, these are substituted into a weighted summation formula to generate the current massage score. Based on this score, personalized massage teaching guidance suggestions are then generated. For example, if the current massage score is below the first scoring threshold, the specific core parameters indicating differences are identified, and corresponding massage teaching guidance suggestions are retrieved from the teaching guidance suggestion library.

[0164] Example 2

[0165] Please refer to Figure 3 The present invention provides a massage teaching system 1 based on multimodal data collaboration, including a memory 3, a processor 2, and a computer program stored on the memory 3 and run on the processor 2. When the processor 2 executes the computer program, it implements the steps in Embodiment 1.

[0166] Since the systems / devices described in the above embodiments of the present invention are systems / devices used to implement the methods of the above embodiments of the present invention, those skilled in the art can understand the specific structure and modifications of the systems / devices based on the methods described in the above embodiments of the present invention, and therefore will not be repeated here. All systems / devices used in the methods of the above embodiments of the present invention fall within the scope of protection of the present invention.

[0167] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0168] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0169] It should be noted that any reference numerals placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims that enumerate several means, several of these means may be embodied by the same hardware. The use of the terms first, second, third, etc., is merely for convenience of expression and does not indicate any order. These terms can be understood as part of the component names.

[0170] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0171] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted to include both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0172] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.

Claims

1. A massage teaching method based on multimodal data collaboration, characterized in that, include: The system collects pressure distribution data and kinematic data during the massage process in real time. The pressure distribution data and kinematic data are preprocessed to obtain preprocessed pressure distribution data and preprocessed kinematic data, respectively. Pressure features are extracted from the preprocessed pressure distribution data to obtain pressure parameters. Simultaneously, motion features are extracted from the preprocessed kinematic data to obtain motion parameters. The kinematic data includes overall palm motion data and fingertip local motion data. The pressure parameters include overall palm pressure parameters and fingertip local pressure parameters. The motion parameters include overall palm motion parameters and fingertip local motion parameters. Using the motion cycle of the overall hand movement corresponding to the preprocessed overall hand movement data as the time reference, the overall hand movement is divided into overall stages to obtain the overall stage time sequence. Based on the fingertip local movement parameters, the fingertip local movement corresponding to the preprocessed fingertip local movement data is divided into local stages to obtain the local stage time sequence. The local stage time sequence is associated and mapped with the overall stage time sequence to construct the stage association mapping rule between the overall hand movement and the fingertip local movement. Based on the stage association mapping rule and the current collection timestamp, the overall palm motion parameters and the local fingertip motion parameters are sliced ​​according to a preset time slice to obtain the overall palm motion parameters and local fingertip motion parameters under all time slices. A three-dimensional coordinate system for the hand is established with the center coordinate of the back of the hand in the overall hand motion parameters paired in each time slice as the origin coordinate. Based on the three-dimensional coordinate system, the fingertip coordinates in the corresponding fingertip local motion parameters are converted into relative coordinates to achieve spatial calibration of the overall hand motion parameters and fingertip local motion parameters paired in each time slice, so as to obtain the spatially calibrated overall hand motion parameters and fingertip local motion parameters paired in all time slices. The overall hand motion parameters and local fingertip motion parameters paired under all time slices after spatial calibration are input one by one into the pre-trained motion association model to calculate the motion association strength, thereby obtaining the motion association strength corresponding to all time slices. Based on the motion association strength corresponding to all time slices, the overall hand motion parameters and local fingertip motion parameters that coordinate the motion are selected. The overall motion parameters of the hand and the local motion parameters of the fingertips are fused with the pressure parameters to obtain dual-mode fusion features. The dual-mode fusion features include the motion synchronization rate of the hand and fingertips and the force-velocity synchronization rate of pressure and speed. The dual-mode fusion features, pressure parameters, overall hand motion parameters, and fingertip motion parameters are used as input to a pre-trained massage technique recognition model to identify the massage technique. This process yields the current massage technique type. Based on this type, matching standard massage parameters are retrieved from a pre-built standard technique library. The difference between the standard massage parameters and the current massage parameters is calculated. A current massage score is generated based on the difference score, and personalized massage teaching guidance suggestions are generated based on the current massage score.

2. The massage teaching method based on multimodal data collaboration as described in claim 1, characterized in that, The preprocessing includes noise reduction. The pressure distribution data and the kinematic data are preprocessed separately to obtain preprocessed pressure distribution data and preprocessed kinematic data. Pressure features are extracted from the preprocessed pressure distribution data to obtain pressure parameters. Simultaneously, motion features are extracted from the preprocessed kinematic data to obtain motion parameters, including: The pressure distribution data is denoised by median filtering and Gaussian smoothing respectively to obtain denoised pressure distribution data. Based on the preset hand contact area division rules, the denoised pressure distribution data is divided into palm area pressure distribution data and fingertip local pressure distribution data. The pressure distribution data of the palm region and the local pressure distribution data of the fingertips were converted into two-dimensional pressure heat maps using bilinear interpolation, resulting in a two-dimensional pressure heat map of the entire palm and a two-dimensional pressure distribution map of the local fingertips. Then, voxel reconstruction technology was used to reconstruct the two-dimensional pressure heat maps of the entire palm and the local pressure distribution maps of the local fingertips into three-dimensional pressure heat maps, resulting in a three-dimensional pressure heat map of the entire palm and a three-dimensional pressure heat map of the local fingertips. Pressure features were extracted from the three-dimensional pressure heat maps of the entire palm and the local pressure heat maps of the local fingertips to obtain the overall pressure parameters of the palm and the local pressure parameters of the fingertips. The overall hand motion data is denoised using Kalman filtering to obtain denoised overall hand motion data. Simultaneously, the local fingertip motion data is denoised using particle filtering to obtain denoised local fingertip motion data. Motion features are extracted from both the denoised overall hand motion data and the denoised local fingertip motion data to obtain the corresponding overall hand motion parameters and local fingertip motion parameters.

3. The massage teaching method based on multimodal data collaboration as described in claim 1, characterized in that, The process involves inputting the paired overall hand motion parameters and fingertip local motion parameters from all time slices after spatial calibration into a pre-trained motion association model to calculate the motion association strength, resulting in the motion association strength for all time slices, including: The overall hand motion parameters and fingertip local motion parameters paired under all time slices after spatial calibration are input one by one into the pre-trained motion association model. The constraint deviation of the overall hand motion parameters and fingertip local motion parameters paired under each time slice is calculated by the four types of preset constraints built into the motion association model, so as to obtain the four types of constraint deviations corresponding to all time slices. The four types of preset constraints include displacement constraints, velocity constraints, acceleration constraints and posture constraints. The four types of constraint deviations include displacement constraint deviation, velocity constraint deviation, acceleration constraint deviation and posture constraint deviation. The displacement constraint condition is as follows: ； in, Indicates displacement constraint deviation. This represents the angle between the tangent directions of the overall hand motion trajectory in the paired overall hand motion parameters at time slice i. This represents the angle between the tangent directions of the fingertip local motion trajectory in the paired fingertip local motion parameters under time slice i. Indicates the first threshold; The speed constraint condition is: ； ; in, This represents the overall hand velocity in the overall hand motion parameters paired at time slice i. This represents the local velocity of the fingertip in the paired local motion parameters for time slice i. Indicates speed constraint deviation. Indicates the first coefficient. Indicates the second coefficient; The acceleration constraint condition is as follows: ； in, Indicates acceleration constraint deviation. This represents the peak value of the overall hand acceleration in the overall hand motion parameters paired at time slice i. This represents the peak value of the local fingertip acceleration in the paired local fingertip motion parameters at time slice i. Indicates the second threshold; The attitude constraint is as follows: ； ； in, This represents the rate of change of the overall hand posture angle in the overall hand motion parameters paired at time slice i. This represents the rate of change of the local fingertip posture angle among the paired local fingertip motion parameters at time slice i. This represents the third threshold. This represents the fourth threshold. Indicates attitude constraint deviation; Substitute the four types of constraint deviations corresponding to all time slices into the weighting formula to calculate the motion association strength, obtaining the motion association strength corresponding to all time slices. The weighting formula is as follows: ； in, Indicates the action correlation strength under time slice i. Indicates the first weight. Indicates the second weight. Indicates the third weight. This indicates the fourth weight.

4. The massage teaching method based on multimodal data collaboration as described in claim 1, characterized in that, The process of fusing the overall hand motion parameters and the local fingertip motion parameters with the pressure parameters to obtain dual-mode fusion features includes: Calculate the time difference between the overall hand motion parameters and the local fingertip motion parameters in the same time slice at the preset key nodes to obtain the time difference for each time slice. Determine whether the time difference for each time slice is lower than the first threshold. If so, the time slice with the time difference lower than the first threshold is taken as the candidate time slice. Calculate the ratio of the overall pressure change rate in the overall palm pressure parameter to the local pressure change rate in the local fingertip pressure parameter under each candidate time slice to obtain the ratio corresponding to each candidate time slice. Determine whether the ratio corresponding to each candidate time slice is within the first preset interval. If not, filter out the candidate time slices whose ratio is not within the first preset interval to obtain the effective time slice. The overall hand motion parameters and local fingertip motion parameters under the effective time slice are fused with the pressure parameters to obtain dual-mode fusion features.

5. The massage teaching method based on multimodal data collaboration as described in claim 4, characterized in that, The process of fusing the overall hand motion parameters and the local fingertip motion parameters under the effective time slice with the pressure parameters to obtain dual-mode fusion features includes: Based on the overall palm motion parameters and the local fingertip motion parameters under the effective time slice, a preliminary technique type is identified to obtain a preliminary technique type. Based on the preliminary technique type, a matching standard pressure distribution entropy is called from a pre-built standard technique library. The overall palm pressure distribution entropy in the overall palm pressure parameters and the local fingertip pressure distribution entropy in the local fingertip pressure parameters are integrated to obtain the current pressure distribution entropy. The current pressure distribution entropy, the standard pressure distribution entropy, and the effective time slice are input into the first synchronization rate formula to calculate the synchronization rate of hand and fingertip movements. The first synchronization rate formula is as follows: ; ； in, This indicates the synchronization rate between the movements of the palm and fingertips. This represents the pressure distribution entropy correction coefficient. This represents the current pressure distribution entropy. Represents the standard pressure distribution entropy. Indicates the number of valid time slices. Indicates the total number of time slices; The average pressure value of the entire palm from the overall palm pressure parameters and the average pressure value of the fingertip from the local fingertip pressure parameters are integrated to obtain the current average pressure value sequence. Based on the preliminary technique type, a matching standard speed sequence is retrieved from a pre-built standard technique library. The standard speed sequence and the current average pressure value sequence are input into a second synchronization rate formula for calculation to obtain the force-velocity synchronization rate of pressure and speed. The second synchronization rate formula is: ； in, The force-velocity synchronization rate, representing pressure and velocity. This represents the Pearson correlation coefficient between the current average pressure series and the standard velocity series. This represents the current average pressure value sequence. Represents a standard velocity sequence. This represents the dynamic time-warped distance between the current average pressure value sequence and the standard velocity sequence. This represents the largest dynamic time warp distance in the standard technique library.

6. The massage teaching method based on multimodal data collaboration as described in claim 1, characterized in that, The process involves inputting the dual-mode fusion features, pressure parameters, overall hand motion parameters, and fingertip motion parameters as the current massage parameters into a pre-trained massage technique recognition model to identify the massage technique type. Based on this current massage technique type, matching standard massage parameters are retrieved from a pre-built standard technique library. The difference between the standard massage parameters and the current massage parameters is then calculated, including: The current massage parameters are normalized to the [0,1] interval using Min-Max normalization to achieve standardization of the current massage parameters and obtain the standardized current massage parameters. The random forest algorithm is used to select N core parameters from the standardized current massage parameters, and the principal component analysis algorithm is used to reduce the dimensionality of the N core parameters to obtain the N core parameters after dimensionality reduction. The N core parameters after dimensionality reduction are input into the pre-trained massage technique recognition model to identify the massage technique and obtain the current massage technique type. The pre-trained massage technique recognition model is built and trained based on an LSTM network. Based on the current massage technique type, N standard massage parameters that match the N core parameters after dimensionality reduction are called from the pre-built standard technique library, and the difference between the N standard massage parameters and the N core parameters after dimensionality reduction is calculated.

7. The massage teaching method based on multimodal data collaboration as described in claim 6, characterized in that, The difference between the calculated N standard massage parameters and the N core parameters after dimensionality reduction includes: Obtain the parameter type for each core parameter. When the parameter type is numerical, use the difference formula to calculate the difference between the core parameter and the corresponding standard massage parameter. When the parameter type is directional, the difference between the core parameter and the corresponding standard massage parameter is calculated using the vector angle cosine theorem. When the parameter type is time series, determine whether the time series is a periodic uniform time series. If yes, use the coefficient of variation method to calculate the difference between the core parameter and the corresponding standard massage parameter. If no, use the waveform similarity algorithm to calculate the difference between the core parameter and the corresponding standard massage parameter. When the parameter type is stable, the difference between the core parameter and the corresponding standard massage parameter is calculated using the standard deviation formula.

8. A massage teaching system based on multimodal data collaboration, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

Images